Bug in OS X with Last Open Date in Spotlight and Quarantine flags

I’ve just filed a bug with Apple that affects one of Trickster’s core features and also affects all users of OS X, thought they might not realize or care about it.

The problems is related to Last Open Date metadata not being updated for files that have quarantine attribute set in the file system (either because they were downloaded from the Internet or because they were saved from a Sandboxed application, which apparently can also set a different kind of quarantine bit.

I first mentioned it at Apple’s developers forums at https://forums.developer.apple.com/message/66280 and have now submitted a proper bug report (rdar://23193803).

Here’s the contents of the bug report:

Summary:

Most files that are saved from sandboxed applications (or downloaded from the Internet) seem to have their quarantine flag set to some value. For some reason, for most of these files, Last Used Date (kMDItemLastUsedDate) metadata won’t be updated by Spotlight when this file is opened.

It appears that this problem started in OS X Lion (see Workarounds for quarantine bug in OS X Lion on CNET). I also noticed that the problem is not always showing itself. For example, if the first 4 digits are something like 0041 or 0042, the last used date will update. If it’s something else like 0001, it won’t.

I started a thread on it at Developer forums some time ago at https://forums.developer.apple.com/message/66280. No response so far.

Steps to Reproduce:

  1. In Finder, go to Downloads folder and add Date Last Opened column and sort by it in descending order
  2. In Safari, go to http://www.apparentsoft.com, right click on ImageFramer icon and “Save Image to Downloads”.
  3. It should appear on the top of the list in Finder as the file is created with the open date = date added.
  4. Open other files in Downloads to move them to the top of the list (though it might not work because of this bug). If no files move to the top, wait until the clock changes to a different minute.
  5. Double click the downloaded PNG from Finder. It should open in default application, like Preview.

Expected Results:

Last Open date of the PNG file is updated to the time in step 5.

Actual Results:

Last Open date doesn’t change.

Version:

10.11.0

Notes:

Removing the quarantine attribute from the file and then opening it will bring it to the top and update its Last Open date.

Removing of the attribute can be done in the Terminal with:

xattr -d  com.apple.quarantine <path/to/file>

TL;DR Programming

TL;DR, an abbreviation for “Too long; didn’t read”, is Internet slang used in reply to a lengthy online message. It is also used at the beginning of a summary of such a message. — Wikipedia.

TL;DR

This essay explores TL;DR in programming context.
It tries to find out if the trend of TL;DR behaviour spreads to software development as well, and what are the dangers of that.
It’s about Bob. It’s not about you. Good programmer don’t TL;DR. Good programmers read.

Enter Bob

Robert Kode1 is a long-time programmer. Bob loves to code. He was programming since his early teens and is fluent in six languages. He speaks only English but can read and write the other five. These five are not used for speech.

Over the years Bob developed good know-how in many areas of software development and has an intuitive understanding of many concepts. He can crunch code fast and his code is fast.

It just so happens that Bob is a TL;DR programmer.

Too Long; Didn’t Read

TL;DR comes from the increasingly short attention span of today’s Internet generation. Do programmers also fall victim to the short attention span which leads to TL;DR?

Bob believes that programming is about writing. Writing code. Writing code comments. Writing commit messages.

Bob was assigned to write a new module for an application. He skimmed the requirements for the module. He skipped half the paragraphs in the email that described some focus points. He didn’t get back to them when he churned out lines of code.

Eventually, Bob’s module implemented half the cases it was supposed to support. In some cases, it did what it wasn’t meant to do. QA filed a ton of bugs and Bob found himself in Tester Driven Development mode.

Now he cannot afford to skip reading the bug reports.

Too Lazy; Didn’t Research

How much research had to be done before writing the first line of code for Apollo Project?

Bob’s got years of programming experience, as I said, so he relies on his intuition. Oftentimes more than he should.

This time our Mr. Kode got the kind of task that he liked. He was supposed to implement a peculiar data processing algorithm for, well, the data.

Bob had prior experience writing some algorithms and was excited about the new challenge. He conceived the general approach of the algorithm as he was reading the requirements. He might have overlooked some of the intricacies of the requirements (see above) so it didn’t help either.

Bob went right ahead to writing it. After half a day it kind of worked. There were some edge cases that required attention. It was also slow. It also didn’t scale well for high amounts of data. Another day went on trying to optimize it. Then another day. Then it started giving the wrong results. And crashing on the edge cases.

Robert didn’t invest the time to think through his idea before developing. It’s not easy, so he trusted his intuition instead.

Bob also didn’t run a relevant search on Google. If he had, he might have stumbled upon a paper, or a blog post, or an answer on Stack Overflow, any of which might have helped him select and implement the right approach. Research is hard. Bob didn’t research. Bob was a bit too lazy.

They say good programmers are lazy. But they say it for a different reason.

Q&!A

Q: Does TL;DR programming lead to the culture of Stack Overflow questions of “please give me code to do X”?

Q: Do we see more bugs today because of TL;DR attitude? Including critical bugs like Boeing 787 bug which could bring the whole plane down in one case? Or more security omissions?

Q: How many programmers just try things out until they work, sometimes without understanding the theoretical basis of why and how? Or choose a simpler, inadequate solution?

Q: Could this also apply to other aspects of the software business, like product and project management, software architecture and market research?

Q: Are you a little like our Bob?
A: I am


  1. Fictional character

Song: App Oddity

I made another musical video. This time I remade David Bowie’s excellent “Space Oddity” with lyrics on the subject of ups and downs of indie app development. It should be more accessible to the general population than my previous video about the programming language Swift.

Lyrics are embedded in the video but here they are in full:

App Oddity
~~~~~~~~~~

App Review to Coder Tom
App Review to Coder Tom
Take your caffeine pills
And put your headphones on

App Review to Coder Tom
Commencing countdown,
you’re logged on.
Check submission
and may Jobs’ love be with you

This is App Review to Coder Tom
You’re featured on first page
And the web sites want to know how much you’ve made
Now it’s time to go out and celebrate

This is Coder Tom to App Review
I’m stepping through the doors
And my head spins in a most peculiar way
And the App Store’s very different today

For here
Am I sitting in my basement
Far from outside world
Why do I feel blue
Is there something I can do?

Though I’m past
one million free downloads
I can’t see many sales
And I think I might've chosen wrong way to go
Tell my customers I care, 
'cause they should know

App Review to Coder Tom
Your app is dead,
there’s something wrong
Can you fix it Coder Tom
Can you fix it Coder Tom
Can you hear us Coder Tom
Can you…

Here am I lying in my basement
Far from outside world
All I feel is blue
And there’s nothing I can do

App Camp for Girls and an interview at AppStories

I’ve been following the App Camp for Girls project since it started not a long time ago. App Camp for Girls is a non-profit founded by my friend Jean MacDonald from Smile. Its mission is:

App Camp For Girls wants to address the gender imbalance among software developers by giving girls the chance to learn how to build apps, to be inspired by women instructors, and to get exposure to software development as a career.

They’re currently raising funds for the project, so consider donating to this fine endeavor. While it’s already reached its initial funding goal, they’re thinking about expanding the program to additional locations, so any contribution will help.

To inspire young people to learn application development they started App Stories, where they interview developers about their first steps in software development.

There are already several interesting interviews available from developers and designers, including an interview with yours truly. More interviews with seasoned developers are going to be published on the site later, so you might want to add it to your RSS feed.

Have I already asked you to donate? Just a reminder. 🙂

Toggle hosts file for less distractions (in Haskell)

I decided to edit the /etc/hosts file to block access to some domains that I compulsory visit when working, in order to reduce my distractions. The usual suspects are Twitter, Facebook, Hacker News, and some real-world news sites.

So I edited the file and added the following lines:

# -- WORK --
127.0.0.1     news.ycombinator.com
127.0.0.1     facebook.com
127.0.0.1     twitter.com

Now they’re blocked. But I also wanted the option to quickly comment this section out and back again.

I could do it quickly in Ruby but I thought that it would be a great way to exercise my basic Haskell skills.

It took me quite a while to get it right — my Haskell is rusty. But after it compiled, I only had one little logical bug, which I quickly found out and fixed.

The final code is below. It also backs up hosts file (just in case) and prints the resulting hosts file. Both of these steps could be removed, as they’re not important for its main functionality.

import Data.List

-- Starts toggling the comment after a line that starts with "# --"

processLine switch line
    | length line == 0                          = (switch, line)
    | switch == False && take 4 line == "# --"  = (True, line)
    | switch == True                            = (True, toggleLine line)
    | otherwise                                 = (switch, line)
      where toggleLine l
              | head l == '#' = tail l
              | otherwise     = "#" ++ l

process = snd . mapAccumL processLine False

main = do
  hosts <- readFile "/etc/hosts"
  writeFile "/etc/hosts.backup" hosts
  let newHosts = unlines $ process (lines hosts)
  putStrLn newHosts
  writeFile "/etc/hosts" newHosts

I’m sure this could be done shorter, as everything in Haskell, but I’m just a beginner.

I then compiled the code and put the binary in my user’s bin folder.

Since it edits the /etc/hosts file, it has to be run with sudo. I added a line to sudo permissions (run sudo visudo to edit the file) to allow me to run it without a password:

jacob ALL = NOPASSWD: /Users/jacob/bin/switch_hosts

Then, I added an action to Keyboard Maestro to run sudo /Users/jacob/bin/switch_hosts using a trigger in its system menu. You could do a hotkey trigger or use a TextExpander expansion, or use LaunchBar, or Alfred, or just type it in the Terminal.

It just was this kind of day. I couldn’t do real work, but I could play a little to make my environment better for when I’ll be able to work.

Of course, there are commercial products that can help with distraction-free work environment, like Concentrate, which offers this and more. But I prefer to run less applications, if possible.

Introducing ReactiveCoreData

Lately I’ve became seriously interested in the fantastic projects that several bright guys at GitHub open-sourced. I’m talking about ReactiveCocoa and some of its derivatives, like ReactiveCocoaLayout. If you don’t know what these are, first go and read about them. I’ll wait.

While working on UI code for Cashculator 2, I decided to utilize both of above frameworks and, so far, I’m very happy with my decision. Especially I loved how ReactiveCocoaLayout allowed me to create behaviors similar to Cocoa’s AutoLayout, which I can’t use in Cashculator for performance reasons.

Like many applications, Cashculator uses Core Data for data persistence. So I had the idea of trying to bring Core Data into the ReactiveCocoa world.

Thus, ReactiveCoreData (RCD) was born. I’ve already implemented most of the basic functionality that I wanted to include in a hypothetical 1.0. Or is it 0.1. There’s also a proof of concept demo application for the Mac, which shows only a little of how to use it.

But there are also specs which verify all the functionality, which include easy fetching, insertion, performing stuff on a background context, saving, merging. All of this in the ReactiveCocoa’s signals domain, so it looks neat and can interoperate with your other ReactiveCocoa world.

A short example:

RAC(self.filteredParents) = [[[[Parent findAll]
  where:@"name" contains:filterText options:@"cd"]
  sortBy:@"name"]
  fetchWithTrigger:objectsChanged];

This will update filteredParents property (an NSArray) to the all the Parents whose name contains text in the signal filterText (that comes from a search field, for example), sort by name and fetch.

It will fetch when either the filterText signal sends a next value or when objectsChanged signal fires (it will fire after new Parent objects were added or removed, for example).

The where: and sortBy: commands modify the NSFetchRequest that’s started with the findAll method of Parent and pass it next. Then, fetch: runs the NSFetchRequest in the current NSManagedObjectContext and sends the NSArray of the result, which gets assigned to self.filteredParents.

There’s more to ReactiveCoreData than that, but it’s a nice example.

Check ReactiveCoreData on GitHub.

I’ll be happy to get any feedback: suggestions, critique, pull requests. Use GitHub, if possible.

If you prefer to give feedback over Twitter, I’m @apparentsoft. But you better use App.net. I’m @apparentsoft over there as well.

Software Architecture cheat sheet

For the past several weeks I’ve been focusing my efforts on learning how to approach software architecture. Despite my experience in developing several applications, I wanted to read and learn more about this to do a better job in the future, for our upcoming project.

I’ve read some articles and a couple of books on this topic. My goal was to extract relevant pieces of knowledge with the final goal of producing one sheet of paper with the most important points. I wanted to stick it onto the wall behind my screen so that I could refer to it with only a glance. It had to have the most important questions that I should ask myself before committing to anything important during software design.

Today, I finally condensed all I wanted to write, prepared this page, printed and glued it to the wall:

I prepared a PDF that you can download and print to put in your workplace if you like this.

Here are the points and their explanations:

Is this a “Good Idea”?

I took this one from Avoid “Good Ideas”

The really insidious thing about “good ideas” is that they are “good.” Everyone can recognize and reject “bad” ideas out of hand – It’s the good ones that slip through and cause trouble with scope, complexity, and sheer wasted effort incorporating something into the application that isn’t necessary to meet the business need.

In other words, think if this is necessary or is it just feature creep. Software complexity increases exponentially so having twice the features makes the code complicated by much more than twice.

DRY. Don’t Repeat Yourself

DRY is a well-known software design principle. Explained in Pragmatic Programmers, it means that “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”

I already have an ingrained instinct about DRY. Still, I wanted to have a good in-the-face reminder so as not to forget it in a middle of a passionate idea.

Orthogonal?

In other words, how independent is this module or block from other parts of the system? Modularity in software is important. It makes life easier later. It’s easy to understand and easy to overlook.

Testable?

Think how you will test this part of the system. Is it designed in such a way that makes testing easy? Testability depends on modularity, complexity, code style. Don’t neglect testability of your system or code. Testing is important. Manual testing is good. Automatic testing is much better.

Is there another way?

This is also from Pragmatic Programmers. It contains good quotes in every chapter. Here’s one by Emil-Auguste Chartier — “Nothing is more dangerous than an idea if it’s the only one you have.”

Often, when thinking about problems we come up with a solution and run to implement it, feeling happy that we have one. Unfortunately, often the first solution won’t be the best. Think again, be creative, try a different approach. When you come up with more ideas, it will often be the case that the first one was too rushed and that the next ones are, in fact, much better.

Costs of changing this later

Consider the decision you’re about to take. What will it cost you to change it at later stages of the project? Can you defer this decision? If you’re not sure about the decision, can you design the system in such a way that changing it later will not break things and cause unnecessary work?

What if I didn’t have this problem?

This one is from Don’t Be a Problem Solver.

Because architects tend to immediately enter problem-solving mode, we forget, or rather have never learned how, to interrogate the problem itself. We must learn, like a telephoto lens, to zoom in and zoom out, in order to ensure the question is really framed properly, and that we’re not merely accepting what we’re given.

Instead of immediately working to solve the problem as presented, see if you can change the problem. Ask yourself, what would the architecture look like if I just didn’t have this problem? This can lead ultimately to more elegant and sustainable solutions.

What are facts and assumptions? Document rationale.

The last two are from Challenge assumptions – especially your own.

Best practices in software architecture state that you should document the rationale behind each decision that is made, especially when that decision involves a trade-off (performance versus maintainability, cost versus time-to-market, and so on)…

This practice is valuable because by way of listing these factors, it helps highlight assumptions that the architects may have that are affecting important decisions regarding the software that is being designed. Very often these assumptions are based on “historical reasons”, opinion, developer lore, FUDs…

In other words, if you’re designing software architecture, write down why specific decisions are made and then think what of these are facts and what are assumptions. Check your assumptions.

Facts and assumptions are the pillars on which your software will be built. Whatever they are, make sure the foundations are solid.

Download the PDF with these question to stick on your wall.

Jacob.
I’m @jacobgorban on Twitter

Using multi-resolution TIFF files with WebKit on Mac for Retina support

I’m working on Retina support for our Mac application Trickster. Most of the things are pretty standard – either creating PDF images instead of PNGs or creating two PNGs for regular and @2x image.

During the build process the latest Xcode will take the two PNGs and create one multi-image TIFF file which is inserted into the built application’s Resource folder.

Trickster uses WebKit to display its Help screens. Two are just text but one is an image.

Previously I only loaded the image directly, without HTML with:

[[NSBundle mainBundle] URLForImageResource:@"Trickster Help"];

To support Retina version of the image, I created the @2x version, added it to the bundle and went to look how to use the media queries and other CSS tricks to display the double-resolution image on Retina display. I had to build an HTML file and load it into WebKit instead of the PNG:

[[NSBundle mainBundle] URLForResource:@"Trickster Help" withExtension:@"html"];

I built the HTML:

<!DOCTYPE html>
<html>
<head>
    <style type="text/css" media="screen">
    body { margin:0 }

    div#help {
        width: 740px;
        height: 562px;
        background-image: url(Trickster%20Help.png);
        background-size: 740px 562px;       
    }

    @media screen and (-webkit-min-device-pixel-ratio: 1.5), screen and (min-device-pixel-ratio: 1.5) {
      div#help {
        background-image: url(Trickster%20Help@2x.png);  
      }
    }

    </style>
</head>
<body>
    <div id="help"></div>
</body>
</html>

It worked great when I checked it in the standalone Safari. Safari even dynamically substituted the image used based on where the window was when I dragged it.

The problem was in the built application. The PNGs are converted to TIFF, by default, and the HTML can’t find them so it doesn’t display anything.

I thought “OK, I’ll just change the CSS to point to TIFF image”. Then I thought – but I only have one TIFF, what do I put in the media query?

The solution was unexpectedly simple: point background-image URL to the TIFF file and don’t use the media query at all. WebKit handles the double-resolution TIFFs automatically and selects the Retina version when the window is on the Retina screen and regular one, when on regular screens. Yes, it even does that dynamically when dragging Trickster’s help window.

Here’s the final HTML:

<!DOCTYPE html>
<html>
<head>
    <style type="text/css" media="screen">
    div#help {
        width: 740px;
        height: 562px;
        background-image: url(Trickster%20Help.tiff);
        background-size: 740px 562px;       
    }
    body {
        margin: 0;
    }

    </style>
</head>
<body>
    <div id="help"></div>
</body>
</html>

To sum it up — using WebKit with Retina images inside your Mac application becomes very easy. It just works if you point your HTML to multi-resolution TIFF files instead of PNGs. And Xcode generates these TIFF files by automatically. Easy win!

It’s probably not an appropriate solution for general Web use as I don’t think that other browser will support multi-resolution TIFFs. It also requires visitors to download large TIFFs even if they don’t use them (and let’s not forget – most people don’t have Retina displays).

I wonder if a similar solution works on iOS. I haven’t tried. Would love to hear result, if you try it.

You can get in touch with me on Twitter at @apparentsoft.