Mike Swanson's Blog

This week, I’ve been working on a radial menu control for...

Fri, 30 Mar 2012 13:32:10 -0700

[Flash 10 is required to watch video.]

This week, I’ve been working on a radial menu control for an upcoming Juicy Bits app. I recorded a short video that illustrates the evolution of the project and shows the menu in action. The radial menu has proven to be a quick and convenient way to select context-sensitive options. Also, because the buttons are equidistant from the center, your finger doesn’t have to travel around the screen as much.

Smaller Apps with Vector Images

Sat, 24 Mar 2012 20:45:00 -0700

3/26/2012: Added an update at the end of the post.

For the past couple of weeks, I’ve been working on new vector processing code for a future update of my Halftone app for iOS. As I’ve mentioned before, Halftone draws a lot of its graphics using vectors, and as a result, it automatically takes advantage of the new iPad Retina display. These recent experiments have forced me to take a closer look at vector handling in iOS, and I thought I’d share what I’ve learned.

By the end of this post, I hope to convince you that a vector version of your app may require only 14.8% of the space that a bitmap-based Retina version requires.

Before I get going, it’s important to mention that—while I love vectors—I love bitmaps too! Halftone contains a lot of bitmap images, and I spend a lot more time in Photoshop than I do in Illustrator. So don’t walk away thinking that I’m recommending vectors for everything.

That said, here’s how I generally break it down:

For small images that aren’t photographic, I use PNG images. This includes icons, UI elements, and logos, to name a few. If every pixel and edge matters, a bitmap provides the most control.
For photographic images, I use JPGs, no matter what the size.
For large images that aren’t photographic, it depends…

Though the iOS SDK has great support for bitmap image formats like JPG and PNG, it doesn’t include any support for vector image formats like SVG or EPS. To be fair, the UIWebView control can be used in a pinch, but it doesn’t provide access to the raw image data (including its alpha channel), and it’s often a much heavier solution than necessary.

The current version of Halftone draws each vector with Objective-C code that’s been exported directly from Adobe Illustrator using a custom plug-in that I’ve built. This method works well, but it requires that the vector image code is compiled into the app itself before it’s submitted to the App Store. One downside of this method is that there’s no way to enable a Halftone user to add their own vector image files. Nor is there a way to include in-app purchase for vector images (apps are not allowed to download code after-the-fact). These limitations led me to consider a new workflow.

Ever since the advent of HTML5, SVG files have enjoyed a bit of a resurgence (the format was invented in 1999), and they’re supported by all modern web browsers. The two dominant vector editing tools, Adobe Illustrator and the Open Source Inkscape, have excellent support for creating and editing SVG documents. This means that any image that can be opened by those tools can easily be saved to an SVG file.

My goal was clear: I needed to find a SVG parser and renderer for iOS!

After some searching, I found a few projects that purported to handle simple SVG files but none that claimed robust support for the format (at least robust enough). So, I downloaded each project, and it only took a few quick tests to find the rough edges. The existing projects are good enough for basic support, and if that’s all you need, you’ll probably be fine with any of them.

Like any good developer, I have a healthy dose of naïveté. If I had realized what I was getting myself into, I probably wouldn’t have started. As it goes, I’ve spent the last couple of weeks nearly memorizing the SVG specification, and I’ve become more than familiar with the rules of the CSS cascade. Thankfully, I have no need for scripting, external style sheets, events, or any of the fancy text-based support, and I’ve been able to avoid those sections of the spec.

I’ve built a test app that allows me to open an SVG file and display the UIWebView/WebKit-rendered version by holding down a “Show Reference” button. There’s also an “onion skin” mode that fades the UIWebView to 50% so that I can see both renderings simultaneously, though it’s not as useful as flipping back-and-forth with the button. As an aside, does anyone remember those advertisements (perhaps from National Geographic?) where they’d show an elephant, then flip between a normal version and a version that didn’t include its trunk? If memory serves, they’d say something like: “did you see that?” Well, that’s me testing my renders against WebKit; I’m constantly looking for pixels that appear or disappear.

To test my parsing and rendering, I’ve used the SVG test suite. The suite is incredibly useful, as there are many complex rules that—no matter how many times you read the spec—can be ambiguous. Many of the tests use crafty or clever techniques, and they’re quick to expose issues. I’ve also collected a set of common SVG files, and as acid tests, I’ve exported some extremely complex vector images from Adobe Illustrator. Here’s a file from the SVG test suite rendered by my test app on the left and by UIWebView/WebKit on the right:

To test the extremes, here’s the Rise Together sample that ships with Adobe Illustrator CS5. The exported SVG file is a full 21MB (!), and it takes my non-optimized test app a full 30 seconds to parse and render it! As before, my version is on the left, and the UIWebView version is on the right. Not sure what UIWebView is up to here, but I’ll take the “win,” as I usually lose to WebKit.

A Rise Together close-up from a Retina display:

Here’s another acid test that uses the Yellowstone Map sample from Illustrator. It’s not as complex as Rise Together, but it includes a lot of text. While my needs don’t require robust text support, it’s good to see that the test app can cover the basics. As a side note, I’ve noticed that UIWebView appears to ignore some font kerning and ligature information while rendering.

A Yellowstone Map close-up from a Retina display:

Here’s a more reasonable example that illustrates a plant cell structure. It’s easy to imagine diagrams like this being featured in reference or magazine apps:

A Plant Cell Structure close-up from a Retina display:

Finally, here’s a popular lion sample that hits the sweet spot. It parses and renders quickly, and it’s easy to imagine the image (along with its alpha channel) being used as an element or character in an app:

Being vector-based, all of these images can be rendered at any size, and they can easily be transformed (positioned, scaled, rotated, skewed, etc.) without any loss of quality.

Another major benefit of vector-based formats is file size. SVG documents are XML documents, and they’re very compressible. On average, my sample files ended up being around 33% of their original size when packaged in an IPA file (the compressed format that’s used to package iOS apps).

To compare format sizes, I used four versions of eight different SVG test files:

The original, untouched SVG file.
A PNG export of the SVG file from Adobe Illustrator (including an alpha channel) that fills the 2,048 x 1,536 pixel dimensions of the new iPad Retina display and preserves its aspect ratio.
A PNG that fills the 1,024 x 768 pixel dimensions of the standard display on an iPad/iPad 2.
A compiled stream of Objective-C code that uses Quartz 2D commands to draw the SVG file. The code is exported by my test app after the parsing phase and it represents the steps that would be required by almost any SVG renderer. It’s not completely optimized, but it’s still useful for comparison. Other than loading a bitmap, it’s about as fast as you can get.

To accurately measure the payload of each image as it would be distributed in an app, I started by creating an image-free reference IPA file. Then, one by laborious one, I built individual IPA files for each sample image using the four different data types (that’s a total of 1 + (8 x 4) = 33 IPA files!). Finally, I calculated the difference between the reference IPA file and each IPA + image file to get a measure of its payload in bytes.

Here are the results for the eight sample files (in bytes):

Example	SVG	SVG in IPA	Retina PNG	Ret PNG in IPA	Standard PNG	Std PNG in IPA	Obj-C Code
Butterfly	31,187	10,860	211,519	189,155	202,590	185,621	22,984
Giza Pyramid Complex	934,950	281,182	320,211	309,018	327,854	355,424	411,178
Lion	18,324	6,917	291,593	281,530	260,699	264,833	36,704
Metadata Example	21,675	8,876	143,347	129,291	97,823	99,179	18,310
Plant Cell Structure	189,161	46,420	436,100	432,548	316,653	333,397	184,332
Rise Together	21,550,150	8,212,273	1,425,383	1,399,053	969,032	986,736	359,420
Tiger	96,719	31,780	402,998	397,482	308,423	302,995	79,902
Yellowstone Map	2,194,130	550,520	723,806	729,014	565,504	563,129	1,978,212

As you can see from the table, images that are comprised of strokes, fills, and gradients can be represented very efficiently as SVG files and compiled Objective-C. The more complex an image becomes (and therefore, the more instructions it takes to describe the image), the more PNG files start to show their strength. Sometimes, it’s just quicker and easier to store the result of a parse/render than it is to draw an image when you need it. If you’re wondering, yes, I double-checked the Giza PNG numbers.

Remember that if you’re shipping an iPad Retina-capable app, you need to include at least two PNG files for each image: one at standard resolution, and one at Retina resolution. With vector images, you only ever have to store one, and it’ll never change in size. This can be a huge payload win. For apps that share artwork with desktop apps, using vectors could also help to future-proof assets in anticipation of possible Retina support in Mountain Lion.

Here’s a look at the relative payload sizes for each of the sample images:

Example	SVG App	PNG App	SVG/PNG
Butterfly	10,860	374,776	2.9%
Giza Pyramid Complex	281,182	664,442	42.3%
Lion	6,917	546,363	1.3%
Metadata Example	8,876	228,470	3.9%
Plant Cell Structure	46,420	765,945	6.1%
Rise Together	8,212,273	2,385,789	344.2%
Tiger	31,780	700,477	4.5%
Yellowstone Map	550,520	1,292,143	42.6%

The last big issue to consider is performance. Bitmap images are pre-baked and ready to go…just decompress the data and pump pixels to the screen. Vector images require additional parsing, computation, and drawing, and they often take much longer to render. For simple images that don’t need to be available instantly, this can be done in real-time (as examples, both the Butterfly and Lion sample images take about 0.28 seconds to load, parse, and render on a 3rd generation iPad using my non-optimized test app). For images that need to be available instantly, this processing overhead may be unacceptable. One strategy is to parse and render the SVG image the first time, then cache the final bitmap to disk. This way, you get the benefit of shipping a much smaller app, the quality of near-infinite-resolution vector images, and the raw performance of a bitmap.

The compiled Objective-C code method saves the payload of the parser/renderer but is just a bit heavier than an embedded SVG file. However, as there is no parsing/cascading step to perform, the code is very fast. It’s an interesting compromise for images that can be baked-in to the app.

Imagine the characters and backgrounds for a game like Angry Birds. These relatively simple line-art images could be drawn off-screen using vectors between levels, then sent to the screen as bitmaps (this is probably what they’re doing anyway, but with OpenGL). Or, they could be rendered and cached to disk the first time the app runs. There are lots of apps that use vector-based art but ship with PNG versions.

To summarize, if we throw out the Rise Together acid test where SVG clearly loses to PNG, on average, a vector-based app that embeds SVG files requires around 14.8% of the space that a Retina-capable app requires with its dual PNG files. Put another way, you can include more than six times the graphical content in the same payload size without losing any quality, all while future-proofing your assets.

Just remember that the right format depends on the nature of your content and the appropriate balance of payload and performance. Only you can know what’s best based on your needs, and as always, test, test, test.

Update on 3/26/2012: I’ve received a lot of questions about the technical nature of the SVG parser and what I intend to do with it. The technical details could probably fill a whole article, but I’ll try to summarize them here.

It wasn’t clear in the original post, but I’ve written a brand new SVG parser and renderer from scratch that isn’t based on any of the referenced projects. It offers near full support for the following SVG elements: circle, clipPath, defs, ellipse, fill, font-face, g, image (including embedded base64-encoded images), line, linearGradient, path, pattern, polygon, polyline, radialGradient, rect, stroke, svg, symbol, text, tspan, and use.

The parser performs an abbreviated CSS cascade that respects both style and presentation attributes (no support for external style sheets, as I have no need for them). It supports all length units and their associated computations, including percentages, ems, exs, points, etc. It has full viewBox support, including preserveAspectRatio and objectBoundingBox units, and support for an arbitrary combination of transforms. IRI/xlink references are also correctly resolved and used.

Additionally, it supports fills and strokes that use solid colors, gradients, and patterns.

The Objective-C generation is based on a “Quartz 2D emulator” that understands contexts and states and outputs the minimum set of commands to render its output.

As far as release plans, I hadn’t given it much thought. My experiments and work are currently focused on Halftone and future Juicy Bits apps, but if there’s enough interest, I’d consider releasing it in some form.

For what it’s worth, I’ve exchanged a few e-mails with the SVGKit folks, and it sounds like they’re pushing for much more robust support in future releases.

Halftone and the iPad Retina Display

Fri, 16 Mar 2012 11:19:00 -0700

Today is delivery day for the new iPad! So, I thought I’d show some differences between the non-Retina display on the original iPad and iPad 2, and the new Retina display on the iPad (3rd generation) using Halftone.

First, it seems common to miscalculate the math behind the new screen. While each dimension has been doubled, because it’s been doubled in both dimensions, there are actually four times (2 x 2 = 4) as many pixels on the screen. This diagram illustrates the difference:

That’s a lot of pixels! If you do the math, you’ll discover that the non-Retina display has 786,432 pixels, and the Retina display has 3,145,728! Not only does the Retina display have four times as many pixels, but those pixels are condensed into the same physical screen size of 9.7 inches (diagonal). Put another way, you could squeeze four iPad 2 screens into the same physical size as one Retina screen on the new iPad. Wow!

So how does it look? As many people have mentioned, it’s tough to show the difference without having an actual screen in front of you, but I’ll do my best with what we have.

Here’s an actual screenshot from Halftone on a non-Retina display. So that it matches the same dimensions as the Retina example below, I’ve used Photoshop’s Nearest Neighbor algorithm to resample it at 200%.

This is how the same area looks on the new iPad Retina display:

If we zoom-in even closer (200%)—again using Nearest Neighbor—you can really start to appreciate the additional detail. Here’s the non-Retina display:

And the same image on a Retina display:

The difference in quality is truly stunning.

For those who are curious, it didn’t take a lot of extra work to add full iPad Retina support to Halftone. Being a Universal app that also runs on iPhone and iPod touch, Halftone has long supported Retina resolution on the iPhone 4/4S. This meant that most of the icons and imagery in Halftone was already at Retina resolution.

Also, Halftone relies heavily on vector graphics for its borders, speech balloons, and stamps. As a result, it required no additional work for these elements to take full advantage of the new display. Another side-effect is that the additional memory in the new iPad also means that larger images can be processed and exported at full quality.

Finally, there’s legitimate concern that Retina-enabled iPad apps will result in much larger app sizes (and therefore require more bandwidth to download and store them on the device). In Halftone’s case, though, the increase is only around 1.5MB, and it’s due to the default screenshots that display when the app launches in portrait or landscape orientation. Otherwise, the vector imagery requires no additional space at all. Aren’t vectors great!?

If you’re a Halftone user, I hope you love how it looks on the new display.

My wife and I visited Andy Keck today to see his amazing 1922...

Sat, 10 Mar 2012 17:00:02 -0800

My wife and I visited Andy Keck today to see his amazing 1922 Intertype Model C machine (named Eloise). Andy and I met via Twitter because of our mutual interest in Linotype: The Film, and we ran into each other at the Seattle premiere this past Wednesday evening. The film is fantastic, and seeing the Intertype machine in-person made it even better (more photos). Thank you, Andy!

Confusing iOS Location Services Prompt

Sat, 25 Feb 2012 13:08:00 -0800

Two days ago, I released an update to my Halftone app for iOS that adds an oft-requested feature: the ability to open photos directly from Facebook, Flickr, and Instagram. The new image picker is the result of more than a month of design and development work, the same code base runs across all devices (with a tailored UI for each device), and it serves as a drop-in replacement for the native iOS image picker. I’m proud of the final product, so I was dismayed to read one of the first tweets about the new version:

One of my favorite apps (Halftone) just updated & added location services as a “key” to continue using the app … WTF?? DELETED!!

Shortly thereafter from another customer:

@juicybits requiring using location services to use app?? 1 star rating

But I’m getting ahead of myself.

Prior to this update, Halftone used the native, built-in UIImagePicker control that every iOS owner is familiar with. It’s the same control that you see when opening an image in most apps. From the app’s perspective, using the image picker is straightforward:

The app asks iOS to display the image picker
You interact with your albums and photos and select an image (or choose to cancel)
The selected image is handed to the app for its use

Note that the app has no direct access to any of the images, and in fact, it doesn’t even know how many albums or images are on your device. All it knows about is the single image that you picked, and when that image is handed to the app, almost none of its metadata is included (like the date and time the photo was taken, exposure information, and GPS coordinates).

For apps that need access to a single image without metadata using the standard look-and-feel, the UIImagePicker control is a fantastic, familiar, and easy to use solution.

However, if an app requires a customized version of this workflow, if it wants to provide a different presentation, or if it needs access to any image metadata, it has to use the ALAssetsLibrary functionality that Apple introduced in iOS 4.0. While ALAssetsLibrary does not provide any standard user interface elements, it does enable an app to access all of the albums and photos on a device.

And—most importantly for this article—ALAssetsLibrary has access to all of the image metadata.

For Halftone, I wanted to build an image picker that would take full advantage of the available screen space, work in all orientations, and have a single codebase for all devices. Not only did it need to pick photos from the device, but it also needed to allow users to just-as-easily pick photos from Facebook, Flickr, and Instagram in a consistent way. So that’s what I built, and as you can now surmise, the new image picker relies on ALAssetsLibrary. Here’s an iPhone screenshot of the new image picker on the Facebook tab:

Many modern cameras and most smartphones (including the iPhone) can automatically embed GPS coordinates that record where a photo was taken. These coordinates are stored in the image metadata, and if you’ve ever used an app that places photos on a map, it’s easy to understand how this location information can be used. It’s likely that you have photos on your iPhone right now that contain these coordinates.

For apps that only use the built-in UIImagePicker to open images, none of the GPS metadata is handed to the app. Not even the date and time the photo was taken. Just the image itself.

For apps that use ALAssetsLibrary, though, all of the image metadata is available, including the GPS location. So, what do you think happens the first time any app tries to use ALAssetsLibrary? Bingo! Because the app could access the location where an image was taken, iOS displays this prompt:

If you’ve read this far, the prompt may not be surprising at all. In fact, if you’re not surprised, I’ll pat myself on the back for a job well done. Good job, Mike!

Well, if you’re among the one million+ casual Halftone users who will never (and should never) read this post, I can tell you that it’s baffling. Especially the misleading initial text: “Halftone” Would Like to Use Your Current Location. No it wouldn’t. It doesn’t do anything with your location. As a matter of fact, it doesn’t even want to see the location metadata in your photo!

Let’s remind ourselves of one user’s response to this prompt:

@juicybits requiring using location services to use app?? 1 star rating

Without any context, this unexpected prompt makes it sound like Halftone is trying to track your location. For users who have heard about the recent misappropriation of address book data by well-known apps, it’s no surprise that they choose “Don’t Allow.” I’d do the exact same thing.

The problem is that by disallowing this permission, Halftone can no longer use ALAssetsLibrary to read photos from the device…at all. From a user’s perspective, Halftone used to be a trusted app that allowed me to open my photos, but this “new and improved” update not only wants to track my location, but it won’t show me the photos on my own device! #epicfail!

To explain why Halftone is prompting for Location Services approval, the new version displays this pop-up just before the prompt:

Normally, I’d never try to explain a complex topic like this to a user who only wants to access their photo library. But, in this situation, it’s important that the user understands why they’re being asked, because the new functionality is dependent on approval. I loathe having to display this text.

Other well-known iOS apps that use ALAssetsLibrary have tried similar approaches. For example, here’s the screen that Dropbox shows just prior to the approval prompt:

For Dropbox users who notice the small little arrow after “Permission is needed to access your photos,” they are treated to a long explanation similar to the one that Halftone uses. They even go so far as to provide a link to the Apple developer documentation for ALAssetsLibrary, which contains this small snippet of text:

Special Considerations
This method will fail with error ALAssetsLibraryAccessGloballyDeniedError if the user has not enabled Location Services (in Settings > General).

Of course, the Dropbox app doesn’t depend on photo access, so denying this permission has no significant impact.

Another example is Mixel, an iPad app by Khoi Vinh that does depend on access to photos. It’s a very well-designed app that takes advantage of ALAssetsLibrary to provide an improved user experience. I envy the simplicity of Mixel’s approach in that there is no attempt to explain anything about Location Services. The permission prompt is displayed by iOS, and if the permission is denied, the following screen is displayed:

Basically, it instructs the user to go into the Settings app to enable Location Services. Until that happens, Mixel has no access to photos on the device.

Here’s a bug report by Sean Heber of Iconfactory explaining similar challenges. In this case, Twitterific uses ALAssetsLibrary to get the most-recent photo from the camera roll so it can easily be attached to a tweet. Based on what you’ve read in this post, you now understand why it’s necessary for Twitterific to use ALAssetsLibrary: it needs access to photos and their dates. From his report:

…after being deployed for only one day, the support issues involving the location services question are flooding in.

Many people reflexively deny location permissions…

We could present the user with an alert if they deny permission explaining what just happened, but that seems annoying, too, and we’re not sure it’d really help since it’d happen after they already denied it meaning that to undo the “Don’t Allow” damage, they’d have to told to leave the app and hunt through Settings to flip a switch back on!

I couldn’t agree more.

Here’s a post by Andreas Lindahl expressing a similar frustration with his recent app. He needed to use ALAssetsLibrary because the standard UIImagePicker control doesn’t support landscape orientation. He’s also seeing negative reviews, and his app has no interest in location metadata.

So, what can be done to improve this situation?

To start, let’s agree that ALAssetsLibrary provides functionality that the majority of photo apps (and other apps like Dropbox) can use to provide an improved user experience. It’s good for users, and it’s good for apps.

Let’s also agree that the current “App” Would Like to Use Your Current Location prompt is at least confusing, more likely misleading, and in some cases, even alarming to users.

For apps that have no interest or use for location metadata (and I would argue that this is the vast majority), there’s no need for this information to be passed to the app.

Here are two solutions that Apple could implement in a future version of iOS that would make the use of ALAssetsLibrary much more appealing to app developers. Only one of them would need to be implemented.

The first option would be to add a parameter to ALAssetsLibrary during initialization that allows the app to request access to location metadata. In Halftone’s case, it would never ask for this permission, and as a result, there would be no Location Services prompt from iOS, there would be no “why are we asking for Location Services approval” explanation, and the new functionality would just work.

The second option would be to continue to prompt for permission for ALAssetsLibrary but with a more appropriate message (e.g. “Halftone would like to use the location information in your photos”). If a user denies this permission, ALAssetsLibrary would still allow access to images, but it would also strip away any location metadata before handing the images to the app.

Of these two options, I personally prefer the first, because it more accurately reflects the intentions of the app. Based on my experience, accurately codifying an app’s intentions will provide Apple with more architectural flexibility as ALAssetsLibrary continues to evolve in future releases.

In the end, either of these improvements will make life better for users, app developers, and Apple. I think we all agree that protecting sensitive and private user information is a requirement, and I support proposals like the iOS permission dialogs that Marco Arment of Instapaper has outlined in his post. However, in the case of ALAssetsLibrary, the current heavy-handed approach is not only confusing users, but it is also preventing developers from providing an improved user experience in their apps.

I Have No Idea

Mon, 06 Feb 2012 10:03:00 -0800

In early December, 2011, I was both surprised and flattered to discover that Apple had named Halftone as one of the best photo apps in its App Store Rewind 2011. Since then, I’ve received a lot of e-mail asking me how I did it. That is, what did I do to “get” Apple to pick Halftone? I don’t know what kind of response they expect, but I’m sure that I disappoint them when I respond:

I have no idea.

I’ve certainly thought a lot about the question, but I have no special insight into the process or criteria that Apple uses to assemble its year-end list. That said, I’d guess that one criterion is quality and another is likely user experience. I invest a lot of time trying to craft an experience that delights and empowers users.

The stamp feature of Halftone serves as a good example.

There are many apps that allow users to add graphic elements to a photo or a document, and in almost every case, the graphic element is a bitmap. There’s nothing inherently wrong with bitmaps, of course, but for Halftone, I wanted to enable users to freely move, rotate, and scale stamps with no loss of quality. That meant that the stamp feature needed to use vector-based elements.

Unfortunately, the iOS SDK has no native support for vector graphics formats (like SVG, EPS, and AI), so I had to develop my own solution.

If you’ve followed me for a while, you’ll know that I’ve written a couple of free plug-ins for Adobe Illustrator. The most recent, Ai->Canvas, adds a new option to the File/Export menu in Illustrator that makes it easy to use vector shapes and basic animation with HTML5 canvas elements (watch this short introductory video if you’re curious).

Remembering that the canvas element was created by Apple for WebKit, and recognizing similar concepts in Quartz 2D, I decided to build a new plug-in that exports Illustrator artwork directly to Objective-C code. I’ve used the plug-in to create the code that draws all of the stamp shapes in Halftone.

For example, this WHAM! artwork in Adobe Illustrator:

Exports to this Objective-C code that I paste into my Halftone Xcode project:

By drawing native vector shapes, Halftone can render its stamps at any resolution with very high quality. That’s why you can turn on the Full Size setting in Halftone and create stamps that look fantastic, even close-up:

I’m confident that most users don’t know (or care) that the stamp feature is based on vectors, but I do. Also, most users don’t zoom-in to full size Halftone images to examine the pixels, but for those who do, they’re going to see the finest output that I know how to create. Whether or not it’s consciously noticed, this attention to the smallest detail certainly contributes to an overall feeling of quality.

I doubt that Apple knows anything about the effort that went into the stamp feature in Halftone, but I like to think that they did recognize its contribution to the overall quality of the app.

A second example—also related to the stamp feature—is that I try to re-think interaction methods when designing for a touchscreen interface. One of the best things about a touchscreen device is that it feels like you’re interacting directly with objects that are just underneath your fingertips. That means that many of the common interactions that were developed for indirect input (like a mouse) should be reconsidered.

In almost every graphics application, you know that you can click on an element to enter an editing mode and use “handles” to scale and rotate the element. These are such common interactions that you might not even think about it. Step back for a moment, though, and you’ll realize that these handles were created for your mouse, and over the years, you’ve been trained to adapt to the computer more than the computer has adapted to you.

To discover a more natural interaction, I placed a sheet of paper on the desk and began to manipulate it with my fingertips…just like you would with a touchscreen device. The great thing about tests like this is how surprisingly obvious the interaction becomes. You’re interacting directly with a physical object, and you just “know” what to do. If there’s a way to duplicate this interaction with touch, it’s likely to seem just as natural to your users.

So, to move a stamp in Halftone, you touch somewhere near the center of the stamp and drag it wherever you’d like. No editing mode. No interacting with artificial interface elements like handles. No visual indicators. Just direct interaction with the stamp itself.

To rotate, you touch near an edge and start spinning the stamp around. Scaling works the same way…just touch and drag an edge away from the center. Even though you can’t scale a physical sheet of paper, it’s easy to imagine what you’d want to do. After you’ve tried these interactions just once, you quickly forget that they’re anything special, because they behave “as expected.” Plus, there are no special editing modes that the user has to learn, track, or maintain.

Out of curiosity, I built a test version that rotated and scaled around a second touch point. This method worked well, but it required two hands and added unnecessary complexity, so I removed it.

Like the vector functionality, I have no idea if Apple noticed or considered these interaction methods in their deliberations, but I can tell you that with over one million users and all of the e-mail I receive, nobody has mentioned any difficulty with these methods. To the contrary, I very frequently hear how easy it is to use Halftone.

I’m proud of the stamps in Halftone. Most people don’t know how much effort it took to make the feature work (nor do they need to know). I do believe, however, that the effort contributed to a more natural and delightful interaction with the app, and I trust that the high output quality is noticed, even if everyone isn’t a pixel peeper.

It would have taken considerably less time to build a feature with standard bitmap images that could be rotated and scaled with typical bounding box handles. But I wanted to push what’s possible with touchscreen devices and to explore new interaction methods.

Often, it’s difficult to know exactly what an app or a feature should or shouldn’t do. It’s difficult because—as developers—we have the burden of professional knowledge: we know how much work it will take.

My advice is to train yourself to recognize and note the small (but important) reactions that you have when you’re working with your own apps. Dismiss your professional knowledge about the effort it will take and consider the experience alone. Only when you’re willing and able to do what’s necessary to perfect a feature will you be accomplishing your best work. Your exit criteria should be when you’re delighted to use your own app or feature and surprised that you were able to pull it off.

At the end of the day, I don’t know exactly why Apple picked Halftone, and I’ll probably never know. But, I like to think that it had something to do with providing users the best experience I knew how to provide.

Ever since I’ve moved to my new blog, the most common...

Fri, 02 Dec 2011 13:15:17 -0800

Ever since I’ve moved to my new blog, the most common request I get (by far) is: “when are you going to post your wallpaper images again?” Well, the answer is: today!

I’ve gone through the 160+ images that I’ve released over the years and culled them down to a set that represents some of the most popular along with some of my favorites. I hope you enjoy them.

Happy Thanksgiving! I was curious about heat transfer while...

Thu, 24 Nov 2011 15:04:00 -0800

Happy Thanksgiving! I was curious about heat transfer while cooking our turkey this year, so I logged the thigh temperature every five minutes. We cooked a 14lb turkey at 350⁰ F in a cooking bag, and it took almost exactly three hours (which includes a 15 minute resting period at the end). Note that I pulled the turkey out of the oven when it reached 180⁰ F, and it continued to warm to 189⁰ F.

The Thermal Imaging Blog has a nice post about cooking turkey, and here’s a post by someone else who tried this last year. If you’re really interested, I found a paper titled Modeling Heat Transfer During Oven Roasting of Unstuffed Turkeys that uses a 2D finite element model to calculate temperatures at different points in the turkey. Cool!

Update on 11/25/2011: I’ve had a few questions about the dotted line in my graph. It’s an exponential trendline that I used to predict when the turkey would reach 180⁰ F.

We received our Nest Learning Thermostat yesterday. The unit and...

Fri, 18 Nov 2011 13:59:10 -0800

We received our Nest Learning Thermostat yesterday. The unit and its packaging are exquisite, simple, and well-designed. Other than a drill, the Nest includes everything you need to install it yourself, including a small screwdriver with four interchangeable bits. It took me about an hour from start to finish, and as you can see in the photo, I used one of the optional mounting plates to cover up the bare wall that was behind our old thermostat. Yes, the screen has a slight red tint to it, because it was heating the house when I took the photo. The free iPhone app (and web site) makes it easy to remotely monitor and control the Nest. So far, so good.

Juicy Bits

Thu, 17 Nov 2011 21:48:53 -0800

Most of my friends know that I’m always working on side projects. And if you don’t know what side project I’m currently working on, that doesn’t mean that I’m taking a break. It just means that I’m being quiet. ;-)

This side project began as an experiment in March, 2009, and over the past 2½ years, it’s become my new full-time job.

In February, 2009, I bought a Mac Mini so that I could port my Adobe Illustrator to XAML plug-in to run on a Mac. As a technologist, I was aware of the Apple App Store and had followed its progress since it opened its doors in July, 2008. I was spending my time in Apple’s developer tools (Xcode) anyway, so I downloaded the iPhone SDK to see what mobile development was all about. I also picked up an iPhone 3G for testing.

After some frustrating weeks learning Objective-C (thankfully, I already knew C/C++), I managed to build an app called 3D Camera that was released in May, 2009. I created a “novelty” app primarily because I didn’t have time to maintain servers or other back-end infrastructure, and I wanted a fun app that nobody would depend on. I also needed an app that the press wouldn’t find very interesting. After all, my day job was still as a Technical Evangelist at Microsoft, and nobody needed that article.

At the time, most of my decisions were based on experimentation, and I never expected that this side project would turn into anything real. I told one of my friends that I’d be happy if I could make enough money to pay for a new camera lens. My company name, Juicy Bits, was concocted on short notice so I could release 3D Camera in the App Store. If I knew that it would eventually become my full-time job, I probably would have put more thought into the name!

Since then, Juicy Bits has served as an after-hours technical playground. I didn’t need it to pay the bills, so I was free to experiment. I continued to build novelty apps, and I tried running sales, creating “lite” versions, altering pricing, using social media, and just about anything that could teach me more about this new marketplace. Not only was I hooked, but I was constantly learning new things, challenging myself, and having fun all at the same time!

But I couldn’t tell anyone about it. Only my wife and one of my closest friends knew what I was doing.

As my skills improved, I was able to create and release more apps. In June, 2009, I packaged-up some of my close-up nature photos and published a gallery of wallpaper images. Those gallery apps have gone through some revisions to become Nature Images, Nature Images HD, and Textures HD. While they’re not big sellers, each release has taught me just a little bit more about how the app marketplace works.

In December, 2009, I released Spy Pix. It’s yet another novelty (and photography) app that uses a technique called steganography to hide one image inside of another. Imagine my surprise when I learned that it’s popular with college kids who like to send…err…special photos to each other. Who knew!?

3D Camera Studio, a more fully-featured version of my first app, 3D Camera, was finally approved by Apple in May, 2010. The iPad had been released in January, and this was my first in-depth experiment with tablet development. It was liberating to have a larger screen and a chance to redesign my original app for this new form factor.

In between the new apps, I was adding features to existing apps, fixing bugs, and starting to promote them more and more. As a result, the business side of Juicy Bits continued to generate more income. My apps were being reviewed in printed magazines, online, and frequently, they were featured by Apple in their New and Noteworthy and Staff Favorites lists.

The more popular they became, the more that other companies would reach out to me with offers to collaborate on future apps and products. Because I needed to keep Juicy Bits under the radar, I had to pass up many exciting opportunities. It was frustrating.

My most recent app, Halftone, was inspired by another technique that makes photos look like they came from old newspapers and comic books. You’d think I have a thing for photography apps, wouldn’t you? Halftone was released in February of this year, and it’s my first Universal app (meaning that the same app runs on iPhone, iPod touch, and iPad).

It will come as no surprise that I like the most recent app the best. Over the summer, Halftone was selected by Apple as its iPhone and iPad App of the Week in both Canada and the UK. It’s spent a lot of time in “top 10” lists, and it’s frequently among the top 100 photography apps in many of the worldwide app stores. It even spent some time above Angry Birds, if you can believe it! Within the next week or two, it’ll tick over the one million user mark. It’s stunning (to me) to think that Halftone has been used to create over 5½ million photos.

So, in the summer of 2011, it hit me: I’m having so much fun writing mobile software for Juicy Bits, why don’t I make it my full time job? After discussing the idea with some trusted friends, I decided that I would leave my job at Microsoft after we completed our BUILD event in September, 2011. After 11½ wonderful years, September 30th, 2011 was my last day at Microsoft.

October 1st, 2011 was the beginning of something new and exciting. While I miss the daily interaction with all of my Microsoft friends and internet followers, I’m having fun applying my full attention to Juicy Bits and what comes next.

There’s another set of Halftone updates coming later this year, and I hope to release a new app that I’ve been working on sometime before Christmas.

If you want a quick tour of Halftone, Adorama TV recently published a nice video review that’s worth watching. Also, crowdSPRING ran an interview that they did with me a few weeks back that contains a few more details.

Welcome to Juicy Bits.

While I left Microsoft of my own accord, Lemonade is a short...

Thu, 27 Oct 2011 15:24:00 -0700

While I left Microsoft of my own accord, Lemonade is a short documentary about 16 people in the advertising industry who lost their jobs and found their calling. It’s just as appropriate today as when I first saw it years ago. Definitely worth watching.

Keoni

Mon, 10 Oct 2011 19:49:54 -0700

Keoni