iPhone migration: restore from backup or setup from scratch?

In moving to my fifth iPhone I decided it was time to try a fresh install instead of transferring state from my previous one. Apple’s transfer process has always worked works seamlessly but that Star Trek-like transportation of your old phone to the new makes the new one seem, well a little less new somehow. Previously I would have feared losing something small but vital by not using the transfer process but with so much now synchronised between devices by iCloud, it seemed that the most painful and important items were covered. My willingness to experiment was also mitigated by the knowledge that if it all went wrong I could still re-install the new phone from my old backup.

Without the need to wait for a gigabyte of data to be transferred to the new phone, I was up and running very quickly and enjoyed seeing what differences there are between an out-of-the box configuration and my own. With the exception of turning off the sound and vibration notifications for new email, I have customised very little to start with: I want to see if there are any settings or features I have previously overlooked because of some decision I made 7 years ago. Working through my list of apps and choosing to only reinstall those I know I have used recently was also enlightening—there were applications I had forgotten existed buried in never-opened folders, and including some that now crash on startup. I noticed others that are no longer being updated but the developers have instead issued a whole new app instead. I learned that the telltale sign for this is when you tap on the application in your “purchased” list and it fails to display anything about it.

Having chosen which apps would survive the migration and downloaded them all, the main pain point was having to re-authenticate in all of them. The reduced number of apps kept this manageable and I would guess about 50% enabled the use of iCloud Keychain which made it a two-tap process. Given the tedious and error prone nature of typing in a complex password onto a phone keyboard I am shocked that the other 50% do not enable this. Those apps which forced me to lookup and retype my password are certainly strong candidates for deletion if there is an alternative.1

By not restoring from backup, any applications that store state locally and not in the cloud would lose their data. I was a little worried about my Angry Birds progress but then realised I had not played the game in a very long time anyway, and so far I have discovered only one application which stores data I would like to keep. It is a free app and its paid for version supports data export, which seems fair enough. WhatsApp messages were restored from its own iCloud backup (which it performs nightly) but I did lose my SMS and iMessage history (going back to 2008). Recent iMessages are on my mac anyway and after reviewing the older messages they were all very temporally contextual—“I am here / outside, where are you?”—type stuff. It is noticeable how much longer messages have become recently, perhaps an inevitable consequence of removing the 160 character limit of SMS.

In conclusion, unless you have apps or games that depend on local data, setting up from scratch was not at all painful and it felt really good to have a spring clean. Deleting something always requires metal energy to assess its value, or downside of its loss, and with storage so cheap it is rarely worth the effort. Consequently we accumulate bits and bytes rather too easily, and sometimes a platform migration is a good moment to assess what is no longer required, and start afresh.

  1. Anecdotally, many of those which failed to implement keychain were companion apps from established non-digital companies, for example, airlines and hotel chains. Developers competing for users on the strength of their app alone were far more likely to have implemented it. [back]

Too Busy To… available on Apple News

Subscribing to a blog via RSS remains a niche and decidedly geeky activity. Having said that, Apple devices now come with a built-in News.app which, for all the usual Apple gloss, does use RSS. To view this blog in Apple News, visit this link on your iPhone or iPad and tap the + sign in the top right hand corner to subscribe.

If you do not use News.app, Safari’s shared links feature (the @ tab in the bookmarks pane) also allows you to subscribe to this and many other blogs.

Photography: An unorthodox position in the RAW/JPEG debate

In my post, A macOS Photo Editing Workflow, I admitted to taking the somewhat unorthodox position of capturing both RAW and JPEG versions in-camera and normally working with only the JPEG version. The conventional wisdom in photography is that RAW is the more flexible option, providing greater opportunity for correcting and improving the image in post-processing than when the camera is allowed to make all the decisions on your behalf. There are traditional reasons for not using RAW files such as the increased file size, but these are normally only relevant in specific situations—there seems to be few generic reasons not to use RAW unless you are operating under specific constraints.

I was therefore intrigued to see a recent article entitled, Is RAW Dead? in Photography Week. In addition to examining all the usual issues, it points out that in-camera JPEGs are now so good that in the majority of situations the camera will probably do a better, or at least no-worse, job of rendering the final image than you. This makes sense: unless you actively tweak the settings in your RAW processor then you are simply handing the conversion over from one algorithm (your camera) to another (your RAW processor), and while in the past the processing ability of a camera was limited compared to that of a PC, the advances in mobile chips and image processing have removed the PC’s previous advantage. Moreover, since the JPEG output is the version seen by potential buyers and new owners of their products, the camera companies themselves are highly incentivised to make the JPEG output as attractive as possible. The article does list a specific benefit of RAW as being able to recover an additional 1-2 stops of highlights and shadow detail compared to JPEG, although it also points out that a wider range can be obtained using the camera’s automatic exposure bracketing or even built-in HDR.

For the moment I will be having my cake and eating it by using RAW+JPEG: JPEG for convenience but with the RAW available as an option should the trade-offs and technology change in the future.

A macOS Photo Editing Workflow

This is part of a series of posts describing my move from Apple’s Aperture to Photos.app for managing and editing digital photographs. Previous entries include a feature comparison, third party applications and extensions.

Although Aperture has not received a significant update in three years, I did not feel it was deficient in any way. Having now used a number of recently released other applications1 in the last few months I have now become aware that Aperture does lag behind the state of the art and it is time to update my toolbox. Another consequence of using different tools is that I now recognise that Aperture is in fact three products in one: RAW converter, photo manager and editor. Given the convenience and advantages of using Apple’s built-in image management, including the iCloud library for device synchronisation, it seems obvious that Apple’s Photos.app should be the starting point for a replacement solution.

When editing, for many photographs I found I could produce the same or better results in Photos.app compared to Aperture, unless I needed to tweak the RAW conversion. Photos’ superior plugin implementation also means that when it cannot get the job done alone it only takes one click to call out to another application. Consequently while Aperture has more built-in tools than Photos.app, the high quality of external applications means that Photos.app plus plugins can get better overall results with no loss of efficiency.

Photos.app handles RAW files but offers no editing abilities above and beyond what is offered for JPEGs. Given the additional time and bandwidth required to upload them to iCloud, along with the greater space consumption on iCloud storage, there seems to be no benefit to offset those costs and I have decided not to store RAW files in Photos.app. Storing my RAW files in a standard folder hierarchy without the obfuscation of a management application also makes it very easy to move these files between tools2 and platforms in the future.

I have been using Nikon’s Capture NX-D software to convert the RAW files into JPEGs. This is free from Nikon and produces lovely JPEGs which match what is shown on the camera screen, but I found too slow and too difficult to use to make even the most basic edits. I tried a 30 day demo of DxO Optics Pro 10 which produced reasonable JPEGs and was very good at improving certain aspects of a photo (noise removal and some lighting scenarios) but for the average photo is no improvement on the built-in Apple RAW conversion, and not as aesthetically pleasing as Capture NX-D.

In light of these findings, on recent shoots when SD card space has not been an issue, I have set my camera to store RAW+JPEG rather than RAW-only. I have then imported the camera JPEGs directly into Photos.app and reserved RAW edits for only those where the in-camera JPEG is not good enough. In that situation I do then import the RAW into Photos.app and use the excellent DxO Optics Pro for Photos plugin if I think that will solve the problem, or resign myself to fiddle with Capture NX-D if necessary.3

  1. Nikon’s Capture NX-D, On1 Photo 10, Polarr and DxO Optics Pro. [back]
  2. Such as the recently announced On1 Photo RAW. [back]
  3. I have on occasion also loaded the RAW into Aperture to compare the results with the other two, and in four out of five cases found Aperture could not do a better job. [back]

Pixelmator Retouch Extension

Previously I noted that moving from Aperture to Photos.app meant giving up the ability to selectively apply enhancements using a brush. I speculated this advanced functionality would be something third parties could provide via the extensions API, and the latest version of the superb Pixelmator provides all the brushes I remember having actually used in Aperture: burning, dodging, colour and sharpening. The retouch extension also provides access to Pixelmator’s very impressive repair tool for removing unwanted objects, a serious upgrade to the built-in retouch function.

Also worthy of mention is the Mac version of the Polarr Photo Editor. A slick and efficient editor in its own right, it makes the popular curves adjustment easy to apply without leaving Photos and its de-haze slider is also useful for increasing the clarity of hazy or misty photos.

Photo Editing Apps for Mac

As a follow up to my previous post, I have been evaluating some of the many photo editing apps available for Mac to see how they might fit into my post-Aperture workflow.

For the problem of comparing sets of images the best tool I have found so far is MacPhun’s Snap Select. Although it can only display two photos side by side and not the 8 that Aperture can display simultaneously, I have found it to be effective at picking a single “best shot” from a selection of similar ones, which is my usual use case.

The majority of other apps appear to be focussed on editing and not managing a collection. On1 Photo 10 has a ‘browse mode’ which supports star ratings but it lacks side-by-side viewing and can only display thumbnails—the main focus of the suite is editing. It includes an easy to use portrait editing as well as a versatile effects module with a flexible masking option to apply effects to only parts of an image. I originally came across them because they offered a free set of presets for Aperture which I have used quite a bit. Having purchased this app on the Mac App Store—where it is currently much cheaper than direct from their website, and apparently the same full version although this version does not integrate with applications other than Photos.app. I also found the Resize module to be very useful in making a high resolution 30×20″ print from an older photo I only had as a 16:9-ratio jpeg. I had always thought that up-scaling an image in this way was bound to lead to poor quality but it turns out there are clever algorithms that can be applied to maintain quality. Resize also supports generating an extra border for the image which is then “wrapped” around the edges of a canvas. Without this “gallery wrap” feature, an image can lose a substantial section from the edges when the canvas is 3-4cm thick.

The power of On1’s Effects can be seen in this video. I have yet to really explore this module but initial experiments showed that while using smart layers brings some file format compatibility with photoshop, the resulting files are around 200MB each. This is not relevant if the module is used as a plugin to Photos.app because Photos does not allow edits to be modified further once the plugin has exited, but for advanced work a more compact file format is needed to avoid making the user decide whether they want to ever revisit a set of edits again in future.

My favourite Photos extension so far is definitely DxO Optics Pro. It only offers a small number of enhancements, including noise and haze reduction, but the noise reduction is excellent and worth the money alone. It works best with RAW files, but I have achieved some pleasing results even with camera phone photographs. Highly recommended, and on the strength of the extension I will be trialling their full DxO Optics Pro software when I next have a good batch of shots to process.

New website look and feel

This blog is run using WordPress, a very powerful piece of open source software that now claims to be used on about 25% of the web. I had been using the previous theme since early 2012, and since then I have come to believe that a good website should work well, and load quickly, on a greater variety of screen sizes as well as the need to optimise for higher latency and less reliable mobile networks. The new theme is a customised version of WordPress’s latest, Twenty Sixteen, which I hope brings a modern feel to all platforms while remaining an evolution of this site’s own identity. Using Safari’s “responsive design mode” was very useful for understanding how a page would look on a variety of device screen sizes, including different orientations.

While updating the theme I also spent some time looking at how to optimise the site’s Web Page Test score. The most significant gain seems to have been made by switching the caching from WP Super Cache to W3 Total Cache. Both plugins were configured using the default/recommended settings but W3 Total Cache seems to be able to significantly reduce the First Byte Time, taking the score from an “F” to an “A”. The high number of photo posts, plus the embedded youtube videos, means that the front page is currently just under 10MB.

Either the new theme, the new cache plugin, or a combination of both, did not work with the SyntaxHighlighter Evolved plugin. It has been disabled and may or may not be replaced at a future date.

Aperture v Photos.app, El Capitan Edition

I have been a very happy user of Apple’s Aperture since 2011. I remember spending a few months using trial versions of both Lightroom and Aperture, before deciding Aperture suited me a little bit better—I think its integrated editing tools were slightly more powerful at the time and it seemed likely that Adobe would be keen not to make the built-in tools so powerful as to compete with their premier product. I was not overly worried when Apple first announced that Aperture would be discontinued because they have a track record of introducing revolutionary new products without every advanced feature in the initial version but then iterating consistently year-on-year until the gap is closed. With that in mind there was much anticipation around this Autumn’s release of OS X El Capitan, and consequently considerable disappointment that support for external editing extensions was the only addition. However it is important to remember that Photos.app was only released six months prior to El Capitan and so this was not a full year of iteration.

Although I had dismissed the initial version of Photos, with the availability of external editors to replace any missing editing tools, perhaps it is time to start thinking about the transition. Photos already has a few features in its favour, for instance it feels faster at navigating between photos, and does not spin for 30 seconds whenever it encounters a video. The advanced tools are comprehensive once you drill down enough to find them, and kudos to Apple for building a textbook example of how to make an interface that provides such a progressive user learning experience. In a taste of what new functionality is being denied Aperture hold outs, the noise reduction in Photos appears much improved over the equivalent in Aperture.

Perhaps the most significant omission is that there is no mechanism to apply an edit to only part of a photo, an equivalent of Aperture’s brushes. This may be a gap that Apple expects editing extensions to fill although I find Apertures’s non-destructive edits much quicker and simpler to use than the layer-mask approach of Photoshop and Pixelmator. There is also no way to display hot/cold regions to see under- and over-exposed areas. Photos.app also has greatly simplified camera-data and EXIF display. These are useful when trying to improve one’s photography “post-mortem” by looking at how the camera was setup for a particular (usually unsuccessful) photograph so their removal is disappointing.

Star ratings are also no longer available. In Aperture each photograph could be assigned a colour and a flag as well as a star rating, so it is easy to see that there were probably too many options for the majority of users who will be much happier with a simple “favourite” function. But when dealing with a large library having different ways to annotate a photograph is useful, a flag can signify a photograph that needs more attention before it is “finished” while star ratings can be used when dealing with many similar photographs of a single subject to distinguish the great from the merely good, and then the 1-star, “bad but captures some unique moment so worth keeping.” Another problem when dealing with multiple exposures of the same subject is that inability to do a side-by-side comparison of two-or-more photos. In Aperture it is possible to bring up all the photographs of a particular subject in a single view and eliminate them one-by-one. This seems to be completely impossible in Photos.app and therefore a complete showstopper for me because I use this feature a great deal when deciding which images from a set to discard and which to publish here.

Which of these missing features Apple will eventually bring to Photos.app is difficult to guess. Star ratings seem the most unlikely to return since only obsessive-compulsive types rate their possessions on a five-point scale, plus Apple already expended the effort to convert the ratings to keywords/tags. I sincerely hope side-by-side comparison returns since that is the raison d’être of both photograph management software and large screens. Brushes require a significantly more complex user interface, especially on an iOS, but are also an ideal use case for Apple’s new iPad Pro Pencil. Or Apple may decide that such advanced editing functionality has no place in a management app, and cede the space to third party developers via the extensions API.

Too Busy To… Flickr Galleries

My most recent photoblog posts here have been hosted using flickr. The upload experience is slick and efficient, and the 1TB of online storage means I can freely upload my files at full resolution and let flickr figure out the best way to serve it to a given client, whether a bandwidth constrained smartphone or a 30 inch display with a “fat pipe”.

flickr can also take care of other optimisations. Two of the key findings when I analysed my website was to use the progressive jpeg format and a content-distribution network. I could do the former myself by adding a manual step1 to my export-and-upload workflow, but the latter requires a considerable amount of effort to setup and maintain. Another unexpected but welcome benefit has been flickr’s social aspect: my photographs are seen by more people than when I host them solely here on my own site.

Hosting on flickr does have some disadvantages. flickr has no supported way to embed a gallery of images in a webpage. The photographs are being displayed here using the very good Flickr Justified Gallery plugin, although the full size image display is not quite as nice as the one used for native WordPress galleries. Entrusting my content to a third party service also carries risks for the long term longevity—will flickr still be serving my photographs in 2025? That risk is somewhat present even using the open source software that hosts this blog should it stopped being maintained, although the size of the WordPress user community means there is a good chance of an export path. While not quite the same, flickr also has a large community and good API support, so for the moment the benefits I listed above seem to outweigh this risk.


One of the more tedious parts of uploading a batch of photographs is captioning. Most websites, flickr and WordPress included, will take the filename as the primary title. The problem with this is that since filenames must be unique within a directory, multiple files with the same title end up with a extraneous number that must be removed.2 Any text in Aperture’s “caption” field becomes the description in flickr. On a album-orientated website such as this, the title does not need to include any context since that is provided by the album itself, for example an image entitled San Telmo in an album about Buenos Aires is unambiguous even if there are multiple cities in the world with a district of the same name. However flickr’s stream-orientated approach to images means that at least some visitors may arrive at the photograph without the context provided by an album. To complicate matters, only the title is shown on this website when displaying a gallery. I include this information here purely for reference since if a few months go by between uploads then I struggle to remember the exact mappings and have to spend time fixing up the titles manually!

  1. $ jpegtran -copy all -progressive -outfile $x.new $x [back]
  2. flickr makes this very quick and easy, which is why I described its upload process as efficient above. [back]

OS X Notes.app and IMAP Accounts

On a fresh install of OS X Yosemite, the notes.app was unable to see the notes stored on my IMAP server. The account was working properly in Mail, and notes.app worked fine with other accounts.

I recall having this same problem with a previous version of OS X, and that it was related to the “IMAP Path Prefix” advanced setting within the Internet Accounts system preference panel. The prefix is set correctly so I was about to give up on this as being an annoying–but–ignorable bug when the very last post in this forum discussion indicated that a cargo-cultish approach of changing the prefix, opening and closing notes, then reverting the setting had fixed it. I can confirm that this solution also worked for me. I kept Mail.app closed for the duration to prevent it being confused, and observed that simply unsetting the value is insufficient, it must temporarily be set to another value, such as “none”, to work.

Which email addresses receive spam?

One of the advantages of owning a domain name is the ability to create a limitless number of email aliases. I use this to allocate each company that requests an email address a unique one, which makes it a lot easier to spot phishing emails, and track whether a company has used it according to my expectations. A recent browse through my spam email folder showed some egregiously bad spam (obvious frauds, scams, etc) being sent to aliases assigned to companies.

  • Vision Express
  • Tumblr (the micro-blogging platform)
  • JET Photographic, Cambridge
  • Adobe — suffered a well publicised data theft
  • LinkedIn — likely someone with whom I am connected since they would then see this email and could import it into their personal address book
  • Dropbox — dropbox includes this email address when I share files and links with others via its service so again the leakage is probably from a third party

Another surprising result of my browse is that the email address I publish on this website does not get very much automated spam, although it does get the occasional offer of “sponsored posts”.

Resolving mixed content errors with WordPress

This blog has in theory been available via a secure (“https“) connection for about 2 years. I say “in theory” because some of the images were being loaded from insecure connections which meant there still ways to easily circumvent that security. After some digging it seems this is a long-standing known problem with the WordPress software that runs this blog, and despite some recent activity, still not fixed in last week’s 4.0 release.

Fortunately the discussion in the bug report does provide a one-line workaround. There was no advice on where to put that one liner, so I decided to write a plugin as it would then be easy to toggle on and off if required.

 * @package fix_ssl_attachment_url
 * @version 1.0
Plugin Name: Fix SSL Attachment URL
Plugin URI: https://core.trac.wordpress.org/ticket/15928
Description: Hacky fix for wp_get_attachment_url function not checking for https. 
Taken from the bug report referenced above.
Version: 1.0
Author URI: http://www.toobusyto.org.uk

add_filter( 'wp_get_attachment_url', 'set_url_scheme' );

The next challenge was that the instapress plugin I had been using to display my Instagram photographs in the side bar was also using insecure content. It seems that instapress is no longer supported, and although worked for me, might not continue to work for much longer so I upgraded to Simple Instagram. This was a straightforward drop-in replacement (once I had successfully made an Instagram developer account) but displayed three Instagrams per row which I found a bit small. The author appears to be very active and helpful on the forums, providing these hints on how to customise it, but initially I could not get this to work for me when I put the settings in my custom theme’s style.css. The problem is that the first CSS class is now .si_feed_list and I found I needed to mark the customisation as !important in order to override the default.

/* For use with the Simple Instagram plugin */
.si_feed_list .si_item {
  width: 50% !important;

Finally I had to disable the Simple Facebook Connect plugin. Like instapress, this was reported as being broken and discontinued by the author.

Managing cron.d with chef

I have recently been playing with the chef configuration management system. I was looking for a way to manage files in a directory such that any that were created by chef would be cleaned up again when they were no longer needed. A classic use case is the /etc/cron.d directory which may be populated by files from multiple sources. There appeared to be no established pattern for this but since chef allows the use of ruby in its recipes, I was able to construct the following. It assumes the use of the cron cookbook.

[code language=”ruby”]
cron_d ‘usercron.chef’ do
minute 0
hour 23
command ‘/bin/true’
user ‘myuser’

Dir.glob("/etc/cron.d/*.chef") do |f|
name = f.split(‘/’)[-1]
t = resources("cron_d[#{name}]")
rescue Chef::Exceptions::ResourceNotFound
cron_d name do
action :delete

OS X Terminal.app, bash and UK Keyboards

On an Apple UK keyboard, the # symbol is accessed by pressing ⌥-3 (pronounced option 3). Unfortunately the terminal application is only useable when option has been mapped to the UNIX meta-key, which takes precedence over “special” characters such as #. Thanks to this tumblr post, it is possible to work around this problem:

$ cat .inputrc
"\e3": "#"

Unfortunately .inputrc is a bash-specific configuration file and this does not solve the problem for terminal-based applications.

Google Maps and iOS Background App Refresh

I am posting this to the web in case it helps anyone else trying to troubleshoot a similar problem.

Recently I noticed my iPhone’s battery was ending each day significantly lower than usual, causing me to have to charge it every night instead of every couple of days. At first I suspected the extra consumption was caused by communicating to my Pebble smart watch but quickly eliminated that possibility when turning the Pebble off for a day had no effect.

After some experimentation, the change that restored my battery usage to its previous norm was disabling background apps refresh for Google Maps. This was an application I had recently installed because it works very, very, nicely with the Pebble, sending turn-by-turn navigation directions to your wrist as you walk. This completely removes the need to take the phone out of its pocket every few minutes to double check that the road you just passed was not the one you were supposed to turn down! However I do not use it frequently enough to justify doubling my daily power consumption…