Tracking truth

As I mentioned in my previous post, organization on today’s computers is getting harder and harder, given rapidly increasing disk space and more stuff we want to store.

One of the more important techniques I’ve developed to combat this problem is something I’m calling “designating the truth.” It’s a simple concept with strong real-world parallels, but it often gets neglected in the computer realm.

“Truth” is based on terminology used by Apple’s Sync Services API. The concept of Sync Services is invisible: Mac OS X keeps a variety of data, including calendar events, contacts, and tasks in internal databases that users can’t get to. This data is used to synchronize any software, web sites, and portable devices that want to make use of. Software that the everyday user thinks of as the place for their information – iCal, Address Book, and so forth – are just pretty Sync Services clients.

Once everything has its place, the trick to staying organized is to designate which copy of a given piece of information is authoritative. This copy has a variety of different names. For example, “truth” is Apple’s moniker, while “golden master” is used in software manufacturing.

Here’s where I keep my designated truth for different types of information:

Type Location of “truth” Additional copies
Appointments OS X Sync Services (iCal) Palm PDA
Contacts OS X Sync Services (Address Book) Palm PDA
School work Subversion Working copies on a shell server and my Mac
Music FLAC files on a home server iTunes, Palm PDA, Creative Rio, and more

The beauty of this approach is that I can backup to my heart’s desire, and I no longer lose track of which version is which. This is particularly important for my school work – now, if I want to do an assignment at school, it travels to me in a Subversion working copy on a USB key. When I get home, I check in changes to my server, and no longer have to worry about which copy is more recent.

I also can add new gadgets (the iPod is tempting), and since I know what my truth copy is, I’ll never overwrite it, knock wood.

Where’s My Stuff?

I’ve become convinced that the most pressing challenge of computer usability in the very-near future is organizing the mass of data we’re now capable of accumulating into something we’ll be able to use effectively later. With hard drives becoming larger and cheaper at the same time, the amount of data that a person can accumulate is becoming mind-boggling.

Take me, for example. Between a thumb drive, Memory Stick, SD card, external hard drive, two drives in a Linux server, Gmail account, web hosting provider’s space, and my Mac’s hard drive, I’m about to break the terabyte barrier. (I already have, if you count the dozens of CD-R’s and DVD-R’s lying around). Consider that my first computer had a 20 MB drive (that was an upgrade), and it’s amazing how far the technology has come.

The problem, I’ve noticed, is that with all of this space, things get lost. Developing ways to organize my files has been challenging. Apple helps me somewhat by integrating rich metadata support into the OS, but I’m of the school of thought that says that making an effort to sort files makes for easier retrieval in the long run.

Keeping Synchronized

Another related problem has been keeping track of multiple copies of the same file. This was a nightmare – I work from multiple locations, and having to track changes made to different documents on different media at different times is a monumental undertaking. Thankfully, through the slightly eccentric but surprisingly useful advice of Debian Linux developer Joey Hess, I’ve found a solution that I’ve been adopting: version control.

Version control is useful for programmers, but it has a major strength for non-programmers: it forces users to designate a “golden copy” of a file that all other copies are updated against. Suddenly, the task of figuring out which file has which change disappears, because the version control software (in my case, Subversion) takes care of all that.

I’m not quite up to Joey’s level of reliance on Subversion (he keeps everything in it), but I’m getting there. Currently, almost all school work is in Subversion – the major exceptions being large media files that are almost certain not to change. As I continue working with Subversion, I expect that my setup will eventually look a lot like Joey’s.

Microsoft/Apple: The Letdown

Perhaps the primary reason we’re so far behind in the file organization war is the less-than-stellar tools provided by Microsoft and Apple. Apple currently holds a commanding lead with Spotlight and iLife, but there’s definite room for improvement. For example, opening a shell window to query Spotlight with specific metadata is a usability disaster, as is trying to figure out how to organize images that aren’t photos (importing them into iPhoto seems like a waste here).

And as for Microsoft? If they had spent the last years concentrating on WinFS instead of buggy, draconian copy protection technologies that nobody actually wants, I might not have had to write this post.