tucuxi.org

OS X and VFAT Support

I had a surprise encounter with one of of OS X's lesser-known interactions with the rest of the world a little while back, specifically, the way it presents a POSIX-flavoured veneer atop filesystems that don't support certain features.

Like many people, I keep my photos on a laptop before I'm able to dump them on a fileserver. As part of the processing I do for panoramas or time-lapse videos, I'll often select subsets of the images to work with, and what better way on a Unix-like OS to do this than to use symlinks? After doing an initial run over the photos, I typically copy them to an external HDD.

As part of my backup routine, i just completed doing a rsync of about 1T worth of photos, and noticed some of the files were very, very small for raw photos. So small they couldn't have been real. Insert a brief moment of panic.

I saw a bunch of 1067 byte files with contents like this:

XSym
0015
83ea4b35483415f19ee2fc3a636a3cef
../_MG_8931.CR2

My first thought was that my fileserver had induced corruption in these images - I'd recently moved to XFS for one of my fileserver volumes, and, hey, that 'XSym' seems remarkably like what they'd call symlink metadata for XFS, right?

As it turns out, nope. Not even close.

It took quite a while of searching, but I eventually came across this little gem of a result: Mac OS X Internals' description of MS-DOS filesystem support under OS X.

It turns out, OS X has a particularly horrible way of pretending it supports symlinks on FAT filesystems - it'll create a file with contents similar to the above, and transparently interpret that as a symlink. Which is all well and good, until you try to use that external HDD on another system, say, a Linux fileserver. Not to mention, this method is extremely poorly documented - searching for "OS X XSym" didn't turn up any Apple technical notes, or other official documentation.

Lying about the capabilities of a filesystem is a very Apple way to do things, and in my opinion, just plain wrong. If the FS doesn't truly support symlinks, any attempt to create a symlink should fail. The semantics of the data on a filesystem should not differ across OSes, or else you'll get problems like this - I'd much rather have two copies of a photo wasting space on a disk, than two files with the same name, and different contents. That can be particularly dangerous if you don't have any safeguards around de-duplication, or re-organising your fileserver.

No data was harmed in the making of this post.