Three Architectural Goals for LTFS

Creating a simple, readable and multi-platform tape file system

LTFS 1 Creation: Segment 2


Jay Livens:
It's clear that one of the big principles was thinking about using tape as a file system. When you thought more broadly about the features and functions of what you wanted to include, what were the other sort of architectural goals or principles that you had as part of the process of creating the LTFS technology and the LTFS spec?

Michael Richmond:
First and foremost, this idea of building a file system was the key underpinning. In terms of how we achieve that file system, there are probably 2 buckets of things that we worried about. We worried about the standard data storage issues of how do we make sure that this is rock-solid reliable, so that once data gets written to the file system, it stays there until the user doesn't want it there. That kind of issue just comes with trying to build a file system. In terms of specifically for a file system on top of data tape, the goal was to make data tape easier to use.

Ultimately one of the complications in working with data tape historically is the fact that you have to either have a paper label stuck to the tape back in the early days, or nowadays some kind of tape management system which maintains a database of the content of what's on the tape, so that you can find that content again. We really had to eliminate that database. That meant moving that index information, the file metadata on to the tape itself. In moving that data on to a tape, we then had the possibility of making the tape completely self-contained. This idea of being self-contained ended up being very important, because it allowed us to start talking about use cases where data tape was used to exchange data between 2 remote sites, in addition to data tape as an archive destination.

Another guiding principle that we settled on fairly early was that the file system itself and tapes written in this format must work across platforms. Our prototype work started off on Linux late towards the end of the prototyping phase. I personally ported the Linux implementation to OS X. As soon as we start a productizing, we laid out plans to release Linux, OS X, and Windows in short order, which immediately meant we had to make the format itself completely cross-platform compliant, and eliminate anything that crept into our work that would tie the LTFS to a particular platform, or operating system, or set of software.

A third thing that was very important, at least in the conceptual phase, was we wanted to have both a transparent format and an uncomplicated format. This was in part, because we were aiming for an archive market. We wanted it to be easy for future developers to pick up either the implementation, or a tape itself, and reverse engineer the format, and be able to read the data off. For the archiving market, that felt extremely important. The uncomplicated format part of it also drove a use case that we talked about a lot while we were designing the format. This use case is that if a software developer wants to extend an existing software or write a new piece of software that just produces an LTFS volume. For example, if Avid decided that they wanted to support LTFS, export to LTFS, it should be trivial for them to write out the format, since they're just trying to blast the data out to tape.

We came up with this model where writing out an LTFS volume, just from a blank tape, should be as simple as writing down a label indicating that the tape is an LTFS tape, writing out all of the file content that you want to put on to a tape, and then laying down whatever data structures LTFS needs, and that's it. No additional complexity, other than just dropping the data onto a tape in the appropriate format.

The Speakers:

Michael Richmond
Jay Livens