LTFS Creation – A Grassroots Effort

Moving from a prototype to a polished technology

LTFS 1 Creation: Segment 4


As you can imagine, the specifications and the design can change. You talked about over time going proprietary vs. open. When you think about the whole process and you think about where you started and then where you ended, what is the one thing that sort of most surprises you perhaps from what you ended up with from where you started? I'm just curious.

The thing that surprised me the most was that we completely changed the format, the on-tape format. When the prototype was put together, we felt pretty good about the tape layout that we had, that we were using. At a very, very high level the tape format is the same. There's the prototype used, two partitions, stored indexes along the file content. That's the end of the commonality between the prototype and the final version.

The final version maintains one copy of each index when it writes the index. I should be clear here. With LTFS the current index is always written to the data partition and the index partition when the tape is ejected from the drive. With the prototype we wrote two copies of the index to the data partition and two copies of the index to the index partition because we ultimately, at the time, we felt that tape media itself may be unreliable – and this was coming from the software side. So we wanted to be able to write down two copies to insure that we would have a good copy to read back.

On talking with the tape experts and really researching the characteristics of the media and the way the drives worked, we realized that while we didn't, I shouldn't say realize, we learned that if the data for the index gets on the tape, it's going to be tape for years. So we didn't need to for our own comfort level. Ultimately our final specification completely captures the original vision largely because the specification was written after we wrote the product version of the software. The software was in early alpha testing, or possibly beta testing, by the time we sat down and banged out a draft of the specification over a two or three day period. We then needed to update the specification a couple of times to support some of the future directions that IBM wanted to take LTFS in for product reasons.

And we haven't talked today about product strategy but I probably should outline it a little bit. IBM's LTFS single-drive edition is an open-source release. IBM has built out, and we built out during the early LTFS project, some future phases of LTFS work that will be closed-source proprietary products that IBM releases. And those proprietary releases are the library edition which was released in 2012. And then some potentially longer-ranging enterprise editions for large-scale tape installations.

So the idea was that we develop this technology for a single drive. It can release that implementation as an open-source release and then build on that open-source release to make products that IBM will sell to larger-scale customers to support IBM's tape organization and drive revenue for IBM. It was a bit of, let's do something good for the community and then build on that goodness to make up our products. And so what we learned after releasing the first format specification was that there were some pieces we had omitted in the single-drive implementation because we could compute them dynamically when the tape was mounted that we needed to actually pin down and store on the tape if we are going to support library deployments.

And so we revised the specification a couple of times, mostly dealing with changing phrasing or adding clarity. And then with the LTFS specification V2 we introduced some support that is more applicable in the enterprise space such as support for sparse files and persistent file identifiers, and then back-ported those changes to the open-source release.

We also built into the specification some rules for how the version number of the specification will change. These rules effectively give a way of decoding the specification so that you can say that tapes that are written to be compliant with version X of the specification will or will not be compatible with software that implements a later version of the specification based on which numbers in the specification version have been updated. And this is just about trying to make it easier for both customers and developers to work out when the LTFS format and LTFS tapes are going to with a given piece of software or not work with a given piece of software.

Interesting, so it gets to the backwards compatibility element that has always been sort of architected into LTO. It sounds like you were thinking about the same thing with version numbering and such so it was consistent and that people could access previous version numbers and such. This that right? Am I interpreting that correctly?

That's exactly right. And ultimately the version number is three digits; for example, version 2.0.0. If the lowest digit, if we think of this as x.y.z. If the z number changes in the specification then this indicates that the specification has been updated texturally but this text that doesn't change the intent and meaning of the specification. This is more fixing typos or adding clarification where there is unintended ambiguity. If the y number has incremented then it has been a change in the format on the tape or a change in the meaning of a piece of data on the tape, but the change should not break implementations of the preceding version.

So, for example, a 2.0.0 implementation of LTFS, a tape written by a 2.0.0 should be readable by a 2.1.0 or 2.1.whatever without any negative consequences. If the X, if the first digit changes, then this indicates a major revision in the specification where the format has changed sufficiently that a preceding version of the specification will not understand how to read this format.

Interesting. You know that's so interesting to me that you put so much thought into even something simple like version numbering. But it just makes logical, intuitive sense given the constant changes. So I'm curious, I imagine you first released version 1.0.0 I would imagine. Do you know version we are shipping these days?

So actually we didn't release version 1.0.0 initially. We added this versioning into the specification specifically because we needed to rev the format very soon after releasing LTFS. And we realized the problem that existed so we baked in ... as we were changing the specification we added in these rules for how the version number would change to ensure that we didn't have this problem again.

And at the time we argued, or made the argument to ourselves, that LTFS is brand new, people are still trying to wrap their heads around the technology, so we have a small window where can actually make this change that is actually going to break backwards compatibility for those versions, but that's because we knew all of the people using LTFS at the time on a first-name basis. So we could call them individually and say, we're really sorry but we need to make this change.

Hah! Fair enough. That's sort of I think the really interesting benefit, right, of essentially starting a grassroots effort like you did where you and your team were essentially the thought leaders in driving it in partnership with the other industry players so that the group of you as sort of initial adopters, if you will, had a lot of flexibility early on but of course that was the ground-setting stage and now obviously with it being adopted so widely I would imagine a lot of these changes, versioning, all that stuff is very much set in stone and not the same kind of flexibility that perhaps there once was. But that's probably also a reflection naturally of the maturity.

You're exactly right. And to actually answer your direct question we’re currently shipping software is complaint with the 2.0.0 specification, there was a revision of a specification 2.0.1 which clarified a couple of typos which strictly speaking is what the software today is in compliance with but there's no substantive changes between 2.0.0 and 2.0.1.

Fantastic. That's great. So I think that this has been a great conversation. I really enjoyed it. I appreciate your time and you know I look forward to continuing our discussion in our future podcasts where we'll have another one talking about technology, another one talking about the future.

But this has been a great summary of sort of how the process was around creating LTFS. And I still find it great – it's not really a question, but I think it's really interesting how you guys did such a grassroots effort inside your team to really bring this to fruition and even doing things like writing the spec after you've already developed the code. It's sort of unusual, I think, process but certainly there's no doubt that you make a lot of progress. You've created a really interesting technology that I think will see adoption now and will accelerate going forward.

Thanks Jay. It's been really a pleasure talking with you about LTFS. And I look forward to our conversations about the technology behind it and how it works. This is really where I'm really looking forward to our next conversation. That's really where my head is usually.

Fantastic. Thanks.

The Speakers:

Michael Richmond
Jay Livens