Proprietary vs. Open-source Technology

Fighting to keep LTFS open-source

LTFS 1 Creation: Segment 3


Jay Livens:
You talked about LTFS and the desire to have it to be open and readable with a simple, flexible design. That’s I think a really powerful paradigm that pretty much mirrors what we’ve seen historically from physical tape because LTO tape has a standard that can be read by pretty much any vendor’s tape drive hardware regardless of manufacturing, and similarly regardless of who manufactured the media itself. That’s really I think beneficial with LTFS, but how did you manage the process of gaining agreement with other vendors, and other groups and such to adapt this technology especially since at the time, of course, you work for IBM who has a very strong business in this space. Obviously, it’s different when a single vendor does it versus a consortium, say like LTO, the LTO consortium or similar cross-industry group.

Michael Richmond:
That’s a really interesting question, Jay. The way that LTFS came to being is that a couple of geeks got interested in the technology, and threw it together, so that we could have a convincing demo. An engineer at IBM was working on the Linux implementation. We had planned to demo LTFS to a small number of customers who IBM had a good relationship with at NAB in 2009. Sometime in early March of 2009, I learned more about this project, and suggested that if we only showed a Linux implementation, then it would be difficult to have credibility in the media and entertainment space.

My manager at the time looked at the Mac laptop on my desk and said, “Well, you have six weeks. Can you port the software?” We were really just throwing the technology together at the time. In fact, the implementation of the prototype’s write functionality was written by me as we are driving across the Mojave Desert with a SUV full of hardware to get to NAB. It was really pulled together quite quickly and was very much in flux at the time. The feedback from that demo was overwhelmingly positive. We then used that positive feedback to make the pitch to IBM management that we should actually develop this technology as a product. That approval took a few months to work through the process, then we started productizing the technology which ultimately meant completely rewriting the file system, completely changing the format because what we have learned during the prototype was that our software guys really didn’t understand how to work with tape. We had started off just by throwing SCSI commands at the tape drives until we got something working.

From the prototype, we learned what we had done wrong and how to better design the file system, and that better design went into the actual reimplementation. Throughout the beginning of this productization, IBM was thinking of this as a proprietary file system, as a closed-source technology that we would release, and we would sell the way that other vendors actually sell their own technology that works over tape using proprietary formats on tape. During this development time and the working out our plans to go to market, myself and my manager at IBM, we’re arguing pretty strongly that if we release this as a proprietary format, we would get nowhere in the market because there are dozens of proprietary formats out there, and this would just be yet another proprietary format without much new being added. There was certainly a compelling use case of making tape easier to use, but if you were locked in to IBM as a vendor, then customers would be unlikely to be willing to adopt the technology.

What we did was we argued internally, and made the pitch to release the implementation as an open-source implementation for no cost. We also strongly advocated for the idea that we should be documenting the format of an LTFS volume in a specification and publishing that specification. In the back of our minds, we were thinking about how we actually get to a standardization process; but getting industry standards up and running, moving them forward and getting to our productive output is generally long drawn out process because standardization is both about the written document that defines whatever is being agreed upon. The other part of the standardization is the actual agreement between industry competitors who are distrustful of each other just because they are competing with each other in the industry. Once we got to this idea of we would release the technology as open-source, we would write up this format specification, we then started having conversations with other vendors in the space, particularly HP and Quantum, to give them a heads-up about this technology that we are planning to release. There were a lot of careful legal negotiations there to protect both sides, but ultimately, we got to a point where we could share the specification document with HP and Quantum.

I had numerous phone calls with their tape experts to describe what we are trying to do. They were excited about the technology, but understandably concerned about what IBM’s motivation was. This was largely a conversation that happened through December, January, February in 2010. Probably about a month before NAB, we had all of the pieces in place where we could share a better version of the open-source code with HP and Quantum, so that their engineers could start actually looking at what’s really behind everything and how it works. This was ultimately massively successful because we were able to announce LTFS in April at NAB, April of 2010. At the same show, HP was able to announce that they would have an LTFS implementation available I think it was within a couple of weeks of the show ending. The implementations by all three companies, IBM, Quantum, and HP are all from the same codebase. HP has tweaked the code a little bit for their particular drives, and implemented low-level drive management operations that are custom for their drives. But the code was architected, so that these drive level operations could be easily swapped in.

I believe later in the same year or maybe in the following year, Oracle announced support for LTFS on their tape hardware which completely blew me away because we had … Since Oracle is not part of the LTO consortium, operates completely different tape technology, they picked up and ran with this technology based on the publicly available information which was exactly what we are trying to do by releasing the technology as open-source for single drive and releasing the open specification.

Jay Livens:
That’s a great story. What I personally love about it too is how it sounds almost like entrepreneurial, like you, and your team, and your manager are working really hard on not only developing the technology, but also thinking about how you go to market, and whether it’s proprietary or open-source. It sounds like an interesting and exciting time for sure as you’re working through all these different potential go-to market models, and technology strategies, and such.

Michael Richmond:
It was. It was a lot of busy work and an extremely busy time for me personally. The porting efforts that I did over those six weeks before the first NAB, or the NAB in 2009 I should say, was conducted between 6pm and 12pm each day because I still had job responsibilities on my existing project. It ended up being something that the team of engineers that we had working early on really got their teeth into this problem; really were living and breathing it. The go to market conversation was challenging largely in terms of we were technologists. We were really saying, “This is the way that tape should be used. This massively simplifies things.” But we had to work through all of the difficult conversations where there’s natural suspicion or uncertainty about what our goals were. Because it was very easy to see how people could think of this as IBM is trying to release a format, get people to adopt it, and then IBM would change the format out from underneath everybody, because we’re in a competitive space, and the tape market has been shrinking over years.

This was an honest attempt to try and expand the tape market with the view that if IBM expands the tape market across the industry, a rising tide lifts all boats. That particular phrase was something that was uttered in pretty much every meeting that we had about our go to market strategy. One of the oddities that happened during this phase that was unfortunate and a slight misstep, but ultimately hasn’t harmed us too much, is that IBM felt uncomfortable with the word “tape” in the linear tape file system name.

When IBM went to market with V1, we released the linear tape file system specification, but our product was called the Long Term File System. This was a way of dancing around concerns about tape being a negative thing. As technologist, we kept saying, “But tape is the valuable thing here,” but this was a disconnect. Ultimately, it meant that as the architect of LTFS, both the specification and the software, I went to a couple of tradeshows where customers would come to me and say, “Well, this Long Term File System, is it compatible with LTFS?”

Jay Livens:
That’s funny.

Michael Richmond:
My answer was always yes, but it was a challenging conversation to have until we could get everyone on the same page that naming the technology a consistent name is vital for customers to be able to understand what we’re doing here.

Jay Livens:
Fantastic. It is interesting how … It’s nice the letters worked out, right? The LT could be Long Term or Linear Tape in the final iteration, so that is funny how that worked out in a favorable way.

Michael Richmond:

The Speakers:

Michael Richmond
Jay Livens