Lustre is a GPLed cluster file system for Linux that is currently being tested on three of the world's largest Linux supercomputers, each with more than 1,000 nodes. In the past 18 months we've tried many tactics to scale to these limits, and the first half of this paper will discuss some of our successes and failures. The second half will explore some of the changes that we plan to make over the next year, as we scale towards tens of thousands of clients and petabytes of data.

...

Download PDF.