Comment: They characterize four parallel applications: sort, matrix multiply, seismic migration, and video server, in terms of their I/O activity.

Abstract: With a vast number of resources being committed to reformatting into digital form, we need to consider how we can ensure that digital information will continue to be accessible over a prolonged period of time. This chapter first outlines the general problem of information in digital form disappearing. It then looks closely at five key factors that pose problems for digital longevity. Finally, we turn our attention to a series of suggestions that are likely to improve the longevity of digital information, focusing primarily on metadata. This chapter was written for the library, museum, and archives communities. However, the observations will be useful for all communities wishing to ensure the longevity of any type of digital information.

Abstract: The promise has been made: ``Digital information is forever. It doesn't deteriorate and requires little in the way of material media.'' So said one of the chieftains of the emerging digital age, computer-chip maker Andy Grove, the head of the Intel Corporation. Another chieftain, Librarian of Congress James H. Billington, has set about digitizing the world?s largest library so that its contents can become accessible by anyone, from anywhere, forever. But a shadow has fallen. ``It is only slightly facetious,'' wrote RAND researcher Jeff Rothenberg in Scientific American, ``to say that digital information lasts forever—or five years, whichever comes first.''

Abstract: Managing large objects with high data-rate requirements is difficult for current computing systems. We describe an Input/Output architecture, called Swift, that addresses the problem of storing and retrieving very large data objects from slow secondary storage at very high data-rates. Applications that require this capability are poorly supported in current systems, even though they are made possible by high-speed networks. These range from storage and visualization of scientific computations to recodring and play-back of color video in real-time. Swift addresses the problem of providing the data rates required by digital video by exploiting the available interconnection capacity and by using several slower storage devices in parallel. We have done two studies to validate the Swift architecture: a simulation study and an Ethernet-based proof-of-concept implementation. Both studies indicate that the aggregation principle proposed in Swift can yield very high data-rates. We present a brief summary of these studies.

Comment: SHA-1

Comment: SHA-1, SHA-256, SHA-384, and SHA-512. Supersedes FIPS 180-1.

Abstract: An accurate estimate of host reliability is important for correct analysis of many fault-tolerance and replication mechanisms. In a previous study, we estimated host system reliability by querying a large number of hosts to find how long they had been functioning, estimating the mean time-to-failure (MTTF) and availability from those measures, and in turn deriving an estimate of the mean time-to-repair (MTTR). However, this approach had a bias towards more reliable hosts that could result in overestimating MTTR and underestimating availability. To address this bias we have conducted a second experiment, using a fault-tolerant replicated monitoring tool. This tool directly measures TTF, TTR, and availability by polling many sites frequently from several locations. We find that these more accurate results generally confirm and improve our earlier estimates. We also find that failure and repair are unlikely to follow Poisson processes.

Abstract: We present an I/O architecture, called Swift, that addresses the problem of data-rate mismatches between the requirements of an application, the maximum data-rate of the storage devices, and the data-rate of the interconnection medium. The goal of Swift is to support integrated continuous multimedia in general purpose distributed systems. In installations with a high-speed interconnection medium, Swift will provide high data-rate transfers by using multiple slower storage devices in parallel. The data-rates obtained with this approach scale well when using multiple storage devices and multiple interconnections. Swift has the flexibility to use any appropriate storage technology, including disk arrays. The ability to adapt to technological advances will allow Swift to provide for ever increasing I/O demands. To address the problem of partial failures, Swift stores data redundantly. Using the UNIX operating system, we have constructed a simplified prototype of the Swift architecture. Using a single Ethernet-based local-area network and three servers, the prototype provides data-rates that are almost three times as fast as access to the local SCSI disk in the case of writes. When compared with NFS, the Swift prototype provides double the data-rate for reads and eight times the data-rate for writes. The data-rate of our prototype scales almost linearly in the number of servers and the number of network segments. Its performance is shown to be limited by the speed of the Ethernet-based local-area network. We also constructed a simulation model to show how the Swift architecture can exploit storage, communication, and processor advances, and to locate the components that will limit I/O performance. In a simulated gigabit/second token ring local-area network the data-rates are seen to scale proportionally to the size of the transfer unit and to the number of storage agents.

Keyword: parallel I/O, multiprocessor file system, pario bib

Comment: They took the Unix file system from SunOS and extended it to run on the CM-5. This involved handling non-power-of-two block sizes, parallel I/O calls, large file sizes, and more encouragement for extents to be allocated. The hardware is particularly suited to RAID 3 with a 16 byte striping unit, although in theory the software could do anything it wants. Geared to data-parallel model. Proc nodes (PNs) contact the timesharing daemon (TD) on the control processor (CP), who gets block lists from the file system, which runs on one of the CPs. The TD then arranges with the disk storage nodes (DSNs) to do the transfer directly with the PNs. Each DSN has 8 MB of buffer space, 8 disk drives, 4 SCSI busses, and a SPARC as controller. Partition managers mount non-local sfs via NFS. Performance results good. Up to 185 MB/s on 118 (2 MB/s) disks.

Abstract: Commercial publishing interests are presenting the future of the book in the digital world through the promotion of e-book reading appliances and software. Implicit in this is a very complex and problematic agenda that re-establishes the book as a digital cultural artifact within a context of intellectual property rights management enforced by hardware and software systems. With the convergence of different types of content into a common digital bit-stream, developments in industries such as music are establishing precedents that may define our view of digital books. At the same time we find scholars exploring the ways in which the digital medium can enhance the traditional communication functions of the printed work, moving far beyond literal translations of the pages of printed books into the digital world. This paper examines competing visions for the future of the book in the digital environment, with particular attention to questions about the social implications of controls over intellectual property, such as continuity of cultural memory.

Abstract: DFSTrace is a system to collect and analyze long-term file reference data in a distributed UNIX workstation environment. The design of DFSTrace is unique in that it pays particular attention to efficiency, extensibility, and the logistics of long-term trace data collection in a distributed environment. The components of DFSTrace are a set of kernel hooks, a kernel buffer mechanism, a data extraction agent, a set of collection servers, and post-processing tools. Our experience with DFSTrace has been highly positive. Tracing has been virtually unnoticeable, degrading performance 3-7\%, depending on the level of detail of tracing. We have collected file reference traces from approximately 30 workstations continuously for over two years. We have implemented a post-processing library to provide a convenient programmer interface to the traces, and have created an on-line database of results from a suite of analysis programs to aid trace selection. Our data has been used for a wide variety of purposes, including file system studies, performance measurement and tuning, and debugging. Extensions of DFSTrace have enabled its use in applications such as field reliability testing and determining disk geometry. This paper presents the design, implementation, and evaluation of DFSTrace and associated tools, and describes how they have been used.

Comment: LBFS

Keyword: parallel I/O, hypercube, Intel iPSC/2, multiprocessor file system

Comment: Intel iPSC/2 Concurrent File System. Chose to tailor system for high performance for large files, read in large chunks. Uniform logical file system view, Unix stdio interface. Blocks scattered over all disks, but not striped. Blocksize 4K optimizes message-passing performance without using blocks that are too big. Tree-directory is stored in ONE file and managed by ONE process, so opens are bottlenecked, but that is not their emphasis. File headers, however, are scattered. The file header info contains a list of blocks. File header is managed by disk process on its I/O node. Data caching is done only at the I/O node of the originating disk drive. Read-ahead is used but not detailed here.

Abstract: In September 2001, the University of Virginia was awarded a grant from the Andrew W. Mellon Foundation to develop the first digital object repository management system based on the Flexible Extensible Digital Object and Repository Architecture (Fedora) [1]. The Digital Library Research Group at Cornell University originally developed Fedora under a National Science Foundation Grant [1-3]. Fedora is one of a number of repository architectures that have been proposed over recent years for use in digital libraries. Related architectures have included those described in Kahn and Wilensky [4], Arms, et.al. [5], Mnch [6], and Nelson and Maly [7]. The Digital Library Research and Development Department at the University of Virginia Library reinterpreted the architecture and built a prototype that demonstrated the feasibility of the architecture with their large and diverse digital collections [8]. Together, the two groups make up the development team for the project.