Implementing Offline Digital Video Storage
using XenData Software
XenData software manages data tape drives, optionally combined with a tape library, on a Windows Server 2003 platform to create an attractive offline storage solution for professional video applications. Individual files are written to and read from data tape as though from a magnetic disk. The system retains file and directory information for tape cartridges that are held 'on the shelf'. When a tape cartridge is ejected from a drive, the files that it contains continue to appear in the file system. If access is attempted to a file located on an offline tape, the system will provide notification that identifies the particular cartridge on which the file is held.
In addition to convenient file-based access, the solution delivers outstanding performance, capacity and data longevity.
This paper discusses implementing an offline digital video storage solution using XenData software.
1. Introduction
XenData software solutions are available for both fully automated digital video archives and offline file-based storage. This paper describes implementation of offline digital video storage using the following software:
Video Edition of XenData Archive Series software for managing stand-alone drives and a tape library
XenData Alert Module
The XenData software manages one or more data tape drives, optionally combined with a tape library, on a Windows server. Contents of the tapes appear as one file system under a standard Windows logical drive letter. Files are written to and read from tape as though from a magnetic disk.
This unique solution is optimized for managing an offline digital video archive. A magnetic disk cache is employed to provide file caching for performance reasons. It is also used to retain file and directory information for tape cartridges that are held 'on the shelf'. When a tape cartridge is ejected from a drive, the files that it contains continue to appear in the file system. If access is attempted to a file located on an offline tape cartridge, the system will provide notification that identifies the particular tape cartridge on which the file is held.
XenData software is also available for fully automated digital video archives using tape libraries that scale to multiple petabytes. The tapes generated by the offline solution described in this paper are fully compatible with such an automated system.
2. Supported Data Tape Formats
The software supports the following tape formats: AIT-2, AIT-3, AIT-4, LTO-2, LTO-3, SAIT-1 and SDLT 600. Of these, SAIT and LTO-3 are ideal for high performance digital video applications and their characteristics are summarized below.
SAIT-1 LTO-3
Capacity per cartridge
(Native capacity i.e. without compression) 500 GB 400 GB Equivalent hours recording per cartridge at 25 Mbps 44.4 hours 35.5 hours Equivalent hours recording per cartridge at 50 Mbps 22.2 hours 17.7 hours Maximum Data Transfer Rate in Megabytes per second
(without compression) 30 MB/s 80 MB/s Media Archival Lifetime 30 years 30 years1
SAIT (Super Advanced Intelligent Tape) was developed by Sony and has a strong market share in digital video applications. At 500 GB, it offers the highest capacity per cartridge. Using a recording bit rate of 25 Mbps, this is equivalent to 44.4 hours per cartridge. More information about this format is available from www.sait.com.
LTO (Linear Tape Open), also known as Ultrium, is now in its third generation. It is the most popular mid-range tape format in the general IT market and combines high capacity with very high data transfer rates. LTO-3 cartridges have a capacity of 400 GB, equivalent to over 35 hours recording at 25 Mbps. Tape
drives are manufactured by three vendors: Certance, HP and IBM. More information is available from www.lto-technology.com.
Compared to video tapes, SAIT and LTO-3 data tapes are much lower cost per hour of recording and, with lifetimes 30 years, are much more stable.
3. A Non-Proprietary Solution
Digital video archiving in broadcast has conventionally involved complicated architectures, proprietary formats and proprietary interfaces. This has resulted in solutions which have been expensive to install and difficult to maintain.
In contrast, XenData software creates a digital archive on a Windows Server 2003 platform with a straightforward architecture and non-proprietary interfaces. The software is tightly integrated into the Windows server operating system which means it delivers high performance with a simple configuration. The XenData archive has a standard file system interface appearing as a single Windows logical drive letter. The solution is optimized for use with the standard Windows network protocol (CIFS) or FTP file transfers. This non-proprietary approach to the interface means that the archive can be used by multiple applications and it does not tie the user to any particular asset management or automation solution. In addition to its standard file system interface, the solution offers many other non-proprietary features:
¾ All file types can be archived on the system and partial file restore has been implemented in a way that is not specific to the file type.
¾ The system uses the open standard TAR format for recording to data tape.
¾ The system fully complies with the Microsoft security model based on Active Directory, which means that tedious special administration of file permissions is avoided.
4. Recommended Configurations
Two Tape Drives
Although a single tape drive is supported, a two drive system is recommended. To state the obvious, one drive cannot simultaneously write to the latest tape and read files from another tape. A configuration with two drives allows files to be written to the archive at all times using a drive dedicated for recording. The second drive is then available for retrieving files from other tapes.
When accessing a file that is on a tape within one of the drives, it is simply read over the network. However, if there is an attempt to read a file that is ‘on the shelf’, an on-screen message or e-mail alert is sent that identifies the tape cartridge that contains the
Copyright XenData Limited 2006
Windows 2003 File Server with onboard
magnetic disk cache
1 Gbit Ethernet LAN
Recommended Configuration with Two Tape Drives
Two tape drives connected via SCSI
One drive is dedicated to writing and the other is used for reading The server runs the Video Edition of
XenData Archive Series software and the XenData Alert Module.
requested file. This cartridge can then be inserted into a drive making the file available to the network. When the latest tape that is being written comes close to being full, an e-mail or on-screen alert is issued. This occurs when the tape is 95% full by default and it provides a warning that the tape should be ejected and replaced with a new blank tape cartridge.
A further feature of XenData software is the automatic generation of one or more additional tape cartridge replicas for offsite retention and disaster recovery purposes. However a configuration with two drives is not recommended for automatic tape cartridge replication – a configuration such as the system described below should be used.
Tape Drive and Autoloader
This configuration consists of a tape autoloader and stand-alone tape drive. (An ‘autoloader’ is the term for a small tape library with only one internal tape drive.)
The use of an autoloader automates the writing process and supports tape cartridge replication. The autoloader holds the following tape cartridges:
¾ Latest tape that is being written and any replicas ¾ Blank tapes
¾ Fully written tapes that are ready for export
The system automatically generates replicas. Also as soon as a tape becomes full, the system automatically initialises and starts using the next blank tape. If the autoloader is short of blank tapes and the latest tape is becoming full, an e-mail or on-screen alert will be issued.
The stand-alone tape drive is used for reading files from full tapes that have been exported from the autoloader.
Server Hardware
With large files written to a server running XenData software, the data transfer rates for both writing and reading are typically limited only by network speed or by the tape drives. For example, with SAIT drives, I GB Ethernet and a 2.4 GHz dual processor, sustained transfer rates over the network of 200 to 300 Mbits/s are easily achievable.
The minimum server hardware requirements are available from www.xendata.com. Even a modest specification server will deliver high performance. However, it is important that the server has sufficient RAM. A minimum of 1 GB is required, but 2 GB or more is recommended.
Copyright XenData Limited 2006
Windows 2003 File Server with onboard
magnetic disk cache
1 Gbit Ethernet LAN
Recommended Configuration with Autoloader and Drive
Autoloader and tape drive connected via SCSI
The server runs the Video Edition of XenData Archive Series software and the XenData Alert Module.
5. Convenient File Access
Windows Offline Attribute
After a file has been successfully written to tape, the Windows offline attribute is set. The Microsoft offline bit changes network timeout periods to allow retrieval of the file from media with long access times. It also changes the appearance of a file within Windows Explorer – a small clock is added to the bottom left of the file icon as shown below.
For files that are on tape cartridges either in a tape drive or an autoloader, the file is read just as it would be from magnetic disk. The only difference is that it might take a minute or so to access the file.
If an application tries to read a file that is on an offline tape cartridge, it will display a message such as “The remote storage service was not able to recall the file”. Simultaneously an on-screen message or email alert will be issued that identifies the particular tape cartridge that contains the file.
Keeping Files Online
The system can be configured by the administrator to keep some files on the magnetic disk cache for defined periods. One use of this feature is to keep low resolution proxies online permanently. Of course, the magnetic disk cache needs to have sufficient capacity to retain the online files.
6. Summary
This paper describes a unique storage solution that addresses the needs of professional digital video in a convenient and cost effective manner. The solution provides high performance file-based access using low cost standard data tapes with lifetimes specified at 30 years.
7. Glossary
Autoloader Term for a tape library containing only one tape drive.
CIFS 'Common Internet File System'. It is the standard protocol used by Windows computers to
communicate on a network.
Fibre Channel A technology for transmitting data between computer devices at data rates of up to 2
Gbps. Fibre Channel is especially suited for connecting computer servers to shared storage devices and for interconnecting storage controllers and drives. In addition to SCSI, XenData software supports Fibre Channel interfaces.
FTP 'File Transfer Protocol'. FTP is a protocol commonly used to copy files between two computers on
the Internet. Both computers must support their respective FTP roles - one must be an FTP client and the other an FTP server.
LTO 'Linear Tape Open'. This is the most popular mid-range tape format which is also known as Ultrium. NAS 'Network Attached Storage'. XenData Archive Series software may be installed on NAS file servers
running Windows Storage Server 2003.
RAID 'Redundant Array of Independent Disks' RAM 'Random Access Memory'
SAIT 'Super Advanced Intelligent Tape'. This is a popular mid-range tape format developed by Sony. SCSI ‘Small Computer System Interface’. This is a common interface for connecting computers and
peripherals.
Tape Alert Tape drives and other hardware employ a mechanism called Tape Alert to send maintenance
information to external devices. XenData Archive Series software periodically monitors Tape Alert information and informs the user or takes other appropriate action where necessary.
TAR A term derived from 'Tape ARchive'. It is a popular open standard data format optimized for
archiving to tape and is the format used by XenData Archive Series software.
Ultrium Mid-range tape format, also known as LTO.
For more information, please visit www.xendata.com or contact XenData:
