Few data files are as large, costly, or more frequently reused than those created for video programming. One of the nation’s largest production studios develops and stores hundreds of terabytes of entertainment shows, promos, commercials, sports, documentaries, broadband content, network IDs, and interstitials for leading cable programmers.
Initially, the studio stored its content on virtual and non-virtual tape libraries. However, these solutions were slow and ineffective. When original files were lost, tight production schedules were compromised as administrators labored to retrieve backups.
Read more about how this award-winning production studio used JetStor RAID Arrays to combine the economies of tape libraries with the speed and efficiencies of disk storage.
Leading Production Studio Safeguards Programming with JetStor® RAID Solutions
THE ORGANIZATION This full-service production studio, located in the southeast, is one of the largest in the U.S. Founded some 30 years ago, it produces entertainment shows, promos, commercials, live sports, documentaries, broadband content, network IDs, interstitials, and DVDs for leading cable programming providers. Its commitment to entertainment excellence is exemplified by the many industry awards its work has garnered over the years.
THE CHALLENGE The studio found that success led to a critical challenge—safeguarding its work. Every week, its producers, artists, and executives generate data, especially large multimedia files, that must be stored and backed up. Initially, it relied on robotic tape libraries for back-ups due to their low per-gigabyte storage costs, but this strategy presented shortcomings. With their limited I/O capabilities, the tape drives ran constantly to keep pace with the flow of production data.
Moreover, restoring files was arduous. To retrieve a file, the entire monthly back-up of the file’s directory had to be restored, followed by incremental restores of snapshots until the day the file was last saved. Administrators tried virtual tape libraries, which provided faster I/O speeds, but retrievals of files still remained difficult. “We needed a new storage strategy if we were to protect our work according to our business needs,” said the studio’s storage engineer. “It had to combine economical and extremely reliable storage solutions with a means for rapidly accessing backed-up data.”
THE SOLUTION Twenty JetStor® SATA 516F 16-bay RAID Arrays from Advanced Computer & Network Corporation (AC&NC)
SYSTEM CONFIGURATION • ZFS file system running on the Solaris operating system
• Twenty JetStor SATA 516F 16 bay RAID Arrays, each with 32Tb of storage
• QLogic fibre channel switches linked to the JetStor Arrays
BENEFITS IMMEDIATELY REALIZED The studios’ strategy was to implement a primary storage repository for users’ data, and an identical site for back-up. Each site is anchored by ten JetStor SATA 516F RAID Arrays that house 233 terabytes of data. Should disaster strike, administrators can transform the back-up site into the live repository in minutes, enabling users to store and access their files routinely. The infrastructure greatly improves operational efficiencies over tape and virtual tape libraries.
By using ZFS, a public-domain file system for managing enterprise data storage, administrators view all the data stored on the JetStor solutions as unified virtual storage pools, permitting rapid access to back-up files. “Production deadlines are no longer jeopardized because of the time needed to restore data,” said the studio’s engineer, “and our staff no longer has to contend with work orders for laborious tape recoveries.”
Additionally, the stored data is up-to-date. Administrators use rsync to back-up 27 production SAN and NAS servers onto the JetStor Arrays at the primary storage facility. Every several minutes, the ZFS file system makes incremental block-level updates of the files on the primary site and saves them to the JetStors at the back-up site.
This strategy ensures that users can restore the latest versions of files should the primary site become unavailable. “Moreover, when we update or replace servers, we no longer have to temporarily park data on devices strewn across our network,” said the studio’s engineer. “Now, we simply attach a new server, direct it to our storage pool, and then remove the old device, saving us a lot of time and trouble.”
The JetStor solutions each provide robust RAID 6 protection, which means no data is lost even if two disks fail. The devices feature additional safeguards such as redundant, hot-swappable power supplies with two independent power inputs. Administrators can easily configure, manage, and monitor the solutions using the web-based JetStor GUI RAID Manager.
For extra protection, they immediately receive e-mail notifications should any issues arise with the solutions. “Low cost and scalability are further benefits,” explained the engineer. “Our ZFS system is free and scales to an astronomical billion terabytes. Our JetStor Arrays can each store 32 terabytes of data in one compact enclosure at a fraction of the cost of large storage arrays. And our system’s operating expenses are all but negligible in our overall IT budget.”
HOW WE DID IT The engineer initially developed the innovative storage strategy on a small scale. Using ZFS, he created an experimental virtual storage pool, called a zpool, by linking Dell and HP servers to legacy JetStor RAID Arrays. ZFS seamlessly operated with the servers and presented a unified view of the stored data, allowing fast access to individual files. When he had to remove data from a server that needed an upgrade, he stored the files in another zpool on JetStor and other storage systems. Although the data was scattered across storage devices, he was able to reclaim all the files easily. The engineer used this same approach to back up an Apple Xsan file system from a Dell server onto a JetStor SATA 516F RAID Array.
When a studio artist lost 500 gigabytes of data on a time-sensitive project, the engineer simply directed the project’s manager to the storage pool and the back-up files were retrieved within minutes. “The manager was all but stunned to recover the data so quickly,” the engineer noted.
Administrators then built the first primary storage site with 5 JetStor RAID Arrays, each with one Tb drives in its 16 bays. They connected production servers to fibre channel switches linked to the JetStor solutions and control the flow of data with virtual LANs (VLANs). To ensure effective disaster recovery, the engineer and his team built another primary storage site at a different location and linked it to the older site, now designated as the back-up facility, via a 10 gigabyte connection.
The new primary site featured 9 JetStor RAID Arrays, but with 2 Tb drives in their bays that doubled their capacity to 32 Tb per array. Administrators mirrored the two storage sites by adding four JetStor RAID Arrays with 2 Tb drives at the back-up site. They replaced the 1 TB drives in the site’s existing arrays with 2 Tb drives. They finished, for the time being at least, by adding another JetStor chassis to each site, bringing the total to 20 JetStor devices.
CONCLUSION The studio demonstrated that the most effective solution is not always the most expensive. “By relying on ZFS and JetStor, we’ve met our business and IT needs well into the future,” said the engineer. “We have a storage repository and a DR site with redundancy, flexibility, and scalability, all at very competitive costs.”