Science Group at Columbia University Safeguards Troves of Environmental Research Data with JetStor
Since its creation in 1949, the Lamont-Doherty Earth Observatory (LDEO) at Columbia University’s Earth Institute has been a leader in Earth science. Its mission is to gain “knowledge about the origin, evolution and future of the natural world.” Located along the Hudson River just north of Manhattan, LDEO’s scientists were the first to map the seafloor; develop a computer model that could predict an El Niño weather event; prove the theory of plate tectonics; and reveal the oceans’ role in triggering climate change.
With support from the National Science Foundation, LDEO provides a central repository where scientists can store their data on Earth’s marine, terrestrial and atmospheric systems. Their efforts, however, produce huge amounts of information. For example, a fleet of 25 oceanographic research vessels sails around the globe year round with sensors that measure everything from the atmosphere to the water column to the sea floor. Datasets, which are stored indefinitely, range from a small spreadsheet of rock samples, to 400 Gb of sonar data from an oceanographic expedition, to terabytes of video data from a volcano observatory.
“Curating data for the national Earth science research community is a big responsibility,” said Robert A., Lead Systems Analyst/Programmer at LDEO. “Collecting the datasets is costly and time consuming, and they must always be safe. We can’t lose anything.”
This critical role in science research is becoming even more challenging. Scientists are deploying more and more sensors on ships and elsewhere that create exponentially growing troves of data. “Datasets are exploding in size and we need enterprise solutions to store them reliably and quickly,” added Robert.
Four JetStor SAS 716FD 16-bay RAID Arrays from Advanced Computer & Network Corporation (AC&NC).
SYSTEM CONFIGURATION (per cluster)
• Two JetStor SAS 716FD 16-bay RAID Arrays, each providing 128Tb of storage capacity using sixteen 8Tb SAS drives
• Five rack-mounted Dell PowerEdge 420 Servers, two with QLogic QLE2562 (8Gb) adapters
• Two QLogic SANbox 5802V switches with 10Gb Fibre Channel links
BENEFITS IMMEDIATELY REALIZED
LDEO is addressing its burgeoning datasets with JetStor SAS 716FD storage arrays from AC&NC. LDEO has long stored research data on JetStor RAID storage systems. “AC&NC has been one of Columbia University’s more popular vendors and there are many JetStor solutions on our main campus,” explained Robert.
The JetStor 716FD solutions greatly expand LDEO’s storage strategy. To ensure the safety of every dataset, LDEO maintains two matching clusters of JetStor RAID chassis, each in separate buildings on its campus. All data are replicated across the two clusters. Each cluster had consisted of five JetStor arrays, but the introduction of the JetStor 716FD systems increases both their capacity and performance.
The JetStor 716FD solutions each feature 128 terabytes of storage, significantly larger than LDEO’s legacy arrays. Additionally, these systems link with 8Gb Fibre Channel speeds rather than the gigabit iSCSI or SCSI connections of the earlier chassis.
“By switching to the high-speed fiber connectivity of the JetStor 716FD arrays, we gain the outstanding throughput needed to ingest even the largest datastores,” said Robert. “Researchers also have much faster random access to their data, avoiding bottlenecks and improving productivity.”
By turning to solutions like the JetStor 716FD systems, LDEO is also easing the burden of storing so much research data. “It’s simpler to maintain a few larger arrays than many smaller ones,” said Robert. “Moreover, we’ll trim our operating costs by consuming less electricity.”
Perhaps most importantly, by continuing to rely on JetStor solutions, LDEO meets its primary objective of safeguarding data critical to the scientific community.
“Our JetStor RAID arrays always deliver the dependability we demand,” said Robert. “In fact, until we replaced them with the JetStor 716FD solutions, the first JetStor chassis we bought in 2006 were still providing us trouble-free operation, which is remarkable after all these years. The solutions are rock solid and the staff at AC&NC continues to be responsive to our every need. I don’t have to submit a request ticket. Instead, they respond promptly to email, which we greatly appreciate.”
HOW WE DID IT
LDEO upgraded its storage clusters at each site by replacing a JetStor 412S chassis and a JetStor 516iS chassis with two JetStor 716FD RAID solutions. It installed QLogic QLE2562 adapters in two of the five rack-mounted Dell PowerEdge 420 Servers that link to the JetStor chassis in each cluster, providing the new JetStor systems with robust 8Gb Fibre Channel connectivity to the servers. The organization also deployed a four-port QLogic 5802V switch at each cluster and links them to the fiber-capable servers and the JetStor 716FDs via Fibre Channel.
All the JetStor platforms are configured for RAID 6, which provides block-level data striping and avoids data loss even if two disks fail within an array.
The 16-bay JetStor 716FD arrays are provisioned with 8Tb SAS drives, increasing the operational capacity of each cluster to over 330Tb. The solutions are engineered for non-stop operation. A fully cableless design improves reliability. Dual hot-swappable power supplies with integrated turbo-cooling and separate power connections, and dual hot-swappable controllers provide system tolerance of any component failure. Robert frequently uses the systems’ RAID Manager browser-based management and monitoring software to remotely configure and manage all of the JetStor systems. The application will forward e-mail notifications in the unlikely event that issues arise, helping to ensure availability.
With its fibre-capable, enterprise-class JetStor 716FD arrays, LDEO will continue to support scientists’ efforts to better understand our planet and its systems. “Our new JetStor chassis provide the research community with greater performance, more capacity, and the same availability we’ve enjoyed since we began using JetStor products nearly a decade ago,” concluded Robert. “We’re proud and pleased to know we’ll continue our mission well into the future.”
ABOUT ADVANCED COMPUTER & NETWORK CORPORATION (AC&NC)
Advanced Computer & Network Corporation designs, manufactures and markets high-performance, competitively priced data storage RAID systems. With an extensive line of Fibre Channel, NAS, DAS, and hybrid flash storage solutions for any computing platform and operating system, comprehensive free technical support, and fast on-time product deliveries, AC&NC brings freedom of choice to all clients, from workgroup to enterprise. Our RAID systems ensure rapid, round-the-clock access to all of the information our customers rely on to move forward in the 21st century.