Data intensive supercomputer JASMIN to double available storage to more than 44 petabytes

JASMIN supercomputer

A major upgrade is being made to double the storage available in one of the UK’s national supercomputers.

The upgrade will double the available storage to more than 44 petabytes, equivalent to over 10 billion photos. It will also add around 40% to the processing capability, with 11,500 cores on 600 nodes, similar to adding the power of several thousand high-end laptops. This means that the 1,700 registered users of JASMIN can process and analyse big datasets simultaneously and in very little time.

When JASMIN was brought online six years ago with just 4·5 petabytes of storage, it revolutionised access to data for the environmental science community in the UK. Researchers can access the data, which is typically too big for them to download to their own computers, and process it rapidly, reducing the time it takes to test new ideas and get results, from months or weeks to days or hours.

The Centre for Environmental Data Analysis (CEDA) manages the software and data available on JASMIN. Dr Victoria Bennett, Head of CEDA, said:

“We are excited to be expanding JASMIN to manage the increasingly large datasets, from satellites, climate models and other sources. The current Sentinel satellites alone are producing 10 terabytes of data every day and this will grow as more are launched as part of the European Commission’s Copernicus programme. This upgrade will allow us to build on the successes we’ve already seen in enabling our users in the science community to efficiently process and analyse these datasets.”

Funded with a multi-million pound investment from NERC, the upgraded system will support the global analysis of the next generation of climate models, provide a venue for UK academia and industry to exploit Earth observation data, and continue to provide the ‘UK environmental data commons’ – an online collaborative space bringing together data, services and expertise – underpinning much of academic environmental science.

NERC Associate Director of National Capability & Capital Dr Liz Fellman, said:

“NERC welcomes this major upgrade to a world-class facility. The JASMIN supercomputer is central to delivering NERC science across its portfolio and provides a globally unique and increasingly powerful capability for the UK’s environmental science community, enabling significant improvement of predictive environmental science to benefit the UK and beyond.”

Professor Pier Luigi Vidale from the University of Reading has been using JASMIN since 2012 to store and analyse high-resolution global climate model data. Professor Vidale welcomed the JASMIN upgrade:

“The project we’re currently leading involves 21 institutions across Europe and will output more than 4 petabytes of data. The upgrade will allow us to store all data and to do most of the analysis online, thus dramatically speeding up the extraction of science, at unprecedented resolution and enabling scientific publication at a far higher rate. We would not have embarked on the project without the enhanced JASMIN.”

As a ‘customised’ data intensive supercomputer, the JASMIN upgrade involves the integration of computing equipment from many suppliers, a specialised new network, the development and deployment of new software, and the migration of petabytes of archived data from old hardware in need of retirement, to new. The entire process will take many months, from the integration of the first new equipment in March until the last of the old storage is retired. Completion is expected by the end of 2018. The system integration is being led by the Science & Technology Facilities Council’s Scientific Computing Department (SCD), and the software and data management by CEDA.

Jonathan Churchill, JASMIN Systems Architect & Manager for SCD, said:

“We are terrifically excited to see how the JASMIN 4 upgrade, which we designed and are now installing, will be exploited by the ever expanding JASMIN science communities. Not only have we dramatically scaled out JASMIN storage, processing capability and networking, but the new storage and networking technologies will improve the user experience and provide capabilities that we have never been able to make available to users before. The computer upgrade will provide not only much needed extra batch computing cores but also provide the deep, on-demand cloud computing capacity and flexibility that releases new analysis environments to our science communities.”

JASMIN is jointly managed on behalf of NERC by CEDA (part of RAL Space) and SCD, all based within the Science & Technology Facilities Council (STFC) at Harwell Campus in Oxfordshire.

Further information

Mary Goodchild
NERC News & Media Officer
01793 411939
07710 147485
marodc@nerc.ac.uk

Notes

1. JASMIN is a globally unique data intensive supercomputer for environmental science. Jointly managed by CEDA and STFC SCD.

2. RAL Space is an integral part of STFC’s Rutherford Appleton Laboratory (RAL). RAL Space carries out world-class space research and technology development with involvement in over 210 space missions. The Centre for Environmental Data Analysis, based within RAL Space, serves the environmental science community through three data centres, data analysis environments and participation in a host of relevant research projects.

3. SCD manages high performance computing facilities, services and infrastructure, supporting some of UK’s most advanced scientific facilities. SCD staff, which is made up of approximately 160 computing experts, offer services and products that help the scientific community to make vital discoveries and deliver progress. SCD are leaders in the advancement and support of scientific computing, carrying out research and development to gather and interpret results quickly and clearly. Whether it’s creating new computational science software, helping to visualize complex scientific results, or developing the infrastructure that allows SCD to process huge amounts of data, research and development from SCD is driving improvements across the scientific research landscape.

In addition to our wider R&D activities, we are also trouble-shooters and problem-solvers, providing day-to-day products and services that help meet scientific needs. This includes hardware – both computational and storage facilities – as well as data management, networks, software, skilled people and expertise, policies and all of the other behind-the-scenes components vital to scientific investigation.

4. NERC is the UK’s main agency for funding and managing research, training and knowledge exchange in the environmental sciences. Our work covers the full range of atmospheric, Earth, biological, terrestrial and aquatic science, from the deep oceans to the upper atmosphere and from the poles to the equator. We coordinate some of the world’s most exciting research projects, tackling major issues such as climate change, environmental influences on human health, the genetic make-up of life on Earth, and much more. NERC is a non-departmental public body. We receive around £330 million of annual funding from the Department for Business, Energy & Industrial Strategy.

SOURCE: NERC