RESEARCH ON OUR SITE
Changer la langue en Francais    SITE MAP | ACCESS | DIRECTORY | CONTACTS | PARTNERS | LINKS
  
 

Storage of the Experimental Data
Instrumentation Contents > Computing > Experimental Data Storage Synchrotron SOLEIL : Active Circle - NeXus - TWIST
To store the data generated by the beamlines experiments, the data storage infrastructure is based on the concept of cellular storage "Active Circle" (ACTIVE CIRCLE) and uses standard hardware.
This choice has been driven by the following requirements : availability in order to run even in case of a failure of the storage network, reliability to be able to access a stored data whatever it happens, security of data which should be accessible only to the "beneficiaries" and scalability with the progressive opening of the beamlines.
The infrastructure is operationnal since december 2006. The storage capacity was increased at the beginning of 2008, in order to answer the beamlines needs.
Principle
Each user has a direct access to a circle which looks like an unlimited and virtualised file system, the allocation and management of the storage resources being transparent for him. This circle is composed of a whole of servers called cells, connected via IP, without any hierarchy between them. 
Each cell is an entrance point to the file system which permanently verifies the presence of the other cells of the circle. Thus, the cells share the storage tasks, the data being automatically replicated and distributed on other cells, and automatically react to the events like incidents, cells appearance or disappearance, new resources....
Infrastructure
In case of SOLEIL, there is :
  • a local storage cell for each beamline for the most recent data also called "close data". Its capacity is adapted to the needs of the beamline. Fifteen local storage cells are already installed on the beamlines : the phase I beamlines (AILES, CASSIOPEE, CRISTAL, DESIRS, DIFFABS, ODE, PROXIMA1, SAMBA, SMIS, SWING) and first phase II beamlines (ANTARES, DISCO, MARS, PLEIADES).
  • two cells located in each of the two computer rooms – RGI1 in the central building and RGI2 in the synchrotron building – driving respectively an EMC disk library (115 TB) and a GRAU capacitive tape library. Data are replicated in each of the two rooms and automatically migrated from disks to tapes according to a precise strategy for each beamline. Basically, "recent data" remain for hundred days on disk while "long term data" remain from one to five years on tape following the volume of data of the beamline. The used tapes can contain 400 Go to 1,3 To depending on the type (LTO3 or SAIT1) and the compression level.

 
Complementary tools

Each file of experimental data is recorded on the local storage cell, via the TANGO control and data acquisition system of the beamline. The NeXus format has been chosen as the SOLEIL standard file format. To be able to qualify the recorded data,  the path file and data describing the acquisition context are referenced in a database. Besides, when recorded, an authentification mechanism via LDAP is used in order to know which right access to give to the file and to ensure that the data remain speficic to the project and its experimenters.

A Web access tool, https://twist.synchrotron-soleil.fr, allows to easily search data files, with the ability to explore their content, and to extract data into an ASCII or binary format. --> News about TWIST

More details
Accueil