This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
linux:datamanagement [2017/11/27 12:34] – deul | linux:datamanagement [2020/11/24 07:48] (current) – [Data generated in the Astrophysical and Optical labs] deul | ||
---|---|---|---|
Line 1: | Line 1: | ||
======Data Management====== | ======Data Management====== | ||
- | Your data is stored on the STRW data storage media which can be either the local disk of your workstation, | + | Your data is stored on the STRW/IL data storage media which can be either the local disk of your workstation, |
+ | |||
+ | The Observatory has a semi filled in template for {{ : | ||
=====Sources of data===== | =====Sources of data===== | ||
Line 11: | Line 13: | ||
====Data from the big international observatories==== | ====Data from the big international observatories==== | ||
- | At the big international observatories data acquisition takes place in a controlled fashion. Data processing is done in the head quarters computing centers and is stored and archived at the storage facilities of sites. Is some cases the that are proprietory | + | At the big international observatories data acquisition takes place in a controlled fashion. Data processing is done in the head quarters computing centers and is stored and archived at the storage facilities of sites. Is some cases the that are proprietary |
- | Each international observatory has a portal that can be accessed to download observational data. So the processed ' | + | Each international observatory has a portal that can be accessed to download observational data. So the processed ' |
- | ====Data generated | + | Depending on the processing unit data can be stored on the local disks at your workstation or on the disks attached to the server that is used for processing. This provides the best data-to-cpu throughtput. data stored at the local workstation should be stored on the '' |
+ | In general final data products are published and made available to the community. This can be done in several ways, depending on the size of the data or the type of product. For small data set the Centre Donnees Stelleair at Strassbourg is the standard place when published in European journals. For larger data sets one might opt to make the data available through a project website (which can be hosted by the Observatory) or through the associated big international observatory. | ||
+ | |||
+ | Data coming out of local research will always be stored on local disks (bulk storage) at the Obsrevatory. Just contact the computergroup so that they can copy the data to bulk storage and make it available for further access. | ||
+ | |||
+ | ====Data generated in the Astrophysical and Optical labs==== | ||
+ | At the two Observatory laboratories data is produced by the instruments acquisition board and stored locally on lab computers. Data that will be used in further processing can be stored on the '' | ||
+ | |||
+ | This share is accessible both from the data acquisition equipment and from the personal desktops of the lab associates. The storage device is on a special secure medium and is snapshotted regularly as a form of backup. | ||
+ | |||
+ | Users are responsible for keeping track of the data and retaining its integrity. The Windows accounts and access rights can help in this task. | ||
====Data produced by theoretical work and simulations==== | ====Data produced by theoretical work and simulations==== | ||
+ | As a result of theoretical calculations or model simulations, | ||
+ | |||
+ | Similarly to the data processing for data coming from the big observatories it is the users responsibility keep track of the data and its integrity. As most of this processing is done on the Linux computing environment, |