Please do not hesitate to contact us if you have any questions.
FAQ about DHC-DC client
How to browse and download data from the HC-DC client ?
The procedure to retrieve and install the client can be found here :
http://sphere-dc.osug.fr:8080/sphere-server/
To browse data :
- install and launch the client
- Login : public_user and password : public_user
- To browse individual files (mostly interesting for calibration files, data are in the workspace “calibrations_reduites”), click on “Data Browse”
- To browse sets of files produced at the same time during reduction (mostly interesting for level 1 and 2 data in the PUBLIC_reduced_science workspace), click on “Process Browse”.
- Select the Workspace you are interested as well as additionnal filters. Two workspaces of public data are currently available:
- calibrations_reduites : reduced calibration files
- PUBLIC_reduced_science : Level 1 and 2 reduced science data
To download some data or process outputs:
- Select the file(s) (browse data case) or processes (browse process case) you wish to retrieve
- Right click on your selection and create a download script for these files on your local computer
- Make the file executable (for example with chmod x+u script)
- Launch the script to retrieve the data
Complete information can be found in the on-line manual
Important tips on browsing on the SPHERE-DC client
The java client provides a graphical interface from which you can select datasets, by filtering them on many available criteria (RA, dec, date, program ID, target, filter, etc…), and then creating a download script for all your selected data by right clicking on the selection and use the “Download script” option (see next FAQ question about downloading).
You have 2 ways to make a selection, either by browsing data (i.e. files) or by browsing processes (i.e complete data set at a given reduction step). To download reduced science data, we strongly advise to use “browse process” because by construction it provides consistent datasets. It also allows you to visualize which algorithm has been used.
Using Browse process : click on the “browse process” tab on the main DC-client. The 2 levels of reduced science data made available by the SPHERE-DC are outputs of 2 routines.
- To get level 1 reduced data – Reduced master cubes with wavelength, flux and parallactic rotation calibrations file-, in the filtering tab “Recipe”, type “convert”, wait for the client to propose you a choice
- select the recipe name “ird_convert_recenter_dc2021” to get IRDIS data.
- select the recipe name “ifs_convert_dc2” to get IFS data.
- To get level 2 reduced data – calibrated stacks after various ADI algorithms have been applied, as well as associated detection limits-, in the filtering tab “Recipe”, type “specal”, wait for the client to propose you a choice and then
- select the recipe name “ird_specal_dc” to get IRDIS data.
- select the recipe name “ifs_specal_dc” to get IFS data .
Filtering.
- You can naturally apply any other available filtering criteria to narrow down your selection.
- Do not forget to click on “submit” after introducing a new filter, to be sure the client has actually updated the selection.
- You can get more information about the browsing and filtering options within the DC client by reading section 3 of the DC manual , or just hover your mouse on any field in the « filtering » tab to get dedicated info on this filtering parameter.
- Also for most parameters in the DC filtering interface you can enter partial info and the DC will propose you relevant completion, so it is better to look for GJ_285 by typing “285” and selecting GJ_285 rather than typing GJ285, without the “_” and getting nothing that matches ! The “Object” field comes from the fits header, hence what was manually written written in the OB. Some typos may have been made during the observation, so if you do not find your target, use the box “target”, which will look in the database for the simbad resolved names (any of the Simbad names will work) associated with the coordinates of the observation, or directly use RA/dec filter.
How to download data from the SPHERE-DC client ?
You can select the files (see tips about browsing above and indications to reach the different data levels) as usual : you can manually select subset of the displayed selection by clicking on specific lines of the dataset. As for any selection you can make a multiple selection by pressing “ctrl” and clicking or select a range with the “shift” key.
A right click on one line or your selection allows you to retrieve a script to download the data. Note that if you are interested in a given data level only you may want to choose “outputs only” (full will include inputs as well, i.e. the previous level data). At this stage you can choose a script applying to your selection or to all the displayed data.
The script must be made executable (for example, for linux and mac: “chmod u+x script_name.sh”) before being launched.
What is not in this public release ?
- Later than P95 Zimpol data
- Polarimetric IRDIS data are not reduced, they will be on a short term basis
- IRDIS LSS mode data are not reduced yet
- IRDIS Observations without star centering sequence or observations with saturated or otherwise invalid star centering sequence are only available at level 1 only (without relative recentering of the left and right channels)
Note that a few PIs may have requested to have a delay before their reduced data is publicly released, those will be available after some delay for recent public data.
Finally, if you are interested by a dataset and think it should be released, don’t hesitate to ask us and we’ll see what we can do for you.
What are the limitations of the final reduced images ?
Here are a few warnings and informations on the limitations of the data:
- It is well-known that the cADI and TLOCI image processing introduces some patterns in the images. It is therefore necessary to use dedicated techniques (for example by fitting the pattern) to extract photometric informations on a detection.
- The final reduced images make use of an automated frame selection to avoid bad frames within a data cube. Some science cases might benefit from more or less aggressive frame sorting that our automated selection.
- More about data quality here !
FAQ about DIVA+
Search Syntax and Operators for Querying the Database
Queries in this database operate on two logical levels:
- Within a field: If multiple values are specified for a single field, the query will return results that match any of those values (i.e., an or operation).
- Across fields: If multiple fields are filled, the query will return only the results that match all of the specified field values (i.e., an and operation).
For example, if you enter the following parameters:
- instr_setup: IRDIS IFS
- band: G
The query will return all images that match both of the following conditions:
- The instr_setup field is either “IRDIS” or “IFS”
- The band field is “G”
This means that the band parameter will be applied to both IRDIS and IFS images.
In fields with the option of NULL or NOT NULL, the following applies:
NULL means that the database will retrieve all records of “reductions” that do not have a value for this particular parameter. In other words, any record where this field is empty or unknown will be included in the results.
NOT NULL, on the other hand, means that the database will only retrieve records of “reductions” that have a value for this parameter. This means that any record where this field is empty or unknown will be excluded from the results.
By default, the NULL and NOT NULL options are not enabled. Please contact us if you need these options to be activated for any field.
Dates in diva+
Fields that use a date in DIVA+ utilize the module described here: https://valor-software.com/ngx-bootstrap/#/components/datepicker?tab=overview
This datepicker module provides a user-friendly interface for selecting dates, with options for setting the format, minimum and maximum date.
To change the year in the graphical interface, simply click on the year displayed. After selecting a range of dates using the graphical interface, you can also modify them manually. It may be easier to enter two very distant dates (e.g., 2015-02-01 – 2024-01-01) to quickly set a wide range of years, and then adjust the start and end dates as needed.
Create an account in DIVA+
You can fully use DIVA+ without an account. However, to download large archives (e.g., the entire SHINE survey), creating an account is recommended. Generating large archives may take time, and with an account, you can leave the DIVA+ page while your archive is being prepared. You will receive an email notification when it’s ready to download. To access your archive, simply log in, click the icon at the top right, and select ‘Archive list’.
In the future, we will introduce more features for using DIVA+ with your account. Feel free to contact us if you have any questions or are interested in specific developments.