3. Reduction using API¶
There may be cases where you would be interested in accessing the DRAGONS’ Application Program Interface (API) directly instead of using the command line wrappers to reduce your data. Here we show you how to do the same reduction we did in the previous chapter but using the API.
3.1. The dataset¶
If you have not already, download and unpack the tutorial’s data package. Refer to Downloading the tutorial datasets for the links and simple instructions.
The dataset specific to this example is described in:
Here is a copy of the table for quick reference.
Science | N20170614S0201-205
|
10 s, i-band
|
Bias | N20170613S0180-184
N20170615S0534-538
|
|
Twilight Flats | N20170702S0178-182
|
40 to 16 s, i-band
|
3.2. Setting Up¶
3.2.1. Importing Libraries¶
We first import the necessary modules and classes:
1 2 3 4 5 | import glob
from gempy.adlibrary import dataselect
from recipe_system import cal_service
from recipe_system.reduction.coreReduce import Reduce
|
Importing print_function
is for compatibility with the Python 2.7 print
statement. If you are working with Python 3, it is not needed, but importing
it will not break anything.
glob
is Python built-in packages. It will be used to return a
list
with the input file names.
dataselect
will be used to create file lists for the
darks, the flats and the science observations. The
cal_service
package is our interface with the local
calibration database. Finally, the
Reduce
class is used to set up
and run the data reduction.
3.2.2. Setting up the logger¶
We recommend using the DRAGONS logger. (See also Double messaging issue.)
8 9 | from gempy.utils import logutils
logutils.config(file_name='gmos_data_reduction.log')
|
3.2.3. Setting up the Calibration Service¶
Before we continue, let’s be sure we have properly setup our calibration database and the calibration association service.
First, check that you have already a rsys.cfg
file inside the
~/.geminidr/
. It should contain:
[calibs]
standalone = True
database_dir = /path_to_my_data/gmosimg_tutorial_api/playground
This tells the system where to put the calibration database. This database will keep track of the processed calibrations as we add them to it.
Note
The tilde (~
) in the path above refers to your home directory.
Also, mind the dot in .geminidr
.
The calibration database is initialized and the calibration service is configured as follow:
10 11 12 13 14 | caldb = cal_service.CalibrationService()
caldb.config()
caldb.init()
cal_service.set_calservice()
|
The calibration service is now ready to use. If you need more details, check the Using the caldb API in the Recipe System User’s Manual .
3.3. Create list of files¶
The next step is to create lists of files that will be used as input to each of the
data reduction steps. Let us start by creating a list
of all the
FITS files in the directory ../playdata/
.
15 16 | all_files = glob.glob('../playdata/*.fits')
all_files.sort()
|
The sort()
method simply re-organize the list with the file names
and is an optional step. Before you carry on, you might want to do
print(all_files)
to check if they were properly read.
Now we can use the all_files
list
as an input to
select_data()
. The
dataselect.select_data()
function signature is:
select_data(inputs, tags=[], xtags=[], expression='True')
3.3.1. List of Biases¶
Let us, now, select the files that will be used to create a master bias:
17 18 19 20 21 | list_of_biases = dataselect.select_data(
all_files,
['BIAS'],
[]
)
|
Note the empty list []
in line 20. This positional argument receives a list
of tags that will be used to exclude any files with the matching tag from our
selection (i.e., equivalent to the --xtags
option).
3.3.2. List of Flats¶
Next we create a list of twilight flats for each filter. The expression specifying the filter name is needed only if you have data from multiple filters. It is not really needed in this case.
22 23 24 25 26 27 | list_of_flats = dataselect.select_data(
all_files,
['FLAT'],
[],
dataselect.expr_parser('filter_name=="i"')
)
|
3.3.3. List of Science Data¶
Finally, the science data can be selected using:
27 28 29 30 31 32 | list_of_science = dataselect.select_data(
all_files,
[],
['CAL'],
dataselect.expr_parser('(observation_class=="science" and filter_name=="i")')
)
|
Here we left the tags
argument as an empty list and passed the tag
'CAL'
as an exclusion tag through the xtags
argument.
We also added a fourth argument which is not necessary for our current dataset
but that can be useful for others. It contains an expression that has to be
parsed by expr_parser()
, and which ensures
that we are getting science frames obtained with the i-band filter.
3.4. Make Master Bias¶
We create the master bias and add it to the calibration manager as follow:
33 34 35 36 37 | reduce_bias = Reduce()
reduce_bias.files.extend(list_of_biases)
reduce_bias.runr()
caldb.add_cal(reduce_bias.output_filenames[0])
|
The Reduce
class is our reduction
“controller”. This is where we collect all the information necessary for
the reduction. In this case, the only information necessary is the list of
input files which we add to the files
attribute. The
runr()
method is where the
recipe search is triggered and where it is executed.
Once runr()
is finished, we add the master bias to the calibration
manager (line 37).
3.5. Make Master Flat¶
We create the master flat field and add it to the calibration database as follow:
38 39 40 41 42 | reduce_flats = Reduce()
reduce_flats.files.extend(list_of_flats)
reduce_flats.runr()
caldb.add_cal(reduce_flats.output_filenames[0])
|
3.6. Make Master Fringe Frame¶
Warning
The dataset used in this tutorial does not require fringe correction so we skip this step. To find out how to produce a master fringe frame, see Create Master Fringe Frame in the Tips and Tricks chapter.
3.7. Reduce Science Images¶
We use similar statements as before to initiate a new reduction to reduce the science data:
43 44 45 | reduce_science = Reduce()
reduce_science.files.extend(list_of_science)
reduce_science.runr()
|
The output stack units are in electrons (header keyword BUNIT=electrons). The output stack is stored in a multi-extension FITS (MEF) file. The science signal is in the “SCI” extension, the variance is in the “VAR” extension, and the data quality plane (mask) is in the “DQ” extension.