GALEX Data Available on AWS

tl;dr Data releases 6 and 7 of the GALEX mission are now available to download from STScI’s public s3 bucket hosted by Amazon Web Services (AWS). In this post we provide an example on how to download data from GR6 and GR7 anonymously.

The Galaxy Evolution Explorer (GALEX) is a NASA mission that observed between July 7, 2003 and February 7, 2012 with the goal of investigating star formation evolution in galaxies from the early Universe to present day. With a 1.25 degree field-of-view, the telescope took direct images in the near-UV (NUV) and far-UV (FUV) wavelengths using microchannel plate detectors, and low resolution spectroscopy was obtained using a grism. Data was collected as time-tagged photon events and science data was only taken while behind Earth’s shadow (referred to as an “eclipse”).

These data are now available in the public S3 bucket on AWS and can be accessed anonymously through astroquery.mast.Observations. The S3 path for GALEX data is s3://stpubdata/galex and the data volume hosted there is approximately 21 TB of GR 6 and 7 data cumulatively.

Which data are available? How often will they be updated?

The data currently available are Galaxy Release (GR) 6 and 7. Users can verify whether or not their products are in the cloud by making a call to astroquery.mast.Observations.get_cloud_uri (for a single product) or astroquery.mast.Observations.get_cloud_uris (for multiple products).

Example data access with astroquery observations

We begin by querying for program 31411 using the query_criteria method in astroquery.mast.Observations.

from astroquery.mast import Observations

observations = Observations.query_criteria(obsid="31411")

The output to this query_criteria call is seen below. It is a GALEX program that observed the target M1.

<Table masked=True length=1>
dataproduct_type calib_level obs_collection ... intentType obsid objID
      str5          int64         str5      ...    str7     str5  str5
---------------- ----------- -------------- ... ---------- ----- -----
           image           2          GALEX ...    science 31411 54974

The next goal is to find the products that correspond to this observation which we will do with the get_product_list method in astroquery.mast.Observations. Products will be filtered further using filter_products to return only science products.

>>> products = Observations.get_product_list(observations[0])
>>> filtered = Observations.filter_products(products, productType='SCIENCE')
>>> print(filtered)
<Table masked=True length=6>
obsID  obs_collection dataproduct_type ... parent_obsid dataRights calib_level
 str6       str5            str5       ...     str5        str6       int64
------ -------------- ---------------- ... ------------ ---------- -----------
113214          GALEX            image ...        31411     PUBLIC           2
113215          GALEX            image ...        31411     PUBLIC           2
113221          GALEX            image ...        31415     PUBLIC           2
113222          GALEX            image ...        31415     PUBLIC           2
 31411          GALEX            image ...        31411     PUBLIC           2
 31415          GALEX            image ...        31415     PUBLIC           2

From here, users can download the desired products from the cloud by calling astroquery.mast.Observations.download_products and setting cloud_only to True. Remember that to pull GALEX data from the cloud, anonymous cloud access must be enabled via the Observations.enable_cloud_dataset() call, otherwise the download will default to pulling from the MAST Portal.

>>> # Enabling anonymous cloud access to public s3 bucket
>>> Observations.enable_cloud_dataset()
INFO: Using the S3 STScI public dataset []

>>> # Downloading file
>>> Observations.download_products(filtered, cloud_only=True)
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/00-visits/0001-img/07-try/AIS_309_0001_sg28-nd-int.fits.gz to ./mastDownload/GALEX/6381787619088400384/AIS_309_0001_sg28-nd-int.fits.gz ...
|==========================================|  10M/ 10M (100.00%)         0s
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/00-visits/0002-img/07-try/AIS_309_0002_sg28-nd-int.fits.gz to ./mastDownload/GALEX/6381787619121954816/AIS_309_0002_sg28-nd-int.fits.gz ...
|==========================================| 9.7M/9.7M (100.00%)         1s
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/00-visits/0001-img/07-try/AIS_309_0001_sg36-nd-int.fits.gz to ./mastDownload/GALEX/6381787627678334976/AIS_309_0001_sg36-nd-int.fits.gz ...
|==========================================|  10M/ 10M (100.00%)         0s
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/00-visits/0002-img/07-try/AIS_309_0002_sg36-nd-int.fits.gz to ./mastDownload/GALEX/6381787627711889408/AIS_309_0002_sg36-nd-int.fits.gz ...
|==========================================|  10M/ 10M (100.00%)         0s
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/01-main/0001-img/07-try/AIS_309_sg28-nd-int.fits.gz to ./mastDownload/GALEX/6381787756527353856/AIS_309_sg28-nd-int.fits.gz ...
|==========================================|  13M/ 13M (100.00%)         0s
Downloading URL s3://stpubdata/galex/GR6/pipe/02-vsn/50309-AIS_309/d/01-main/0001-img/07-try/AIS_309_sg36-nd-int.fits.gz to ./mastDownload/GALEX/6381787765117288448/AIS_309_sg36-nd-int.fits.gz ...
|==========================================|  13M/ 13M (100.00%)         0s
<Table length=6>
                                Local Path                                ...
                                  str73                                   ...
------------------------------------------------------------------------- ...
./mastDownload/GALEX/6381787619088400384/AIS_309_0001_sg28-nd-int.fits.gz ...
./mastDownload/GALEX/6381787619121954816/AIS_309_0002_sg28-nd-int.fits.gz ...
./mastDownload/GALEX/6381787627678334976/AIS_309_0001_sg36-nd-int.fits.gz ...
./mastDownload/GALEX/6381787627711889408/AIS_309_0002_sg36-nd-int.fits.gz ...
     ./mastDownload/GALEX/6381787756527353856/AIS_309_sg28-nd-int.fits.gz ...
     ./mastDownload/GALEX/6381787765117288448/AIS_309_sg36-nd-int.fits.gz ...

The data will be downloaded compressed via GNU zip, but content can be handled with the same way a non-compressed FITS file would be handled. See here for more information on handling compressed files with

from import fits

file = "./mastDownload/GALEX/6381787619088400384/AIS_309_0001_sg28-nd-int.fits.gz"

with as hdulist:
     data = hdulist[1].data

Brought to you by Jenny V. Medina