NASA Space Physics Data Facility (SPDF) Heliophysics Archive

Catalogs directory https://spdf.gsfc.nasa.gov/pub/catalogs/

Contents:

- all.xml : Description in XML form of the datasets in the SPDF archive including those shown in CDAWeb.
The list is incomplete and doesn't include many older datasets and more recent lower-level datasets.
Included in the descriptions are templates for interpreting the dates in the filenames and subdirectories (see below).
This enables automated processing of diretory listings for selected files within a specified time range.


- filelist.gz : complete list of the files in the public archive; created periodically
- filelist.patch.gz : difference of the filelist from the previous one
- filelist.txt : description of the filelist process
- spdf_filelist.sh : sample script to download the filelist and compare to the previously downloaded one for
     finding new files and files removed since the previous check
- spdf-plotwalk-catalog.json : JSON list containing objects with URI templates for pre-generated plots; 
     powers the Plot Walk application (https://spdf.gsfc.nasa.gov/plot_walk)
- spdf-plotwalk-dates.json : JSON object that maps mission name to a list of plot types, time ranges, 
     and start/stop dates for pre-generated plots; this is also used by the Plot Walk application, and 
     is updated periodically


SPDF also has extensive web services for getting data files, plots, listings, orbits:

CDAWeb <https://cdaweb.gsfc.nasa.gov/WebServices/REST/>
SSCweb <https://sscweb.gsfc.nasa.gov/WebServices/REST/> for orbits


Filename template scheme used in all.xml:

Describe the dataset's filenaming scheme using constants and % variables as defined in ISO/IEC 9899:1990 (ISO C89) and ISO 8601:1988. with extensions as used by Unix strftime/strptime functions, and two over-rides: %Q for variable (wildcard) fields and %q for alternative numeric measure of time. Example filenaming: ac_h0_mfi_%Y%m%d_%Q.cdf
Several formats may be provided if needed, although separating into several dataset descriptions may be more appropriate.
 
# %Q matches variable amount of alphanumeric characters (wildcard)
# %q alternative numeric measure of time, with naming_units of Bartel_Rots, crres_orbs, gme_ints, timed_orbs, etc.
# BORROWED FROM strptime module
# %a Locale's abbreviated weekday name.
# %A Locale's full weekday name.
# %b Locale's abbreviated month name.
# %B Locale's full month name.
# %c Locale's appropriate date and time representation.
# %d Day of the month as a decimal number [01,31].
# %H Hour (24-hour clock) as a decimal number [00,23].
# %I Hour (12-hour clock) as a decimal number [01,12].
# %j Day of the year as a decimal number [001,366].
# %m Month as a decimal number [01,12].
# %M Minute as a decimal number [00,59].
# %p Locale's equivalent of either AM or PM.
# %S Second as a decimal number [00,61]. (1)
# %U Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.
# %w Weekday as a decimal number [0(Sunday),6].
# %W Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0.
# %x Locale's appropriate date representation.
# %X Locale's appropriate time representation.
# %y Year without century as a decimal number [00,99].
# %Y Year with century as a decimal number.
# %Z Time zone name (no characters if no time zone exists).
# %% A literal "%" character.

Very old Python code https://spdf.gsfc.nasa.gov/pub/software/old/ftpfinder/


Robert.M.Candey@nasa.gov
updated: 2024 May 07