ISTP Metadata Guidelines: How to Structure Data in CDF

We have identified three types of variables to be included in ISTP CDF files: data variables of primary importance (e.g., density, magnetic_field, particle_flux), support_data variables of secondary importance (e.g., time, coordinate_system, energy_bands associated with particle_flux) and
label metadata variables (e.g., a variable holding “Bx,By,Bz” to describe magnetic field). The support_data variables and the label metadata variables are attached to the data variables through the use of variable attributes (see below).

We describe each of these kinds of variables with specific examples taken from Space Physics instrument types. After understanding this structure look at the full list of required descriptions in ISTP Standard Attributes.

Examples of the three types of variables commonly found in ISTP investigations are shown below. Their corresponding dimensions and sizes in CDF are also shown.

Instrument Types

We have identified a number of instrument types for Space Physics. Choose one from the list below to see example variables specific to the instrument type and their corresponding dimensions and sizes in CDF.

Electric Fields (space)

Example data variables from the Geotail Electric Field Detector are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

Because the two variables, Sunward and Duskward Electric Field and Spacecraft Potential from 2 probes, each have two components and are likely to be plotted as separate time-series plots, then they each need two labels. For example:

In order to keep the labels distinct in CDF, they are stored in dimension 1, size 2 metadata variables ("label_E", "label_Vs") attached to the data variables using the attribute LABL_PTR_1 (1 for dimension 1). For example:

In this case, these labels also define the dimension for each of the data variables. This dependency of the data variables on the dimension is made explicit by the use of the attribute DEPEND_1 (1 for dimension 1). A full example of a data variable attached via attributes to its support_data and metadata variables is shown below:

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Magnetic Fields (space)

Example data variables from the Wind Magnetic Fields Investigation are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

Because the four magnetic field vectors each have three components and are likely to be plotted as separate time-series plots, then they each need three labels. Additionally, they each need their own unique label to distinguish one from another. For example:

The labels are stored in dimension 1, size 3 metadata variables ("label1_BGSM", "label2_BGSE") attached to the data variables using the attribute LABL_PTR_1 (1 for dimension 1). For example:

The dimension for each of the data variables is defined by the coordinate system, either Cartesian (x,y,z) or polar (r,theta,phi). The data variables are tied to the coordinate system by the use of the attribute DEPEND_1 (1 for dimension 1). A full example of a data variable attached via attributes to its support_data and metadata variables is shown below:

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Plasma and Solar Wind

Example data variables from the Wind Solar Wind Experiment are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

Because the three velocity vectors each have three components and are likely to be plotted as separate time-series plots, then they each need three labels. Additionally, they each need their own unique label to distinguish one from another. For example:

The labels are stored in dimension 1, size 3 metadata variables ("label_V_GSE", "label_V_polar") attached to the data variables using the attribute LABL_PTR_1 (1 for dimension 1). For example:

The dimension for each of the data variables is defined by the coordinate system, either Cartesian (x,y,z) or polar (r,theta,phi). The data variables are tied to the coordinate system by the use of the attribute DEPEND_1 (1 for dimension 1). A full example of a data variable attached via attributes to its support_data and metadata variables is shown below:

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Particles (space)

Example data variables from the Geotail Energetic Particles Instrument are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

This data set has several 1 dimensional variables of varying sizes from 2 to 12. Each of the 1 dimensional variables has an additional dependency (other than time). Ion Differential Intensity and its uncertainty both depend on energy values (support_data). These variables are attached to the data variables using the attribute DEPEND_1. For example:

The Ion Differential Intensity and its uncertainty would most naturally be displayed as spectrograms. Individual labels on each intensity or energy level are not required.

He/H Ratio and its uncertainty each have two components and are likely to be plotted as separate time-series plots so they each need two labels. Additionally, they each need their own unique label to distinguish one from another. Electron anisotropy parameters and its uncertainty, each have five components and are likely to be plotted as separate time-series plots, then they each need five labels. For example:

In order to keep the labels distinct in CDF, they are stored in metadata variables of dimension 1, size 2 (e.g., "He/H_label") or dimension 1, size 5 (e.g., "labels_anisot") attached to the data variables using the attribute LABL_PTR_1 (1 for dimension 1). For example:

In this case, these labels also define the dimension for each of the data variables. This dependency of the data variables on the dimension is made explicit by the use of the attribute DEPEND_1 (1 for dimension 1). A full example of a data variable attached via attributes to its support_data, and metadata variables is shown below:

For this example each of the main data variables have an associated uncertainty. The uncertainty variables are explicitly tied to their corresponding variable via attributes DELTA_PLUS_VAR and DELTA_MINUS_VAR. The uncertainty is stored as a (+/-) on the value of the original data variable. In this case Ion differential Intensity is at the exact center of the range so that only one variable is defined which is half of the range. If Ion differential Intensity were not at the exact center, two uncertainty variables, plus_Uncertainty and minus_Uncertainty would need to be defined. A full example of a data variable attached via attributes to its support_data, metadata variables, and to its uncertainties is shown below:

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team.
Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Radio and Plasma Waves

Example data variables from the Wind Radio/Plasma Waves Instrument are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

There are two example 1 dimensional variables of size 76. Both of these variables (data) also depend of frequency values (support_data). The frequency variable is attached to both data variables using the attribute DEPEND_1. For example:

The Electric field average intensity and Electric field background would most naturally be displayed as spectrograms. Individual labels on each intensity or energy level are not required.

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Ground-Based HF-Radars

Example data variables from the Darn Goose Bay Radar are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

This variable is 2-dimensional where the first dimension holds the ionospheric convection velocity components (East, North) and the second holds the 30 radar scan indices. These indices are also tied to the geographic latitude and longitude positions so that in effect, the geographic position is tied to the velocity. The velocity labels (support_data) also define the first dimension while the indices (support_data) define the second dimension; they are tied to the velocity (data) via DEPEND attributes as shown below. The velocity labels are also attached via the LABL_PTR_1 attribute (1 for first dimension).

Special plotting software developed at the Johns Hopkins Applied Physics Laboratory and slightly modified for use with the CDAWeb system is used to plot this data. Plots can be viewed through the CDAWeb system using the ISTP Key Parameters publicly available data database.

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Ground-Based Imagers

Example data variables from the Canopus All Sky Camera are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

The image data variable also depends on geodetic latitude and longitude (support_data); these variables are attached to the data variables using the attributes DEPEND_1 and DEPEND_2. For example:

The Max. Intensity of High-latitude ionospheric would most naturally be displayed as an image with a super-imposed latitude and longitude grid already specified above. Individual labels on each pixel are not required.

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Ground-Based Magnetometers, Riometers, Sounders

Example data variables from the Canopus Magnetometer and Riometer Array are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

These data variables are all scalars with no dimension and thus have no structure other than their dependency on time.

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Ground-Based VLF/ELF/ULF, Photometers

Example data variables from the Canopus Meridian Photometer Array are listed below with their corresponding dimensions and sizes in CDF.

These data variables have an associated time tag, that is, they each depend on time. In CDF the ISTP standard time Epoch (support_data) is attached to the data variables using the attribute DEPEND_0. For example:

Epoch (support_data, O dimension in CDF) is listed as, e.g., 01-Jan-1997 10:20:00.000 but stored as a Real*8 scalar variable for ease of manipulation across day, month, and year boundary.

The 42 Values of 5577A Intensities (data) depend on their associated geodetic latitude . The latitude variable (support_data) is attached to the intensity variable via the attribute DEPEND_1:

The 42 Values of 5577A Intensities from Geodetic Lat 46-67, Long=265 would most naturally be displayed as a spectrogram. Individual labels on each intensity or latitude value are not required.

This fully defines the structure associated with these variables. Other descriptions also attached via attributes, are required to fully define the variables such that they are correctly and independently usable by someone not connected with the instrument team. Look at the full list of required descriptions in ISTP Standard Attributes.

*Return to list of Instrument Types

Return to Top

Return to ISTP Metadata Guidelines

CDF home page