Examples of Input Data Files Readable by makeCDF

The following examples of input data files represent some of the formats that are readable by makeCDF. Each input data file example lists the important characteristics of the file. These characteristics are used to create a translation file. A translation file is an ASCII file that describes the input data file to makeCDF. A link to an example of a translation file is included with each input data file example.

Table of Contents for Input Data File Examples

• Simple ASCII data file

  • Simplest flat file

  • with unwanted header records

  • with 1-dimensional variables

  • with 2-dimensional variables

  • with records spanning more than one line (physical record)

• Complex ASCII data file

  • with header variables

  • with embedded variables (embedded in a cycle of repeating sub-records)

Return to makeCDF on-line help

Simple ASCII Data File

What can we say about this data file fragment?

94 101  3.25 0.440  215.6  6060.8
94 101  5.75 0.659  195.3  5026.2
94 101  8.15 0.604  185.1  5177.3
94 101 11.20 0.576  167.6  3847.0
94 101 14.65 0.378  151.9  988.2

• The data file is a text file in ASCII format.
• Each horizontal line in the file contains a "pattern" of 6 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• A time algorithm must be identified for converting to ISTP standard Epoch time

• Each number identified as a variable for the CDF data file must be assigned a name (same as in the CDF data file).
  • For this example, each number is a variable.
  • We assign variable names Year, DOY, seconds, Density, Speed, and Temp.
• For this example, each variable is a scalar, i.e., one component (size=1).
• The format of the variables is, respectively, 2 integers and 6 reals.
• The last variable, Temp, is followed by a line-feed.
• The record begins with Year and ends with Temp.

Look at translation file for this example

Return to table of contents

Unwanted Header Records

What can we say about this data file fragment?

---------------------------------
Yr Day  Secs Density  V     Temp
---------------------------------
94 101  3.25 0.440  215.6  6060.8
94 101  5.75 0.659  195.3  5026.2
94 101  8.15 0.604  185.1  5177.3
94 101 11.20 0.576  167.6  3847.0
94 101 14.65 0.378  151.9  988.2

• The data is a text file in ASCII format.
• The first 3 lines contain header information not needed explicitly in the CDF.
  • No problem in this particular case: any records in the input data file that do not match the "pattern" provided in the translation file are skipped by makeCDF. These headers don't contain numbers, so won't match the desired "pattern".
  • There is a danger of confusion between a header record and the first data variable if they are of the same type, such as both real numbers, or both character strings. For such cases, see the more complicated header record example below.
• The other horizontal lines in the file contain a "pattern" of 6 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• A time algorithm must be identified for converting to ISTP standard Epoch time

• Each number identified as a variable for the CDF data file must be assigned a name (same as in the CDF data file).
  • For this example, each number is a variable.
  • We assign variable names Year, DOY, seconds, Density, Speed, and Temp.
• For this example, each variable is a scalar, i.e., one component (size=1).
• The format of the variables is, respectively, 2 integers and 6 reals.
• The last variable, Temp, is followed by a line-feed.
• The record begins with Year and ends with Temp.

Look at translation file for this example

Return to table of contents

More-Complicated Case of Unwanted Header Records

What can we say about this data file fragment?

--------------------------------------------
DD Mon Year hh:mm:ss   Density  V     Temp
--------------------------------------------
11-Apr-1994 00:00:03.25 0.440  215.6  6060.8
11-Apr-1994 00:00:05.75 0.659  195.3  5026.2
11-Apr-1994 00:00:08.15 0.604  185.1  5177.3
11-Apr-1994 00:00:11.20 0.576  167.6  3847.0
11-Apr-1994 00:00:14.65 0.378  151.9  988.2

• The data is a text file in ASCII format.
• The first 3 lines contain header information.
• The other horizontal lines in the file contain a "pattern" of 4 items: a formatted character string that represents the time, followed by three numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.
• In this case, there is danger of confusion between a header record (text) and the first data variable (character string). If the first variable is not a character variable, see the simple header record example above.

• Time is displayed as a formatted character string (dd-mon-year hh:mm:ss).
• A time algorithm must be identified for converting to ISTP standard Epoch time

• Each item identified as a variable for the CDF data file must be assigned a name (same as in the CDF data file).
  • Time items must also be named if they are to be used in the computation of the EPOCH variable in the CDF, even if they don't actually appear in the CDF.
  • A variable must be identified in each unwanted header record.
• We assign variable names Dum1, Dum2, Dum3, Timestring, Density, Speed, and Temp.
• For this example, each non-time data item of interest is a scalar, i.e., one component (size=1).
• The format of the variables is, respectively, 3 string variables, 1 character variable, and 3 reals.
• The first three variables, Dum1, Dum2, and Dum3, each are a string that is part of an unwanted header record.
• The last variable, Temp, is followed by a line-feed.
• After the header records, each data record begins with Timestring and ends with Temp.

Look at translation file for this example

Return to table of contents

1-dimensional variables

What can we say about this data file fragment?

----------------------------------------------
Yr Day  Secs Density   Vx    Vy    Vz    Temp
----------------------------------------------
94 101  3.25 0.440  -215.6  34.5   7.1  6060.8
94 101  5.75 0.659  -195.3  56.4   4.8  5026.2
94 101  8.15 0.604  -185.1  18.9   6.8  5177.3
94 101 11.20 0.576  -167.6  63.5  23.5  3847.0
94 101 14.65 0.378  -151.9  47.1  34.1  988.2

• The data is a text file in ASCII format.
• The first 3 lines contain header information.
• The other horizontal lines in the file contain a "pattern" of 8 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• There is a logical relationship between the 3 time variables.
• A time algorithm must be identified for converting to ISTP standard Epoch time

• For this example identify:
  • 3 time variables
  • 1 density variable
  • 1 velocity variable with 3 components
  • 1 temperature variable
• All variables are scalars (one component) except velocity which has 3 components.
• The format of the variables is, respectively, 2 integers and 6 reals.
• The last variable, Temp, is followed by a line-feed.
• The record begins with Year and ends with Temp.

Look at translation file for this example

Return to table of contents

2-dimensional variables

What can we say about this data file fragment?

------------------------------------------------------------------
Yr Day  Secs  F11  F12  F13  F21  F22  F23  F31  F32  F33  Temp
------------------------------------------------------------------
94 101  3.25   1.2  4.5 11.3 17.1 31.2 48.4  7.1 10.9 17.4  6060.8
94 101  5.75   0.7  5.2 12.9 16.3 33.4 47.9  6.3 11.7 16.9  5026.2
94 101  8.15   1.1  3.8 14.2 16.7 32.7 49.5  6.1 13.2 19.7  5177.3
94 101 11.20   1.3  6.3 10.8 14.9 34.6 46.8  5.7 12.6 18.5  3847.0
94 101 14.65   0.9  5.5 9.3 15.5 30.8 47.4  7.0 11.9 19.4  988.2

• The data is a text file in ASCII format.
• The first 3 lines contain header information.
• The other horizontal lines in the file contain a "pattern" of 13 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• There is a logical relationship between the 3 time variables.
• A time algorithm must be identified for converting to ISTP standard Epoch time

• For this example identify:
  • 3 time variables
  • 1 flux variable with 9 components
  • 1 temperature variable
• The time and temperature variables are scalars, while the flux variable is a 3x3 array whose elements are given in row-major order.
• The format of the variables is, respectively, 2 integers and 11 reals.
• The last variable, Temp, is followed by a line-feed.
• The record begins with Year and ends with Temp.

Look at translation file for this example

Return to table of contents

Records Spanning More Than One Line

What can we say about this data file fragment?

--------------------------------
Yr Day  Secs Density <lf>
Vx    Vy    Vz    Temp <lf>
--------------------------------
94 101  3.25 0.440 <lf> 
-215.6  34.5   7.1  6060.8 <lf>
94 101  5.75 0.659 <lf>
-195.3  56.4   4.8  5026.2 <lf>
94 101  8.15 0.604 <lf>
-185.1  18.9   6.8  5177.3 <lf>

• The data is a text file in ASCII format.
• The first 4 lines contain header information.
• Two horizontal lines in the file contain a "pattern" of 8 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• There is a logical relationship between the 3 time variables.
• A time algorithm must be identified for converting to ISTP standard Epoch time

• For this example identify:
  • 3 time variables
  • 1 density variable
  • 1 velocity variable with 3 components
  • 1 temperature variable
• All variables are scalars, i.e., one component, except velocity which has 3 components.
• The format of the variables is, respectively, 2 integers and 6 reals.
• The variable Density, is followed by a line-feed.
• The last variable, Temp, is followed by a line-feed.
• The record begins with Year and ends with Temp.

Look at translation file for this example

Return to table of contents

Header Variables

What can we say about this data file fragment?

Year= 94 Day= 101
3.25  0.440  -215.6  34.5   7.1  6060.8
5.75  0.659  -195.3  56.4   4.8  5026.2
8.15  0.604  -185.1  18.9   6.8  5177.3
11.20 0.576  -167.6  63.5  23.5  3847.0
14.65 0.378  -151.9  47.1  34.1  988.2

• The data is a text file in ASCII format.
• The first line contains 4 header variables.
• This defines a header record, (not repeated).
• The other horizontal lines in the file contain a "pattern" of 8 numbers.
• This "pattern" defines a record.
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and decimal Seconds (two integers and a real).
• There is a logical relationship between the 3 time variables.
• A time algorithm must be identified for converting to ISTP standard Epoch time

• For this example, identify 4 scalar "header variables".
• The format of these are string, integer, string, integer.
• Also identify:
  • 3 time variables
  • 1 density variable
  • 1 velocity variable with 3 components
  • 1 temperature variable
• All variables are scalars, i.e., one component, except velocity which has 3 components.
• The format of the variables is, respectively, 2 integers and 6 reals.
• Day of Year is followed by a line-feed.
• The last variable, Temp, is followed by a line-feed.
• The header record begins with Year and ends with Day of Year.
• The data record begins with Seconds and ends with Temp.

Look at translation file for this example

Return to table of contents

Embedded Variables (embedded in a cycle of repeating sub-records)

What can we say about this data file fragment?

94 101 1 -215.6  34.5   7.1 6060.8  ...  -195.3   56.4   4.8  5026.2
94 102 2 -195.3  56.4   4.8 5100.1  ...    -15.3   96.4  12.3  3456.7
94 103 3 -185.1  18.9   6.8 5177.3  ...  -300.1 100.1   4.8  1234.5

• The data is a text file in ASCII format.
• Each horizontal line in the file contains 3 time variables followed by 60 instances of a "pattern" of 4 numbers.
• This "pattern" of 3 + 60*4 defines the overall record.
• The 3 time variables constitute a sub-record.
• The repeating "pattern" of 4 numbers constitutes an embedded (sub)record.
• There is an implied time offset of 1 minute (60000 msec) between the instances of the "pattern" of 4 numbers. The time of the first pattern is the time given by the time variables specified in the algorithm operands; the offset times are applied to the subsequent sub-records. (The first set of 4 numbers occurs at 1 hour 0 minute, the next at 1 hour 1 minute, the next at 1 hour 2 minutes, etc.).
• The length of each number (bytes or #characters) varies from number to number and from record to record.
• Each of the numbers is separated by one or more blanks.

• Time is displayed as Year, Day of Year, and hour, with an implied minute offset between the "pattern" of 4 numbers (three integers).
• There is a logical relationship between the 3 time variables.
• A time algorithm must be identified for converting to ISTP standard Epoch time

• For this example identify:
  • 3 time variables as subrecord 1
  • subrecord 2 (use 2 in Translation Table for easier reading), repeated for a total of 60 times
    • 1 velocity variable with 3 components
    • 1 temperature variable
• All variables are scalars, i.e., one component, except velocity which has 3 components.
• The format of the variables is, respectively, 3 integers and 4 reals.
• The last variable, Temp, is followed by a line-feed.
• The first subrecord begins with Year and ends with hour.
• The second subrecord begins with velocity and ends with Temp.

Look at translation file for this example

Return to table of contents

---

Author and Curator

H. Kent Hills, howard.k.hills@nasa.gov, (301)286-9431
ADNET / NSSDC, Code 690.1, NASA/Goddard Space Flight Center
Greenbelt, MD 20771, USA


Last Updated: December 2008, TJK

---

NASA Official: R.E. McGuire, Head, Space Physics Data Facility (Code 670, NASA/GSFC)
Robert.E.McGuire@gsfc.nasa.gov, (301)286-7794.


---