15.36s IMP 8 Magnetic Field data - Version 3 1. Introduction 2. Input data sets 3. Time tagging of V3 data records,time difference between new and old records 4. Cleaning of V2 data used in V3 data set 5. IMP 8 position vector 6. Solar wind flag 7. Paths to the data 8. Format of V3 data records 1. Introduction This data set contains 15.36s averaged values of magnetic field data obtained by the magnetometer flown on the IMP 8 spacecraft. The data span October 30, 1973, through June 10, 2000. Data records include magnetic field Cartesian components in GSE and GSM coordinates, magnetic field magnitude, variances in the component averages (xx, xy, xz, yy, yz, zz), GSE and GSM Cartesian components of the IMP 8 position vector, and three additional flags discussed below. 2. Input data sets This data set was created in 2009 from two input data sources. The first was a newly cleaned data set of 320ms resolution IMP 8 magnetic field data, and the second was the long-accessible Version 2 15.36s IMP 8 magnetic field data set. The 320ms data set is accessible from ftp://omniweb.gsfc.nasa.gov/imp8/mag/320ms_ascii/cleaned/, from http://ftpbrowser.gsfc.nasa.gov/imp_mag_320ms_cln.html, and from http://cdaweb.gsfc.nasa.gov/, and is extensively documented at ftp://omniweb.gsfc.nasa.gov/imp8/mag/320ms_ascii/cleaned/ doc/imp8_mag_320ms_proc.txt. 15.36s averages created from this 320ms data set constitute the great majority (~94%) of data in the Version 3 of the 15.36s data set. However, as noted in the 320ms data documentation file, there are intervals when the old V2 15.36s data are preferable to the new data and other intervals having old data but not new data. For such intervals that exceed 30 minutes, newly cleaned (see below) old data are used in the V3 data set rather than averages based on the 320ms data. There is a flag in each V3 15.36s data record indicating whether the data of that record come from the 320ms data or from the old V2 15.36s data set. It remains to discuss the time tagging of the data records and the cleaning applied to the old V2 15.36s data prior to their inclusion in the new V3 data set. 3. Time tagging of V3 data records V3 data records are tagged with the start times of 15.36s averaging intervals. For every daily file in the V3 data set, there are 86400/15.36 = 5625 15.36s data records. Some of these have no magnetic field data. The V2 15.36s data set's records were time-tagged with the start time of the last of the 1.28s averages contributing to that record's 15.36s averages. Each such record also contained the number of contributing 1.28s averages, with a maximum value of 12. We assumed that for cases where this number was <12, the contributing 1.28s averages were from contiguous 1.28s intervals. We then computed the inferred start time of the first contributing 1.28s average. Then, for each V3 output record, we used the old V2 record whose inferred start time was closest to the time tag of the V3 record. The difference (Tn - To) between the time tag Tn of the V3 output record and the inferred starting-time (To) of the input old record had to be in the range minus to plus 15.36/2 (=7.68) seconds. The difference value Tn-To is carried in each V3 data record, although it is non-zero only for that minority of records (~6%) holding data from the old V2 15.36s data set. 4. Cleaning of V2 data used in V3 data set Next we address the cleaning of the old 15.36s data set. As discussed in the above-cited documentation of the input 320ms data set, we removed intervals of significant data sparseness. For consistency, we also "desparsed" the old 15.36s data set by requiring (a) that each 15.36s value had at least five 1.28s contributing averages and (b) that each 30-min data interval had at least fifty 15.36s averages that satisfied condition (a). We then sought and eliminated data spikes using two algorithms in sequence. First, for each GSE Cartesian component, in each successive 30-min interval, we computed means and standard deviations. We then took the two points furthest from the means and computed new means and standard deviations excluding the two extreme points. We then tested the two extreme points for possible exclusion by asking whether their differences from the second mean exceeded 6 times the second standard deviation. If either extreme value did, for any Cartesian component, that data point (record) was excluded as a spike. Second, we tested each old 15.36s data point by asking, for each Cartesian component, whether the difference between that value and the mean of the two points before and two points after (i.e., a 4-point mean) exceeded 4 times the standard deviation in the 4-point mean. If it did, for any component, that value (record) was excluded as a spike. One requirement for this test was that the first and last point used in testing a central point had to be within 15 minutes of each other, otherwise this test would not be applied. This test is discussed at http://omniweb.gsfc.nasa.gov/html/HROdocum.html#ap3. 5. IMP 8 position vector Over the years, the IMP 8 position vector was improved. IMP 8 position information included in this V3 15.36s magnetic field data set correspond to the final, best available IMP 8 position data set. The data were obtained from the Space Physics Data Facility's SSCWeb system (http://sscweb.gsfc.nasa.gov/). 6. Solar wind flag We have included in each record a "solar wind flag" based on the IMP 8 bow shock crossing database accessible from http://ftpbrowser.gsfc.nasa.gov/bowshock.html. This is a one-minute resolution flag applied to all 15.36s records whose time tags belong to a given minute. The flag indicates whether IMP 8 was (flag = 0), or was not (flag = 1), in the solar wind for that minute. 7. Paths to the data There are multiple paths to these data: as daily ASCII files: ftp://nssdcftp.gsfc.nasa.gov/spacecraft_data/imp/imp8/mag/ 15s_ascii_v3. with subsetting and graphic display: http://ftpbrowser.gsfc.nasa.gov/imp_mag15s.html As daily CDF files via ftp: ftp://cdaweb.gsfc.nasa.gov/pub/istp/imp8/mag_15sec/ via CDAWeb: http://cdaweb.gsfc.nasa.gov 8. Format of V3 data records The format of the V3 15.36s ASCII data set's records is: ITEM Description (Type)(Size) Units Notes ============================================================================ 1* Year I4 Four-digit year 2* Day-of-year I4 days January 1 = 1 3* Seconds of day F9.2 sec Elapsed seconds of day 4** Data source flag(sf) I2 See below 5*** # of fine scale pts I4 See below 6 X GSE component of F6.2 Earth radii IMP 8 position vector 7 Y GSE component of F6.2 Earth radii IMP 8 position vector 8 Z GSE component of F6.2 Earth radii IMP 8 position vector 9 Y GSM component of F6.2 Earth radii IMP 8 position vector X GSM is the same as X GSE 10 Z GSM component of F6.2 Earth radii IMP 8 position vector 11 Field Magnitude (F1) F8.2 nT 15.36s average of fine scale field magnitudes 12 Bx_GSE F8.2 nT 15.36s averages 13 By_GSE F8.2 nT 14 Bz_GSE F8.2 nT 15 By_GSM F8.2 nT Bx_GSE is the same as Bx_GSM. 16 Bz_GSM F8.2 nT 17 Variance Bxx GSE F8.2 nT**2 18 Variance Byy GSE F8.2 nT**2 19 Variance Bzz GSE F8.2 nT**2 20 Variance Byx GSE F8.2 nT**2 21 Variance Bzx GSE F8.2 nT**2 22 Variance Bzy GSE F8.2 nT**2 23**** Time shift F6.1 seconds See below 24 Solar wind flag I2 =0 if IMP in solar wind =1 otherwise * See above text for information on time tagging ** sf = 1 if based on 320ms data, sf = 2 if based on old V2 15.36s data *** Maximum = 48 for source flag (sf)= 1 records, = 12 for sf = 2 records **** Difference between record time tag and start of average; see Section 3 above.