Community-Wide Workshop on NASA’s Space Physics Data System

---

           Space Physics Division          Office of Space Science         December 1993

Executive Summary

Approximately 75 scientists gathered for a community-wide workshop on the Space Physics Data System (SPDS) in early June at Rice University, Houston. The workshop considered community needs for access to data and made recommendations to guide NASA’s Space Physics Division (SPD) in managing space physics data in the 1990s and beyond. It was organized around four panels (Policy, Data Issues, Data Systems, and Software). Participants were asked to review the SPDS Concept Document (prepared by the SPDS Steering Committee) and to register either approval or an alternative view to any or all the ideas discussed in the document, through one or more of the appropriate panels. In addition, the panels were charged to identify and prioritize key requirements in their area of interest and to comment on relevant issues of implementation.

The resulting recommendations from the four panels are broadly consistent in the overall sense of an SPDS in areas of joint interest. The panel structure worked well in facilitating a combined set of recommendations that constitute a major step forward to define the scope and an approach to SPDS that would most effectively support the NASA space physics research community while stillbeing realistic in an austere budget environment.

The following are the key “policy-level” observations and recommendations of the workshop:

1. It was agreed that the creation of an SPDS would help fulfill the urgent data exchange requirements presented by upcoming new missions, as well as preserve and improve access to critically important data sets from both past and present programs. SPDS should support all four SPD branch user communities (Cosmic and Heliospheric Physics; Solar Physics; Magnetospheric Physics; and Ionospheric, Thermospheric, and Mesospheric Physics) in ways appropriate to each of those communities. The SPD should initiate the SPDS now, in spite of very limited funding. As the scientific productivity of SPDSis demonstrated and recognized by a broader scientific community, NASA will need and ought to provide significant new resources to the Division to supportSPDS.

2. The five functional objectives identified in the SPDS Concept Document areappropriate, namely:

   • to curate data sets important to current or future research;
   • to provide information about space physics data sets to potential users;
   • to provide efficient access to the data and ancillary information;
   • to enable cost-effective and efficient analysis of data; and
   • to act as a focus for the evaluation and introduction of effective new data technologies into space physics.

   The most important SPDS objective now however should be to provide the scientific research community with access to high quality data products. Support for the dissemination of data analysis tools is important but of secondary priority. Awareness of recent development in data management technologies is important for SPDS but is the lowest of the five functional objectives.

3. The workshop endorsed the Concept Document’s emphasis on a distributed data system designed, developed, and operated by scientific users of space physics data. The SPDS should make maximum use of existing facilities including data centers, project data management units, individual centers at Principal Investigator (PI) institutions, and existing computer networks. The current paradigm envisions use of the NASA Master Directory to indicate what data are available and where, the use of existing software and human systems to retrieve data, and the uses of data format translators to import data into an investigator’s own analysis system. Funding should be awarded competitively through peer review. In the initial stage, this could be through an augmentation or supplement to scientific investigations selected under existing peer review programs.

4. Consistent with a philosophy of a grass-roots effort built from existing capabilities and operating as a community-distributed service, SPDS should emphasize the concept of “coordination” and “coordinators” as a management approach to accomplish its larger goals. Initially, there should be Project Coordinator(s) and Discipline Coordinator(s) representing each of the four Space Physics Division branch-discipline user communities. The Discipline Coordinators should be scientists within their community who presently work with, and exchange, data. The Project Coordinator(s) will provide contact between the SPDS and other NASA data systems activities, with other governmentagencies and the international community as well as work with the Discipline Coordinators. Formal SPDS management authority should remain with MO&DA Program of the Division. It was suggested that a follow-on SPDS Users’ Workshop should be held in about a year.

5. Participation in SPDS by all current and future division-related investigations, and their Principal Investigators (PIs) during the active phase of missions, should be encouraged. This will require recognition of rights of PIs to a period of exclusive use of the data from their instruments.

   The workshop panels also made specific recommendations in their specific areas of interest to both requirements and implementation. Highlights and recommendations from the individual reports (grouped by panel) are:

6. In the area of Data Issues:

   • For SPDS to be effective, a listing of all data sets that are available is needed. Catalogues should contain information concerning the instrument, satellite, format of the data, a timeline of instrument and satellite events, and the location of the ephemeris. Directories should serve as pointers to distributed inventories in a standard format, that contain sufficient information to characterize the data holding to allow the average researcher to decide whether they are appropriate and if so, to identify the parts of the data that meet his/her particular requirements.
   • To include a maximum number of participants, the use of the SPDS should require only widely available technology and hardware and should need minimal technical knowledge of computer systems. There should be few “requirements” to which data suppliers or users would need to conform. However, there should be many “suggestions” which can be used to guide those who wish to participate.
   • Some criteria for determining which data sets should be a part of SPDS include: uniqueness, the difficulty with which they were obtained, and their potential for being lost. However, the primary criterion must be heavy use orthe possibility of heavy use. Archiving and documenting data sets must generally assume priority over improving formats or placing data sets on-line. Archiving Level 0 data is highly desirable.
   • The curatorial role of the original institution where the PI is (was) affiliated should be recognized and developed. SPDS should investigate the use of universities and institutions which have mechanisms that are designed to provide long-term curation of scholarly data bases and raw research materials in other disciplines such as libraries and museums.
   • SPDS should include non-NASA data sets such as those funded by NSF, DoE, other federal institutions, and universities. Both US and non-US data suppliers and users should be included. The needs of developing nations should be considered.

7. In the area of Data Systems:

   • The critical system functionalities identified for SPDS are: the delivery of self-documenting data; the existence of a matrix of translators between several standard formats; and a network-based capability for browsing and examining inventory records for the system’s data holdings.
   • Considering the austere funding environment, as well as the diversity of systems in place at the nodes listed in the concept document and potential subnodes, the panel specified as system level standards only TCP/IP network interfaces and the seven file formats identified in the Concept Document as initial system-wide standards. The overriding consideration for any SPDS interface should be ease of use by a technically competent investigator. As an aid to SPDS nodes developing appropriate software interfaces, the space physics community should be polled on such subjects as preferred interface, monitor display capabilities, data analysis software of choice, and availabledata communications. SPD should express its support for NASA’s efforts to leadin the implementation technologies, such as ATM, that would significantly increase the bandwidth of data communications with currently bandwidth-limitedsites.
   • NSSDC besides continuing its traditional roles as a Master Directory server, central data copying facility, and deep archiving facility, should also serve the SPDS user community by providing a bulletin board and/or mailing list for announcements of general interest, and anonymous FTP servicesfor data format translators, models, and other software tools of general interest.

8. In the area of Software:

   • The SPDS should state a policy that new application development should meet some minimum standards. Adherence to this policy should become a contractual obligation. However, it is also believed that NASA’s current software development policy imposes too much overhead on software projects.
   • NASA should provide funding to extend and enhance existing systems to meet its requirements and needs. If no software currently exists that can meet specific SPDS needs, then development of new software should be funded. SPDS should encourage the sharing of existing software and actively work to make it easier to obtain existing software. Freely distributed software was considered a high priority activity, but the choice of Commercial Off-The-Shelf (COTS) software should be left up to the individual. It was also noted that no COTS technology assessment group should be a part of the SPDS.
   • A catalog of existing software should be implemented. Providing direct access to a central repository (library) of software is as not important as acatalog. Software should include data processing, translation, analysis, and visualization routines, and could include ancillary software such as models, simulation results, etc. Acting as a software repository is not considered as high a priority as dissemination of information, comments, and experiences.
   • NASA should assume a capital equipment responsibility and provide funds to insure that the research community has a computing environment that does not limit the general distribution, use, and development of software systems. However, the current hardware and operating system market is not sufficiently stable nor influenced by the SPDS community by itself to the extent that choosing a standard platform would be viable. It is recognized that the current trend is towards UNIX systems that run X-windows on a variety of architectures and Windows-NT running on Intel architectures. The emphasis should be on developing portable code so that it may easily be migrated as new technologies are introduced.

Since the workshop and in direct response to its recommendations, the Space Physics Division has called for volunteers and now named an “SPDS Coordinators”Working Group (SPDS/CWG) that will be composed of:

• an SPDS Project Scientist,
• four SPDS Lead Discipline Coordinators, and
• two SPDS Project (Co-)Coordinator(s).

The Lead Discipline Coordinators will be supported by four Discipline Coordination Teams composed of volunteers from their communities. The Lead Discipline Coordinators will represent the Teams on the SPDS/CWG. The SPDS/CWG has the broad responsibility to move the definition of an SPDS program forward from the Concept Document and the recommendations of this workshop, to start a first working version of SPDS and to serve as focal points for advertising SPDS, compiling and disseminating information about available data and data access, and establishing a beginning set of “rules of the road” for data exchange.

The prototype SPDSdeveloped for the workshop remains operational. The workshop participants in the data systems panel also defined an experiment they would undertake to see whether DITDOS, an inventory tracking system developed by PDS, could be used easily and inexpensively to manage existing space physics inventories. The CEDAR project inventory established at NCAR was selected as a test case. The Institute of Geophysics and Planetary Physics (IGPP) at UCLA has established a list server to serve the SPDS community as a forum for discussing SPDS related issues.