Architectures Toward Reusable Science Data Systems


Science Data Systems (SDS) comprise an important class of data processing systems that support product generation from remote sensors and in-situ observations. These systems enable research into new science data products, replication of experiments and verification of results. NASA has been building systems for satellite data processing since the first Earth observing satellites launched and is continuing development of systems to support NASA science research and NOAA’s Earth observing satellite operations. The basic data processing workflows and scenarios continue to be valid for simple remote sensor observation research projects as well as for the complex multi-instrument operational satellite data systems being built today.  System functions such as ingest, product generation and distribution need to be configured and performed in a consistent and repeatable way with an emphasis on scalability. This paper will examine the key architectural frameworks of two satellite data processing systems currently in operation and under development that make them suitable for scaling and reuse. By highlighting key elements and implementation experience we expect to find architectural components that will outlast their original application and be readily adaptable and reusable. Concepts and principles are explored for guiding SDS developers and strategists.