Reference Frameworks for Assessing Maturity of Earth Science Data Products: Part 1
When managing and providing effective stewardship to digital Earth Science data products in the era of Big Data and Open Data, reference frameworks for assessing dataset quality are key components in helping with the major challenges of consistently describing and systematically integrating quality information to diverse audiences of data users. The natural evolution of user base and data tools over time require continuous monitoring and flexible--but consistent--methods of rating and signaling data quality and maturity.
This workshop consists of two sessions.
The first session will bring together a panel of experts in the fields of exploring quantifiable measurements of product, stewardship, and service maturity levels for digital data in order to help data managers, data producers, or institutions to identify the strength and gaps in their procedures or practices applied to individual data products.The panel will examine the current states of the cutting-edge research, followed by Q&A and panel discussion.
The second session (http://commons.esipfed.org/node/9035) will provide updates on use case studies of various maturity assessment models, followed by training/working time for attendees to get familiar with and apply the assessment models, including the data stewardship maturity model, to their datasets.
Inivited Panelists & Presentation Focus Areas
John Bates, NOAA/NCEI - Reference Framework for Product Maturity Assessment
Anna Privette, Climate Data Solution & NASA - Relevance of maturity identification to enhance the usability of the federal data
Michael Brewer and Derek Arndt, NOAA/NCEI - Assessing Service Maturity through End User Engagement and Climate Monitoring
Grace Peng, NCAR - Conveying data quality and suitability to diverse audiences of data users
- The presentations from the panel are attached.
- Peng provided an overview of the session and the background of the Dataset Maturity Assessment, including:
- Optimal goal is to have consistent measuring maturity of individual Earth Science Data Products.
- Presentation #1: "Status and Future of Data Maturity Models" by John Bates of John Bates Consulting
- There are many related efforts including AGU's maturity model.
- Maturity Models - Status and Next Steps:
- In addition to AGU's program using Data Management Maturity model by CMMI institute, publications and federal government also have related requirements that will help promote and facilitate data management maturity progress.
- Presentation #2: "Stewardship Maturity in the Climate Data Initiative" by Ana Privette of Climate Data Solutions/NASA
- Key catalyst for data.gov: Executive Order - Making Open and Machine Readable the New Default for Government Information.
- The main, targeted audience for the Climate Data Initiative (CDI) and data.gov was innovative data stakeholders.
- There are still areas that can be improved further and ultimately, the following are the areas of stewardship maturity that CDI would like to review and achieve: Preservability, accessibility, usability, production sustainability, data quality assurance, data quality control/monitoring, data quality assessment, transparency/traceability, and data integrity.
- Presentation #3: "Assessing Service Maturity through End User Engagement and Climate Monitoring" by Deke Arndt & Mike Brewer NOAA’s National Centers for Environmental Information
- Knowing what the people need and what their context is for the data usage are crucial for providing data services.
- Several customer requirements and use cases were presented.
- This includes data services provided in order to support climate monitoring related effort.
- 5 levels of customer service, customer engagement, impacts, and monitoring are also discussed.
- Presentation #4: "Conveying Data Quality and Suitability to Diverse Audiences of Users" by Grace Peng of Research Data Archive at the National Center for Atmospheric Research
- The user base for the Research Data Archive (RDA) is heterogeneous from internal staff to commercial sectors and k-12 educators.
- RDA attempts meet the diverse needs from their potential users.
- RDA has added additional communication methods to meet the users' needs including videos, blogs, and RSS.
- Question: Please could the panel elaborate more on discoverability?
- Answer: Finding data depends on a lot of additional components that enabled the data's discoverability to be possible, including metadata, infrastructure, and advertisement/presentation to the users.
- Comment: Analytic potential - how many users could potentially be using the datasets; the result of this study might not entirely solve the maturity assessment issue, but it could help inform the assessment.
- Question: Is the service maturity matrix currently being used to assess anything in the NOAA data catalog and is there any plan to incorporate the Product Maturity Matrix and the Service Maturity Matrix into the NOAA OneStop project?
- Answer: The maturity matrix is still being developed, but the development team for the matrix is confident that the matrix will become part of the formal process when it is in the format/structure that is appropriate. Additionally, not all data will have the same use context; as a result, it might be necessary for all datasets to achieve the highest maturity level.
- Comment: There are additional efforts that might also help complement the data maturity, such as adding semantic relationships/annotations to improve search relevancy. These efforts might be combined to achieve greater impact for data services.
- Question: Will the Service Maturity Matrix be applied to all individual datasets?
- Answer: Yes; the intent is to apply the service maturity to each individual datasets, so that all datasets would have the same maturity assessments.
- Connecting users to the products that a data center/archive/repository offers is the definition of "service" that the panel is using in this discussion.
- Question: Is the panel working with each other to develop a more general maturity matrix model that is applicable to the majority part of the community?
- Answer: Yes; although the panel cannot work with all the possible parties, the pane endeavors to involve as many representatives of the community as possible in the development and validation process of the maturity matrix.
- The panel recognizes the importance of community collaboration and its influence in enabling outcomes that are well agreed and easier to adopt by the community as a whole.
| Attachment | Size |
|---|---|
 ESIP_Ref_Framework_Panel_Intro_Final.pdf | 2.95 MB |
| Bates_DMM overview ESIP 7-16.pdf | 2.76 MB |
| Privette_ESIP_summer2016.pdf | 892.81 KB |
| Arndt_Brewer_ESIP_Service_Maturity.pdf | 5 MB |
| GracePengDiverseAudiences.pdf | 8.81 MB |