Software Metrics

How do we understand how our software is being used and what our users need?  Could the software itself tell us?

This breakout will explore the benefits and issues surrounding the idea that science software used for Earth science research can better support the research community’s needs through automatic and unobtrusive collection of trace data recorded as software executes. Data collected in this way can complement traditional user requirements and lead user interactions, improving coverage and reducing the time taken.

We’ll motivate this idea, including the need for a much better understanding of how (and which) research software is actually used, with which other software it is used, and the need for reproducible science methods.

There are a host of issues raised in this model, including privacy and IP issues. The purpose of this breakout is to explore the issues, the possible models for implementation, and the potential benefits of adding automated configuration reporting to earth science research software.

James  Howison will  draw on his empirical research on the organization of scientific software development and science policy to provide high level motivation for such a system.

James Frew will discuss the relationship between this service and broader goals of scientific information provenance.

Chris Mattmann will identify how software trace information may be collected in the vein of software packaging information collected by pen source packages including Maven, PyPI, and by CPAN, and other related issues.

Sky Bristol will provide some perspectives on scientific software sage reporting used in the context of disaster response, drawing on experiences from Deepwater Horizon to paint a “what if we’d done it differently” picture.

Bruce Caron will help lead the discussion.

All ESIP members are encouraged to bring their ideas to this forum.