A Framework to Evaluate the Return on Investment (ROI) of a Data Repository

To the staff:  I am unable to put this session in the correct collaboration area, because ours, Sustainable Data Management, does not appear in the drop down list above.    I note that it also does not appear in the list of collaboration areas on http://commons.esipfed.org/.
 

• Bob provided a brief introduction regarding the Sustainable Data Management cluster (please see attached slides for further details).
• Using JISC's "The Value and Impact of Data Sharing and Curation" (attached as a PDF file), Anne provided an overview of the information that could be leveraged from this report to evaluate the ROI of data archives/repositories.
  • Key portions highlighted from the report include the following:
    • Figure 1: Methods for exploring the economic value and impacts of research data centres on Page 9 - provides 5 different areas of the method.
    • Table 2: Data and approaches used in the three studies on Page 11 - examples of how ROI is evaluated.
      • Figure 3: The value and impacts of the three UK data centres - summarizes the study using the areas from Figure 1.
        • This comparison indicates that while general guidelines could be used, each evaluation case needs to develop its own specific details.  However, overall, all studies benefitted from the ROI evaluations.
    • Qualitative benefits of the ROI evaluation are demonstrated through Figure 2: The KRDS Benefits Framework on page 10.
    • Additional recommendations are also provided; review of the attached PDF files is recommended.
• Bob also presented the following - "Review of the Beagrie and Houghton Report: The Value and Impact of the European Bioinformatics EBI Institute" (presentation file is attached).
  • Access value is defined as the value perceived by those who accessed the data.
  • The presentation showed the following from Beagrie, N. & Houghton, J. 2016. The Value and Impact of the European Bioinformatics Institute. Charles Beagrie Ltd: Salisbury: 
    • Methodology used for the report (including analysis of log data, user surveys, and literature reviews).
      • There were several caveats related to the methodology, such as annual expense based on 3-year average minus research costs, limited statistics for some services, and unique accesses based on monthly reports across services.
        • These caveats suggested that additional attention/improvement could be added if the Sustainable Data Management cluster were to perform its own study.
      • There were also specific key characteristics associated with the methods used for deriving the usage-related value, contingent value, and research production value.
        • Again, similar to the caveats, the information relating to the methods for the above values indicated that these are areas for further review when devising one's own ROI studies.
    • Key quantitative and qualitative findings from the report were also shown.
    • In general, when designing ROI studies, it is important to consider and plan for potential pitfalls and limitations of the methods used for the studies.
• Next, Ruth presented "ROI: Why a Framework is needed?" (Ruth's presentation is attached as a PDF file).
  • The following terms are defined in Ruth's presentation for the purpose of this session's discussions:
    • Storage, archiving, preservation, and curation.
  • The value of a repository and the value of a data/dataset both influence the discussion of ROI of a data repository.
    • Maturity assessment and analytic potential are two complementary frameworks that could help in informing the values of repositories and data/dataset.
    • Metadata could also play a role in differentiating repositories. Likewise, the activities relating preservation and curation could also differentiate repositories.
    • Ultimately, who should be paying in order to ensure high quality for these characteristics could be achieved?
      • This directly impacts ROI, so it is important to take all these differentiators into account for ROI framework.
• As Sophie was moderating for the last portion of the meeting and could not take notes simultaneously, please refer to the recording for further details regarding the final portion of the session.
  • For the recording, please go to: http://tinyurl.com/zezvcvn
  • The sign in is: [email protected], and the password is: Earth111
    Select: “July 21 PM Junior Ballroom A” (this session will start at the second half of the recording).