HDF Communities

Abstract/Agenda: 

The Hierarchical Data Format is used in many Earth Science communities as well as many other disciplines, and these communities have a lot to learn from each other. This session will expose a few diverse HDF users, and how they are working with the technology. The focus here is to create discussion across disciplines to expose common challenges and solutions. 

Notes: 

1.    Peter Bunting, Aberystwyth University (Wales, UK)
***video missing***
    b.    SPD File Format
    c.    Sorting geographically and Indexing
    d.    Compression
    e.    History of KEA
        i.    Rasters storage in HDF5
        ii.    Size and speed
    f.    Updated version of SPD (v4) is available.

2.    Danny Price, UC Berkley – “Breakthrough Listen”
    a.    Likely to become one of the largest astronomy databases.
        i.    Many PB of data
        ii.    1 day of breakthrough listen = 1 year of previous research
    b.    Data Rates
        i.    20gb/sec
        ii.    reduced to ~660gb/hr
    c.    History of FITS
        i.    Common format for astonomonical data
        ii.    Developed in 1979
    d.    Porting FITS to HDF5

3.    Aashish Choudhary, Kitware – “Scientific Computing and Visualization using HDF”
    b.    HDF at Kitware
        i.    Climate Community
    c.    ACME
    d.    OpenNEX

4.    David Durden, Battelle/NEON – “NEON HDF5”
    a.    NEON is the National Ecological Observatory Network
    b.    Chose HDF5 due to:
        i.    Fast and efficient
        ii.    Compressible
        iii.    Readable to human eye

Citation:
Powers, L.; HDF Communities; 2016 ESIP Summer Meeting. ESIP Commons , March 2016