EarthCube - Brokering: A Way Forward for Global Multi-disciplinary Data Sharing


Brokering is a means of mediating interactions across heterogeneous multidisciplinary data sources and services. A distinguishing characteristic is that brokering shifts the burdens of enabling interoperability from users and providers to a third party service. Brokering can take many forms and involve many different types of services: translating and interpreting between search protocols, controlled vocabularies and natural languages, business processing models, etc. This session will report on current EarthCube activities related to brokering, explore some of the current brokering approaches and implementations, and look a bit into some of the future types of brokering possibilities that may help in fully achieving global multi-disciplinary data sharing.


EarthCube – brokering – a way forward for global multi-disciplinary data sharing

Steven F. Browdy

google doc session:

  • Earthcube goal – to transform the conduct of research in geosciences by supporting community-based cyberinfrastructure to integrate data and information for knowledge management across the geosciences
    • If you built it – they will not come – so talk to the community
  • Create a road map
    • How will work in the larger governance and other groups
  • Expectation – want to engage the long tail folks
    • This is one of the primary aspects – all groups have this important
    • Long tail folks are the scientists on their own – outside of the larger data centers
    • Do pilot studies and hack-a-thons
      • Hack-a-thons are quick – 1 off experiments
  • History of what done
    • Worked on roadmap – still evolving & align with other roadmaps
    • Had first hack-a-thon – used Euro GEOSS broker
    • Pilots – NSIDC arctic Pilot – ongoing – moving towards operation status
    • Attend June Charrette
    • Cross-fertilize with other earthcube groups
  • Roadmap
    • Community
    • Information sciences and
    • Theorectical input
  • Meta-summary
    • Infrastructure is a complex, dynamic process
    • Benefits – lower barriers, enhances multi-disciplinary  (requirement of earthcube)
  • Community engagement
    • Use use cases to target science communities – need to do this in a flexible and evolutionary way
    • Look at small community problems
    • After cycling – look at larger problems – inter-community not just intra-community
  • Hacka-a-Thon1
    • EuroGEOSS – info on ning site
    • Broker needs to be configured to access metadata and data
    • Provider of data had to provide the metadata and the data
    • Users did not have to register anything with the broker – just use tools
    • Talked about successes and failure during webex
    • Brokering is part of a robust, adaptive cyberinfrastructure
    • Next
      • Deal with governance
      • Continue and develop pilots and community engagement
    • People wanted broker to stay open – it has
    • Set up a wiki for discussion
    • Too narrowly focused – will include old hack-a-thon people
    • Extend length of hack-a-thon – but have meeting in through
  • Pilot
    • NSIDC Arctic Polar Pilot
  • There are lots of different ideas about what brokering is
    • Worked on common use cases
  • Possible broker approaches
    • Single broker
    • Single broker in cloud
    • Multiple instances of same broker in cloud
    • Deploy in community – no communication between communities
  • EuroGEOSS Broker
    • Deployed in many situations – GEOSS common infrastructure, WIS (WMO information system) access, and light semantic mediation
    • New accessor – interoperate with data providers
    • Uses ISO 19115 data mode
  • Broker use cases
    • Tradition query, semantics-embed query, access on a common grid, third-party clients
  • Brokering can include – publishing, discovery broker, MNL component, semantic broker
  • Q – if you want to involve more sophisticated sementatic inside or add to broker
    • It would depend on the broker – can build it in, but from architecture – not sure why want to hide behind the broker
  • Principals – whereas:
    • Diverse, use data differentially, different tools, access differently, providers are asked to provide data outside normal communities
    • Brokers should facilitate access to both big end and long tail
  • We believe:
    • No one technology/standard will serve all user needs
    • Brokers can be effect at achieving flexibility
    • Brokering/framework – provides cyberinfrastructure
    • Maintained by brokering not by data users or provides
  • Definitions of brokering framework
    • Brokers are middleware connect client and server in earthcube infrastructure
    • Facilitate run-time interconnection
    • Different components and functionality (ex. Data access, processing, publishing)
  • Definition 2
    • Enables at runtime service components
  • Q – how loosely or tightly coupled the components will they be
    • This is not defining that – semantic capability – it does not have to be there
    • Semantics should be outside the broker
    • In earthcube – is there a broker or a type of broker
    • In the future –have interoperable brokers that are used by earthcube
  • It would be useful to access broker from different systems – what is the level of decoupling for different types of brokers – in semantic it would make sense to couple
    • Want decoupled – don’t know design yet
  • Current USGS broker – it has light components – both internal and external components
  • When you talk about interoperability – genetic – talk about data – but semantic is goal – is it important to define level of interoperability
    • There are lots of other types of interoperability, semantic interoperability is left to semantic group
  • Q – everyone seems to be waiting on some other group
    • Needs to be community driven – so need to engage community to make progress
    • Governance team – will have something in place this year… will take time
  • Q – is reference architecture – same as other groups showed
  • Reference architecture
    • Domain infrasture – has service bus – includes metadata, polices, models
      • Within community – serve long tail
    • Cube – consists of multiple components
      • Broker, governance and stands, c
    • Q (Kelly) Should cross-domain registries be diff color – not as certain as other sections of infrastructure
    • Collaborative environments – social network
    • Brown cloud behind grey cloud
      • Brown cloud – high performance platforms and additional protocols
        • Will access/consume earthcube infrastructure (but issue of how infrastructure will work with high performance)
      • Grey cloud is web
  • Governance questions
    • How should the brokers be deployed and maintained
    • Policies for brokering across boundaries
    • Published data, information, and knowledge standards – is there a convergent set of standards
    • What is process for evolving and innovating CI
  • Use cases
    • Need to be interdisciplinary
    • Wants to use full range
    • Q – do we have common use cases for earthcubeu
      • Some groups have and some haven’t
      • Working on compiling the use cases
      • Need to be a primary activity to inform
  • Future brokering possibilities
    • 2 main area – add functionality and different architecture/framework of how to deploy
    • Broker interoperability
      • May need new standards, or publishing capabilities, more governance issues
  • Q – have you thought about existing standards for brokers ex. Wps and there are existing mappings on how to chain brokers
    • Existing standards/processes need to be use – but haven’t done this yet
  • Q do you have a registry of brokers
    • It is not a large list – has been around for a long time – there are other areas other than science that have used them for a while – but not long list
  • Q – can you list 3 brokers
  • Q – is ECHO a broker
    • No
  • Future – second and 3rd hack-a-thon mid-Aug and mid-Sept, more pilots
  • ACTION – if have broker email [email protected]
  • There is also a list serve
Office presentation icon ESIP Brokering Session.ppt5.53 MB
Duerr, R.; Browdy, S.; EarthCube - Brokering: A Way Forward for Global Multi-disciplinary Data Sharing; Summer Meeting 2012. ESIP Commons , June 2012


sjskhalsa's picture

I've created a google doc for real-time comments and questions: