Exploring New Opportunities for Data Access at Scale with Drupal 8 and ElasticSearch

Drupal, an open-source web framework for collaboration and content management, has filled an important niche enabling data management teams to organize, curate and archive Earth science metadata then publish it in a variety of forms - for humans (HTML) and machines (RESTful Web Services, RDFa, RDF, JSON-LD). Whilst the current Drupal version (7) has been an effective solution - powering WHOI’s Biological and Chemical Oceanography Data Management Office (BCO-DMO) metadata portal, amongst others - the recently released Drupal 8 innovates significantly to make data access from Drupal via semantic search and web services easier to implement and scale.

Meanwhile, ElasticSearch has emerged as a highly scalable RESTful data store with fast fulltext search and data analytics baked in. Bridging Drupal metadata documents to ElasticSearch is now feasible and opens up even wider avenues for data access at scale - coupling Drupal’s flexible content models and UIs for data management teams, with high-availability real-time RESTful search and analytics from ElasticSearch. ElasticSearch is also a powerful tool for driving faceted browsing and geospatial search. 

This session will feature, firstly, a run-down of the new web services and semantic innovations in Drupal 8. In the second half, we’ll introduce ElasticSearch and present a case study of its use by RPI for faceted browsing at the Deep Carbon Observatory (DCO, see https://tw.rpi.edu/web/project/DCO-DS/Technology). We’ll finish up by demonstrating how to integrate Drupal with Elasticsearch, based on work done for BCO-DMO by Blue Dot Lab. There will be ample time for Q&A and discussion of how these new technologies might be applied to further improve data access in the Earth Science community.

Talks:

1. REST in Drupal 8 by Kosta Harlan

2. Elasticsearch-powered Faceted Search in the Deep Carbon Observatory Data Portal by Stephan Zednik, Patrick West, Peter Fox

3. ElasticSearch + Linked Data for Improving Data Discovery at BCO-DMO by Adam Shepherd