Energy and Climate WG: Session I. Renewables - Dynamic Decision Tools Catalog and Community of Practice

Agencies interested in this project will be invited to attend

Session Notes (Q=question, C=comment)

Alvaro Graves - DEMO of decision support tool catalog

Motivation: many tools related to energy projects but hard to find and what they do

Have central location to find and discuss tools

Provide a dynamic catalog and establish community

Provides one location for interoperability discussions and clarification

Developed using Drupal, open source content management system, and Solr, open source search platform

Users: anonymous, registered, developers, admins

Different levels of access

The Tool Catalog

faceted browser, can filter results based on different categories

Once log-in, provides further options for filtering and examining tools

Tool descriptions use some controlled vocabulary, can add new tags

Q: how do people get tools into catalog?

Developers can access forms via admin, request an account, and ask for permission to submit a tool

Tools can be sorted by what audience they are relevant to (e.g. students, government)

There is a tool forum, discussion of each tool as well as a general forum

Q: would you want to tag a discussion to relate it to each tool?

Most questions will most likely be related to one tool, so did forum to focus it by tool. Interesting idea to tag discussion -- unsure if people would do it or not.

Q: Controlled vocals, did they survey for existing ontologies? How did you populate controlled vocab?

Some were left open, like audience so don't force people from didn't domains to use other domains' terminology. Same for tags. But for types of data, defined by the group via discussion, and there may need to be revisions in the future/expand set of terms. Should be easy to do, but need to discover what is missing. Did not want to try to cover every single case, too much, will need to be user feedback on what they really want.

C: When began discussion of what this should be, comparing tools, QA/QC, got too huge. Had to go back to step one and start simple pilot to then build on. Did use expert input on initial options. 

C: because way compare cameras is standardized, easy to compare. Would be nice eventually to standardize these tools, and unbiased.

C: May want to develop criteria over time

C: this will be successful if you are able to maintain it, generally the issue with this kind of project

C: If successfully adopted, this methodology could be adopted by other groups, for instance air quality. Also idea of bringing in private industry.

Q: is posting to forums open to public?

Have to be registered to view and post in the forums

No cost to be a member

Q: When developer has a tool, what's criteria? How is the quality assured?

Not sure, discussion that needs to happen

Q: is there a place in metadata where publish QA?

Not yet. Could be a searchable tab of 'approved by ____' 'peer reviewed' 'privately submitted'. Just need to come up with the filters.

Developers get weekly digest of all tools' discussions on the forums to review. Unlikely to regularly visit the forum otherwise.

Q: do developers get email if post in their forum?

Developer will also get specific email

Q: are the forums moderated? who is the expert?

Each tool developer is the moderator of their forum; invested in their tool. Might need to make explicit that that is their developer

Q: When building code repository, what's the value added beyond a simple internet search?

Feedback is that people don't know what's available or where to find them, so value of centralizing. Can be hidden in a government agency used internally and hard to find. Forum also provides who you talk to and where to ask questions

C: And then how do people find it exists? Crosslinks? How keep it from being a code graveyard, keeping the continuing investment and the value added?

C: reviews and what others thinks, like on Amazon, is what could add vaue to this site. Has a rating system for the tools. Number of hits is even useful.

C: Combination of ratings and tool comparisons would be big step in value.. The PR aspect needs to come after getting the tool ready to be used. 

C: need to do search engine optimization so in first 5 hits

C: and maybe a more understandable name

Q: Is there metadata format to describe tools? Eventually want DOI for landing page for a tool, because when publish paper want to reference the correct tool. New DOIs for new updates in tool versions. If making published claims want to be able to repeat. 

Might affect how developers version tools (e.g. using date or code sequence). Very unique and not easy to race versions. Agree important/nice to have but not simple to implement yet.

Q: in design what is distinction data sets and tools? How handle associated metadata?

Portal focused on tools. How describe tools with purpose for wind energy projects. Data in tools is not currently exposed, nor the sources, but could expose. Worst case can dump the database to expose metadata.

C: users try to expose data through the tools rather than go to the data source. Issues with data can probably be propagated by tool.

Would require more collaboration with developers.

Is this scope of the project? Right now it's a catalog, but not the Keeper of the tools. 

Real point is to identify the tool and who has it, go to that person.

This catalog is the general comparison and for details go to the tool website/organization.

C: The DOI for tools: if service can offer identifiers, can crawl web for when it has been used in publications. Value added, can advertise the use of it.

Great but problem is cite publication that describes a tool, not the tool itself. Hard to make sure people cite the DOI of a tool or publication. 

C: Maybe part of registration, ask that they do cite it. 

C: Give stamp to creator of resource when put it in catalog, link back (for the CLEAN project do this)

Q: Who is the end user?

Q: When going to approach user communities?

This is the unveiling of the prototype. Have sent it out to some regulators to take a look. 

Q: going to label tools for specific procedures/assessments?

Long way off, out of scope. Want to provide as much information as possible but won't say if you use this tool you get your permit. Could work with tools and the regulators during review process to have dialogue.

Q: What about tagging end-user by tool?

You register and can then access tool, and sort by audience

Q: could you do a map of domain? For instance a tool for kansas? And the resolution of the tool?

So far just have geographic cover like state or national. Can use the tags to have your tool labeled with keywords such as which state. In the future may be better to have more structure for the user. Could put in when developer is describing the tool. 

Called it a catalog because it's a list, can't provide everything, but do want to go further with peer reviews and comparisons. 

C: Need to keep links/URLs updated. Program self-healing links?