Making Evaluation Work: Taking the Right Steps for Your Program & Funders

We are also looking for 3-4 individuals willing volunteer their program/project as uses case for the small group  discussions during the workshop. We are looking for uses cases across all of ESIP (data management, applied science, education, etc). Your project/program will not be evaluated, rather this is an opportunity for free help/feedback on your project !!, to gain a better understanding of the steps and thinking involved in project evaluation. Please indicate your interest in the survey https://www.surveymonkey.com/s/ESIP_Evaluation_PreSurvey_Winter2014 AND contact Ana Prados at [email protected]
 

Meeting Notes:

4pm - 5:30pm Wednesday Session:

Breakout groups discussed the Current States and the Desired Ends of the project

Table leads shared the results - where the conversation took them:

1.

Current State: Lot of data – what is the best fit quickly for people

Desired end: find the best data for their application asap

Defining the heterogenous nature of users- don’t have the same needs applications and approaches

Difficult to be both knowledgable and neutral

Other people in group:

• Scoping issues- how to focus efforts – “sphere of influence”
• Sometimes clarity of the end goal needs refinement
• Set of activities are going to take criteria into account

2.

Current State:

Don’t have any system to analyze the current data

Need something to drive data governance

Perspective that management is expensive and decentralized

ROI – should provide a picture for why this is important

Cost saving, policy implications, increasing the transparency

Desired end goals:

Useful, effective, public, complete, comply with the executive order

What are your observations of the process?

1. 1. specification of the simple goals
2. 2. Hard to find assets – technical assets (tend for positive things to be harder to see)
3. 3. Reflection with other people is important to the process

Advice to groups:

1. Look for the Words that trigger questions
2. Spend a little more time talking about audience
3. Thinking in the abstract- play little scenarios – explore potentials
4. There will be people issues not just technical

Moving to discuss assets and barriers and questions (Step 2 and 4 of the template)

Break out group discussions

Session ended and will continue in the AM

--------

1:30 - 3:00pm Thursday Session
By Joshua Joseph and Ana Prados
*Part 2, continuing yesterday's session on Evaluation*

Josh provided overview of today's part of the workshop

Review of Day 1 and Part 1:

Josh asked for questions from participants who were at the first part of the workshop yesterday.
-Comment: It is important to define the question that you are seeking to solve.
-Comment: If we cannot solve an entire problem and you start looking at solutions, you end up going backwards.
-Q: Are there strategies for dividing large problems up into smaller pieces that can be attacked individually.
-A: Josh: Can define a large problem but look at the part you can solve. "Big enough to matter, small enough to do". Can get overwhelmed if you attack a large problem as a whole. Break problem into smaller pieces and achieve smaller victories to build momentum.

-Comment: Reflection process is important for clarification and understanding.
-Josh: Reflection allows you to see the larger picture and begin to appreciate things more.

Review of 5 Key Questions:
-Use when looking at where you are at and where you want to end up. Helps fill in intermediate steps.
1)Current State
2)Key barriers and assets. Need to identify where to focus attention
3)Your approach and why will it work. Project insights.
4) How can you tell if it's working. Indicators of progress along the way? Don't get into 'all or nothing' situation where you must wait until end to see if idea succeeded.
5) What is the desired state? End goals and outcomes.

5 Key questions are presented on a handout 'planning sheet'

Part 2: Assets, Barriers and Progress

Josh:
-Presented an idealized case study on how the 5 questions are used along with the planning template.
-It is hard for a team close to an issue to synthesize information cleanly.
-Issue: Asymmetry of knowledge within teams. Planning sheet captures important pieces of conversation to keep for future and give to future group participants.
-Easy to come up with barriers but don't have to address all of them. Many could be addressed with a single solution while others need to be 'flagged' and gone around.
-This planning tool is very helpful for retreat planning and prioritizing discussions.

Barriers:
-3 Key Questions:
1)is it valuable? (to audience, to whom?)
2)is it workable? (technical challenges, people power challenges, fear of unknown)
3)is it compelling? (other more immediate things? Why does it need to be done now?)
-Look for technical and social barriers
-Overcoming barriers might = sign of progress?
-What is smallest change that represents a meaningful advance?
-Signs that project is "off track" or not working?

Assets:
-What and Who can help?

* Break up into groups for exercise: Apply step 2 to projects that were defined by representatives from yesterday's groups.
* Group exercise lead by Group Leaders and facilitated by Josh

Josh presented a wrap-up of this session and discussion of best points that he overheard during group exercises.

Each group also presented a summary of new ideas discussed for each of the 5 questions.

Comment was made that this 5 step process would be very helpful when writing proposals.
--------

3:30 - 5:00pm Thursday Session