impact-measurement-designer

# Impact Measurement Designer

## Overview

Funders and stakeholders increasingly demand evidence of impact—not just what you did, but what changed because of it. Many nonprofits struggle to move beyond counting activities (outputs) to measuring meaningful change (outcomes). This skill provides frameworks for designing and measuring impact.

## The Measurement Challenge

```
WHAT MANY NONPROFITS REPORT:          WHAT FUNDERS WANT TO KNOW:
──────────────────────────────        ─────────────────────────
"We served 500 people"                "What changed for them?"
"We held 12 workshops"                "Did they learn/do anything different?"
"We distributed 1,000 meals"          "Did it reduce hunger?"
"We trained 50 teachers"              "Did teaching improve?"

OUTPUTS (Activities completed)    vs   OUTCOMES (Changes that resulted)
```

## Outcomes Hierarchy

```
┌─────────────────────────────────────────────────────────────────┐
│                    OUTCOMES HIERARCHY                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  LONG-TERM IMPACT                                               │
│  ────────────────                                               │
│  Ultimate change in conditions (5-10+ years)                    │
│  "Poverty reduced in our community"                             │
│                     ▲                                           │
│                     │                                           │
│  INTERMEDIATE OUTCOMES                                          │
│  ─────────────────────                                          │
│  Behavioral/systemic changes (1-5 years)                        │
│  "Participants obtain and retain employment"                    │
│                     ▲                                           │
│                     │                                           │
│  SHORT-TERM OUTCOMES                                            │
│  ────────────────────                                           │
│  Changes in knowledge, skills, attitudes (0-2 years)            │
│  "Participants gain job skills and confidence"                  │
│                     ▲                                           │
│                     │                                           │
│  OUTPUTS                                                        │
│  ───────                                                        │
│  Direct products of activities (immediate)                      │
│  "# trained, # workshops held, # hours of service"              │
│                     ▲                                           │
│                     │                                           │
│  ACTIVITIES                                                     │
│  ──────────                                                     │
│  What you do                                                    │
│  "Job training workshops, resume assistance, mentoring"         │
│                     ▲                                           │
│                     │                                           │
│  INPUTS                                                         │
│  ──────                                                         │
│  Resources invested                                             │
│  "Staff, funding, facilities, volunteers"                       │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

## Logic Model

### What is a Logic Model?

```
A LOGIC MODEL shows the causal chain:

INPUTS → ACTIVITIES → OUTPUTS → OUTCOMES → IMPACT

It answers:
• What resources do we use? (Inputs)
• What do we do? (Activities)
• What do we produce? (Outputs)
• What changes? (Outcomes)
• What's the ultimate goal? (Impact)
```

### Logic Model Template

```
LOGIC MODEL
───────────

PROGRAM: ____________________________
GOAL: _______________________________

┌─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐
│ INPUTS      │ ACTIVITIES  │ OUTPUTS     │ OUTCOMES    │ IMPACT      │
│ Resources   │ What we do  │ What we     │ Short-term  │ Long-term   │
│             │             │ produce     │ & Intermed. │ change      │
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│             │             │             │             │             │
│ • Staff     │ • [Activity │ • # served  │ SHORT-TERM: │             │
│   (X FTE)   │   1]        │ • # events  │ • Increased │ • [Ultimate │
│             │             │   held      │   knowledge │   condition │
│ • Funding   │ • [Activity │ • # hours   │ • Improved  │   change]   │
│   ($X)      │   2]        │   of service│   skills    │             │
│             │             │             │ • Changed   │             │
│ • Facilities│ • [Activity │ • # products│   attitudes │             │
│             │   3]        │   distributed│             │             │
│ • Partners  │             │             │ INTERMEDIATE│             │
│             │ • [Activity │ • #         │ • Changed   │             │
│ • Volunteers│   4]        │   completions│  behavior  │             │
│   (X hours) │             │             │ • Improved  │             │
│             │             │             │   conditions│             │
│             │             │             │             │             │
└─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

ASSUMPTIONS: [What must be true for this logic to work]
EXTERNAL FACTORS: [What outside factors affect success]
```

### Logic Model Example

```
EXAMPLE: Youth Job Training Program

INPUTS          ACTIVITIES       OUTPUTS          OUTCOMES          IMPACT
───────         ──────────       ───────          ────────          ──────
• 3 FTE staff   • Job skills     • 200 youth     SHORT-TERM:       Reduced
• $300K budget    workshops        served        • 80% complete    youth
• Training      • Resume         • 40 workshops    training        unemployment
  facility        assistance     • 800 hours     • 90% report      in our
• 50 employer   • Mock             of training     increased       community
  partners        interviews     • 150 resumes     confidence
• Curriculum    • Job            • 100 mock      INTERMEDIATE:
                  placement        interviews    • 70% obtain
                  support                          employment
                                                 • 50% retain
                                                   job 6+ months

ASSUMPTIONS: Youth are ready to work; employers are hiring
EXTERNAL FACTORS: Economic conditions; transportation access
```

## Outcome Indicators

### Designing Good Indicators

```
SMART INDICATORS
────────────────

S - Specific: Clear, unambiguous
M - Measurable: Can be quantified or assessed
A - Achievable: Realistic to collect data
R - Relevant: Connected to outcome
T - Time-bound: Measured at defined intervals

GOOD INDICATOR:
"% of participants employed within 6 months of program completion"

POOR INDICATOR:
"Participants get jobs" (not specific, not time-bound)
```

### Indicator Types

| Outcome Type | Indicator Type | Example |
|--------------|----------------|---------|
| Knowledge | Pre/post test scores | % increase in test score |
| Skills | Demonstrated competency | % demonstrating skill |
| Attitude | Self-report survey | % reporting increased confidence |
| Behavior | Observed/reported action | % engaging in behavior |
| Condition | Status change | % achieving status (employed, housed) |

### Indicator Development Worksheet

```
OUTCOME INDICATOR WORKSHEET
───────────────────────────

OUTCOME STATEMENT:
[What change do you expect?]
_________________________________________________

INDICATOR:
[How will you measure this change?]
_________________________________________________

DATA SOURCE:
[Where will data come from?]
□ Survey  □ Assessment  □ Administrative data  □ Observation  □ Interview

DATA COLLECTION:
Who collects: _________________
When collected: □ Pre □ Post □ Follow-up: ____
How often: ____________________

TARGET:
Baseline (current): ____________
Target: ____________ by [date]

DISAGGREGATION:
Break down by: □ Demographics □ Site □ Cohort □ Other: ______
```

## Data Collection

### Data Collection Methods

| Method | Best For | Pros | Cons |
|--------|----------|------|------|
| **Surveys** | Attitudes, satisfaction, self-report | Scalable, standardized | Response bias |
| **Pre/Post Tests** | Knowledge, skills | Direct measurement | Test anxiety, teaching to test |
| **Interviews** | Depth, context, stories | Rich data | Time-intensive, not scalable |
| **Focus Groups** | Exploring themes | Multiple perspectives | Groupthink |
| **Observation** | Behavior, skills | Direct, objective | Observer effects |
| **Administrative Data** | Status, completion | Already collected | Limited scope |
| **Document Review** | Work products | Tangible evidence | Interpretation needed |

### Survey Design Tips

```
SURVEY BEST PRACTICES
─────────────────────

DO:
✓ Keep it short (5-10 minutes max)
✓ Use simple, clear language
✓ Ask one thing per question
✓ Include both closed and open questions
✓ Test before deploying
✓ Provide incentives (if appropriate)
✓ Ensure anonymity/confidentiality

DON'T:
✗ Use jargon or technical terms
✗ Ask leading questions
✗ Ask double-barreled questions
✗ Assume literacy or tech access
✗ Make all questions required
✗ Skip the "other" option
```

### Common Survey Scales

```
SATISFACTION:
Very dissatisfied – Dissatisfied – Neutral – Satisfied – Very satisfied

AGREEMENT:
Strongly disagree – Disagree – Neutral – Agree – Strongly agree

FREQUENCY:
Never – Rarely – Sometimes – Often – Always

CONFIDENCE:
Not at all confident – Slightly – Moderately – Very – Extremely confident

IMPROVEMENT:
Much worse – Worse – No change – Better – Much better

LIKELIHOOD:
Very unlikely – Unlikely – Neutral – Likely – Very likely
```

## Outcome Tracking

### Participant Tracking

```
PARTICIPANT OUTCOME TRACKER
───────────────────────────

Program: ___________________
Cohort: ____________________

┌────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ ID     │ Start   │ Completed│ Pre-    │ Post-   │ 6-mo    │ Status  │
│        │ Date    │ Program  │ Score   │ Score   │ Follow  │         │
├────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│        │         │ □        │         │         │         │         │
│        │         │ □        │         │         │         │         │
│        │         │ □        │         │         │         │         │
└────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

STATUS KEY:
• Active  • Completed  • Dropped  • Lost to follow-up
```

### Outcome Dashboard

```
OUTCOME DASHBOARD
─────────────────

Program: ___________________
Period: ____________________

OUTPUT METRICS:
┌────────────────────────────┬──────────┬──────────┬───────────┐
│ Output                     │ Goal     │ Actual   │ % of Goal │
├────────────────────────────┼──────────┼──────────┼───────────┤
│ # Participants enrolled    │          │          │     %     │
│ # Sessions delivered       │          │          │     %     │
│ # Completing program       │          │          │     %     │
│ Completion rate            │     %    │     %    │           │
└────────────────────────────┴──────────┴──────────┴───────────┘

OUTCOME METRICS:
┌────────────────────────────┬──────────┬──────────┬───────────┐
│ Outcome                    │ Target   │ Actual   │ Status    │
├────────────────────────────┼──────────┼──────────┼───────────┤
│ % showing knowledge gain   │     %    │     %    │ ● ● ●     │
│ % reporting skill increase │     %    │     %    │ ● ● ●     │
│ % achieving [outcome]      │     %    │     %    │ ● ● ●     │
│ % at 6-month follow-up     │     %    │     %    │ ● ● ●     │
└────────────────────────────┴──────────┴──────────┴───────────┘

STATUS: ● On track  ● At risk  ● Off track
```

## Impact Reporting

### Funder Report Framework

```
IMPACT REPORT STRUCTURE
───────────────────────

EXECUTIVE SUMMARY (1 page)
• Key accomplishments
• Headline outcomes
• Challenges and responses
• Looking ahead

PROGRAM OVERVIEW (1-2 pages)
• Goals and objectives
• Target population
• Activities conducted
• Partnerships

OUTPUTS (1 page)
• People served (with demographics)
• Services delivered
• Resources distributed
• Comparison to targets

OUTCOMES (2-3 pages)
• Short-term outcomes achieved
• Intermediate outcomes progress
• Data and evidence
• Success stories/case studies

CHALLENGES AND LEARNING (1 page)
• What didn't work
• What we learned
• Adjustments made

SUSTAINABILITY AND NEXT STEPS (1 page)
• How outcomes will be sustained
• Plans for improvement
• Future goals

FINANCIALS (1 page)
• Budget summary
• Cost per outcome (if applicable)
```

### Telling the Impact Story

```
IMPACT STORYTELLING FRAMEWORK
─────────────────────────────

1. THE NEED
   What problem exists? Why does it matter?
   Use data: "X people in our community face Y challenge"

2. THE APPROACH
   What did you do about it?
   Be specific: "We provided X service through Y approach"

3. THE RESULTS
   What changed?
   Lead with outcomes: "As a result, X% of participants achieved Y"

4. THE EVIDENCE
   How do you know?
   Show your work: "Based on pre/post surveys, participants..."

5. THE STORY
   Make it real
   Individual example: "Maria came to us struggling with..."

6. THE FUTURE
   What's next?
   Connect to sustainability: "Building on this success, we plan to..."
```

### Individual Success Story Template

```
PARTICIPANT SUCCESS STORY
─────────────────────────

NAME: [First name or pseudonym]
PROGRAM: [Program name]

THE CHALLENGE:
[2-3 sentences describing the person's situation before]

THE JOURNEY:
[2-3 sentences about their experience in the program]

THE OUTCOME:
[2-3 sentences about what changed for them]

IN THEIR WORDS:
"[Direct quote from the participant]"

BY THE NUMBERS:
[Connect to broader outcome data]
[Name] is one of [X] participants who [achieved outcome].

PHOTO: □ Release signed
```

## Evaluation Design

### Types of Evaluation

| Type | Purpose | When | Resources |
|------|---------|------|-----------|
| **Needs Assessment** | Understand the problem | Before program design | Low-Medium |
| **Formative** | Improve during implementation | During program | Low-Medium |
| **Process** | Document what happened | During/after | Low |
| **Outcome** | Measure results | After program | Medium |
| **Impact** | Prove causation | After program | High |

### Evaluation Plan Template

```
EVALUATION PLAN
───────────────

PROGRAM: ____________________________
EVALUATION PERIOD: __________________

EVALUATION QUESTIONS:
1. [What do you want to learn?]
2. [What do you want to learn?]
3. [What do you want to learn?]

EVALUATION DESIGN:
□ Pre/Post only
□ Pre/Post with comparison group
□ Post only
□ Longitudinal (follow-up)
□ Mixed methods

DATA COLLECTION:
┌────────────────┬─────────────┬─────────┬───────────┬───────────┐
│ Question       │ Data Source │ Method  │ When      │ Who       │
├────────────────┼─────────────┼─────────┼───────────┼───────────┤
│ [Q1]           │             │         │           │           │
│ [Q2]           │             │         │           │           │
│ [Q3]           │             │         │           │           │
└────────────────┴─────────────┴─────────┴───────────┴───────────┘

ANALYSIS PLAN:
• Quantitative: [Describe approach]
• Qualitative: [Describe approach]

REPORTING:
• Internal reports: [Frequency]
• Funder reports: [Per grant requirements]
• Public reporting: [Annual report, website]

ETHICAL CONSIDERATIONS:
□ IRB required? □ Yes □ No
□ Consent forms prepared
□ Data protection plan
□ Confidentiality measures
```

## Common Frameworks

### Theory of Change

```
THEORY OF CHANGE
────────────────

A Theory of Change is a comprehensive description of how 
and why a desired change is expected to happen.

COMPONENTS:
1. Long-term goal (impact)
2. Pathway of change (outcomes leading to goal)
3. Interventions (what you do)
4. Assumptions (what must be true)
5. Indicators (how you'll measure)

SIMPLIFIED FORMAT:
IF we [do these activities]
THEN [these outputs will be produced]
WHICH will lead to [these short-term outcomes]
WHICH will lead to [these long-term outcomes]
BECAUSE [these assumptions are true]

EXAMPLE:
IF we provide job training and placement support to 
unemployed youth, THEN they will gain marketable skills 
and connections to employers, WHICH will lead to 
increased employment and income, WHICH will lead to 
economic self-sufficiency and reduced poverty in our 
community, BECAUSE youth are motivated and employers 
are willing to hire program graduates.
```

### Common Outcomes by Sector

```
EDUCATION PROGRAMS:
• Increased knowledge/skills
• Improved academic performance
• Higher graduation rates
• Increased educational attainment

WORKFORCE PROGRAMS:
• Gained job skills
• Obtained employment
• Increased income
• Retained employment (6/12 months)

HEALTH PROGRAMS:
• Increased health knowledge
• Adopted healthy behaviors
• Improved health indicators
• Reduced disease incidence

HOUSING PROGRAMS:
• Obtained stable housing
• Maintained housing (6/12 months)
• Improved housing conditions
• Reduced homelessness

YOUTH DEVELOPMENT:
• Improved social-emotional skills
• Reduced risky behaviors
• Increased positive connections
• Improved life outcomes
```

## Resources

### references/
- **outcome-bank.md** — Common outcomes and indicators by sector
- **survey-question-library.md** — Pre-tested survey questions
- **evaluation-glossary.md** — Key evaluation terms defined

### scripts/
- **outcome-calculator.py** — Calculates outcome percentages
- **pre-post-analyzer.py** — Analyzes pre/post test changes

### assets/
- **logic-model-template.pptx** — Editable logic model template
- **evaluation-plan-template.docx** — Evaluation plan workbook
- **outcome-tracker.xlsx** — Participant outcome tracking spreadsheet