Types of Program Evaluation

Why the type of evaluation matters

Ask five people what a program evaluation is and you will get five answers. A funder may mean an outcome report. A board member may mean a judgment on whether to keep the program. A program manager may mean feedback to make next month go better. They are all talking about evaluation, but about different types.

The Government of Canada’s Policy on Results offers a useful anchor. It defines evaluation as the systematic and neutral collection and analysis of evidence to judge merit, worth or value, and says evaluation informs decision making, improvements, innovation and accountability. It notes that evaluations typically examine questions about relevance, effectiveness and efficiency.

Choosing the right type means you collect the evidence that actually answers your question, at a cost that fits your organization. Our complete guide to program evaluation covers the full process. This article goes deeper on the types and how to choose between them.

“The best evaluation is not the most rigorous one. It is the one that answers the question you actually need answered.”

Three ways to sort evaluation types

Evaluation types overlap, which is why the terms get confusing. It helps to sort them three ways:

A single evaluation can sit in all three groups at once. A formative process evaluation designed with participants is formative by timing, process by focus and participatory by approach.

Needs assessment

A needs assessment happens before a program starts, or when you are considering a major change. It asks whether there is a real need, who experiences it, how big it is and what already exists to meet it.

• Who is affected, and how?
• What are the root causes, as people experiencing the issue describe them?
• What services already exist, and where are the gaps?
• What strengths and assets does the community already have?

A good needs assessment prevents one of the most expensive mistakes in the sector: building a program for a need that is smaller, different or already met. Our guide to designing a nonprofit program shows how it feeds into program design.

Evaluability assessment

An evaluability assessment asks a simple question before you invest in a larger evaluation: is this program ready to be evaluated, and would an evaluation be useful?

• Is there a clear, shared understanding of what the program is trying to achieve?
• Is there a logic model or theory of change that links activities to outcomes?
• Is data being collected, or could it be?
• Do the people who will use the findings agree on the questions?

If the answers are no, the best next step is usually to clarify the program’s logic first. Our guides to the logic model and theory of change help you do that.

Formative evaluation

Formative evaluation is about improvement. It happens while a program is being developed or in its early delivery, and it gives you feedback you can act on quickly.

• Are participants finding the program useful and accessible?
• Which parts are working, and which are confusing or underused?
• What should change before the program grows?

Formative evaluation is often light and fast: short surveys, conversations with participants, staff reflections and simple tracking. Its value comes from acting on what you learn, not from a polished report.

Share what you learn with participants too. Seeing their feedback acted on builds trust and better feedback next time.

Process or implementation evaluation

Process evaluation asks whether the program is being delivered as intended, and how well. It is the foundation for every other type, because you cannot judge outcomes if you do not know what was actually delivered.

• Reach. Are you reaching the people you intended to serve?
• Fidelity. Is the program delivered as designed, and if not, why?
• Quality. How do participants and partners experience it?
• Dose. How much of the program do participants actually receive?

Outcome evaluation

Outcome evaluation looks at the changes participants experience: in knowledge, skills, behaviour, circumstances or wellbeing. It is what most funders mean when they ask about results.

It relies on clear outcomes and indicators, set out before data collection begins, and often on comparing participants’ situation before and after the program. Our guide to nonprofit outcome measurement covers how to choose indicators and tools.

Be honest about what outcome evaluation can and cannot show. It can show that participants changed. On its own, it usually cannot prove that your program caused the change. That is the job of impact evaluation.

Impact evaluation

Impact evaluation asks how much of the change can be attributed to your program, compared with what would have happened without it. Evaluators call that alternative scenario the counterfactual.

Answering it well usually requires a comparison: a similar group that did not take part, a waiting list, or in some cases random assignment. These designs take time, money, technical skill and careful ethical review.

One alternative is contribution analysis, which builds a reasoned, evidence-based case for how a program contributed to observed change, without needing a comparison group. It works well alongside a solid theory of change.

Summative evaluation

Summative evaluation looks back at the end of a program cycle to judge its overall merit, worth or value. It informs big decisions: continue, expand, change or stop.

• Did the program achieve what it set out to do?
• Was it worth the investment?
• What should happen next?

Summative evaluations usually draw on process and outcome data collected along the way. That is why it pays to plan evaluation from the start rather than scrambling at the end.

A summative evaluation is also a chance to tell the full story of a program to your board, funders and community: what it set out to do, what happened, what it cost and what you learned along the way.

Economic evaluation

Economic evaluation compares a program’s costs with its results. It answers questions boards and funders care about: is this a good use of resources?

These methods can be powerful, but they depend heavily on assumptions, especially when putting a money value on social outcomes. Be transparent about those assumptions, and treat precise-looking ratios with healthy caution.

Developmental evaluation

Developmental evaluation is designed for innovation. The J.W. McConnell Family Foundation’s Developmental Evaluation Companion opens with a line from its earlier primer: developmental evaluation supports the process of innovation. It adds that adaptation and complexity are at the core of the approach.

It fits when a program is still being shaped, the problem is complex and the path is uncertain: a new collaboration, a systems change effort or a pilot responding to a fast-changing situation. Instead of judging a fixed model, a developmental evaluator works alongside the team, gathering and interpreting evidence in real time to guide adaptation.

It is not a shortcut. It requires a skilled evaluator, an organization willing to learn openly and a different kind of reporting, often frequent and conversational rather than a single final report.

Participatory and community-led evaluation

Participatory evaluation involves the people a program serves, and often staff and partners, in designing the evaluation, collecting data and making sense of the findings. It can make findings more relevant, more trusted and more likely to be used.

For work with First Nations communities, data ownership and control matter. The First Nations Information Governance Centre’s principles of OCAP®, which stand for ownership, control, access and possession, establish how First Nations’ data and information will be collected, protected, used or shared. Build them into the evaluation from the start, in partnership with the community.

Our program evaluation guide includes more on participatory and equity-focused approaches.

Utilization-focused evaluation

Utilization-focused evaluation starts from a simple principle: an evaluation should be designed around how its intended users will actually use it. Before choosing methods, you identify who will use the findings, what decisions they face and what evidence would help.

It is less a separate type than a discipline you can apply to any evaluation. If you cannot name who will use the findings and for what, pause before collecting any data.

Writing good evaluation questions

The type of evaluation follows from the questions. Good questions are specific, answerable with the time and data you have, and tied to a decision someone will make.

Made-up examples by type:

• Needs. What barriers do newcomer parents in our area face in accessing child care?
• Process. Are we reaching the families we designed the program for, and if not, who is missing?
• Outcome. Do participants report more confidence in job searching after completing the workshops?
• Impact. How do employment rates compare between participants and a similar group on our waiting list?
• Economic. What does it cost us to support one participant into stable employment?

Notice how each question points to a type and a design. Write your questions first, and the rest of the evaluation becomes much easier to plan.

Mixing up types: a common trap

A common trap is a mismatch between the question and the type of evaluation. An organization wants to know whether its program works, but collects only satisfaction surveys. Or a funder asks for impact, and the organization reports outputs: sessions held, people served.

Satisfaction, outputs, outcomes and impact are all useful, but they answer different questions. Be clear about which one you are measuring, and do not present one as another. If a funder asks for impact and you can only show outcomes, say so honestly and explain how you will strengthen the evidence over time.

Internal or external evaluator?

Who conducts the evaluation affects its cost, credibility and usefulness.

Formative and process work often suits internal teams. Impact, summative and economic evaluations, or anything high-stakes, often benefit from external expertise.

Evaluation designs, explained simply

Types describe what an evaluation looks at. Designs describe how it is structured to answer the question. Common designs include:

• Post-only. Data collected after the program. Simple, but tells you little about change.
• Pre and post. Data before and after, showing change in participants. A common choice for outcome evaluation.
• Comparison group. Participants compared with a similar group that did not take part, which strengthens claims about impact.
• Randomized design. People assigned at random to take part or not. Strong for attribution, but often impractical or inappropriate for community programs.
• Case study. In-depth look at one site, group or story, useful for understanding how and why.
• Mixed methods. Numbers and stories combined, so each strengthens the other.

Match the design to the question and the stakes. A strong design for the wrong question is still the wrong evaluation.

Being honest about limitations

Every evaluation has limits: small numbers, missing data, people who did not respond, or changes outside your control. Naming them openly makes your findings more credible, not less.

Explain what the evaluation can and cannot tell you, and avoid stretching findings beyond what the evidence supports. A clear statement of limitations protects your credibility with funders and helps everyone use the findings wisely.

Where the data comes from

Every type of evaluation draws on some mix of these sources:

• Program records. Attendance, intake forms, referrals and service logs.
• Surveys. Short questionnaires, ideally tested with a few participants first.
• Interviews and focus groups. Depth on how and why things happened.
• Observation. Seeing the program in action.
• Partner and public data. Information from partners or published statistics, used with care.

Collect only what you will use, protect personal information, and be clear with participants about how their information will be used. A well-set-up database or CRM can make program records far easier to analyze.

How to choose the right type

1. Name the purpose. Improvement, accountability, a decision about the future, or learning about something new?
2. Identify the users. Who will use the findings, and for which decisions?
3. Write the questions. Three to five key questions are plenty for most small organizations.
4. Check the program’s stage. New and changing programs need formative or developmental approaches; stable ones can support outcome or summative work.
5. Look at your data and resources. What data exists, what budget is available and who has the skills?
6. Consider ethics and consent. How will you protect participants and their information?
7. Pick the type and design. Choose the simplest approach that credibly answers your questions.

If two types seem to fit, ask which one would change a decision. That is usually the one to do first.

When to plan your evaluation

The best time to plan an evaluation is when you design the program, not when the funder’s report is due. Planning early lets you:

• agree on outcomes and indicators while the program is being shaped
• collect baseline data before participants start
• build data collection into everyday program routines
• include evaluation time and costs in budgets and grant applications
• schedule check-ins at the midpoint, not just the end

If your program is already running without an evaluation plan, start now. A light process review and a clear set of questions for the next cycle are far better than waiting for the perfect moment.

Combining types over a program’s life

Most programs benefit from different types of evaluation at different stages. A made-up example for a three-year employment program:

• Before launch. A needs assessment and a logic model.
• Year one. Formative and process evaluation to refine delivery.
• Year two. Outcome evaluation using pre and post measures.
• Year three. A summative evaluation, with a cost analysis, to inform renewal.

Planning the sequence early means each stage collects data the next stage needs.

A small-organization example

Here is a made-up example of a proportionate approach for a small drop-in program for seniors.

• Process. Simple attendance tracking and a quarterly check on who is and is not attending.
• Formative. A short conversation circle every few months to ask what participants value and what they would change.
• Outcome. A brief pre and post questionnaire on social connection, used once a year.
• Reporting. A two-page summary for the board and funders, with a few quotes and the changes made.

None of this needs a large budget. It needs consistency, clear questions and a habit of using what you learn.

Reporting different types of evaluation

Different types of evaluation call for different kinds of reporting:

• Formative. Quick notes or short meetings, so changes happen while they still matter.
• Process. Simple dashboards or regular updates on reach and delivery.
• Outcome and impact. Structured reports with methods, findings, limitations and recommendations.
• Developmental. Frequent sense-making sessions with the team, documented as you go.
• Economic. Clear tables of costs and results, with every assumption spelled out.

Whatever the format, include what you learned and what you will change. Funders and boards respond well to honesty about what did not work.

What evaluation takes

Evaluation needs time, skills and systems. Be realistic about what your organization can manage.

• Staff time for data collection and reflection, built into workloads.
• Data systems that capture what you need without heavy manual work.
• Skills in survey design, interviewing and analysis, inside or outside the team.
• Budget for an external evaluator when independence or specialized skills matter.

Proportion matters. A small program needs a small, focused evaluation. Some funders allow evaluation costs in grant budgets, so ask, and include them when you plan.

Standards and ethics

Good evaluation follows recognized standards. The Program Evaluation Standards, developed by the Joint Committee on Standards for Educational Evaluation, group good practice under five attributes: utility, feasibility, propriety, accuracy and evaluation accountability.

• Utility. Is the evaluation useful to the people who need it?
• Feasibility. Is it realistic and efficient?
• Propriety. Is it proper, fair, legal, right and just?
• Accuracy. Are the findings dependable and truthful?
• Evaluation accountability. Is the evaluation documented and open to review?

Ethics run through all of it: informed consent, privacy, cultural safety and doing no harm to participants. When in doubt, slow down and ask.

Working with funder requirements

Funders often specify what they want evaluated and how they want it reported. Read those requirements early, and propose an evaluation that meets them without overloading your team.

• Clarify which outcomes and indicators the funder expects.
• Agree a realistic scope and timeline in the application, not after.
• Include evaluation costs in the budget where allowed.
• Share findings honestly, including what did not work and what you changed.

Our guide to grant reporting after you win covers how to turn evaluation data into reports funders value, and impact reporting shows how to share results more widely.

Building a learning culture

The type of evaluation matters less than whether your organization uses what it learns. A learning culture makes evaluation a habit rather than an event.

• Leaders ask what the evidence says before major decisions.
• Staff have time to reflect on data, not just collect it.
• Mistakes and disappointing results are discussed openly.
• Participants see how their feedback changed the program.

Small, regular habits build this culture faster than any single large evaluation. Our program evaluation guide has more on building evaluation capacity in small nonprofits.

Common mistakes

• Jumping to impact before the program is stable or well understood.
• Collecting data nobody uses.
• No baseline, so change cannot be measured.
• Ignoring process data, then misreading disappointing outcomes.
• Evaluation as an afterthought, planned in the final month of a grant.
• Overstating findings to please funders.

Getting support

Choosing and running the right evaluation is easier with an experienced partner. For Good Consultants supports Canadian nonprofits with program design and evaluation, from logic models and evaluation plans to data collection tools and reporting.

Whatever type you choose, start with clear questions, match the approach to your program’s stage and use what you learn. That is what turns evaluation from a requirement into a real advantage.

A quick glossary

• Output. What the program produces or delivers, such as sessions held or people served.
• Outcome. A change for participants, such as new skills, behaviour or circumstances.
• Indicator. A measurable sign that shows whether an outcome is happening.
• Baseline. The starting point, measured before the program begins.
• Counterfactual. What would have happened without the program.
• Attribution. Showing that the program caused a change.
• Contribution. Showing how the program contributed to a change, alongside other factors.
• Logic model. A one-page picture linking resources, activities, outputs and outcomes.

Sharing a glossary like this with staff and board members helps everyone talk about evaluation in the same language.

Frequently asked questions

What are the main types of program evaluation?

Needs assessment, formative and process evaluation, outcome evaluation, impact evaluation, summative evaluation and economic evaluation. Developmental evaluation is an approach suited to innovative programs in complex settings.

What is the difference between formative and summative evaluation?

Formative evaluation happens during development or delivery to improve the program. Summative evaluation happens at the end of a cycle to judge its overall merit and inform decisions about its future.

What is the difference between outcome and impact evaluation?

Outcome evaluation looks at the changes participants experience. Impact evaluation asks how much of that change the program caused, usually by comparing with what would have happened without it.

What is process evaluation?

An evaluation of how a program is delivered: whether it reaches the intended people, is delivered as designed, and how participants experience it.

When should we use developmental evaluation?

When a program is innovative or still being shaped in a complex, changing situation. The McConnell Foundation’s companion notes that adaptation and complexity are at the core of the approach.

How do we choose the right type of evaluation?

Start with the purpose, the intended users and your key questions, then consider the program’s stage, your data and resources, and ethics. Choose the simplest approach that credibly answers your questions.

How does the Government of Canada define evaluation?

The Treasury Board’s Policy on Results defines it as the systematic and neutral collection and analysis of evidence to judge merit, worth or value, typically examining relevance, effectiveness and efficiency.

What standards guide good evaluation?

The Program Evaluation Standards group good practice under utility, feasibility, propriety, accuracy and evaluation accountability.