Key Messages

Evaluation helps programs ensure that participants are showing improvement in the outcomes expected by the program and that the program is being implemented as intended.
Evaluation can help decision makers understand how the implementation process is working and can help a program to improve and mature.
Often, the term evaluation has negative connotations (e.g., that it is a test). It is important to clearly communicate to program staff that evaluations help with performance and training.
It is important to identify and involve stakeholders who will use the evaluation data and will need to be involved in its collection early on.
When undertaking an evaluation, it is important to locate resources. Determining whether an evaluation will be conducted by an internal or external team member is an important first step.
To determine which evaluation components are necessary, it is important to start by identifying the purpose of the evaluation.
Crucial evaluation components include a theory of change logic model, fidelity indicators, and outcome measurements.
These evaluation components are well resourced and templates are often available for them; it is important to draw on these tools to avoid replication.
A good reporting process will ensure that evaluation data ends up in the hands of the appropriate stakeholders, and that evaluation information is used for program improvement.

Why Evaluate the Importance of Evaluation and Performance Measurement?

Evaluation helps programs ensure participants are demonstrating improvements in expected program outcomes. Evaluation helps answer questions such as:

Are participants stably housed?
Is the quality of life improving for participants?
Which participants aren’t improving and need a different approach?

Evaluation also helps ensure that the program is actually in place, answering such questions as:

Is the target population being reached?
Are participants receiving supports consistent with the Housing First model?
Are participants receiving support on a consistent basis?

Further, it can help decision makers understand how the implementation process is working, and provide insight into questions such as:

Is the program being implemented as intended?
Is the program being implemented with a high degree of fidelity to the principles of the Housing First approach?
Are there adequate financial and human resources to implement the program?
Are staff receiving appropriate training and support?
Is the program adapted appropriately to the local context without comprising the basic principles of the Housing First approach?

Evaluation can also help a program to improve, mature, and answer questions like:

How can the program better serve participants?
In what ways can the program be enhanced and lead to greater improvement in outcomes for participants?

Finally, evaluation can determine if the program is being provided in an efficient manner, answering questions such as:

What are the costs and potential cost savings of the program?
Are there ways that the same level of outcomes can be achieved at a lower cost?

In summary, evaluation or performance measurement is a tool that helps clinicians do their job better. It’s also a tool that helps managers understand their programs’ strengths and work with staff to build upon these strengths. Evaluation can be used by program staff for public relations and advocacy, helping them to sell the program to policy makers and funders. Evaluation also helps all stakeholders to continually improve the program. Finally, evaluation is a way to engage individuals receiving Housing First services. For all of these reasons, assessing program fidelity and outcomes is part of the reporting procedures specified by the Homelessness Partnering Strategy. This module of the toolkit provides you with the information and resources to get started and move in the right direction to implement an evaluation approach for your Housing First program.

Myths About Evaluation

There are a number of myths surrounding evaluation. Stakeholders may have to address the following misconceptions at an early stage.

Myth

Evaluations are “a test.”

Myth Busted

The term “evaluation” itself may contribute to this notion. It is important to frame program evaluation as a tool for program improvement. Some programs have avoided the term “evaluation” and have used the term “progress report” instead. The field of program evaluation has evolved to use terms such as “developmental evaluation.” This emphasizes an approach that sees program improvement as evolving. Developmental evaluation ensures that the intervention continually adapts to its environment, rather than seeing evaluation as the “last word” as to whether the program works or not.

Myth

Evaluations take resources away from clinical work.

Myth Busted

It does take time to complete forms and questionnaires, but when done properly, evaluation is a tool that ensures clinicians are actually achieving the goals that program participants articulate; evaluations can make use of outcome measures that are also clinically relevant. Program evaluation can also offer clinicians the opportunity to give their opinions on how the program is working. It can also be used to flag systemic issues that are interfering with clinical goals, such as a lack of resources, incomplete implementation of the model, or inefficient teamwork.

Myth

Evaluations are only of use to external bodies and funding decisions.

Myth Busted

Past experience with accreditation processes may have given stakeholders the impression that evaluation is only about going through a rigorous process in order to receive a “stamp of approval” on a program. Given the realities of insecure or limited funding, evaluation is necessary and valuable, particularly when it is also used as a springboard for developing internal quality improvement capacity.