Health Transition Fund Project NA012: Diabetes Community/Home
Support Services for First Nations and Inuit
Program Evaluation
Introduction to Program Logic Models
Program Logic Models can be used as a framework in both program
planning and evaluation. A logic model can:
- provide a 'snap-shot' of a program by outlining the program
activities, the direct outputs from the activities and the short
and long term results;
- illustrate the links between the components of a program; and
- highlight the logic or reasons behind the planning and actions.
A logic model is client-centered. The completed model is a drawing
that represents the logical relationships amongst each of the components.
The development of the model is a dynamic process and as groups
work through the model development process, they will:
- build a group vision and consensus; and
- become aware of potential issues in their program design and
can work through these as part of the model building process.
"In my view, more days of training
are needed in order to become skilled in designing a program
evaluation logic model. Some of our forms will need to be
revised in the coming years since it is highly unlikely that
they comply with evaluation guidelines."
Wendake First Nation Project Coordinator
Future Evaluation Planning
An initial program evaluation plan was submitted with the service
delivery plan for each pilot site. The communities were not given
any tools or resources to assist them in program evaluation. It
was jointly decided at the mid point of the Project to pilot a
program logic model with performance indicators. It was hoped that
this evaluation model would provide a tool for longer-term work
by the pilot communities.
"Start with the community first,
with information and posters. Build a team and involve people
with diabetes then get the services in place. The key to
success is to involve people who have diabetes. Get input
and direction from them."
Pilot Team - Red Earth First Nation
- to develop an understanding of program logic models as an evaluation
approach;
- to practice using all elements of logic models;
- to build a template program logic model; and
- to take home practical, user-friendly tools for discussion
and application in their community to further develop and refine
their own logic model.
A generic template for a home care diabetes logic model was developed
at the workshop and individualized by the pilot team members from
each community. The teams then went back to their communities to
finish their models and to develop specific performance indicators.
Results to Date
Each of the communities developed a draft program logic model
with performance indicators. In the final report evaluation, participants
were asked to comment on the usefulness of this model for program
evaluation and the support needs for its development.
All four communities stated that they found the logic model a
useful tool in planning their long term program evaluation. One
participant noted that this approach was helpful in understanding
the rationale for nursing care and treatments and that it was also
possible to backtrack from the indicator to find the gaps in services.
Another community team member stated that the logic model assisted
with the process of developing statistical records and the tools
required to monitor the progress of the program. It was generally
found to be a useful approach for evaluating both intermediate
and long term outcomes and linking the program activities to the
outcomes.
"It is essential to increase
the staff to be able to commit time and effort to program
planning. It takes additional time to build a comprehensive
Home Care program that balances all service delivery." Red
Earth First Nation - Project Coordinator
"Before you start to work with
the clients, you need to have a good understanding of the
needs of the clients - not talking about statistics but to
really know what each of the needs of clients, both community
needs and individual needs perspective."
Health Director - Wendake First Nation
Some cautions however were noted by participants from the pilot
communities. Most found that this model required extra support
to familiarize the project teams with the concepts and to assist
with the individual model development. It was felt that additional
training days and one-to-one support would have been helpful in
promoting confidence and skill required to effectively use this
model. The communities also identified the need for more support
regarding the development of performance indicators and tracking
tools.
Conclusion
All communities were able to apply the workshop information and
develop their own draft program logic model. However, resource
materials and resource personnel were required to facilitate learning
and use.
"The support of leadership
is essential. In the Fall of 1999, we received moral and
tangible support from the managerial team of the Huron Wendat
Nation through the Council and Grand Chief Wellie Picard,
... They participated in the needs assessment and agreed
to speak to their families and urge them to take part in
the program." Wendake
First Nation Project Team
|
