 |
|
9. Lessons Learned
In this section of the report, we summarize the main findings of the evaluation.
9.1 Insights Relevant to the Design and Delivery of Programs to Single-Parents on Income Assistance
Single parents on income assistance face substantial barriers to economic independence. We can examine economic independence from two perspectives:
- The supply side of economic independence - how income assistance clients improve their education and training, as well as the supports they have to increase their ability to manage personal and family challenges; and
- Demand side - the overall growth in the economy and the willingness of employers to hire income assistance clients.
Most human resource policies address the supply side, under the assumption that changing the qualifications of income assistance clients and helping them manage personal/family problems will increase their employability. Dealing with supply-side issues is a necessary but not a sufficient condition for increasing the rate of employment among income assistance clients. One of the main findings from this study is that job-ready clients can enter the workforce provided the economy is growing. The fact that up to 25 percent in the Control Group leave income assistance with the assumption they find work within nine months, shows a "natural" process of entry and withdrawal from assistance. Some clients are temporarily on income assistance for a single spell, others cycle in and out, and still others are permanent recipients.
Even the most effective program we studied (Employment Connections) shows a long period of time before the average client comes off income assistance (14 months). Job-ready clients gain employment quickly, but the remaining clients come off assistance slowly. Quick fixes are not available for this group.
With little doubt, the success rates of the last two years will decline. The early successes of labour market interventions are unlikely to be continued as the job-ready pool of income assistance clients diminishes. Should economic growth slow, income assistance clients who have found work may be laid off, and those currently receiving assistance may enter the job market less quickly.
Income assistance programs include clients who have recently come into the program as a result of temporary adversity and those who have received welfare for an extended period. In the impact analysis we found that the total time a client had been on welfare increased the monthly post-intervention income assistance payments and the length of time he or she remained on assistance after leaving a program. This indicates how a more elapsed time on assistance creates greater barriers to employment and training.
The costs of intervention are significant. Including the income assistance payments while clients are receiving training, results in the Program costs ranging between $2,500 and $8,500 per client. Programs that offer collateral supports, such as day-care, typically have even higher costs.
Day care is essential to encouraging single-parent clients on assistance to enter training programs. The day-care services offered by Taking Charge! are important for clients using the computer lab and taking personal development courses at its offices. However, most client training occurs at service providers that often do not have day care associated with the service. To increase access for their clients, programs need to offer more on-site services, such as day-care, throughout the day and on weekends.
9.2 Insights on Individual Outcomes
All the programs we reviewed encouraged some clients to find employment. Average increases in post-intervention wages ranged from $37 per month for the Control Group to $114 for Taking Charge!, and $126 for Comparison Group clients. Training programs also increased the number of hours worked above the modest increase found for the Control Group.
Correspondingly, the training programs also reduced the time on assistance, increased benefit reductions resulting from employment, and increased the duration of employment. The reduction in benefits paid to income assistance clients ranged from $258 for Taking Charge! clients, to $208 for income assistance Comparison Group clients, to $90 for the Control Group. The reduction in average time on assistance after the intervention over the 23 months for the Control Group, was 20 months for Taking Charge! and 16 months for the Comparison Group.
The interventions we studied have with little doubt had a positive effect on the employability of income assistance clients. Typically, the results for Employment Connections are the most favourable in terms of duration on employment and numbers of clients employed after a specific period of time.
These results offer the following insights into training interventions:
- Even without intervention, a fraction of income assistance clients leave the program and return to work. After nine months, 25 percent of income assistance clients without any intervention had gained some form of employment. Any intervention must be compared against doing nothing. Income assistance clients in the Control Group are typically better educated and have had prior jobs - in effect, they are returning to work.
- Not surprisingly, job-ready clients are the easiest to train and place. Taking Charge! and Employment Connections have had the most success with job-ready clients. Employment Connections has showed the most success in moving clients from income assistance to employment because it deals only with job-ready clients.
- Interventions that stress work/job search, trade skills, and job placement are the most significantly associated with increased wages and hours worked, as well as reduced income assistance benefits.
- Aboriginal clients and those with children less than six years of age face serious barriers to employment. These clients need special attention to enhance their employability. The province does not impose work expectations on clients with children less than six years of age, but many still enter programs. Their success rates are lower than for clients in general. The low rate of employment for Aboriginal clients reflects their generally lower education and lack of previous job experience.
9.3 Design and Delivery Lessons - the Effectiveness of Third-Party Delivery
Taking Charge! is a non-profit organization that prides itself on its separateness from government. This is claimed to offer greater responsiveness to clients, the freedom to design programs tailored to income assistance recipient needs, and the ability to attract clients who may be wary of government programs.
The last claim is probably true, but suggestions that Taking Charge! offers more flexibility than government programs are overdrawn. Under the new business planning/performance measurement processes and the devolution of federal training programs, provincial managers have increased authority to design responses for specific clients. Government programs can now respond quickly to client needs and enter into individualized contracts to offer service.75
The federal and provincial governments created Taking Charge!, with considerable publicity, and realigned the funding to training services. This disturbed the existing funding arrangements and ways of doing business. The initial reaction of many community groups, training service providers, and some government departments was a degree of wariness. At the same time, the Board at Taking Charge! set an independent course and purposely designed its programming to run separately from existing services.
This separation has dissipated. Taking Charge! now cooperates more closely with provincial departments and funds training programs offered by both for-profit and nonprofit organizations. At the same time, the initial isolation of Taking Charge! from the existing training providers is unfortunate. One of the reasons Taking Charge! has had only a modest impact on the client group is the delay in developing training services that were integrated with existing programming.
Taking Charge! also became ensnared between two competing objectives. As a Strategic Initiative, a central goal for Taking Charge! was to discover innovative training for single parents on income assistance. At the same time, as a well-funded program created partly by redirecting funding from other programs, Taking Charge! decided it had to meet objectives that would serve 900 clients a year and place 500 into employment. It satisfied the latter goal by becoming a broker and funder of training programs in a conventional sense. It set aside the process of designing and assessing innovative training programs. In this vein, it has not developed an effective tracking process, used databases and information systems to evaluate alternative programs, or experimented with training. Most of the resources have been directed to running conventional contract training, with special emphasis on collateral services for single parents.
Some of the problems encountered by Taking Charge! can be attributed to circumstances outside its control:
- Delays in appointing Board members slowed program start-up. Since Board appointments are by government, this delay is directly assignable to government.
- Government should consider appointing some public servants to the Board to increase communication and accountability.
- As we note above, Taking Charge! should address the issue of how training service providers might best provide input to the Board. Alternatives to directly sitting on the Board may be a useful way to combine this valuable input and avoid the appearance of a conflict of interest.
9.4 Partnership and Community Involvement
A partnership is an association of two independent entities that have mutual goals, that commit joint resources to meet those goals, and that share risk. Taking Charge! staff and management often speak of partnerships, when in fact very few of their relationships are partnerships. The most important relationships are contractual between Taking Charge! and service providers.
Taking Charge! has weak relationships with the traditional nonprofit community. Local community groups told us that Taking Charge! does not figure in their service provision and has not developed partnerships to deliver services to their clients.
Taking Charge! has been unable to develop relationships with the private sector. In part this may be because the Board, management, and staff do not come from a business culture. Quite reasonably, staff and management are client-oriented, but this probably impedes their ability to "connect" with the business community. We found many staff to have unrealistic expectations about the ability of business to hire program graduates in any quantity. The proposed business council has potential for increasing this relationship, but support from government in approaching potential members of this council or even making Board appointments would accelerate this considerably.
Once again, Taking Charge! was impelled to meet quotas for client service and employment. Once the Board made this decision, Taking Charge! became a training-service broker and changed its relationship with the community. As a funder, it found it difficult to develop partnerships with a community that saw it as a source of money.
9.5 Does the Taxpayer Receive a Return?
It is difficult to claim that Taking Charge! offers the taxpayer a return. In relation to the Comparison Group, Taking Charge! is more expensive and less effective. However, this conclusion is based on short follow-up periods and the success that Employment Connections has had by training job-ready clients in a growing economy.
Whether the programs that concentrate on increasing the education of income assistance clients and working on motivation have a long-term payoff will not be apparent for two or three years.
9.6 Responding to the Evaluation Framework
Table 82 responds to each issue and question.
Table 82
Evaluation Questions |
| Relevance/
rationale |
| 1. What is the legislative base for Taking
Charge! - both levels of government? |
Taking Charge! is based on agreements signed
between Canada and Manitoba and an agreement between Manitoba and Taking Charge!. |
2. Has Taking Charge! met the criteria
established for Strategic Initiatives:
- innovations/experimentation potential?
- relevancy to SI objectives?
- evaluation/information potential for social reform, etc.?
|
As designed, Taking Charge! meets the
requirements of the Strategic Initiatives (SI) in all respects.
As executed, important gaps exist between how Taking Charge! operates and
the fulfilment of SI objectives. Taking
Charge! has not led the development of innovative services,
choosing instead to rely on proposals offered by training service
providers. Although it is starting to
subject service provider's proposals to more scrutiny, Taking Charge!
does not set the training agenda.
The database created by Taking Charge! can support some evaluation, and
while its completeness has improved
over the formative evaluation, Taking Charge! management
appears not to use the information to plan service delivery. Employment
facilitators primarily use it to match client
service with provider programs.
Taking Charge! has provided a basis for evaluating social reform and
assessing the relative merits of
different programs and interventions. However, this evaluation has depended
on the existence of SAMIN, a provincial information system that tracks
Income Assistance recipients. The CAMS
database created and maintained by Taking Charge! has limited usefulness
for assessing outcomes or evaluating training options. |
| 3. How many target-group members are likely
to be in need of Taking Charge!? |
Based on current income assistance data,
potential caseloads are about 7,300 single parents each month. Based on
current caseloads, Taking Charge! serves about 1,200 clients
per year or about 100 clients per month. This represents about 1.5 percent
of the client potential. |
| 4. To what extent does Taking Charge! reach
the intended target group? Do participants represent the target group? If
not, for what reasons do discrepancies occur? |
Taking Charge!'s clientele represents the
target group exactly. All clients are single parents income assistance. To
increase its market penetration, Taking Charge! has relied
on service providers to recruit, assess, and train clients. E& IA
counselors are working more closely
with Taking Charge! than previously, especially since the creation of
an E& IA office on site. Taking Charge! also uses advertising to raise
awareness among the target group. All
of these approaches give Taking Charge! a high profile among
the target client group. |
| 5. Are the services/interventions provided
responsive to, and consistent with participant needs? |
Taking Charge! has relied on service
providers to propose interventions. While Taking Charge!
does present ideas to service providers and develop pilot projects, it has
yet to define client needs and call
upon the service providers to meet those needs. Taking Charge! is
rejecting more proposals than in the first years, especially those that do
not lead to employment. In this sense,
it is assuming more control over the training agenda.
Taking Charge! has also worked with some service providers to amend
existing courses to meet client needs.
In areas such as network administration and technical trade
courses, service providers have
adjusted the pace of the training to increase the
likelihood of success for Taking Charge! clients.
Training service providers complete 37 percent of assessment and
independence planning. They have an
obvious incentive to assess clients to fill their available training
spaces, as opposed to creating new programs
for different clients. Without Taking Charge!
doing the assessment and independence planning, employment facilitators
are in a weak position to place clients
into appropriate programs or to design new training to
meet client needs. A service provider will typically want to offer an
existing program, rather than assume
the costs and risks of developing a new course. |
| 6. What similar services are being provided
by other existing |
Taking Charge! funds a range of services
similar to projects offered by other programs? provincial
programs. The job search training and job specific training/employment
programs of Employment Connections and Taking
Charge! are similar in structure and approach.
Taking Jobs!, the employment placement service in Taking Charge!, is
essentially the same service provided by
Employment Connections. The work placement process
and the basic education programs are common to Community Partnerships,
Pathways to Success, and Taking Charge!, but
directed to different clients.
The Departments of Education and Training and Family Services are now
meeting with Taking Charge! more
frequently to minimize duplication. However, the fact remains, Taking
Charge! offers services that are available in other programs. |
| 7. What gaps in the ongoing/existing array
of services/programs are being filled by Taking Charge!? |
Taking Charge! offers a supportive
environment for single-parents on assistance. As such
it is a "welcoming" first step to independence for clients who
have low expectations of themselves,
weak self-confidence and have spent much of their adult lives
in the "system." Its being apart from E& IA and other
government agencies may also encourage
certain clients to enrol. Because
service provider value has not been systematically assessed by Taking
Charge! (and other provincial programs),
service gaps are difficult to discern. Taking Charge!
has not defined a spectrum of services needed to move clients along the
employability spectrum. For that matter, the
province has also not articulated such a concept
(see Section 10). Therefore, it is not reasonable to expect Taking Charge!
to be in a position to identify service gaps.
In fact, Taking Charge! has adopted the approach
of creating new services rapidly to meet possible needs. The Taking Jobs!
program is an example. |
|
Design and delivery |
| 1. What are the design and delivery features
of the program (i. e., components, activities, and relationships between
components and activities)? |
Taking Charge! has the flexibility to
respond to client needs. It can fund ideas quickly and
well, thereby creating a good "test bed" for innovative
approaches to employment training for
this client group. At the same time,
Taking Charge! has not used this potential fully. It relies on service
providers to propose courses and does not set
the training agenda. Much of the substantive
job preparation occurs outside Taking Charge! by service providers. In
contrast, the original conception saw Taking
Charge! as the primary coordinator of client
assessment, training design, and intervention coordinator for programs
directed to the target group. Taking
Charge! is now one of several interventions funded by the province
and has become a funder and a broker of training services. |
| 2. What are the strengths and weaknesses of
the program design? |
Strengths:
- Flexible, responsive, and generous funding for innovative programming
- Staff and location create a welcoming, non-threatening, environment
for clients
- Its status as an independent agency allows Taking Charge! to fund
programs quickly,
but this advantage is weakening as government programs acquire delegated
authority to make decisions
- In-house cafeteria-style training allows clients an accessible
facility in which to upgrade skills
- Staff have a strong understanding of client needs
- Collateral supports such as day care reduce barriers to participation
in training for the target group
- A single location for service coordination (one-stop shop), with
clients assigned an employment
facilitator
Weaknesses:
- Poor contact with the business community
- High dependency on service providers to recruit, assess, and follow-
up on training creates a situation
where service providers have too much control over the training agenda
- Insufficient occupational forecasting and planning to create training
that supports long-term outcomes for
clients
- Poor information on project outcomes means that new approaches to
social security reform are not being
assessed by Taking Charge! management
- Coordination with government has improved, but service planning still
is not integrated closely with key
departments
- Broad spectrum of services tries to meet all the employment needs of
many clients as opposed to a targeted
niche strategy or experimentation
- Appearance of a conflict of interest with some service providers who
serve on the Board
|
| 3. What are the strengths and weaknesses of
the pilot-project organizational structure? Are the roles and
responsibilities of the various partners and service providers (e. g.,
delivery agent, management committee, Board members, staff, volunteers)
clearly enunciated? |
Too much time passed in the initial planning
stages. In part, this was because the creation
of Taking Charge! with significant funding (some diverted from other
programs) prompted rivalry among
various community organizations as well as between Taking Charge!
and some government departments. This time was never made up.
Taking Charge! essentially abandoned the pilot-program concept when it
decided to meet the original
expectations for service delivery and employment success. At this point,
it ceded some of its mandate to service providers who were contracted to
recruit, assess, and train clients. Too
much is left to the service provider organization in the form
of recruitment, individualized job planning, assessment, intervention
delivery, evaluation, and follow-up.
Taking Charge! has not created the evaluation and information structure
needed to assess the value offered by
service providers. Although employment facilitators are now
reviewing proposals more critically than in the early years, independent
verification of outcomes is not routine. This
is true of all training interventions funded by
the Province. Reports offered by service providers need independent
verification to confirm outcomes.
The strength of a pilot structure is that it tests new approaches.
However, most staff are not seconded
and quickly come to view the Program as a permanent entity to be funded
and defended against all criticism. Understandably, Taking Charge! management
decided to meet the employment expectations of the original agreement
as opposed to testing innovative training approaches. This is easy to
understand in light of the criticism directed
at the Program in the first two years that it was
slow in starting and funded at the expense of other community
organizations and labour market
interventions. |
| 4. How are individualized plans with
participants developed and how do these plans meet their needs? |
Individualized plans are created by Taking
Charge! employment facilitators and by service
providers. Variation clearly exists in how well these plans are completed
and to what extent they meet client needs. The
plans create employability assessments that
categorize income assistance clients as Level 1, 2, or 3. These
assessments are completed by many
individuals and are unreliable for placing clients in training. Service
providers offered quite sharp criticism of
this aspect of Taking Charge!. |
| 5. Does Taking Charge! provide sufficient
and appropriate resources (human, financial, physical) to participants,
service providers, and employers? |
Participants are satisfied with the
counseling process and the supports such as day care,
and the other services offered by Taking Charge!.
For the most part, service providers find Taking Charge! supportive.
Taking Charge! can fund service
providers well. We found training courses as high as $12,000 per graduate
and as low as a few hundred dollars.
Contact with employers is low outside work placement programs. Employers
are generally pleased with these
placements, but because of constraints on this survey,
we could not compare their perceptions of Taking Charge! with other
interventions. |
| 6. To what extent did participants
discontinue before their anticipated completion dates? What were the main
reasons for discontinuation? |
Taking Charge! staff report that training
participants quit programs because of personal
and family barriers. Some clients have unrealistic expectations of
themselves and the training. For other
clients, the idea of training and employment is novel and they
need to adjust to a new lifestyle. Finally, timing is critical. Reports
from clients (follow-up survey)
indicate that training must occur when a client is both willing and
able to take advantage of the intervention.
The quit rate varies by type of intervention, the
job readiness of clients and the nature of personal and family barriers
that impede participation. No one we
interviewed was able to offer a credible estimate of the percentage
of those who quit programs. About 14 percent of Taking Charge! participants
reported leaving the program prior to its conclusion. This compares with
9 percent for Employment Connections and 24
percent for Opportunities for Employment.
Unfortunately, our understanding of why clients leave is based on
interviews. Taking Charge! records the
disposition of clients (completed, terminated, withdrew) on its database
with discussion notes, but these are not complete or coded to support
systematic reporting. Contract files report
graduation rates from 75 percent to 95
percent.
A few service providers have had contracts terminated or not renewed
because of unacceptable completion
rates. The CAMS database is used to record the outcome of training
courses, but we found little evidence that Taking Charge! has used this
information systematically. Taking Charge! is
certainly assessing service proposals more
completely now compared with a year ago, and counselors review performance
to identify contracts that have high drop-out
rates. |
7. To what extent is the community involved
in service delivery and development? What are the linkages, how have they
been developed, and how successfully have they been developed?
- Voluntary sector
- Service providers
- Employers
|
When government created Taking Charge!, it
created a major change in the funding environment
for training. Community organizations that had received funding from
government, initially viewed Taking Charge! as
a competitor. This changed as Taking Charge!
announced it would accept proposals for funding. Many community-based
organizations then became contractors offering
service for a fee. The service providers now
include non-profit organizations, educational institutions and private
vocational schools/contractors. The
relationship that Taking Charge! has with many community-based
organizations is contractual and symbiotic. The service providers deliver
the programming and Taking Charge!
funds them.
In an important sense the voluntary sector and service providers have
merged for Taking Charge! At the same
time, Taking Charge! has not made many in roads into the mainstream
voluntary sector that obtains revenues from traditional sources (private
donations). Based on key informant interviews,
it is also apparent that its relationships with
local organizations are tenuous.
Aside from the work placement programs, Taking Charge! has not developed
strong relationships with employers.
One barrier is that many employers need workers with a grade
12 education, and often Taking Charge! clients do not have this. A GED is
equivalent to grade 12 for employers in
manufacturing and technology areas. The training
literature also strongly confirms the need for a grade 12 education to
compete in the "new" economy. |
8. a. What tracking/monitoring mechanisms
have been put in place to collect information on participants and
interventions?
- Are these adequate for measuring project impacts? b. Have Control/
Comparison Groups been identified? What criteria has been used?
|
Taking Charge! makes follow-up calls, using
volunteers (its own clients) to determine whether
clients need further assistance. Follow-ups to determine the outcomes of
training and to verify employment, create a
difficult problem for the Program. Part of its presentation
is that Taking Charge! operates separately from government. A follow-up
campaign to verify current employment status
resembles the investigation process to eliminate
income assistance fraud. This would impede Taking Charge! in presenting a
"welcoming" environment to clients.
Follow-up is integrated with the volunteer program within Taking Charge!
and is used to give clients work
experience and to improve their phone skills. Taking Charge! also relies
on final reports submitted by service providers, often in conjunction with
an application for project renewal or
the submission of a new project proposal.
No third-party verification exists on project outcomes, except for the
income assistance client follow-up
survey completed as part of the Phase 2 evaluation. The province is
presently not performing a follow-up on all
training programs although follow-up is maintained
on some, such as Opportunities for Employment.
For the evaluation, we defined Treatment, Comparison, and Control Groups
(see Section 6). |
| 9. Have any operational/legislative/
regulatory constraints been identified that impinge on the ability of the
project (or single parents) to achieve objectives? Are the project design
features (i. e., operational guidelines that define eligibility criteria,
funding limits, etc.) consistent with
the stated objectives of the project?
|
Few operational, regulatory, or legislative
constraints exist in the program. If Taking Charge!
were to be more closely associated with government, it may lose an
important feature that makes it
attractive to clients.
Also the follow-up process cannot be investigatory in nature, since that
may discourage some clients.
|
| Project Success -
Individual Outcomes |
1. To what extent has the project prepared
new participants for achieving self-sufficiency?
a. Increased their motivation and self-esteem
b. Helped them develop career action plans
c. Upgraded their educational skills
d. Provided them with occupational skills
e. Provided them with pre-employment training
f. Provided them with work experience
g. Provided them with self-employment/business skills
h. Provided them with mentors/role models |
The follow-up survey of income assistance
clients showed that Taking Charge! clients, as
well as clients in other interventions, are satisfied with their programs.
Most clients reported that the programs
in which they participated: increased their motivation
and self-esteem, helped them develop career action plans, and upgraded
their educational skills. It also provided
them with occupational skills, pre-employment training,
work experience, self-employment/business skills, and mentors/role
models.
|
2. What is the net impact on the
participants' employability, by type of program/intervention and for
completers/non-completers? a. Improvement in labour market attachment
b. Improvement in employment earnings
c. Improvement in quality of jobs secured |
Post-intervention employability is not
evaluated by any program or E& IA and we cannot directly
assess employability. In terms of actual employment, 26 percent of the
Control Group have left income
assistance after 9 months; presumably most have found work. About
35 percent of Taking Charge! clients have found work after 9 months
compared to 42 percent of the
Comparison Group.
Typical post-intervention earning increases are $114/month for Taking
Charge!, $126 for the Comparison Group,
and $37 for the Control Group.
Most clients who find jobs obtain employment in the service sector,
retail, and light manufacturing at
wages between $6.50 and $8.00 per hour.
From the client survey we found that most who were employed after training
were satisfied with the jobs they had
obtained. Little variation in satisfaction occurred across the
programs. |
3. What is the net impact on participants'
self-sufficiency, by type of program/intervention and for completers/
non-completers? a. Change in number and duration of spells on income
assistance?
b. Change in amount of income assistance and Employment Insurance received
c. Type and amount of income assistance and Employment
Insurance received
d. Reason for continued dependency on assistance. |
Very few clients in the database received
employment insurance during the study period
and we have not tracked this program among the income assistance clients.
Separating completers and non-completers is also hard for all programs.
This would result in very small sample sizes
and we did not make this distinction. The
data on non-completion are unreliable and based on service provider
reports and follow-up by the programs.
Because the effective follow-up ranged from 20 months (for those who
completed their intervention in June
1996) to 9 months for those who completed in September 1997, we could
not standardize the number of spells or their duration. This has not been
tracked.
Post-intervention duration on income assistance ranged from an average of
23 months or the Control Group, 20
months for Taking Charge! and 16 months for the Comparison Group.
We did not classify the components of income assistance received, since
that is not recorded by SAMIN.
Continued dependency on income assistance reflects the presence of clients
whose educational levels are too low to
complete job-oriented training and to compete in the labour
market. Unless they have family or personal problems, most job-ready
clients (those with grade 12 and some
work experience) find work and come off assistance. |
| 4. Has the project brought about any changes
in participant's home/family life (e. g., family interrelationships,
health status, involvement in other community activities, use of other
community services, types of recreation engaged to, etc.)? |
The follow-up survey and case studies
revealed that the interventions have a beneficial effect
on home and family life. We limited the follow-up surveys to specific
issues related to clients' abilities
to participate in the labour market and further education. Issues
such as health status, use of community services, and recreation were set
aside by agreement with the Evaluation
Steering Committee to focus on the specific labour
market outcomes of Taking Charge! |
5. To what extent has the project assisted
participants to achieve self-sufficiency?
a. What project activities or interventions were most effective and for
which type of clients?
b. For completers/non-completers?
c. For what reasons do some participants remain unemployed and on
income support after the program?
d. Did Taking Charge! motivate participants to go on to further training? |
Training interventions associated with
immediate job placement and job search have been
the most effective in placing clients of Taking Charge! and the Comparison
Group. Those who are Aboriginal and
have children under 6 years of age face the most significant
barriers to labour market participation. Education appears not to be
related to post-intervention wages,
but work expectations are.
Definitions of self-sufficiency varied
among key informants. Success
ranges from overcoming abuse, to
getting a job, to increasing job earnings, to reduced dependency
(from 100 percent income assistance to 20 percent with 80 percent employment
income). |
| 6. To what extent did the project improve
the long-term prospects of participants and their families escaping the
cycle of poverty? |
With a 9 month follow-up, long-term
prospects are impossible to assess. The evaluation literature
clearly shows that a 3-year follow-up is needed to determine changes to
long-term prospects. Under the argument that
any education, training, and job experience
is beneficial, the training programs we studied have probably enhanced
long-term job prospects. |
| Project Success -
Delivery System Outcomes |
| 1. To what extent has the project succeeded
in developing successful partnerships among the various levels of
government, employers, and community groups? To what extent has it
succeeded in integrating federal/provincial dual services? |
If we view partnerships as an agreement
between two entities to achieve a mutual goal and
with each sharing risk to attain that goal, Taking Charge! has not
developed effective partnerships. Those
community organizations that train for a fee are not partners,
they are contractors performing a service. It has not formed partnerships
with the traditional non-profit
community and some local community groups view Taking Charge!
as detached from themselves. Relationships with employers are weak,
although a business advisory council is being
developed.
The relationship with the provincial government has improved significantly
and is serving to integrate Taking
Charge! services with E& IA counseling and Education and Training.
Taking Charge! has had little involvement in the devolution of training
and labour market services from the
federal to the provincial government. It has improved its
relationship with the Department of Education and Training. |
| 2. How satisfied are participants with
various aspects of the project (e. g., application and selection, services
provided, etc.)? |
Participants are very satisfied with Taking
Charge! and the other interventions in the Comparison
Group. |
| 3. To what extent has the project succeeded
in removing disincentives to employment and training (e. g., changing
income assistance regulations, etc.)? Or alternatively, in increasing
incentives (e. g., earned income supplements, day care, nurturing, health
benefits, etc.)? |
Taking Charge! has had no active role in the
Making Welfare Work program. However,
because it offers a welcoming and friendly face, it can encourage clients
to seek training. The existence of collateral services such as day care
is very important to supporting the harder
"edge" of welfare reform, such as the imposition
of a work expectation. Therefore, a program such as Taking Charge! can
be a valuable adjunct to welfare reform, especially for income assistance
clients with poor education and job
experience. |
| 4. To what extent did participants displace
employees already on an employer's workforce? Were these jobs permanent? |
With the barriers to the employers survey
(see Appendix 3) we were unable to address
this issue directly. Given that Manitoba has had a high rate of job
creation (7,000 new jobs in June 1998),
it is unlikely that Taking Charge! and its approximately 450
graduates who become employed in a year, displace full-time workers. |
| Project Success -
Cost-effectiveness |
| 1. What are the costs of individual
components comprising the program? |
Taking
Charge! spends 60 percent of its annual $5 million budget on training
service contracts. Another 19 percent
is spent on indirect costs (administration, rent, etc.), and 21
percent is spent on direct client services (computer lab, day care, etc.). |
| 2. What is the per diem cost per
participant? Per component? |
Taking Charge! cost $3,112 per client, while
Employment Connections (about 50
percent for the Comparison Group) costs $1,341. |
| 3. What is the cost per participant, per
program completion, per employed participant? |
Taking Charge! costs $6,905 per employed
client (judged after 9 months), compared with
$2,439 for Employment Connections and $4,000 for Opportunities for
Employment. |
| 4. What is the net cost or saving due to the
difference in subsequent level of income assistance and EI dependency
attributable to program participation? |
On average, Taking Charge! clients have
lower income assistance payments of $258
per month, compared with $208 for the Comparison Group and $90 for the
Control Group. A certain number of Control
Group clients get jobs without any intervention,
and this leads to their income assistance payments being reduced. |
| 5. What is the benefit cost ratio and
payback period? |
The benefit-cost ratio captures all
benefits (net income assistance savings, reduced time
on assistance and taxes on earnings ) and all costs (costs of income
assistance while on training, cost of
the training and special assistance for training). Despite the fact
that Taking Charge! clients typically reduce income assistance payments by
government more than the Comparison Group (see
above), because its training is longer
and more costly, it has a benefit-cost ratio between 0.51 and 0.96 (and
no computable payback period). The
Comparison Group has a benefit-cost ratio of 1.54. Assuming
that all benefits are generated within the 16 months average duration on
assistance, the payback for the Comparison
Group cost is about 9 months. Once again,
the Comparison Group programs tend to have a higher proportion of job-
ready clients than Taking Charge! which
accounts for their higher measured returns. |
| 6. Is the Taking Charge! model a cost-
effective way of achieving project objectives? Are there more cost
effective methods of achieving the same objectives? How do results compare
with those of other programs with similar objectives (e. g., brokered
services, collocation)? |
In terms of meeting employment for job-
ready income assistance clients, Taking Charge! represents
poor value compared with programs such as Employment Connections and
Opportunities for Employment. Taking Charge! has evolved into a broker of
training services, but funds relatively
high cost interventions. This conclusion must be qualified short
post-intervention by the observation period and the fact that Taking
Charge! has a much broader spectrum of
clients than those in the Comparison Group. |
| 7. What lessons can be learned from Taking
Charge! on interventions to assist the target group? To what extent does
it contribute to the development of a
policy framework for social security reform? Does Taking
Charge! lead to a more efficient delivery of services? To what
extent can this experience be extended to/adopted by other
jurisdictions? |
(See Section 10) |
|
![[To Top]](/web/20060127002914im_/http://www11.hrdc-drhc.gc.ca/edd-img/top.gif){kind=small}
![[Previous Page]](/web/20060127002914im_/http://www11.hrdc-drhc.gc.ca/edd-img/back.gif){kind=small}
![[Table of Contents]](/web/20060127002914im_/http://www11.hrdc-drhc.gc.ca/edd-img/content.gif)
![[Next Page]](/web/20060127002914im_/http://www11.hrdc-drhc.gc.ca/edd-img/next.gif){kind=small}