Home : Reports and Publications : Audit & Evaluation : Evaluation of the WCBSN – Sept 2004
3.4 Monitoring of WCBSN
Monitoring and Accountability
Do the reports provided by networks members to WD timely? Do they allow WD to effectively monitor the performance of the network? What exactly should be reported?
Overall Findings
Many WD and community representatives felt that the reporting process needs to be streamlined. The need to collect more follow-up and results-based information was also a common concern. Many WD and community representatives, however, recognized the challenges in collecting follow-up information. Community representatives also expressed concerns that the reporting process "does not accurately capture their work." Sixty-four (64) per cent of survey respondents agree that the quarterly reports reflect the achievements of their organizations. Eighty (80) per cent of survey respondents report that their organizations have the capacity to produce quarterly reports in a timely manner.
WD key informants also cited the need for additional resources to adequately roll up and analyze the reports. Another key area of concern cited by both WD and community representatives relates to the lack of consistency in reports. Community
representatives indicate that there needs to be clearer guidelines as to what types of information to include in the reports in order to enhance consistency of reports among network members.
Detailed Findings
At present information is gathered through quarterly and annual reports. Many WD key informants state that the reporting process needs to be streamlined. One key informant states that the quarterly reports includes 43 target areas. There were some mixed opinions expressed by focus group participants as to whether the reporting process was unnecessarily burdensome. While many focus group participants commented that the reporting process was lengthy and complex, some participants felt that reports had to be specific enough for accountability purposes.
There are mixed opinions as to the usefulness of reports provided to WD by WCBSN members. Some key informants suggest the number of indicators should be reduced to collect only the most useful information (i.e. results-based information). Additional information is needed with respect to outcomes such as number of jobs created (e.g. follow-up six months after loan is provided). Some WD key informants also point out that they need a better understanding of what short-term indicators are most likely to lead to longer term results. Some WD members also note that WD does not use or sufficiently analyze present information, particularly at the local level. For example, there is information available as to the extent of loan uptake. If loans are not being used, then WD should take action on this information. Some community representatives perceive that the real challenge is in "rolling up all the information" so that it is useful.
Consistency in reporting was also cited as an issue by some WD key informants and network representatives. Community representatives state that there needs to be clear guidelines as to what types of information to include in the reports in order to enhance consistency of reports. For example, the respondents state that there is a need for clarity as to what types of activities are considered to be "successful marketing initiatives." Some focus group participants also noted that there have been a number of changes to the reporting process leading to some confusion as to requirements.
Information that is cited by key informants and focus group participants as most useful includes:
- Results based and follow-up information with respect to:
- Number of jobs created and maintained;
- Loan performance activity;
- Loan fund turnover ratio;
- Number of exports; and
- Number of businesses established and maintained.
Useful activity and output information:
Type of CED projects;
- How clients are reached (e.g. advertisements);
- New market development;
- Number and value of loans;
- Targets of loan funds;
- Number of pan western activities;
- Number of mentoring events;
- Number of training sessions;
- Number and type of services provided;
- Number of marketing/visible events;
- Dollars leveraged; and
- Number of self-serve interactions/number of officially assisted interactions.
WD key informants had a number of suggestions for improving performance monitoring. These include:
- Provide more follow-up information – what happens to loans, what happens to clients (but question how feasible this is)?
- An improved understanding of how short-term results lead to longer term results.
- More consistent reporting in terms of quality and types of information to be included in quarterly and annual reports.
- More timely and accurate reporting.
- The system for gathering information could be improved. Some WD informants cite the need for better automated system, such as a database management system (benefits – ability to store information at one central location). On-line reports were cited as another option for improving data collection.
- Additional time and resources to sufficiently analyze data.
- Annual reporting from key stakeholders with respect to – top three things that they have achieved (and how can WD help them to achieve them?); and how can WD and stakeholders work together to make a difference?