• +855 (0)17 737 519
  • (0)68 251 790
  • contact@pepyempoweringyouth.org
PEPY transparentPEPY transparentPEPY transparentPEPY transparent
  • About Us
    • Our Story
    • Our Team
    • Our Board
    • Where we work
    • Our Financials and Reporting
  • Our Programs
    • Dream Management Project
    • Scholarship for Higher Education Project
    • Learning Center Program
    • Youth Innovators’ Space and Incubator Program
    • New Initiatives
  • Our News
    • Our Reports
    • Our Blog
    • Newsletter
    • Media Highlights
  • Contact Us
  • Get Involved
    • DONATE
    • Tours and Exchanges
    • Become a PEPY Ambassador
    • Corporate Partnerships
    • Volunteer With Us
    • Join Our Team
    • Upcoming Events
DONATE
✕
Child Clubs adopt changes for the new program year
November 20, 2012
Do you wanna ride with us? YES!
November 21, 2012

Lessons learned – our external evaluation

Published by PEPY on November 21, 2012
Categories
  • Lessons Learned in Development
Tags

Earlier this year, PEPY hired an external consultant to conduct an impact assessment of our programs. You can read a summary of the final report and also the response to the report from the PEPY Board and Senior Management team via the links below.
Executive Summary of Program Evaluation 2012
Board and Senior Management response
While the impact assessment did provide some insight into our programs, our biggest learning from the experience was with regards to the value of conducting an external evaluation. The advantages of hiring an outside evaluator are that the assessment can be more objective and provide insights that team members might not be able to contribute themselves – therefore as a whole providing a more credible analysis.
Our experience, however, was that these advantages are not necessarily guaranteed, and were also far outweighed by other disadvantages – some of which we had understood in advance and accepted, and some of which we had not anticipated.

  • An external evaluator is not always able to understand an organization’s beneficiaries and programs as well as the organization itself. Unlike some organizations, a large proportion of our staff not only spend large amounts of time in the communities they work with, but also are actually from those communities themselves. Therefore many of the findings and recommendations of the assessment did not really tell us anything new.
  • At one point we had to encourage the evaluator to be “more critical” and just “tell us the bad bits” – we didn’t want a glowing report to give to our donors, we were keen for insight into what we could do better. In retrospect we would probably have been more critical of ourselves.
  • In our opinion, the evaluator  had limited time at our projects sites, which restricted his awareness of the scope of our work, and therefore his evaluation, to be in a position to write a report about certain projects. Methods of gathering information focused heavily on interviews with our staff and with beneficiaries that we interact with everyday, as well as gathering information from our published reports. Again, much of this information we therefore knew already and did not provide new insights.
  • External evaluators are expensive – while we are committed to understanding our impact and communicating this to our donors, in retrospect we are not convinced that this method was the best way to spend that amount of money.
  • We do not have reliable baseline data for many of our projects so there was a limit to the amount of quantitative evaluation that could be conducted. This we were obviously aware of in advance but it therefore meant a need for rigorous qualitative evaluation which we felt was not conducted strenuously enough.

So what is our take-away learning from this? Well, there are two main things:

  • If we hire an external evaluator again, we need to spend more time and energy in briefing this person. We would need to do a better job at ensuring a thorough understanding of PEPY, making our expectations clearer and being firmer about our expectations about how the evaluation would be conducted.
  • Most likely, though, we probably wouldn’t hire an external evaluator in the same way again! We would be more likely to hire, for lower cost, a consultant from a respected partner organization who is familiar with our work and has a better understanding of what kind of organization we are and what we are hoping to gain from an evaluation. While some could argue that this may be a less objective evaluation, we believe it would actually be more critical and more useful to us, and be a more efficient way of spending our funds.

Karina Kloos, a member of our board Programs Committee also noted: “PEPY’s experience in this case seems to highlight some of the problems with evaluation pressures from donors across the industry. NGOs can often feel obliged to accept the idea that more “professional” assessments are necessary and legitimate the evaluation process and results. This is not only often not true, but also can devalue local staff in their ability to critically examine their own projects, report mistakes and share lessons learned. Relying on external evaluators is not always a way to ensure that organizations are being realistic and honest. It’s also worth noting that placing demands on NGOs to invest their limited resources in evaluation fuels the “Catch 22” of NGO assessment which criticizes organizations for spending too much money on overheads”.
For those interested in reading more about the pros and cons of external evaluations for NGOs, you might be interested in this article. Produced by Christina Laybourn, formerly of Bond, a UK membership body for NGOs working in international development, it gives a good overview of the pros and cons of differing approaches used in the development industry.
We’d love to hear your thoughts on this or any related experiences you may have to share!
 

Share

Related posts

December 22, 2022

PEPY Team Insights: New Plans and Goals For 2023! 


Read more
November 21, 2022

Get Involved at PEPY: Ways to Contribute to a better future in Cambodia!


Read more
October 24, 2022

Why Supporting Youth Development in Cambodia Matters


Read more

Comments are closed.

Join us in building a better future for Cambodian youth today!
Contact Us

contact@pepyempoweringyouth.org

+855 (0)17 73 7519 | (0)68 251 790

Headquarters: No 28, St. Neak Poan, Borey Proem Prey, Trapang Ses Village, Kok Chok Commune, Siem Reap District, Cambodia

Sign up for our newsletter!

Select list(s) to subscribe to


Quick Links

Tours and Exchanges

New Initiatives

DONATE


Our Policies and Practices

Child Protection Policy (English)

Child Protection Policy (Khmer)

Environmental Friendly Policy

Copyright © | By PEPY Empowering Youth

DONATE
  • No translations available for this page