Visit Website

Stay in Touch

  Share a Referral

  Contact Us

Subscribers:

  Forward

  Subscribe

  Unsubscribe

March 2006 eNews

Dear Friends:

As I mentioned last month, od180 has brought together an alliance of consultants, each one being quite gifted and accomplished in a particular area of organizational life. With this team approach, od180 is able to provide non-profit leaders with much of the help they need to meet their organization’s challenges in a seamless, and therefore, efficient and cost-effective way. 

This month I want to introduce you to Stephen Velez-Confer, who has vast experience in program evaluation and has published numerous articles on the topic. Learn more about his professional background in the “About Us” area of our website. Look also under “Case Studies” where you will find some of his insightful stories about turn-around strategies, building community and effectiveness, and demonstrating overall program effectiveness (Case Studies 6, 7, and 8 respectively).

If your organization has a program that you know is not operating as well as it could, Stephen can help. And if you are designing a new program that you want to be as good as it can be from the very start, he can help with that too.

I invite you to read his article below, “Evaluation at the End Depends on Questions Asked at the Beginning.” Program Managers and Board Program Review Committees will find it enlightening.

Peace,
David Norgard

Evaluation at the End Depends on Questions Asked at the Beginning (Part 1)

By Stephen Velez-Confer, MNM

(Accepted for publication by the ASTD Newsletter, January 2006)

Calculating a return on investment (ROI) from a training program is clearly in vogue. Training professionals and business managers seek to use evaluation data of this nature to demonstrate training interventions contribute to an organization’s bottom line. The approach is to demonstrate that the cost of the intervention is significantly less than the benefits derived.

However, this approach raises the evaluation issue of causality. Can you really show that your intervention actually produces the desired effect? This is a thorny issue at best.

In fact, there is probably no way to demonstrate with absolute certainty that a specific action or intervention caused a specific outcome or result. There are too many other possibilities that might contribute to or intervene in the same outcome. There are ways to demonstrate that there is a strong probability that an intervention caused a certain result or outcome. The process for doing so is to maintain a chain of data and analysis that begins before the intervention and continues afterward. First, design and conduct a strong data based and analytical needs analysis. Second, document successful levels of learning and application. In this first article, we will describe questions to guide an effective needs analysis.

• What is the problem? Defining the needs is clearly the first step in a needs analysis. In our case, we are looking for performance deficiencies that are gaps between what we want and what is actually occurring. We are not looking for a list of training activities in a needs analysis. A professional conducting a needs analysis will do so as a researcher. Ideally, a researcher conducting a needs analysis will describe the problem quantitatively and not yet in terms of skills or knowledge presumed lacked by the population under consideration.
• What kind of problem is it? Training does not solve all performance problems. Do the employees know what to do? Do they know the standards? Do they receive feedback? If the population you are considering does not know what to do, the standards to achieve, and do not receive feedback on how close they are to achieving those outcomes; it is very unlikely those outcomes will be achieved. These are problems best solved by establishing systems to communicate standards and progress toward achieving them. Further, the motivation to perform effectively and to the standards is a function of incentives. Do incentives connect directly to the desired performance? Are the incentives valued? Finally, are the tools and supplies required to perform to the standards available when and where they are needed?
• Who has the problem? For example, if a tool required to perform a job is not available, training someone how to use the tool does not solve the performance problem. A subgroup and not the total population may be the cause of the problem.
• What is the cost of not solving the problem? Is the cost sufficient to justify the expenditure associated with researching and solving the performance problem? This is the foundation data for calculating a return on investment, and the basis for proposing a solution.

In the next article, we will discuss how to establish the nature of the problem. Subsequent articles will discuss continuing the data chain from the intervention to calculating a return on investment.

Stephen Velez-Confer possesses a Masters of Nonprofit Management from Regis University in Colorado, and is currently pursuing a Ph.D. in Public Policy and Administration at Walden University with a specialty in evaluation. He has published in the fields of human resource development with chapters in three books including ASTD books on needs analysis and human research development research as well as several articles.

od180

Suggested Reading

In a different vein, beginning this month I want to recommend to you resources that I believe are quite valuable and accessible. My initial recommendation is a quick read that is full of practical suggestions for organizations which want to grow but have very limited development budgets. (Churches take note!)

There are ways that you can increase your visibility and name recognition without massive investments in paid advertising. Mainly what it takes is a plan and perseverance  That is the essential message in Guerilla Marketing for the New Millennium – Lessons from the Father of Guerrilla Marketing by Jay Conrad Levinson.