Evaluating Coaching Initiatives — Assessing Development Progress

This is the third post in a three part series on evaluating coaching initiatives, a key priority for organizations looking to track the effectiveness of coaching and to drive the strongest ROI on their coaching investment.

The focus here is on development progress assessments, which come in typically during the closeout process for an engagement, but may also be conducted at an interim point in longer engagements.

The other two blog posts in this series focus on:

Interim Evaluations: These are usually conducted once about midway through the expected lifespan of a coaching engagement, but may occur several times in engagements longer than six months. Please see that blog post here.
Post-Coaching Evaluations: It’s common to conduct these within the first few weeks after a coaching engagement has closed, but they are sometimes delayed three to four months, particularly after shorter engagements, to assess the “stickiness” of the coaching and capture longer-term impacts. Please see that blog post here.

About Development Progress Assessments

In addition to evaluations of the coaching experience by the participants in the process, organizations often want to enable a more targeted assessment of individual development progress as a result of coaching.

When conducted during the closeout process for an engagement, progress assessments can serve as a springboard for ongoing development, helping to prioritize focus going forward. When conducted in the middle of a longer-term coaching process (typically once every 9 to 12 months), they can help to re-align development efforts for subsequent coaching. Either way, these assessments help to document and celebrate the progress that a coachee has achieved.

There are two basic approaches:

Second Full 360: re-administer a full 360-degree assessment survey for a “Time 1 – Time 2” comparison of results on all assessment factors
Targeted “Mini” 360: administer an abbreviated 360 that focuses feedback on the specific areas targeted for development in the coaching engagement

Second Full 360

It can be tempting to consider a second administration of a comprehensive 360 survey instrument to be a good gauge of development outcomes from coaching. After all, the instrument is usually already set up and ready to go, it’s inexpensive, and it gives the organization access to aggregate data on coaching outcomes across a population of coachees.

However, there are a number of problems with this approach:

It is overly burdensome for the stakeholders asked to complete the instrument a second time. Feedback providers often feel over-surveyed as it is. As a feedback provider, I have to revisit and reconsider a large number of assessment items, most of which represent things the coachee hasn’t focused on improving.
It’s overly broad in its scope. Rather than focusing on the development areas targeted by the coaching engagement, a full 360 might assess 12 – 20 distinct competencies, of which only one or two might have been the focus of the coaching engagement.
Most significantly, it won’t capture useful data unless the original ratings anchors are replaced with a change scale. In place of the usual 5-point scale (e.g., from “Needs significant improvement” to “Significant strength”), asking whether there have been any improvements or positive changes since the last 360 is more meaningful.

Targeted “Mini” 360

An alternative option that many of our clients have adopted is a targeted or “mini” 360. This can be either done via a survey or, better yet, via 20-30-minute interviews with a limited number of stakeholders. In the interview format, the coach typically interviews the boss and three or four others to gauge the coachee’s development progress and provide additional suggestions and support. This approach has the benefit of being able to focus specifically on the areas the coachee has been working on, and it lays the groundwork for a supportive environment going forward after the coaching engagement has ended.

Here are some questions that we have found useful to ask in typical

One of the things that [coachee] committed to working on in this process was [action item]. What movement, if any, have you seen in this area?
Can you describe some examples of evidence that he/she has (or hasn’t) done something differently in this area?
What has been the business impact of this change?
What does he/she still need to work on in this area?
Do you have any additional suggestions that will help him/her be more effective?

Benefits of this approach – particularly the high-touch interview format – include:

It helps solidify stakeholder recognition of the coachee’s efforts at change and his/her commitment to supporting the coachee going forward
It looks both backward and forward in time, providing the coachee with practical future value
It gathers concrete, anecdotal evidence of coaching impact that can be aggregated by the coaching provider or manager
It promotes the understanding and value of coaching to prospective future coachees and program champions

Although the interviews do add time to the scope of the coaching engagement, coachees find it an immensely valuable element of the process as they transition out of their coaching engagement and take on full responsibility for maintaining their own developmental momentum.

Summary

Ultimately, whatever the approach taken, developmental progress assessments are important because they serve a number of purposes simultaneously:

They document what has been achieved by the parties involved
They demonstrate that others have noticed development and change and encourage the coachee’s continued commitment and effort
They re-engage key stakeholders as sources of support and constructive feedback and advice
They help guide the priorities and focus for individual development going forward