Obsessive Compulsive Metric Disorder

While my writings in the past have strongly pushed for the establishment of better metrics in quantifying social impact, the reliance on such metrics is not immune to negative outcomes. One such negative outcome is when social investors become so concerned with their own metrics that they lose sight of the larger picture — the common goal to provide as much of a difference as possible. The danger of this materialized for writer Kevin Jones of Xigi.net during a conversation he had with an investor at the Skoll World Forum.

The worst news from the Skoll World Forum was from another investor. They were trying to co-invest with a venture philanthropy fund, but found two significant barriers; one that fund does not co-invest, nor release its due diligence reports to even other like-minded institutional funders.

Worse was that this fund had made the social enterprise sign an exclusive deal; they would not take funding from another fund. The reason, it seems, is metrics run amok; they only way to make sure they can measure their impact is to try to restrict other impacts on the enterprises. So less good gets done, less growth of the mission and the company happens in the name of being able to accurately measure and report.

Metrics are important, but they are only as good as the issue they hope to measure. Once the metrics reach a point of control over the analysis where decisions are made solely to feed those metrics, it forms a vicious cycle where organizations become more obsessed with these abstract values than the focusing on actually making a difference.


2 Responses

  1. Makes me wonder whether the metrics was just an excuse for the venture fund, but if the real reason was a lack of willingness to collaborate and deal with other partnerships. Unfortunately, this often happens with the clash of egos in the social space, even with a common goal.

    Related is a wider, and almost ethical, debate with the idea of randomized controlled trials (RCTs), which try control variables and also silo out impacts. Places like the Poverty Action Lab conduct these experiments and analysis in developing countries to eventually have a policy influence. But just like the example posted here with the venture fund, is it better to have a clear impact analysis, or to be more inclusive and have a potentially robust and widespread impact? In the case of the venture fund, I would say that openness, transparency, and a willingness to collaborate would be a more important first step, because they are not conducting a rigorous statistical analysis – they are probably just reporting to their stakeholders. With the case of RCTs, I would have to agree that for a sound scientific analysis, not everything can be included. Another sticky, case-by-case situation.

  2. Shital,

    You bring up a good point with giving the actor the opportunity to elucidate their goals and judging accordingly. Unfortunately the article did not provide this information as the author left the fund anonymous. I would doubt, however, that the venture fund was created as a shell for the metrics.

    Instead I would guess that this fund may have been started by individuals trained in the conventional VC space that are now trying to apply identical investing and measurement practices in the social venture space. Although even that explanation is incomplete as co-investing is an established tool in the traditional VC field as well.

    What I find particularly interesting is that it would seem that the same complications and challenges associated with the creation of metrics for a fund would exist regardless of whether or not another investor is involved. The major hurdles here is how to measure the social impact of the investment itself, which would theoretically be the numerator. Infusing outside cash would seem to only affect one’s denominator that will help determine the per dollar impact.

    Funds, unlike trials, are looking for ROIs that depend on the ability to scale up their investments. Unless the fund believe’s that more money in this specific case would cannibalize the company in some way, more money should be encouraged — which is why this approach makes no sense unless it is employed on a case by case basis.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: