Measuring and Managing Impact

On July 6th, Frontier Incubators hosted their third event in a series of webinars, which aim to connect leaders from global accelerator and incubator programs with program managers who are seeking to serve entrepreneurs in the Asia-Pacific. This webinar explored the importance of measuring impact, what is captured in measurement processes, how to express value, and the need for specific indicators to reflect the impact of different incubators and innovators. Read on below for key strategies and approaches to understanding incubator and accelerator impact from Fledge, Uncharted, Akina, Kinara, Conveners and YGAP.

Incubators

11.07.2018

Five key strategies and approaches for measuring and managing impact

The session opened with an introduction from each expert team explaining how they measure the impact of their programs. This discussion provided a framework for the following questions, and here we provide a brief overview of how these global leaders in incubation and acceleration express the value and impact of their programs.

1.Examples of impact

For Luni Libes from Fledge, the United Nations Sustainable Development Goals (SDGs) are employed as a model by which to classify their companies, and the specific SDG with which they align. Fledge also reports on “what percentage [of the enterprises] are run by women and have a woman on the founding team, and what percentage are targeting women’s rights.” Luni explained that a third layer of metrics is added using The Pinchot Impact Index, which consolidates ‘impact’ in a single numeric expression.

Manita Ray from YGAP explained that they are currently revising the way the organisation measures impact. YGAP have been using IRIS metrics to assess impact across four pillars of improving health, access to health, education and employment — but Manita explained “the impact of employment is not the same as one who is providing sanitary products to your girls attending school — we currently report the whole number of lives improved.”

Akina Foundation’s Guy Redding and Clemmie Baker explained that their approach incorporated both qualitative and quantitative assessment models. They explained that for their ‘Launch Pad’ accelerator, the focus was on “providing employment training opportunities” and measurement of impact included examining the number of jobs created, and “the number of people those companies were proving employment training to — for people most at risk.” The impact assessment also included the value of contracts, funding, investment generated, and satisfaction of the project.

Fajar Anugerah, from Kinara Accelerator offered insights into working with GALI to measure impact. Kinara have incorporated the GALI questionnaire into their selection process so they are able to “monitor companies in our program as well as those who were not accepted.” For Kinara, Fajar described the key measures of impact as:

  1. Whether the company secures investment 1 year after
  2. Whether the team has grown in size
  3. Number of beneficiaries
  4. Growth of revenue

The GALI approach was similarly employed by Uncharted. Banks Benitez outlined how Uncharted measure their impact by assessing “funding growth, revenue growth, we also compare those to control groups — we look at those who were not selected through GALI as well.” Banks added that efficacy was another key measure, as “$1 that funds Uncharted leads to $21 that one of our companies will receive in follow on funding.”

2. Measuring ongoing impact

While discussing resources and research about impact of accelerators on participating companies, Guy highlighted the importance of ongoing engagement and post-accelerating support being included in impact measurement. He questioned, “How do you provide support by continuing to provide networking opportunities and connections and enabling them to access the finance that helps them grow?”

GALI’s framework is critical here for measuring ongoing impact. Their survey is distributed to both cohort alumni and to those who applied but were not accepted into the program to be completed each year after the end of the program. This provides data overtime to show the impact the accelerator had on the cohort participants. One risk that GALI highlights is that it may be that accelerators are just very good at picking winners. As their database grows they hope to show the difference between companies that go through an accelerator compared to those who were accepted but chose not to participate. This would be the ideal control for gauging the true impact of the program on the performance of its’ cohort companies.

3. Transformative impact

The central questions guiding this webinar were:

  • Who do you measure impact for?
  • How do the different audiences influence what gets measured and reported?

In this discussion, Maita highlighted the importance of communication between donors and entrepreneurs, and the question of “are we measuring what we say we are measuring. It is not easy and the big shifts we are having is not being influenced by just what the donors and requesting.” Clemmie similarly outlined the difficulties in capturing and expressing value which mediates between a “top down approach from government an a bottom up community approach.”

In attempting to develop meaningful measures, Banks described the ‘Lives Impacted’ model and Uncharted’s current ’30 Million Lives Impacted’ statistics based upon the their ventures. In this discussion, Banks outlined how ‘Lives Impacted’ fails to “speak to the transformation” and is more of a “vanity metric” than communicating significant information about value. Banks explained “we are looking into lives impacted on a per issue basis — rather than trying to compare clean energy to health to education.”

4. The role of specific measures which best reflect the value of a project

Fajar explained that Kinara conduct a yearly check in, employing three questions

  1. Revenue coming in
  2. Communities the are working with
  3. Company specific metric

Fajar highlighted the importance of adapting measurement methods and questions to cater to the needs of each organisation — “we discuss with them what would be the specific indicator for them and then that is discussed throughout the program — so they can change it based on the strength of the company.” Guy echoed, ‘we might move away from those broad metrics to take the case study approach against the SDG’s.”

This model allows for participants to contribute to modeling their own measurement criteria, as Fajar explained ‘some of the participants themselves may not know what types of impact indicators they can or should use.” Facilitating access to experts in the field for participants can help in shaping appropriate measurement methods.

Conversely, while specific measures are important, Luni outlined the significance of establishing base level indicators. Luni explained “our first KPI is how many companies are still running?”

5. What is the value of measurement?

The webinar concluded with a discussion about the importance of measurement. Clemmie outlined the value of measurement in order to illustrate the efficacy of projects, as partners are “interested in our capacity to report on impact and using us as a case study to increase their interest in commissioning for outcomes.”

Luni explained that for impact investors, measurement is key, as “at a conference the first question is “what is impact” the second is “how do you measure it” — having a report that shows multiple measures. At the end investors really want the stories of the companies they’ve invested in.”

Manita highlighted the value of conveying impact, as “our donors get excited when we talk about how we measure impact. The donors get excited when we talk about how we measure impact. The donors want to be educated as well.” She explained the importance of using innovative way to communicate impact, “we do case studies with really simple language and a picture of an entrepreneur and showing their metrics.”

Banks also spoke about the importance of communicating tangible stories of impact. He explained “alumni speak to very different value speaking about the peer network and the depth and quality of the relationships, Alumni say they will get access to a brilliant community.” Banks highlighted the role of storytelling and personal narratives for communication, as “the best sales people for us are the alumni who speak to the softer metrics that carry strong social capital.”

6. Resources Cheat Sheet