Benchmarks: Averages can be Mean

“There are three kinds of lies: lies, damned lies, and statistics.” - disputed attribution[i]

“It is the mark of a truly intelligent person to be moved by statistics.” - also disputed attribution[ii]

I just got back from the 2014 Nonprofit Technology Conference put on by NTEN in D.C. the second week of March. I was most excited to hear as much about results and data as I could, being a sucker for that sort of info. Luckily, there were more sessions purporting to be about results and data than I had time to attend. And I was able to attend three sessions that included a focus on newly released benchmarks for online giving. The basic, top-line takeaways from each of these sessions are shown in the table below.

"Industry Benchmarks" are (at least to me) always interesting and often useful for:

• convincing your boss (or CEO, or Board, etc.) to start investing, stop investing, or approve or disapprove of some new project

• giving you an idea of how "normal" your organization is

• giving you a lot of anxiety

But remember that averages can be mean…

In fact, by definition they are “mean” (the sum of all values divided by the total number of values). The "flaw of averages" according to Stanford management professor Sam Savage is that "plans based on average conditions are wrong on average." Or, as David Henemeier Hansson, creator of Ruby on Rails and founder of Basecamp puts it, "The problem with averages is that they tell you nothing about the actual incidents and often give you a misleading big picture."

The issue with judging your success or failure only against the averages as presented by various industry benchmarks is that the benchmarks do not reflect the specific investment or lack of investment into growing Direct Response revenue, whether online or off. Nor do they reflect any changes in tactics that might have affected revenue growth or reduction of direct response revenue. Therefore, for an organization to be truly confident in their own growth numbers, they must track and analyze their own performance in tandem with the actual changes to strategy, tactics, and investment that affected that performance.

Simply put, organizations need to keep track of their own benchmarks over a period of years and use those numbers as KPIs for their programs.

As you can see from the table below, which shows actual data for four individual organizations with which we work, there is a significant difference between their organization-specific data and the data averages presented by the benchmarking reports.

The difference is in the details…

• The social service organization shown above was growing online direct response revenue by between 25% and 30% for the three fiscal years prior to 2011, primarily through renewal gifts made by donors already on their file. In 2012, this organization had reached a plateau in online giving, effectively tapping out the current donor file, and growth began to slow. At that point, the organization – having seen the percentage of Direct Response revenue represented by online giving grow from 5% in 2011 to 6.5% in 2013 – is now projecting it to be 7.4% for 2014. Therefore, they made a strategic decision to significantly increase investment in the acquisition of new online donors. This increased investment resulted in the 19% growth shown above while maintaining the ROI of the entire online direct response channel at $3.00.

• The health organization shown above has not significantly increased investment in either online or offline direct response acquisition for some time. Its offline direct mail and telemarketing revenue has remained flat for at least the past two fiscal years. However, an increase in organic subscribers, overall website visits, and a migration of offline donors toward making their annual gifts online – combined with a more integrated overall strategy and more scientific online segmenting than in previous years – has allowed the online direct response revenue to grow by 18%. This has resulted in online revenue becoming a very important piece of the direct response pie, at 39% of all direct response revenue for 2013.

• Investment in direct response fundraising by the international hunger relief organization shown above stayed flat between 2012 and 2013 (it was actually reduced by 3% between these two calendar years), as did direct investment in online donor acquisition. Just looking at the data in the table above, without any additional insight into tactical changes, it appears that offline donors to this organization have been migrating to become online donors. This accounts for the increase in online direct response revenue and for the large proportion of direct response revenue that can be attributed to online giving.

• The wildlife organization shown above was not investing in online direct response fundraising in any significant way before 2012. They ramped up investment in this channel significantly in 2013, creating an online fundraising program in coordination with brand new direct mail creative and tactics. At the same time, they began to invest significantly in online donor acquisition, similarly to what was done by the social services organization. These combined strategic changes, along with a celebrity-sponsored matching gift challenge, led to the incredible 72% growth in online revenue and the impressive overall fundraising growth of 33%.

I conclude with this reminder from my kinsman, Louis D. Brandeis:

“I like the individual case. A man may have six meals one day and none the next, making an average of three meals per day, but that is not a good way to live.”

share

tweet

forward

author

Carrie Miller