Measuring up to the data

I was pretty excited when I saw the title of the recently released joint BDO and The NonProfit Times report, “Nonprofit Standards, A Benchmarking Survey.”   While I was a little surprised that these two organizations would have been the ones to define a set of nonprofit standards, I quickly understood why, and the disappointment just rained down.  Turns out this is a study of the exceptions, not the rule.  So, why am I even wasting my words on this?  Because it says so much about how those who are supposed to serve the sector think about the sector, what they understand it, and who they wish to serve.

This is presumably the first iteration (hence, the benchmarking) of what will be an episodic, on-going survey.  These standards skew heavily in favor of financial items which, in and of itself, is not a bad thing at all, nor a surprise, given BDO’s involvement.  But it is not the only skew in this research, and it these latter issues that raise the question of just how useful this benchmarking survey really is?

The report quotes the co-leaders of BDO’s Nonprofit and Education Practice as saying that this first run “addresses a persistent question among nonprofit leaders:  ‘How does my organization measure up?’” Right up front, the report acknowledges that there are 1.5 million registered nonprofits in the United States, yet this study includes only 105 organizations—that’s .007% of the sector.  (This is why it is so important for everyone to understand how to assess research and not derive credibility simply because of the source.)

What can we learn about a population based on .007% of its members?  There is no way that a mere 105 organizations can be representative of 1.5 million nonprofits.  The mere fact that the sample includes only large nonprofits confirms, at least on this one variable, the lack of sample representativeness.  Of the 105 organizations in the study, 38% of the organizations had annual revenue between $10 million and $25 million; this study refers to them as “small” nonprofits!  The remaining members of the sample had annual revenue of $25 million or more.  According to the Urban Institute, nonprofits with annual revenue of $10 million or more make up only 3.3% of all registered nonprofits that reported any amount of income on their most recent 990s.   I’m okay if you want to study 3.3% of a population; but then don’t for one minute (or one title) suggest that these are standards for all 1,581,445 nonprofits.

I understand that BDO considers those of us with annual income of less than $10 million as not worth an investment of their time and energy.  I can even appreciate that, as we all pick our markets.  But I don’t understand the position of The NonProfit Times.  In this sad era where the existence of independent media seems to be deeply questioned, it would appear that The NonProfit Times showed some of its colors in attaching itself to this project.  It is certainly possible that next week, or next month, The NonProfit Times will come out with another partner and another benchmarking study that will be for the rest of us, but I somehow doubt that.

But I wouldn’t applaud that either.  Annual revenue is but one variable that may influence a nonprofit’s worldview or how it operates, or the challenges it faces.  Wouldn’t it be much better to have a true benchmarking survey that allows us to understand what is going on across the sector, where there are commonalities and where there are none?  Nonprofits spend so much energy and time claiming they are unique and special when, all the while, those of us who work across the sector know that is rarely the case.

One of the standards that this survey postulates is that every nonprofit should have adequate operating reserves—defined as “liquid unrestricted assets.”  But we know that having such assets that would sustain an organization ideally for a year or even half a year let alone a month, is a stretch that many nonprofits think they simply cannot afford or attain.  Yet, it would appear that achieving this goal may be a struggle for all nonprofits, regardless of budget size.  Only 41% of the “small” nonprofits (remember, those are the ones with budgets between $10 million and $25 million) and 40% of the larger ones had one to less than six months of such assets; and only 30% and 26%, respectively, had six to less than 12 months in reserve.

Many of the same concerns expressed by our clients who come from across the spectrum of size, mission, age, etc., are of equal concern to the nonprofits featured in this study.  Worried about staff recruitment and retention?  So are 72% of those in this study, with only a 3% difference between the large and “small” organizations.   Anxious about cutbacks and reduction in funding?  The big guys are, too, with the biggest (incomes of $25 million or more) more worried (52%) than their “smaller” brethren (41%).  And if you are trying to diversify your funding to bring in more individual donors, these larger organizations are working on that, too.  Individual donations represent just 1/5 of the income, give or take, of the “smaller” organizations (take) and larger (give) organizations.

One interesting place where the two groups of organizations in this study diverge is in their worry over attracting paid and volunteer leadership.   More than twice as many of the “smaller” organizations (49%) see a challenge in finding good caliber staff and board members than do those organizations with budgets over $25 million (24%).  In this regard, the “smaller” organizations may be more aligned with their truly smaller colleagues in the rest of the sector, who often feel board members aren’t interested in them because they lack prestige, and senior staff could go elsewhere and earn more money.

No matter how I approach it, I cannot see how this benchmarking study can be of value to the vast majority of the sector.  And I can’t stop thinking about how much this study cost—dollars, time, energy, etc.—and how those resources could have been used for research that could have furthered our understanding on a larger plane.  But, that’s not how it works.  It is the donor’s call as to how her/his money is spent and the scope of the impact s/he wants to have.  Understanding that, nothing says I have to sit quietly by and allow others to see it as useful.