Grant reports are broken. Is it worth trying to fix them?

Back when I was a grantee in the 2000’s, my least enjoyable responsibility was writing reports to our funders explaining what we had done and why they should keep funding us. In my experience as a funder at four different foundations throughout the 2010’s, my least enjoyable responsibility is reading those same reports. Why is grant reporting such a broken, painful process for all parties involved? And can we do anything to make it less so? Dennis Whittle, the CEO and co-founder of FeedbackLabs, and I discuss:

DS: Dennis, earlier in my career, when I was on the grantee side of the table, I had to prepare reports for funders to get my next payment. I hated writing those reports and rarely received any feedback from my funders. I felt like I had to justify the previous year of my life, but rarely did I receive any response. Now I’m on the funder side of the table and I struggle reading the reports. Honestly, they’re pretty boring. Typically, they’re long lists of all activities from the previous year, a few complaints about being over-stretched and under-funded, a few more complaints about the general state of the world, and then an optimistic portrayal of what the grantee expects to accomplish next year and why their work is now more important than ever. But rarely do I learn anything from these reports that I didn’t already know. And I can’t think of a single time when I’ve received actionable feedback from a grantee that helped me do my job better. Earlier in your career you were on the funders’ side of the table, and now you’re a grantee. What has your experience been? Before we start discussing solutions, what do you think are roots of the dysfunctional reporting process we have?

DW: David, in some ways my career has been the reverse of yours: For the first half I was a funder, and for the last half I have been mostly a grantee. But as I think back, I realize that even when I was a funder I had to write reports to my management and board, and those reports often had the same structure and style as the grantee reports you’re describing. That indicates to me that there is a “meta” logic at work. I look forward to digging into that logic with you in this conversation. As a prelude, let me try to boil down almost every single report I have read and written over the last thirty years into one sentence: “Progress has been made, but challenges remain.” Does that resonate with you?

DS: Ha, yes, that does sum up just about every report I’ve read — and those I’ve written myself. I wouldn’t expect it any other way. As Eric Liu is fond of saying, democracy is a garden and the gardener’s work is never finished. Rather, you learn how to be a better gardener over time. You experiment with new plants and techniques — and you learn to adapt to new conditions. But with these grant reports, I’m not learning anything about the metaphorical gardening or how the gardeners are adapting to the changing conditions of politics, social movements and new technologies. Instead, I just get long lists of which seeds they planted and when. It occurs to me that there is a parallel with journal writing — a practice I’ve kept for the past 20 years. Some days I’m tired and I don’t have the energy to write anything more than a list of what I did that day. Other days, however, I’m more introspective and I analyze how my interests, friendships, and work have evolved over time. Some days I question decisions I made in the past, or I write out the pros and cons of decisions I need to make for the future. And this is what I find is missing from most grant reports, the reflection. Why do you think that is? Do you have any suggestions for how to encourage grantees to be more reflective in their reports?

DW:I think there are many reasons. One of of the most important is described in Lant Pritchett’s 2002 paper called “It Pays to Be Ignorant: A Simple Political Economy of Rigorous Program Evaluation.” The main takeaway is that there is an asymmetry between the risks and rewards of being candid. Essentially, aid, philanthropy, and government systems “assume” that all projects should be successful. Success is the “baseline;” if the evaluation says the project worked well, the people working on the project don’t get rewarded with more resources or recognition. However, if the evaluation says the project failed, then the system often penalizes or punishes those in charge. The result is a huge disincentive to candor, and hence to learning. It’s too risky for the grantees and implementers! This is compounded by attribution problems, the absence of systematic perceptual feedback from the people we seek to serve (the “customers or “clients’), and the dearth of relevant quantitative data.

DS: I agree. I think a lack of basic trust is at the root of reporting dysfunction. Just like in any relationship that lacks trust, it’s safer to be polite and vague than direct and honest. My friend Glenn Fajardo gave me some useful advice about what I can do to improve the trust in my relationship with grantees, but I still find it challenging. Let me come back to the issue of trust after ranting a bit about the quality of writing in the reports I receive. We fund some of the brightest, most eloquent individuals I’ve ever met and yet there’s something about a grant report that inspires truly terrible writing. Here’s a typical excerpt from a recent report:

• “One fundamental hypothesis in which the project relies is that the more usable, accessible, timely and open public information is, the better it can help organized -and non-organized- civil society, journalists, researchers, etc. to track how public resources are used, how the actions that come from the public sector actually impact a community, and to detect what can be improved.”

Again, these are brilliant folks that otherwise speak and write compellingly about the importance and impact of their work — with illustrative stories about the individuals who have benefitted. But then when it’s time to ‘report’, all clarity and brevity are sucked off the page and I’m left with repetitive jargon and abstraction. Why is that? And how can I encourage writing that is clearer, less abstract, and more concise?

Here’s one thought: What if in lieu of the standard annual grant report, I write them an email a month or two before the reporting deadline with a list of 5–7 questions to respond to as if they were writing me an introspective letter at the end of the year? Or here’s another idea: what if we give awards to the two or three grantees that write the clearest, most informative and most entertaining reports each year? Finally, do you have any ideas on how to increase trust and change incentives so that, returning to Lant Prichett’s paper, “it pays to be honest and reflective” rather than ignorant?

DW: David, your exchange with Glenn in that post is refreshingly candid as well as insightful. As I reflect on our conversation, I wonder if progress will require us to re-focus the reporting process on Learning (instead of knowing in advance) and Relationships (instead of transactions). Those would both require and maybe even generate a focus on Trust (instead of verification).

Before I turn to those, let me affirm that the grant report excerpt you cite is appalling. And I would be the first to ridicule it if I had not written many passages like it myself. I am looking at a report I wrote at the World Bank in 1999, and almost every phrase from your excerpt appears — albeit in a different order — in my report from 20 years ago! The whole thing reminds me of this paper — accepted into a professional journal — that was generated by computer using stock phrases. I am afraid that given the deep roots and persistence of this problem that I have no easy answers, but here are some thoughts:

• First, philanthropy is about trying to influence systems that are highly complex and emergent. The effect of any intervention is often unknowable in advance. Further, there is often no discernible positive effect of any specific intervention. Lots of times there are unanticipated effects. Sometimes there are unintended negative effects. So, progress requires steady, careful iteration, with thoughtful analysis and judgment to guide the next iteration. The premium should be on demonstrating how well an organization can learn and adapt, since that is what maximizes the chance of progress over time. At GlobalGiving, we developed a crude but effective way to measure this and provide incentives (more opportunities to raise money) to encourage constant discovery and improvement.

Given the desire to change systems and the inherent complexity involved, think about how ill-suited our approach to grant-making and reporting is: potential grantees get rewarded for pitching compelling strategies based on some “Eureka” methodology, tool, or insight; extra points when that pitch is delivered by a charismatic leader who everyone wants to believe in. Given that ex-ante set-up compared to the messiness of the real world, it’s no surprise that obfuscation is the rule rather than the exception when it comes to reporting!

• Second, in a complex emergent environment where organizations need to constantly iterate and adapt, relationships begin to play a much greater role. Social change is sharply different from civil engineering. When a government asks companies to bid on a project to “Build a bridge from here to there, with these specification, within X months,” the main bidders have well established business partner and subcontractor arrangements that allow them to create detailed bids rapidly. Given this, it makes sense to hold them very accountable for the quality, speed, and cost of the construction.

But in the social change world, we don’t know ex-ante how to get “from here to there,” much less how much it will cost or how long it will take. And, crucially, we don’t know who our partners will need to be along the way.” As we iterate, we need rapidly to find new partners that are not only competent but that we can trust. So, organizations that can quickly find and partner with new allies are the ones who — on average over time — are likely to make the most progress.

Critically, these partners will sometimes be foundations themselves, who, in addition to providing money, are also needed to provide strategic support in terms of modeling behavior, funding complementary efforts, and jawboning. Progressive foundations understandably are pulling back from the old top-down style of grant making where they basically tell grantees what to do. Given the huge power imbalances inherent in the grantor-grantee relationship, this is admirable. But sometimes I think that has gone too far, with foundation officers telling grantees, in effect, “Good luck. We are going to keep hands off and wish you all the best!” Yet shouldn’t the opposite be true- at least sometimes? Shouldn’t funders say “We are all in this together. We are committed to this change and we are in it for the long haul. In addition to money, we commit the brainpower and strategic influence of our officers as allies and partners (not bosses).” Maybe that is a crazy and even dangerous idea, but in my current work at Feedback Labs I feel we can’t succeed in changing norms and practices on a broad scale without a full team effort.

So, I guess that brings us back to the subject of final reports. I reluctantly conclude that bureaucracies will always require them, and that we should not be naive about abolishing them altogether. But they will likely never be efficient vessels for conveying information or learning. For those, I wish we could find a way to increase by half the amount of time that program officers and grantees spend together hashing out how things are going, what they are learning (together), and ideas for iterating. I think program officers would find this more rewarding than the constant staff retreats, strategy refreshes, and internal reporting they are caught up in. If we could find a way to do that with an attitude of genuine inquiry (as opposed to evaluation), that would go a long way toward accelerating learning and building the relationships and trust need to enhance the odds our programs will succeed in improving the world.

There are no silver bullets, but here is one idea that might help a bit. What if funders gave “bonuses” for reports that are clear: “Dear ____, Thanks for the report. It vividly summarizes what’s happened over the past year and brings alive all the conversations we’ve had with you as we explore together how to bring about [X] change in the world. Most important, it provides a good foundation of insight, evidence, and judgment for the refined strategy you plan for the coming year. With your permission, we would like to disseminate this internally to our staff, some of our foundation colleagues, and to relevant grantees. And in recognition of your commitment to learning but to candor (so others might learn from your experience), please find enclosed a check for $50,000 of general operating support. We have no doubt you will put this to good use. Look forward to our next in-person conversation. Yours Truly, Program Officer.”

DS: I love that idea and will bring it up with my colleagues. I also appreciate your point about the risk that funders become too detached out of a well-intentioned desire to defer to grantees’ decision-making. It reminds me of an essay written by Gabe Kleinman, a former grantee who now works at a venture capital firm. He points out that for-profit investors have a built-in incentive for their investees to succeed and often go to great lengths to broker helpful relationships, but rarely do program officers go out of our way to help grantees succeed. I think it’s a fair indictment and something I’d like to get better at over time. As Courtney Martin points out in her comment on Gabe’s piece, program officers have to develop our so-called “soft skills” to have any shot at a “deep, respectful relationship with grantees rather than the appearance or performance of such a relationship.” She concludes: “Being a good funder might be as hard as being a good spouse or friend, but why would we expect anything else?”

Dennis, I’m grateful for this discussion. I think its impetus was likely my need for some therapeutic ranting, but it’s become much more actionable and constructive thanks to your ideas and suggestions.

DW: Happy to rant, brainstorm, and re-invent with you anytime, David!