I recently served on an NSF panel, and although I’m apparently not supposed to share the details of which panel and when, It seems like a good idea to share my experiences. I left feeling strangely comforted. Yes- funding rates are 5%, and this means that some very good science gets left unfunded.. But the process is fair, and for me I think there is less stochasticity  in the process that I feared (though I know a few of my panelists would disagree with this assertion). You have great idea, you design appropriate experiments, have a significant amount of preliminary data, and have excellent grantsmanship skills and you have a pretty good chance.. LOL.. That seems totally doable.

OK, the process: First, this is a ton of work. I reviewed 15 proposals. I was assigned 4 that I was lead on, and 11 that I was secondary. I spent at least 1-3 hours in the week(s) leading up to the panel meeting with each proposal, though a few were obviously noncompetitive, and took less time.   At the panel meeting, I synthesized all the reviews and came up with a consensus. This was reasonably challenging, as several proposals had both people that loved and hated it. My job as the primary reviewer was to decide where I thought the proposal should go (High, medium, low priority, or noncompetitive), then present this to the co-reviewers. I think it is at this stage where there is a potential for luck to be involved. You get lucky and have a lead reviewer that really loves the proposal— YES! This is not going to move a noncompetitive proposal to high priority, but from non to low.. from low to med, or med. to high.. this seems really possible and can make a HUGE difference.

I believe that serving on a panel as a young investigator REALLY will help my chances in landing a grand. Reading a bunch of awesome grants (and a few bad ones) really helped me to identify writing strategies that work. Further, working with the POs has to help. I’m no longer an anonymous PI, and this has to be important when they make tough funding decisions. One of the most important things I learned from this panel is how important the POs are- while the panel makes recommendations, the POs make the ultimate decisions and can choose to fund anything, even those placed in noncompetitive.

A few specific points:

Data Management Plan: At the beginning, the PO’s said that it was time to take this review criterion ‘seriously’. I certainly did, and mentioned in my summary when data were not being shared openly. Still, I don’t think DM plan had much impact on the overall placement. It seems like a bad DM plan would never sink a proposal.

Broader Impacts: There is surprisingly quite a lot of variation in the BI of proposals. It seems that there are a group of proposal that incorporate very ordinary BIs.. and another group that has really novel BIs. Like above, while these were evaluated, I didn’t see a case where a wonderful scientific proposal was sunk by poor BIs.

FWIW, the best BIs I read included some education outreach. The panel (or maybe it as just me) seemed to be especially interested in those involving younger children. I personally liked it when BIs included local outreach. You study in some amazing place? Why not teach local citizens about it. This resonated with me and several of the other panel members.

Granstmanship: Insanely important.. I read a lot of proposals, and when your proposal was 11pt font, less-than-single-spaced with absolutely no whitespace..  I probably had trouble reading through your proposal. For me, I did not print proposals- so color figures, and hyperlinked refs really worked for me. Lastly, apparently almost nobody can make a clear (non-blurry)  figure embedded in their document, so when a crisp figure (esp. in color) appeared, it really stood out.

Ok…..This post is getting long, and there is a lot more I want to say.. I’ll leave that for a another post. If people have specific questions, I’d be happy to try and answer.