Sally Rockey reports on grant success rates at NIH

When it comes to grant review, it can be difficult to separate the facts from the hearsay. However, the director of extramural research at the National Institutes of Health, Sally Rockey, tried to do just that in a recent blog post. Rockey addresses the “urban myth” that grants reviewed by the Center for Scientific Review (CSR) are funded at a lower rate than those reviewed by an individual institute or center (I/C).

Rockey first points out that the vast majority of grants are in fact reviewed by the CSR, while only 17 percent are reviewed by individual I/Cs. She also stated that the types of grants reviewed at CSR and I/Cs are fairly different. In general, CSR reviews most R01, fellowship, and small-business applications, while I/Cs review most program-project, training-grant and career-development award applications. Rockey noted that I/Cs do review some R01 grants — typically the ones with I/C-specific features, as well as specific requests for applications.

On the surface, the legend does appear to be true: For fiscal 2010, CSR had a 17 percent success rate compared with a 25 percent success rate at the I/Cs. However, this doesn’t tell the whole story.

“That is really comparing apples to oranges,” Rocky insists. “A closer look shows that there is essentially no difference in your likelihood of getting funded when you compare the same types of applications.”

When one looks at RO1 applications in FY10, CSR had a 19 percent success rate while the I/Cs had a 18 percent success rate. The same is true when comparing success rate for RFAs reviewed at CSR vs. the I/Cs: 24 percent compared with 22 percent, respectively.

In late March, members of ASBMB’s Public Affairs Advisory Committee met with several  I/C directors, including Rockey, to discuss topics such as peer review, funding for investigator-initiated grants and the plans for NIH during these fiscally constrained times. The ASBMB PAAC and Office of Public Affairs will continue to engage with NIH leadership to ensure that the issues important to our members are addressed.

Study finds black researchers less likely to receive NIH funding

Last week, a study released in Science reported a disturbing gap in the funding success rates of black scientists. The study, commissioned by the National Institutes of Health, looked at 83,000 RO1 grant applications from 2000 to 2006 and compared the applicants’ self-identified ethnicity to the probability of receiving an award. The authors hypothesized that scientists with similar research records and affiliations would have a similar likelihood of receiving awards, regardless of ethnicity. However, the study found that black scientists were 10 percentage points less likely to receive NIH funding than white scientists. What was perhaps most striking, was that the significant disparity remained even when potentially confounding factors, such as educational background, training, previous research award success, and publication record, were accounted for.

The authors were unable to determine the exact cause of the disparity, but suggested that it might be the result of small advantages white scientists gain throughout their career. Unfortunately, the gap could also result from biases, however unconscious, toward white scientists. In an interview for NPR on Friday, Raynard Kington, former Deputy Director of NIH and one of the study’s authors, agreed that this data raises concerns over the impartiality of the peer review process stating, “if indeed we are biased in the way that we review some of our applications, that means that the American people’s money may not be going to the strongest scientific ideas.”

These findings are particularly disturbing to the administration at the NIH which has a long history of working to increase the diversity of the biomedical research workforce through the National Institute on Minority Health and Health Disparities and programs such as Research Centers on Minority Institutions, Minority Access to Research Careers, and Diversity Supplements. NIH Director Francis Collins was “deeply dismayed” by the findings of the study. He and Deputy Director Lawrence Tabak co-authored a response in Science detailing NIH efforts to increase diversity in the biomedical workforce and to identify the cause of the disparity. To provide additional suggestions and feedback about this issue, visit the NIH Feedback website. Additionally, Science will hold an online chat on August 25 at 3PM EDT to discuss the study and challenges facing minority and women researchers.

Coburn Report Targets National Science Foundation

In what has become an annual routine, Sen. Tom Coburn, R-Ok., released yet another report last week claiming to identify wasteful government spending, this time going after the National Science Foundation.

In “The National Science Foundation: Under the Microscope,” Coburn criticizes the NSF for “lack[ing] adequate oversight of its grant funding, which has led to mismanagement, fraud, and abuse and lack of knowledge regarding research outcomes.”  He goes on to list several examples of “questionable NSF projects,” a hackneyed method (also used in the infamous “YouCut” project) purporting to identify specious research projects merely from risqué-sounding titles. Coburn also called for the elimination of the Social, Behavioral and Economic Sciences Directorate, questioning whether “any of these social studies represent obvious national priorities.”

One of the more damning claims was that the NSF had not returned $1.7 billion of unspent funds.  A quick investigation revealed that the majority ($1.67 billion) of these funds had been allocated for multi-year grants, and would thus be duly spent as expected over the duration of the grant lifetime. In spite of his distaste for much of the work financed by NSF, Coburn resisted the urge to completely micro-manage the agency, claiming that “ultimately, the decision as to what constitutes ‘transformative’ or ‘potentially transformative’ [research] should be left to the scientific community rather than Congress.”

The report was immediately criticized by scientific organizations, as well as by several of the individual researchers whose work had been targeted in the report.  The NSF released its own statement in response:

“The National Science Foundation is renowned for its gold-standard approach to peer review of each of the more than 40,000 proposals it receives each year. The discoveries and innovations that have resulted from NSF-funded research have advanced the frontiers of science and engineering, improved Americans’ lives, and provided the foundations for countless new industries and jobs. While no agency is without flaws, NSF has been diligent about addressing concerns from members of Congress about workforce and grant management issues.  Indeed, NSF’s excellent record of tracking down waste and prosecuting wrongdoing is apparent from Sen. Coburn’s report, which notes that NSF has aggressively pursued cases of wrongdoing; terminated and even turned over for criminal prosecution employees found to have violated NSF rules or laws; and diligently collected and returned to the U.S. Treasury any fraudulently used funds. We believe that no other funding agency in the world comes close to NSF for giving taxpayers the best return on their investment.”

ASBMB is working with its partner organizations to craft a thorough response. Stay tuned to the Blotter for further updates..

Brouhaha Over NIH’s Revised Policy On Grant Application Resubmissions Grows

On her blog site, National Institutes of Health Deputy Director Sally Rockey last week released the contents of the agency’s response to a petition from Sloan Kettering scientist Robert Benezra that criticized a 2009 decision to lower the allowable number of NIH grant application resubmissions from two to one.

The controversy began in 2008, when the NIH Peer Review Self-Study Report was released.  The report observed that the success rate for initial grant applications (A0 grants) had fallen drastically over the past decade, while the rates for applications resubmitted once (A1 grants) or twice (A2 grants) had risen commensurately. The report suggested that eliminating the A2 grant would remove this funding “queue” and consequently fund more A0 grants.  Following the report’s release, the NIH announced that it was “sunsetting” the A2 grant in January of 2009.

In his letter, Benezra stated that the policy would have a “devastating effect on the biomedical research efforts in this country.”  His main point of contention centered on the inability of the peer review process to adequately fund every meritorious proposal, arguing that reviewers of A0 grants “cannot distinguish a 20th percentile grant from a 5th percentile grant,” and that the system should therefore allow for the resubmission process to fund those proposals that missed the A0 and A1 payline cutoffs.  Benezra added that the policy would “have the consequence in the current funding climate of redirecting the efforts of many of our very best scientists on the basis of what will essentially be an arbitrary criterion,” while simultaneously having a “disproportionately negative impact” on junior faculty.

Rockey responded by pointing out that the decision had produced the desired effect of raising the A0 success rate, which in 2010 climbed above 14 percent for R01-equivalent grants for the first time since 2005.  Rockey also pointed out that young investigators had not been harmed, with the number of awards made to new investigators increasing since 2008.

Benezra had initially circulated his proposal to just a handful of colleagues; however, the email soon went viral and made its way through the scientific community, eventually gathering over 2000 signatures in support.  However, the blogosphere was not unified behind the statement, with some applauding the NIH’s “wise decision” to sunset the A2 grant.  Benezra has given an initial response to the NIH’s letter, and is said to be pondering his next move.

McCain and Coburn single out science as “waste”

In a report released Dec. 9, Sen. John McCain, R-Ariz., and Sen. Tom Coburn, R-Okla., identify 100 projects funded by the stimulus package that they claim are wasteful government spending. Among these projects are 14 research grants funded by the National Science Foundation and the National Institutes of Health.

McCain and Coburn’s report singles out 10 studies from the NSF. Specifically, the report criticizes the use of stimulus funds to study the learning patterns of honey bees and to support educational programs that expose undergraduates to rainforest research.

In need of reviewers, NIH uses ASBMB members for panels

The National Institutes of Health has asked ASBMB to help identify highly qualified, volunteer scientists to help review grant applications. In response, ASBMB is renewing its call, asking members to volunteer to serve on the NIH’s peer-review panels, known as study sections.

“Study sections are where scientific priorities get set,” said Gregory Petsko, professor of biochemistry at Brandeis University and president of ASBMB. A vibrant peer-review process, full of capable and qualified reviewers, is vital to the health of scientific research, he said.

“If good people, people who know good science when they see it…don’t volunteer, then the job of reviewing will be done by the inexperienced, the inept, and/or the inbred,” Petsko said.

The Big Bad Wolves

There has been a lot of congressional huffing and puffing recently challenging the validity of peer review. Some members of Congress are taking controversial stands, claiming that certain types of peer-reviewed, scientific research are a waste of taxpayer dollars.

U.S. Rep. Darrell Issa, R-Calif., added an amendment to NIH funding legislation that would defund three specific NIH peer-reviewed grants related to HIV/AIDS research. Particularly worrisome is that the amendment passed the House. While Issa’s amendment most likely will be removed in final versions of the bill funding NIH in 2010 (Labor/HHS/education appropriations), it is startling that such an amendment would be approved on the House floor.