Subcommittee examines NSF merit review process

On Tuesday, July 26 the Research and Science Education Subcommittee held a hearing on the merit review process, primarily focusing on the National Science Foundation (NSF).  Chairman Mo Brooks (R-Alabama) was joined by Subcommittee Ranking Member Rep. Dan Lipinski (D-Ill), Rep. Larry Bucshon (R-Ind), Rep. Randy Hultgren (R-Ill), and freshman member Rep. Hansen Clarke (D- Mich). In his opening statement Rep. Brooks praised the merit review process, but commented that no human process is flawless and stated that this hearing would focus on the ways in which merit review could be improved, especially in the tight budget environment.  Testifying before the Subcommittee were Dr. Cora Marrett, Deputy Director of NSF, Dr. Keith Yamamoto, Vice Chancellor for Research at the University of California San Francisco, Dr. Nancy Jackson, President of the American Chemical Society, and Dr. Jorge José, Vice President for Research at Indiana University.

The NSF merit review process considers two criteria: “intellectual merit” and “broader impact.” In June the National Science Board (NSB) put forth several revisions to the criteria hoping to further “clarify their intent and how they are to be used in the review process.” The revisions will serve as the basis for a guide that will be used by PIs, reviewers, and NSF staff in the grant review process. The most significant change has been to the broader impact criteria, which now includes nine “national goals.” Projects will be required to demonstrate which national goals are addressed in the proposal and how it will advance them. The proposed changes to the broader impact criteria was one of the main topics of discussion at the merit review hearing.

In their introduction statements, all the witnesses focused on the strengths of the merit review process and its proven track record for selecting the best and most innovative science for funding. Several witnesses included in their testimony their concerns with the changes proposed to the NSF “broader impact” criteria. Dr. Yamamoto in particular was vocal in his opposition to the changes in the broader impact criteria stating that asking scientists to judge the effects of the research on national goals “departs from the scientific merit parameters” and is an “inappropriate consideration of the grant review process.”

During the question period, Rep. Hansen Clarke asked the panel how the success of research should be measured and how does the public know they’re getting a return on their investment in scientific research.  Dr. Marrett explained the difficulties in applying metrics to be able to quantify the broader impacts of scientific research and encouraged members to remember that an investment in science is a long term investment, where the results may not be evident for some time. Dr. Yamamoto agreed and sited early work done by Herbert Boyer, which may not have appeared to “address national goals,” but eventually led to recombinant DNA technology that was the basis for the entire biotech industry.

The issue of transformative research, research that creates new paradigms and often goes against current scientific thinking, was also discussed. Dr. Yamamoto stated that while transformative research is imperative to the advancement of scientific knowledge, it is markedly different from innovative research, which advances the understanding of current scientific thought. He suggested a different review process for transformative projects to ensure that these types of projects receive funding even in difficult economic times when experts tend to favor more conservative approaches. Dr. José also suggested a different review process for transformative projects, in order to give these sometimes “game changing” proposals a greater chance to receive funding.

When Rep. Brooks asked the witnesses for any additional changes they could propose to the merit review process, several suggestions were presented, though all the witnesses agreed that peer review was the best method. Dr. Jackson suggested that the possibility that “weak” proposals, that would clearly not be funded, could be triaged, freeing up time for more careful consideration of the many “very good to excellent” proposals. The issue of a double blind review process was also discussed, but had mixed support among the witnesses. Finally, the possibility of virtual study sections was presented as a way to reduce costs of the review process and maximize participation. Interestingly, studies that would determine the effectiveness of virtual meetings are being conducted by the Directorate for Social, Behavioral, and Economic Sciences (SBE) at NSF which has recently come under attack by Sen. Tom Coburn’s report The National Science Foundation: Under the Microscope.

Finally, Rep. Brooks asked the panel if there was data available that detailed the merit review process, such as a breakdown of the cost per review panel and the amount in dollars tied to each grant funded. The witnesses were also asked to submit a compilation of specific success stories to combat the bad publicity that can surround research on socially controversial topics or with obscure titles. Dr. Marrett said this data was available and could be distributed to the subcommittee. Rep. Lipinski also recommended another hearing in the future that could provide greater detail into the process of how grants are reviewed.

In these difficult economic times, congress is forced to make tough choices on where to make spending cuts. Unfortunately, basic research funding is an area that has come under considerable scrutiny. While it is encouraging that the Research and Science Education Subcommittee held this hearing on merit review to allow experts in the field the opportunity to weigh in on the discussion, only time will tell what the future of research funding will look like.