Sizing Up the National ‘Research Enterprise’
Posted at 5:02 p.m. on July 18, 2014
Is the U.S. measuring its “research enterprise” well enough?
At a Senate Commerce, Science and Transportation Committee hearing Thursday, Stephen Fienberg, a statistics and social science professor at Carnegie Mellon University, talked about some findings in a report the National Academy issued last month:
We found that current measures are inadequate to guide national decisions about what research investments will expand the benefits of science. Moreover, we noted that the U.S. lacks an institutionalized capacity for systematically evaluating the nation’s research enterprise taken as a whole and assessing its performance and developing policy options for federally-funded research.
Fienberg, who was on the National Academies committee who put together the report, said the country does well in measuring performance, like counting outputs, the number of papers scientists publish and patents, but when it comes to measuring how different research programs compare in terms of benefits to society, there was “much room for improvement”:
When we looked across the board at both what the U.S. measures now including the kinds of data gathered by the agency within NSF but more broadly and then we asked what other countries were doing and how they were trying to gauge their own research enterprises and we found an astonishing lack of tools for that kind of assessment. Especially at the program level where you want to ask how a program as a whole is doing as opposed to how an individual scientist or a subset of projects do. And that’s an area where we actually need to invest in new research if we’re gonna get those kinds of performance and evaluation measures in hand to help you and the policy makers in Congress actually be able to assess the kinds of trade offs.
Republican John Thune, R-S.D., asked: isn’t the White House’s Office and Science and Technology Policy well-positioned to doing the systematic research assessment that Feinberg said was lacking?
OSTP could take on that role, but right now, it doesn’t fit into it’s formal mandate and it’s already has much on its plate, Feinberg said. There needs to be an organization tasked with doing such work, and OSTP could be a place to do so, he said.
“Do you think that shortcoming in any way creates an impediment to… economic growth?” Thune asked.
Fienberg replied that he thinks “it has to.”
“If you want to know which of five programs, or which of five different allocations across programs is going to yield the greatest benefit and you have no data and no insights, its very hard to make policy choices on a rational as a opposed to a political, or other gut basis,” he said.