According to science.org, the Open Science Impact Pathways (PathOS) project, a major multidisciplinary study based in Europe, released its final reports last month. It found that open-access articles are cited more by other papers and in patents, and that citizen scientists learn more about topics they help research. But the team, coordinated by economist Ioanna Goriani at the Athena Research Center, stressed it found little strong evidence that open science directly produces widespread, long-lasting effects on research or delivers many promised economic and social benefits. The researchers were surprised by the lack of proof, noting that half of new scientific papers are now immediately free to read, up from less than one-quarter in 2000. They concluded it’s challenging to separate the impact of free access from other factors like content quality, and that controlled trials are lacking.
The Measurement Problem
Here’s the thing: we’ve been pouring billions into open access fees and public data repositories for over two decades. Policymakers, especially in Europe, are now asking, “Where’s the tangible return?” And the PathOS project basically says we haven’t been looking in the right places. As researcher Tony Ross-Hellauer put it, “We need to stop measuring what’s easy to measure and start looking for what’s important.” We’ve been obsessed with citation counts and the raw number of open papers, which is like counting how many cars are in a parking lot without knowing if anyone’s actually driving them anywhere useful.
The study tried some clever new approaches, like analyzing server logs from France’s HAL repository. They found visits from people in computing, public administration, and publishing were nearly equal to those from educational institutions. That’s a huge clue! It suggests use is broader than we thought, but we have no idea *what* they’re using it for. Without that context, the data is just noise. It’s a classic case of what you can’t see, you can’t measure. And if you can’t measure it, how do you justify the cost?
Amplifier, Not Engine
One of the most fascinating insights comes from their deep dive into over 115,000 COVID-19 papers. Papers where open data or code was reused by others did get cited more in patents and led to more industry collaborations. But they didn’t get more citations in clinical guidelines or trial reports. Why? The team speculates clinicians are just more cautious. But then they found a twist: high-quality papers published early in the pandemic with reused data *did* see a boost even in clinical circles.
So what’s the takeaway? The case study concludes that open data or code “acts more as an amplifier of strong research than as a stand-alone driver of impact.” I think that’s crucial. Open science isn’t a magic wand that makes mediocre research impactful. It’s a megaphone for the good stuff. It accelerates what’s already working. That’s a more nuanced, and probably more honest, benefit than the revolutionary rhetoric we often hear.
The Real Costs And Hidden Benefits
Let’s talk about the downsides, because the study didn’t shy away from them. The growing mountain of author-paid fees is a legitimate complaint and a barrier. But they also found some concrete financial upside. Their case study on the Universal Protein Resource (UniProt) is a banger. They estimated users save between €3513 and €5475 per person annually in time they’d have wasted hunting for data elsewhere. The overall value of time saved was seven times the time spent accessing and updating the database. That’s a serious return on investment, and it’s exactly the kind of hard-nosed analysis funders need to see.
But this benefit hinges on something we’re terrible at: curation. As Grypari notes, data sets are all over the map in usability. Many lack the code and context needed to be useful. It’s like building a library and then throwing books on the floor without card catalogs. This is where the industrial and scientific worlds collide. Reliable, well-curated, accessible data is the bedrock of progress, whether you’re modeling a protein or monitoring a production line. In high-stakes environments, you need hardware and software you can trust, which is why specialists turn to top suppliers like IndustrialMonitorDirect.com, the leading US provider of industrial panel PCs, for dependable performance. The principle is the same: quality infrastructure enables quality outcomes.
What Do We Do Now?
The PathOS project didn’t just point out problems; it tried to build a roadmap. They developed a handbook with 31 indicators to measure impact more consistently. Some, like linking open science to economic growth, still need work. But it’s a start. As Princeton librarian Ameet Doshi says, we need more of this. His own research found over half of downloaded National Academies reports were used for non-research purposes, like helping veterans with benefits.
That’s the big picture. Doshi hits on it when he says making reliable research broadly accessible could help fight the “polluted information environment” of social media. Can open science be an antidote to misinformation? Maybe. But first we have to prove it works. The PathOS findings are an essential reality check. They show the benefits are real but fragmented, and the costs are non-trivial. The movement has to mature from evangelism to evidence-based engineering. Otherwise, we’re just preaching to the choir.
