The Decline of Scientific Progress: Analyzing the Slowdown in Innovation

The State of Scientific Progress

Science appears to be thriving at first glance. The number of scientists worldwide has increased rapidly, both in absolute numbers and as a share of the population. Scientific publications are growing exponentially in almost all fields. For example, physics literature doubles approximately every 18 years, while fields like electrical engineering see even faster growth, doubling about every 9 years.

However, a closer examination reveals a concerning trend: the effort put into science is yielding fewer and fewer results. This phenomenon is evident across various metrics and disciplines.

Economic Impact and Productivity

Economists measure productivity by comparing inputs (labor, capital, and technologies) with outputs (goods and services). This total factor productivity helps quantify how much research and development contribute to economic growth.

A 2016 study titled "Are Ideas Getting Harder to Find?" revealed that despite a steep increase in the number of researchers, growth in factor productivity in the United States has remained relatively constant for decades. This suggests that while we're investing more resources into research, it's barely maintaining stable growth.

The study also examined specific sectors:

• Agricultural yields for certain crops showed rapidly increasing numbers of researchers but stagnant results
• The number of new drugs approved by the FDA has not kept pace with the growing number of researchers in the pharmaceutical industry

Patent Novelty and Innovation

Research on US patents shows a decline in innovation:

• Patent novelty, measured by the number of new connections a patent creates between technology sectors, has been decreasing since the 1960s
• The narrowness of inventions has increased over time, indicating less broad-reaching innovation

Research Productivity and Disruptive Ideas

A 2022 paper measured research productivity by the number of top researchers per community, finding a steep decrease since the 1960s.

A study published in Nature analyzed 45 million papers worldwide, quantifying the "disruptiveness" of a paper by how many earlier papers it made redundant. The findings showed a noticeable decline in disruptive ideas across many scientific areas, including:

• Physical sciences
• Chemistry
• Computer science
• Biomedicine

This trend was observed even in top journals such as Nature and Science.

Historical Perspective on Innovation Decline

The slowing pace of innovation isn't a new observation. In 2005, Jonathan Huebner claimed that the rate of innovation (defined as the number of technological events divided by population) has been declining globally since industrialization began.

While Huebner's extrapolation suggesting no technological progress by the end of the century may be extreme, the overall trend of slowing innovation is clear across multiple studies and metrics.

Apparent Contradictions: Rapid Technological Change

Some may argue that the world is changing rapidly, citing advancements in:

• Artificial intelligence
• Quantum computing
• Nuclear fusion
• Autonomous vehicles
• Brain implants

However, these technologies are largely based on breakthrough discoveries made decades ago. Their current development represents the slow process of turning old breakthroughs into practical applications, often requiring enormous investments and time. This slow pace and high cost actually illustrate the problem rather than contradict it.

Possible Causes of Scientific Slowdown

Funding Changes

One might assume that the change in the 1970s, especially in the United States, was due to a decline in R&D funding after the successful moon mission. However, R&D funding as a share of GDP has remained relatively stable over the past decades, though it has increased in absolute terms. The federal contribution has noticeably decreased, but overall funding levels don't explain the global phenomenon.

Diminishing Returns on Research Investment

Some argue that it was never reasonable to expect that more researchers would result in more technological breakthroughs, as knowledge can't be produced like sausages. However, the data show that hiring more people into research has worked to some extent, largely making up for the decrease in productivity per researcher. This doesn't explain the qualitative change in knowledge discovery we're observing.

The "Low-Hanging Fruit" Theory

Another explanation suggests that this is the normal progression of science - the simple discoveries have been made, and we're now tackling more complex problems. John Horgan's book "The End of Science" even proposed that there might be no big breakthroughs left.

However, this argument is implausible because the data clearly show an overall effect that happened simultaneously in many different disciplines, countries, and economic sectors. Something went wrong globally in the 1960s or '70s, indicating a systemic problem.

The Role of Academic Research

The most likely cause of this scientific decline appears to be related to how we organize research, particularly in academia. While most research and development is currently privately funded in the US and EU, most basic research (with long-term horizons expected to deliver breakthroughs) happens in publicly funded academic institutions.

Bureaucracy in Academia

Many, including venture capitalists and entrepreneurs like Elon Musk, point to excessive bureaucracy as a major hindrance to progress. The process of applying for grants, accounting for them, writing reports, and reviewing others' reports can be incredibly time-consuming and inefficient.

However, while bureaucracy certainly wastes time and causes inefficiency, it's hard to believe that its impact would be so dramatic as to explain the entire slowdown in scientific progress.

Publish or Perish Mentality

The pressure to publish frequently, often valuing quantity over quality, has been widely criticized. This problem has been known for decades, yet it persists despite not being in anyone's interest to fund useless research.

Risk Aversion in Research

Academic research is becoming increasingly risk-averse due to intense competition for funding and jobs. Researchers have strong incentives to pick topics that will reliably produce results, encouraging low-risk, mainstream research. This plausibly explains the lack of novelty and disruptive ideas.

However, many funding agencies explicitly aim to support "transformative" or "high-risk, high-payoff" research, yet they often end up funding the same conventional projects.

Public Perception of Science

The slowdown in scientific progress has not yet significantly impacted public trust in science, largely because much of this research is invisible to the average person. However, as awareness grows about the use of public funds for seemingly trivial or unproductive research, this could change.

Media outlets have begun to highlight examples of questionable research funding, such as studies on selfies and happiness or the effects of cocaine on Japanese quail. As these stories become more common, people may start questioning the value of their tax dollars going to academic research that doesn't seem to benefit society.

Scientists' Perspectives

Interestingly, many scientists themselves are dissatisfied with the current state of research. A survey among fast grant recipients in biomedicine found that 78% of respondents would change their research agenda significantly if given the flexibility to do so without changing their budget.

A similar survey among physicists about 20 years ago yielded comparable results, suggesting that a substantial number of scientists in academia feel compelled to work on research they don't consider the most worthwhile.

Implications and Concerns

The slowdown in scientific progress has far-reaching implications:

1. Economic growth: Technological progress drives economic growth, and a slowdown in innovation could lead to stagnation.

2. Solving global challenges: Many pressing issues, such as climate change, require scientific and technological breakthroughs.

3. Existential risks: Protecting humanity from potential catastrophes like super volcanoes, asteroid impacts, or solar flares requires significant technological advancements.

4. Resource allocation: If much of current research is ineffective, it represents a massive misallocation of human and financial resources.

5. Public trust: As awareness of this issue grows, it could erode public trust in science and academic institutions.

Moving Forward

Addressing the slowdown in scientific progress requires acknowledging the problem and seeking solutions. Some potential areas for improvement include:

1. Reforming the academic incentive structure to prioritize quality and impact over quantity of publications.

2. Streamlining bureaucratic processes in research funding and administration.

3. Encouraging more high-risk, high-reward research through targeted funding mechanisms.

4. Improving interdisciplinary collaboration to spark new ideas and approaches.

5. Reevaluating how we measure scientific progress and impact.

6. Fostering a culture that values negative results and replication studies.

7. Increasing transparency in research processes and outcomes.

By tackling these issues, we can hope to reinvigorate scientific progress and unlock the innovations needed to address current and future challenges facing humanity.

Conclusion

The slowdown in scientific progress is a complex and multifaceted problem with no easy solutions. It requires a concerted effort from researchers, institutions, funding agencies, and policymakers to address the systemic issues hindering innovation.

As we move forward, it's crucial to maintain a balance between supporting basic research that may not have immediate applications and ensuring that public funds are used effectively to drive meaningful progress. By acknowledging the challenges and working to overcome them, we can hope to usher in a new era of scientific breakthroughs and technological advancements that will benefit society as a whole.

Article created from: https://youtu.be/QtxjatbVb7M?si=v6MvTpO5YEx1dC3M