Vanishing Acts: Why Well-Meaning Researchers Keep Publishing Computational Science That Cannot Be Rebuilt
Consider a scenario that plays out with uncomfortable regularity in American research institutions: a graduate student, two years removed from a published study, receives an email from an independent team attempting to reproduce the original findings. The student—now a postdoctoral fellow at a different university—opens the methods section of their own paper and realizes, with genuine dismay, that what they wrote describes what they did, but not precisely enough to reconstruct how they did it. The code exists somewhere, perhaps on a decommissioned lab server or a personal laptop with a cracked hinge. The specific version of the statistical package they used is no longer the default. The parameter flags they passed at the command line were never written down.
No fraud occurred. No data was fabricated. The science may well be sound. And yet, for all practical purposes, the computational core of that study is gone.
The Transparency Paradox
The researchers most likely to be caught in this situation are often those who care the most about open science. They post their data to public repositories. They write detailed methods sections. They respond promptly to correspondence. What they frequently fail to do—through no particular negligence—is treat their computational environment as a scientific artifact requiring the same archival rigor as a tissue sample or a survey instrument.
This is the transparency paradox: a researcher can be genuinely committed to reproducibility while still producing work that, at the computational level, cannot be independently verified. The gap between intention and practice is not a character flaw. It reflects a disciplinary culture that has historically distinguished between "the data" and "the analysis," treating the latter as a kind of intellectual service rather than a primary research output.
When a computational biologist runs a differential expression analysis, the R script they use, the version of DESeq2 they invoke, the normalization choices they make, and the thresholds they set are not incidental details. They are, in a meaningful sense, the experiment. Changing any one of them can shift results in ways that range from negligible to publication-altering. Yet journals have only recently begun requiring that these elements be disclosed, and enforcement remains inconsistent.
What Gets Lost and Why
Several distinct failure modes contribute to the disappearance of computational workflows after publication.
Environment drift is perhaps the most insidious. Software packages update continuously, and a script that ran cleanly under Python 3.7 with specific library versions may produce different outputs—or simply fail to execute—under Python 3.11. Researchers rarely document their full dependency trees at the time of submission. Even when they do, they typically do so in formats that are difficult to reconstruct from scratch years later.
Undocumented parameter choices represent a second, equally serious problem. Many computational tools offer dozens of tunable settings, and researchers often arrive at their final configuration through iterative experimentation. The exploratory runs that preceded the final analysis—the ones that shaped the decisions ultimately reported—are almost never archived. What survives in the methods section is the endpoint, stripped of the reasoning that produced it.
Informal code management compounds both issues. A significant proportion of academic computational work is written in scripts that live in unversioned local directories, emailed between collaborators as attachments, or stored in personal cloud folders that are never transferred when a researcher changes institutions. Version control systems such as Git are widely known but inconsistently adopted, particularly among researchers who came to computation as a secondary skill rather than a primary training.
Finally, institutional infrastructure gaps mean that even researchers who want to archive their workflows properly often lack accessible tools to do so. Many universities do not maintain compute environments designed for long-term reproducibility. High-performance computing clusters are upgraded on cycles that prioritize operational performance over backward compatibility. The burden of preservation, in the absence of institutional support, falls on individuals who are already managing multiple competing demands.
The Downstream Consequences
The effects of this quiet erosion extend well beyond the inconvenience of failed replication attempts. In fields where computational analyses inform clinical or policy decisions—genomic medicine, epidemiological modeling, health technology assessment—the inability to audit a workflow is not merely an academic concern. It is a question of evidentiary integrity.
When a machine learning model trained on electronic health records is used to support treatment recommendations, and the training pipeline cannot be reconstructed from the published methods, the scientific basis for those recommendations becomes, at minimum, difficult to scrutinize. Peer reviewers cannot catch what they cannot see. Regulatory bodies cannot evaluate what was never preserved.
The problem also has a compounding effect on the scientific literature itself. Studies that build on prior computational analyses inherit any ambiguities in those analyses. If the original workflow was effectively a black box, subsequent work that extends or applies it rests on an uncertain foundation—one that may never be fully illuminated.
Structural Solutions, Not Individual Fixes
Addressing this problem through appeals to individual researcher responsibility has a poor track record. Checklists and guidelines, however well-designed, cannot overcome structural incentives that reward publication speed over archival completeness.
Several institutional interventions show more promise.
Containerization as a submission requirement would compel researchers to package their computational environments—code, dependencies, and configuration—in reproducible containers such as Docker or Singularity at the point of manuscript submission. A handful of journals have begun moving in this direction, but adoption remains far from universal. Funding agencies, which have more leverage over researcher behavior than journals do, have been slow to follow.
Repository integration at the infrastructure level offers another avenue. When institutional research computing systems are designed to automatically version and archive workflows at defined checkpoints—submission, publication, post-publication—the burden shifts from the individual to the system. Platforms such as Code Ocean and Whole Tale have demonstrated that this model is technically feasible. Scaling it requires institutional investment and, in many cases, changes to how indirect cost recovery funds are allocated.
Curriculum reform in graduate training programs would address the problem at its root. Researchers who learn version control, environment documentation, and workflow management as foundational scientific practices—rather than as optional technical skills—are far less likely to inadvertently produce irreproducible computational science. Several programs across the United States have begun integrating these competencies into their standard methodological training, though progress is uneven.
The Default Must Change
The underlying principle is straightforward: in computational science, the analysis pipeline is as much a part of the scientific record as the data it operates on. Treating it as anything less produces a literature riddled with gaps that no amount of good faith can close after the fact.
The researchers losing their workflows are not, in most cases, indifferent to reproducibility. They are operating within systems that were designed before computational analysis became central to scientific practice, systems that have not yet caught up to the reality of how science is now done. Changing those systems—through policy, infrastructure, and training—is not a peripheral concern for the open science community. It is the work.
Until the default shifts from code disappears to code persists, the methods sections of computational studies will continue to describe, with varying degrees of precision, analyses that can never quite be found again.
