Cited Into Thin Air: The Growing Problem of Scientific Datasets That Exist Only on Paper

In academic publishing, a citation is supposed to be a promise. It tells the reader: this evidence exists, it is traceable, and you may examine it yourself. That contract, implicit in every footnote and reference list, forms the structural backbone of scientific credibility. But a growing body of evidence suggests that for a significant portion of cited datasets—particularly in medical and clinical research—the promise is hollow.

The dataset is listed. The DOI is printed. The supplementary materials section points confidently toward an archive. And yet, when a researcher attempts to retrieve the underlying data, they encounter a dead link, an empty repository folder, or a server error where the evidence is supposed to live.

This is what some in the research community have begun calling the "ghost footnote" phenomenon: citations that perform the function of transparency without delivering it.

A Problem Hiding in Plain Sight

The scale of the issue is difficult to quantify precisely, in part because it requires active verification to detect. A citation that leads nowhere looks identical to a functional one until someone tries to follow it. Several systematic analyses, however, have offered troubling estimates.

Studies examining data availability statements in journals across biomedical and clinical fields have found that a meaningful percentage of datasets described as "available upon request" or "deposited in a public repository" are, in practice, inaccessible. Researchers who attempt to contact corresponding authors for unretrievable data report response rates that are frequently low, and successful data transfers lower still. In some cases, the original investigators no longer retain the files. In others, institutional storage systems were retired without migration. In still others, the data was never formally deposited at all—the citation was, from the outset, aspirational rather than factual.

For a research ecosystem that depends on replication and verification, this is not a minor administrative inconvenience. It is a structural failure.

What Researchers Actually Encounter

Graduate students and early-career investigators tend to discover the problem firsthand, often during literature reviews or meta-analyses when they attempt to access primary data from influential studies. The experience is consistent enough to have developed its own informal vocabulary among researchers who frequent open science forums and preprint communities.

A computational biologist attempting to replicate a frequently cited pharmacogenomics study described spending three weeks attempting to locate a dataset referenced in the paper's methods section. The repository URL returned a 404 error. The journal's data availability statement listed a second archive, which contained a folder with no files. An email to the corresponding author went unanswered for six weeks before generating a reply indicating the data had been stored on a university server that was decommissioned following a departmental restructuring.

The study in question had been cited more than 200 times.

This pattern—high citation count, zero data accessibility—represents a particular distortion of academic credibility. Papers accumulate authority through citation, and that authority is implicitly tied to the verifiability of their claims. When the data behind those claims cannot be retrieved, the citations continue to propagate influence that cannot be substantiated.

The Institutional Mechanics of Disappearance

Datasets do not vanish randomly. Several structural features of how research is funded, conducted, and published create predictable conditions for data loss.

First, there is the transition problem. When a researcher moves between institutions—a common occurrence at the postdoctoral and early faculty stages—data stored on local or institutional servers frequently does not follow them. Universities vary considerably in their policies governing data ownership and portability. In many cases, departing researchers lose access to files before transfer arrangements can be made.

Second, there is the funding cliff. Many datasets are maintained on infrastructure supported by specific grant awards. When funding expires, the servers, subscriptions, or repository accounts that housed the data may lapse. Without a designated successor or migration plan, the data simply ceases to be accessible, even if the papers citing it remain in circulation indefinitely.

Third, there is the standards gap. Until relatively recently, many journals accepted vague data availability statements without verification. A researcher could write that data was "available from the corresponding author upon reasonable request" and satisfy the formal requirement for data disclosure without actually ensuring the data could be retrieved by anyone other than the authors themselves—and sometimes not even by them.

Why Medical Research Carries Particular Risk

The consequences of ghost citation data are not uniform across disciplines. In fields where research informs clinical practice, treatment guidelines, or drug approval decisions, inaccessible datasets carry risks that extend beyond academic inconvenience.

Meta-analyses that incorporate studies with phantom datasets may produce effect size estimates built partly on unverifiable evidence. Systematic reviews that cannot access underlying data cannot assess methodological quality, check for errors, or detect anomalies that might alter their conclusions. Clinical recommendations derived from such reviews inherit these uncertainties without any visible marker indicating that the foundational data could not be confirmed.

The opacity is self-concealing. A clinician reading a treatment guideline has no practical way to determine whether the studies supporting it are backed by accessible, verifiable data or by citations that lead nowhere.

Repository Standards and the Verification Gap

The open science community has made meaningful progress in establishing infrastructure designed to prevent data loss. Persistent identifier systems, long-term archiving repositories, and journal data policies that require verified deposits rather than self-reported availability statements represent genuine improvements over the practices that prevailed a decade ago.

But implementation remains inconsistent. Not all journals require verified deposits. Not all repositories enforce file-presence checks. And the existing literature—the accumulated decades of research published before current standards took hold—remains largely unaudited. Ghost datasets in older papers are rarely flagged, and the studies that cite them continue to accumulate citations of their own.

Some researchers have proposed systematic auditing programs in which citation verification is built into the peer review process for meta-analyses and systematic reviews. Others have advocated for journal-level requirements that data DOIs be checked for accessibility at the time of manuscript submission, rather than taken on faith.

Restoring the Evidentiary Promise

The citation is not merely a courtesy gesture. It is an evidentiary claim—a statement that the referenced material exists and can be examined. When datasets are cited but cannot be retrieved, that claim is false, regardless of whether the authors intended deception or simply failed to anticipate the conditions under which data disappears.

Addressing the ghost footnote problem requires acknowledging it as a systemic issue rather than a collection of individual oversights. Funding agencies, journals, research institutions, and repository operators each hold a portion of the solution. Mandatory deposit verification, persistent funding for archival infrastructure, and institutional data migration policies at researcher transitions would collectively reduce the rate at which evidence evaporates after publication.

Open science depends on more than the aspiration of transparency. It depends on the infrastructure to make that transparency real—and on a shared commitment to treating the citation not as a formality, but as a verifiable promise.