Tropical diseases remain one of the biggest global health challenges of our time. They hit low- and middle-income countries hardest, draining public health systems, slowing economic growth, and deepening inequality. For decades, policymakers and donors have poured resources into interventions, vaccines, medicines, and prevention campaigns, all backed, at least in theory, by scientific evidence.
But is a big share of that evidence credible?
That’s the question we’re exploring in a new project at the Institute for Replication (I4R): how credible, transparent, and reproducible is research on tropical diseases?
Step 1: Setting the Frame
We started with a simple decision: focus on studies published between 2015 and 2024. Why? Because reproduction is a lot easier when authors are still reachable, and data hasn’t vanished into old hard drives. It also builds on earlier lessons from a previous collaboration with GiveWell (https://www.givewell.org/research/trying-to-reproduce-RCTs): contacting authors of older studies often leads to dead ends.
Step 2: Finding the Studies
We went digging through major journals in public health, economics, and general science, think International Journal of Epidemiology, Lancet Public Health, BMJ Global Health, Clinical Infectious Diseases, and top field journals. Using Scopus, we searched for diseases like malaria, dengue, leishmaniasis, schistosomiasis, and Zika virus, among others.
The tally? 9,562 articles:
- 8,230 in public health journals,
- 141 in economics journals,
- 1,191 in general interest journals like Nature/Science.
Step 3: Narrowing Down to What Matters
We weren’t interested in everything, only causal or interventional studies, as these would matter the most for policymakers and donors. That means:
- Randomized trials or quasi-experimental designs (DiD, RD, IV, event studies),
- Studies testing interventions like vaccines, supplements, policies, or programs.
We filtered out studies just investigating disease burden (as they do not involve human-directed interventions), meta-analyses, systematic reviews, and animal-only trials. After several rounds of classification checks, we were left with 598 studies: 553 in non-economic (public health journals + general interest journals) and 45 in economic journals.
Step 4: How Open Is the Evidence?
Here’s where things get interesting, and a bit worrying!
Among 553 “non-economic” studies (see Figure 1):
- Only 19.3% made their dataset public, but with the vast majority of these not including the replication codes.
- Nearly half (45%) had a “supplementary” section, but no actual data or code appended.
- Another 30% either restricted access, had broken links, or had no mention at all of data availability.
Figure 1: Replication Package Availability of Non-economic Studies
Among the 45 economic studies, the picture is better but not perfect (see Figure 2):
- 21 out of 45 provided full replication packages.
- 15 didn’t mention data availability at all.
Figure 2: Replication Package Availability of Economic Studies
In short, 448 out of 553 non-economic studies and 21 out of 45 economic studies require author follow-up to obtain full replication packages (that is, either the data or code was missing in this case).
Why This Matters
It is one thing to debate significance level and effect sizes. It is another when one cannot even check the underlying data. When evidence on tropical diseases, which guides billions in health spending, cannot be reproduced, credibility suffers.
And this is not just about academic niceties. A missing replication package can mean:
- Slower policy response,
- Wasted resources,
- Fewer opportunities to build on previous work.
The Road Ahead
Our next step is ambitious but clear:
- Contact authors to fill the replication gaps.
- Test reproducibility and robustness directly.
- Share results transparently.
Stay tuned!
Ghina Abdul Baki