Our systems are definitely not (and I'd argue cannot be) fraud-proof, but Jonathan was caught in part b/c he published in Am Nat, where he was required (as part of the early efforts to make data sharing mandatory) put up his raw data files which others then found the suspicious patterns in.

Finally, Am Nat does not allow GitHub as a final repository, since it can later be altered or deleted. Instead, we require authors to clone the publication version of their GitHub to a "curated" service like Zenodo, so there is a permanent, stable copy of the code and data associated with the paper.

RE: blind review, most data repository providers (Dryad, Zenodo) let you construct an anonymized and private, available only to the journal and reviewers, repository that can later become public&non-anonymous.

This is basically what Am Nat's Data Editors do (but importantly, they also check that the authors provide sufficient information in their README files so anyone proficient with code can do this with reasonable effort).

To be clear: I think it should be required that authors supply the code and data for review and journals ensure they are complete and work. Not that reviewers check the code for correctness unless they feel like they need to. If they do, having the code saves time, compared to the another R&R cycle.

I do appreciate your context here that review times in PoliSci/Econ are already too long, but I'd say ppl looking at code isn't the cause of that and as folks noted above having the code to refer to can actually speed things up.

I think journals should have a more uniform mechanism than asking reviewers to check a box or comment for ensuring completeness and usability of the data and code. But they should absolutely be provided in review for reviewers who do want to see for themselves how things were done.

And on more than one occasion, having the repository undergo Data Editing caused authors (without any prompting from reviewers) to discover issues that they were able to fix before acceptance. In all, it’s work but it is useful work and I am heartened to see other journals following Am Nat’s lead.

so that someone else can run all your analyses reported without having to ask you for code or data. NOTABLY this process checks for completeness and usability not for correctness of the code. But I have often seen (as reviewer and editor) having the code helps clarifying how things were done.

@asn-amnat.bsky.social@bobmontgomerie.bsky.social) has pioneered “Data Editing,” at least in biology: the idea is that if your paper is allowed into revision, a Data Editor will make sure you included all the data and code and provided adequate documentation …