brilliant! maybe these can be added to the existing cito ontology sparontologies.github.io/cito/current... maybe somewhere between 'derides' and 'parodies' 🙂

Rather, I am worried that our entire publication system may not only be inadequate but possibly quite at odds with what is most sorely needed to confront the biodiversity crisis. We select for novelty and significance. We scrutinize words and claims far more closely than code or sensitivity.

While I applaud these authors (& the editor for showing that it is not about getting three yes votes, and publishing the reviews -- well worth the read!), I don't mean to single out the LPI index or rev 1.

The authors include a line-by-line code review that identifies 30 errors in the published LPI code; though all with much smaller numerical impact than the math issues that receive the focus. (Rev 1: 'I didn't feel it was necessary to review the code because it wouldn't change my assessment.')

reviewer 1 is clearly skeptical of the value here, writing: "In the first sentence of this review report, I purposely refer to this study as a ‘sensitivity analysis’ of the LPI because, to me, that is all that it is."

The authors observe this: "We have explored the code used for the calculation of the LPI. Although [3 cites] provide the basic principle of calculating the LPI, the exact methodological procedure is clear only from the code of the package rlpi (v.0.1.0) in R"

Reviewer #2 speaks directly to the influence, writing: "LPI is probably the single most widely known and used index of human impact on biodiversity [..] that receives exceptionally wide press coverage to one that is embedded into UN conventions and treaties."

Implementing complex indicators such as discussed here is both immensely influential and immensely challenging stuff. Moreover, such metrics are dynamic quantities whose precise definitions are not in the static papers we peer review but the evolving code that computes them.

Note the full peer review and reply chain has been published alongside - kudos to the reviewers and publisher on that! Arguments aside I think it is a great example/resource for new researchers learning to navigate this part.

This rather fantastic recent piece is a great illustration of the value of code review, as well as sensitivity analyses. doi.org/10.1038/s414... But I think it is particularly instructive as commentary on our scientific process today. 🧵

Mathematical biases in the calculation of the Living Planet Index lead to overestimation of vertebrate population decline - Nature Communications

The Living Planet Index is a widely used metric to measure the global population trends of vertebrates. This in-depth analysis of the methodology underlying the index reveals fundamental issues and id...