Even PLoS ONE, a journal that takes open science very seriously, says that “submissions…must demonstrate that the new tool achieves its intended purpose [and that it] is an improvement over existing options.” The fact that it sounds so reasonable is actually what makes it scary.

“Achieving its intended purpose” generally means achieving some minimum score on a quality metric, for example, precision or accuracy. However, one cannot know whether the proposed method will achieve the desired quality or not before doing proper testing.