By replicate I mean run a replication of the study with new data collection. It's not the same data.
Many people do successfully use LLMs for search even though it's noisy and prone to errors.
I agree that this is very well written and is persuasive. The part that someone might disagree with most strongly is "Works are copied [...] without offering any search functionality or other utility to users, let alone criticism or commentary."
You're mentioned specifically in this piece @melaniemitchell.bsky.social
This is the correct analysis. The arguments in opposition are based upon handwaving claims that LLMs are magical and/or brains, which they are not. The real question is if the Second Circuit decides "but Wall Street and Silicon Valley like it, so we need to let AI companies win."
If you simply replicate then it's not exploratory.
for every like this post gets, i'll post one song i like, and one banking password
And there are lots of scientific disciplines where pre-registration makes absolutely no sense at all. We need to be careful not to generalize that what works well for field X to all fields.
This is not at all true. Many of the best ideas in labs where data is cheap are conceived and tested rapidly. And the best analysis plans are often devised after viewing the data and a replication can follow to verify that it's not p-hacking.