I feel like I say this a lot in conversations with you, but I'd love for someone to do a proper history of statistics and "ML" from 1995-2015 or so.

I took an undergrad "ML" class from a statistical learning theory guy in 2015 and he didn't cover neural networks because they were, in his opinion, too data hungry and not robust enough. My impression is just that for a long time people had a lot of room to doubt

Also, for all that people are becoming aware of it _as a phenomenon_, we still just don't understand it. We don't have the right math.

But even though some people have known this to some extent for ~30 years, I think it's reasonable to say that it's only in the past 5-10 years we've realized how robust a phenomenon it is

Well, yes and no, right? I'm relying on @beenwrekt.bsky.social as a historian here, but there were hints about this going back to bagging / boosting and in retrospect the success of SVMs on translation should have made everyone pay attention.

Any chance you know of some better writing on the "fundamental argument"? It's a perspective I'd like to understand better, but this paper just sets my teeth on edge -- as you can maybe tell by the five replies I've made

There is a kernel of legitimacy to the current "hype" around modern ML and it is precisely that models are doing things that that we reasonably thought were formally intractable. Pointing out complexity arguments that say this should be impossible is not useful !

Well, the previous version I saw was just "Act 1"

Oh man, I saw a previous version of this paper and it is so so bad. We have known that learning under arbitrary distributions is NP-hard goes back to Valiant! This is a strawman of ML dressed up in complexity theory

Honestly not too late! Pitch this as an article to the New Yorker or somewhere