Thanks. Any comments welcome.

I’m still digesting, but I keep thinking there’s got to be a performance sweet spot where you use something like a transformer (or Hyena) on subsequences for parallelism, and then something LSTM-style to handle the really non-local behavior