There's some fundamental truths we've come to understand, such as * A detector can be a generator * A pinched network = compression = generalization of a task. * Latent spaces let you do math on concepts themselves, with immense conceptual complexity ... and so forth.

As a young researcher it's very funny to read some of the literature from the 2000s going back and forth on whether generative or discriminative formulations of objectives were better and now it's clear that with enough compute they are dual to each other