Machine learning can build systems that nobody is able to explain, and closing that gap is what this lab works on.
Transformers, diffusion models, and the variants around them work well enough that the question of why rarely gets asked. Existing theory can analyze parts of their behavior once they exist, but it did not predict or produce any of them, and nothing tells us they are the right design.
Our starting position is that computation and learning are physical processes, subject to laws about entropy and energy like anything else. If the right designs can be calculated from those laws, what comes out would be a different sort of system from a transformer, built on different principles.
There is a larger question behind that one. We have seen intelligence in two forms, living brains and the models we have built, and both are particular. If intelligence can be described mathematically without reference to either, then each is one solution to that description rather than the thing itself. Descriptions like that already exist. The best known, published by Legg and Hutter in 2007, cannot actually be calculated, which sounds like a dead end and is not one. It means the useful question stops being what intelligence is, and becomes which physical systems get close to it, how close, and at what cost in energy and time. Those are questions you can run an experiment on. Note 005 sets out the position and what it changes about how we treat the brain.
Start from physical limits
Thermodynamics, statistical mechanics, and information theory put hard limits on any learning system, whatever it is made of. Those limits are narrow enough to be useful: they rule out large classes of design before anyone writes code, which is what makes them a place to start.
Calculate the design
Energy landscapes, wave equations, and entropy functionals are used here as the actual mathematical objects a model is built out of. They are not illustrations of how a model behaves or analogies for explaining it afterwards.
Treat the brain as one example
Brains are the only systems everyone agrees are intelligent, which makes them easy to mistake for what intelligence is. We read them the other way round, as one implementation that physics arrived at under constraints no designer would have chosen. That makes a brain evidence about the general case and a poor blueprint for it.
Build what the theory produces
A piece of theory is finished when it produces something that runs: a design, a procedure, or a prediction an experiment can check. Until it does, it stays on the list of things in progress.