A guided tour of how today's models think, remember, and simulate worlds in hidden representations instead of tokens and pixels, and where researchers are pushing back toward explicit structure.
Glass lens ball refracting neon diagonal stripes, mirrored below · Photo: Nina Storz
Averaging two unrelated passages into one input leaves traces of both in a model's predictions, though pulling them apart takes extra training and still falls short.