←Home

    When Vision Models Learn to Move

    How vision-language models are being turned into robot controllers, either as end-to-end vision-language-action policies or as agents driving a robot through an interface, and how well each approach holds up in the physical world.

    In order

    1 stories
    black wheeled robot with mecanum wheels in a white ribbed hall
    black wheeled robot with mecanum wheels in a white ribbed hall

    Black wheeled robot with mecanum wheels in a white ribbed hall · Photo: Kinsey Wang

    Embodied · VLM robot control

    Off-the-shelf AI models steer robot arms with simple move-and-grip commands

    A 2026 preprint gives a frozen vision-language model a small command vocabulary and at most one example. In simulation it rivals robots trained on task data.

    arXiv preprint · 19 September 2026