Qwen trains one robot model to grasp, navigate and drive different bodies

A single robot-control model can steer simulated robot arms and humanoids and guide a virtual agent through simulated indoor spaces, and on most of the benchmarks it faces it matches or edges past systems tuned for each one. That is the result reported for Qwen-VLA by researchers on the Qwen team, in an arXiv preprint not yet peer reviewed. It is a vision-language-action model, or VLA: software that takes camera images and a written instruction and outputs robot movements. The previous story in this series looked at Ai2's MolmoAct2, an open VLA for low- to medium-cost hardware. (earlier story) Qwen-VLA asks a broader question: whether one model can span manipulation, navigation and different robot bodies at once.

Most embodied AI, the field of machines that act in the physical world, consists of specialists tied to one task family, robot or benchmark, the authors note. The jobs look hard to merge. Manipulation outputs gripper poses or joint angles, navigation outputs waypoints, and first-person video of people's hands yields wrist and hand trajectories, each differing in inputs, control rate and how far ahead it predicts. The authors argue that all share one structure: read images and language, respect the body's limits and predict what comes next.

The model starts from Qwen3.5-4B, a vision-language model that handles images and text together, and adds an 'action expert', a transformer module that generates numeric motor commands by flow matching: it learns to turn random noise into a target action sequence step by step. The robot's embodiment, meaning its body and control set-up, is conveyed only by a short text prompt before the instruction that names the robot, its arms and any mobile base, its control rate and how many steps to predict. All robots' actions share one numerical container, each body filling only the slots it needs. Training mixes robot demonstrations, first-person human video, simulation, navigation and ordinary image-text data across four stages: a text-only warm-up for the action module, broad pretraining (yielding Qwen-VLA-Base), fine-tuning on curated task data, and reinforcement learning, or reward-driven trial and error, in simulation (yielding Qwen-VLA-Instruct).

On four simulated manipulation benchmarks, one Instruct checkpoint faced models tuned for each. It scored 73.7 on Simpler-WidowX against 64.6 for the next-best listed system, and edged ABot-M0 on RoboTwin 2.0, by 0.1 points on the easy tier and 2.2 on the hard one. It trailed ABot-M0 on LIBERO (97.9 against 98.6) and on RoboCasa-GR1, a humanoid kitchen benchmark (56.7 against 58.3). 'Generalist' here does not mean zero-shot, or untrained on the task: Instruct was fine-tuned on simulated manipulation demonstrations, a step the authors call task-specific alignment, so the advantage is one model doing the work of several. No error bars are reported. On the R2R and RxR navigation benchmarks, where an agent follows directions through simulated 3D indoor scenes, it had the best success rates, 57.5% and 59.6% against 56.9% and 52.9% for StreamVLN, which led on some route-quality measures.

The cleanest evidence that broad pretraining helps comes from a real ALOHA robot, with two arms, parallel-jaw grippers and three cameras, driven by a separate model fine-tuned from Qwen-VLA-Base on ALOHA demonstrations. Across six task categories it averaged 83.6% success, against 48.5% for the same architecture trained from scratch and 71.6% for π0.5, which did better at towel folding. Under out-of-distribution conditions, meaning colours, objects, positions, backgrounds or lighting and instructions unseen in training, it reached 76.9%, against 41.5% for π0.5 and 36.2% from scratch. That is one platform, and a model distinct from the unified checkpoint.

Ablations, tests that change one ingredient at a time, suggest the recipe is delicate. On Simpler-WidowX, the text-only warm-up lifted scores from 60.9 to 71.1 when about a fifth of its data was synthetic, but cut them to 51.0 with real data alone, and stretching it from 2,000 to 40,000 steps dragged the score down to 60.4. Adding vision-language data raised RoboCasa-GR1 from 51.1 to 56.0 and RoboTwin from 81.8 to 86.4.

The authors acknowledge that robot action data remains far scarcer and less varied than image-text data, limiting robustness with unusual objects, settings, robot bodies and contact-heavy tasks. Training on everything together brings trade-offs: action-focused training can modestly erode some pure vision-language and navigation results. And most evaluations are short-horizon and benchmark-driven, so long, failure-prone deployment in the real world remains an open test.