A vision-language-action (VLA) model adapts a vision-language model — the kind that captions images and answers questions — so that instead of words, it outputs robot actions. Google DeepMind's RT-2 (2023) established the recipe by co-fine-tuning a VLM on robot trajectories; the robot can then follow a natural-language instruction it was never explicitly programmed for.
The approach went mainstream fast. OpenVLA (June 2024) is an open 7-billion-parameter model trained on about 970,000 episodes from the Open X-Embodiment dataset; Physical Intelligence's π0 (October 2024) uses flow matching to generate smooth, continuous actions. Together they made VLAs the default starting point for generalist robot control.
VLAs let robots inherit the broad world knowledge of internet-scale models rather than learning every skill from scratch, which is why so much capital and research now flows through them. The catch is data: they still need large amounts of real robot demonstration to become reliable — the bottleneck the rest of the industry is racing to solve.
Key Facts
- VLA = a vision-language model adapted to output robot actions
- RT-2 (Google DeepMind, 2023) established the approach
- OpenVLA (June 2024): open 7B model, ~970k episodes (Open X-Embodiment)
- π0 (Physical Intelligence, Oct 2024): flow matching for continuous actions
- Now the default architecture for generalist robot control
Frequently Asked
What is a VLA model?
A vision-language-action model adapts a vision-language model so it outputs robot actions instead of text, letting a robot follow natural-language instructions it was not explicitly programmed for.
Which VLA models matter?
Google DeepMind's RT-2 (2023) established the approach; OpenVLA (2024) is an open 7B model trained on ~970k episodes; and Physical Intelligence's π0 (2024) uses flow matching for smooth continuous actions.
What limits VLA models?
They still need large amounts of real robot demonstration data to be reliable — the data-collection bottleneck the rest of the industry is racing to solve.