Google DeepMindUnited Kingdom2023

RT-2

An AI model: a vision-language-action model (VLA) takes camera images and an instruction and turns them into movements for a robot. This page describes the software, not a robot.

Robotsnot stated

Key figures as stated by the developer

4 of 6 stated · 2 gaps

Model type ⁠B vision-language-action
Controls ⁠B arms and hands
Inputs ⁠A camera language
Access not stated
Parameters not stated
Base model ⁠B PaLI-X or PaLM-E (two versions)

How to read the figures

14kgA30.9lbA: The letter in the grey square after a figure points to the source the figure came from. Robots with several sources carry several letters.
74.5lb: Imperial figures are our own conversion unless the manufacturer states an imperial figure itself. The source letter always belongs to the metric figure — a conversion has no source of its own.

The four states

Versions and notes in the key figures (5)

Model type

Developer wording: "Robotic Transformer 2 (RT-2) is a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control".

Controls

The action is a string of tokens that starts with an end-of-episode flag and then gives "commands to change position and rotation of the end-effector, as well as the desired extension of the robot gripper".

Inputs

"The resulting model takes in robot camera images and directly predicts actions for a robot to perform." "The robot data includes the current image, language command and the robot action at the particular time step."

Parameters

Note One number per version: "RT-2 trained on top of PaLM-E (12B parameters)"; "RT-2 trained on top of PaLI-X (55B parameters)".

Base model

"we adapt Pathways Language and Image model (PaLI-X) and Pathways Language model Embodied (PaLM-E) to act as the backbones of RT-2".

Full specifications

5 of 12 fields stated

The letter points to the source at the foot of the page. Fields without a figure are dashed: the developer does not state them — that is not the same as zero, and not the same as no.
Field	Value	Source and caveats
Input and output
Model type	`vision-language-action`	B Developer wording: "Robotic Transformer 2 (RT-2) is a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control".
Inputs	`camera` `language`	A "The resulting model takes in robot camera images and directly predicts actions for a robot to perform." "The robot data includes the current image, language command and the robot action at the particular time step."
Controls	`arms and hands`	B The action is a string of tokens that starts with an end-of-episode flag and then gives "commands to change position and rotation of the end-effector, as well as the desired extension of the robot gripper".
Degrees of freedom controlled	not stated
Size and speed
Parameters	not stated	Note One number per version: "RT-2 trained on top of PaLM-E (12B parameters)"; "RT-2 trained on top of PaLI-X (55B parameters)". ⁠A
Control rate	not stated
Where it runs and what it is built on
Runs where	not stated
Base model	PaLI-X or PaLM-E (two versions)	B "we adapt Pathways Language and Image model (PaLI-X) and Pathways Language model Embodied (PaLM-E) to act as the backbones of RT-2".
Openness
Access	not stated
License	not stated
The developer’s own claims
Learns new tasks	without examples	B Developer claim: "RT-2 shows improved generalisation capabilities and semantic and visual understanding beyond the robotic data it was exposed to." "This includes interpreting new commands and responding to user commands by performing rudimentary reasoning".
Works across robots	not stated	Note "we show that RT-2 can work on another embodiment, Language Table environment" For this, "we train RT-2 on simulation and real language-table data". ⁠A Yes only if the developer itself says the model runs on different robot models, for example robots from two manufacturers or two of one manufacturer’s models. Several units of the same robot do not count.

Notes

Google DeepMind introduced RT-2 as a vision-language-action model that learns from web data and robot data.
RT-2 comes in two versions, built on two different vision-language models, PaLM-E and PaLI-X.
Google DeepMind published the RT-2 blog post and paper on 28 July 2023.

Sources

A robotics-transformer2.github.io/ · Retrieved 2026-10-13
B Manufacturer product page. deepmind.google/blog/rt-2-new-model-translates-vision-and-language-into-action/ · Retrieved 2026-10-13

One line per source, not per figure: a source has one retrieval date. The letter beside a figure points to the line it came from.