Skip to content
embodied.guide

Back to the AI models

Tsinghua UniversityChina2024

RDT-1B

An AI model: a vision-language-action model (VLA) takes camera images and an instruction and turns them into movements for a robot. This page describes the software, not a robot.

Robotsnot stated

Key figures as stated by the developer

6 of 6 stated
  • Model type ⁠A vision-language-action
  • Controls ⁠A arms and hands
  • Inputs ⁠A camera language robot state
  • Access ⁠A open download
  • Parameters ⁠B 1,200,000,000
  • Base model ⁠A siglip-so400m-patch14-384 (vision) and t5-v1_1-xxl (language)
How to read the figures
14kgA30.9lbA
The letter in the grey square after a figure points to the source the figure came from. Robots with several sources carry several letters.
74.5lb
Imperial figures are our own conversion unless the manufacturer states an imperial figure itself. The source letter always belongs to the metric figure — a conversion has no source of its own.

The four states

Versions and notes in the key figures (6)

Model type

Developer wording: "Task Type: Vision-Language-Action".

Controls

The model card says: "RDT supports control of almost all robot manipulators with the help of the unified action space".

Inputs

Model card: "RDT takes language instruction, RGB images (of up to three views), control frequency (if any), and proprioception as input and predicts the next 64 robot actions."

Access

Model card: "pre-trained model weights, and data are licensed under the MIT license". The weights are on Hugging Face.

Parameters

"We present Robotics Diffusion Transformer with 1.2B parameters (RDT-1B)". The model card and the GitHub page round it to "RDT-1B is a 1B-parameter imitation learning Diffusion Transformer".

Base model

Model card: "Vision Backbone: siglip-so400m-patch14-384"; "Language Model: t5-v1_1-xxl".

Full specifications

10 of 12 fields stated
The letter points to the source at the foot of the page. Fields without a figure are dashed: the developer does not state them — that is not the same as zero, and not the same as no.
Field Value Source and caveats
Input and output
Model typevision-language-actionA

Developer wording: "Task Type: Vision-Language-Action".

Inputscamera language robot stateA

Model card: "RDT takes language instruction, RGB images (of up to three views), control frequency (if any), and proprioception as input and predicts the next 64 robot actions."

Controlsarms and handsA

The model card says: "RDT supports control of almost all robot manipulators with the help of the unified action space".

Degrees of freedom controllednot stated
Size and speed
Parameters1,200,000,000B

"We present Robotics Diffusion Transformer with 1.2B parameters (RDT-1B)". The model card and the GitHub page round it to "RDT-1B is a 1B-parameter imitation learning Diffusion Transformer".

Control ratenot stated

Note Inference speed on the robot's own GPU, not a control rate: "the inference frequency of RDT (6 action chunks/sec or 381 actions/sec)". ⁠B
Where it runs and what it is built on
Runs whereon the robotC

Paper, inference speed measured "on the target robot’s onboard RTX 4090 24GB GPU".

Base modelsiglip-so400m-patch14-384 (vision) and t5-v1_1-xxl (language)A

Model card: "Vision Backbone: siglip-so400m-patch14-384"; "Language Model: t5-v1_1-xxl".
Openness
Accessopen downloadA

Model card: "pre-trained model weights, and data are licensed under the MIT license". The weights are on Hugging Face.

LicenseMITA

Model card: "pre-trained model weights, and data are licensed under the MIT license".
The developer’s own claims
Learns new tasksneeds retrainingB

Developer claim: "The following videos show that RDT can learn new skills with 1-5 demonstrations." The demonstrations are part of the fine-tuning data: "We now list the number of demos related to each task in our fine-tuning dataset". For a new robot platform the model card advises: "we recommend collecting a small dataset of the target robot and then using it to fine-tune RDT".

Works across robotsyesA

Developer claim: "RDT is compatible with almost all modern mobile manipulators, from single-arm to dual-arm". The same card lists a limit: "RDT cannot yet generalize to new robot platforms (not seen in the pre-training datasets)".

Yes only if the developer itself says the model runs on different robot models, for example robots from two manufacturers or two of one manufacturer’s models. Several units of the same robot do not count.

Notes

  • The model card names its developers as the RDT team, researchers from the TSAIL group at Tsinghua University.
  • The RDT-1B paper was first submitted to arXiv on 10 October 2024.
  • The Tsinghua University contact page gives its address as Haidian District, Beijing.

Sources

One line per source, not per figure: a source has one retrieval date. The letter beside a figure points to the line it came from.