Skip to content

Back to the AI models

Stanford UniversityUnited States2024

OpenVLA

An AI model: a vision-language-action model (VLA) takes camera images and an instruction and turns them into movements for a robot. This page describes the software, not a robot.

Robotsnot stated

Key figures as stated by the developer

6 of 6 stated
  • Model type ⁠A vision-language-action
  • Controls ⁠A arms and hands
  • Inputs ⁠A camera language
  • Access ⁠B open download
  • Parameters ⁠C 7,000,000,000
  • Base model ⁠B Prismatic prism-dinosiglip-224px (DINOv2 and SigLIP vision, Llama 2 language)
How to read the figures
14kgA30.9lbA
The letter in the grey square after a figure points to the source the figure came from. Robots with several sources carry several letters.
74.5lb
Imperial figures are our own conversion unless the manufacturer states an imperial figure itself. The source letter always belongs to the metric figure — a conversion has no source of its own.
Versions and notes in the key figures (6)

Model type

Developer wording: "Vision-language-action (language, image => robot actions)".

Controls

The model predicts "robot actions consisting of 7-DoF end-effector deltas".

Inputs

The developer says: "The model takes language instructions and camera images as input and generates robot actions."

Access

Developer: "We release two OpenVLA models trained as part of our work, with checkpoints, configs, and model cards available" on its Hugging Face page.

Parameters

The developer calls it "a 7B-parameter open-source VLA".

Base model

README: "trained from the Prismatic `prism-dinosiglip-224px` VLM (based on a fused DINOv2 and SigLIP vision backbone, and Llama-2 LLM)".

Full specifications

10 of 12 fields stated
The letter points to the source at the foot of the page. Fields without a figure are dashed: the developer does not state them — that is not the same as zero, and not the same as no.
Field Value Source and caveats
Input and output
Model typevision-language-actionA

Developer wording: "Vision-language-action (language, image => robot actions)".

Inputscamera languageA

The developer says: "The model takes language instructions and camera images as input and generates robot actions."

Controlsarms and handsA

The model predicts "robot actions consisting of 7-DoF end-effector deltas".

Degrees of freedom controlled7DoFA

7 degrees of freedom for the setup "BridgeV2 environments with a Widow-X robot"; action space: "7-DoF end-effector deltas of the form (x, y, z, roll, pitch, yaw, gripper)".

Size and speed
Parameters7,000,000,000C

The developer calls it "a 7B-parameter open-source VLA".

Control ratenot stated

Note For fine-tuning data the README advises: "Collect at a control frequency around 5-10Hz". ⁠B

Where it runs and what it is built on
Runs wherenot stated
Base modelPrismatic prism-dinosiglip-224px (DINOv2 and SigLIP vision, Llama 2 language)B

README: "trained from the Prismatic `prism-dinosiglip-224px` VLM (based on a fused DINOv2 and SigLIP vision backbone, and Llama-2 LLM)".

Openness
Accessopen downloadB

Developer: "We release two OpenVLA models trained as part of our work, with checkpoints, configs, and model cards available" on its Hugging Face page.

LicenseMIT; Llama Community License also applies to the pretrained modelsB

Model card: checkpoints "are released under an MIT License"; README: "both the above models are derived from Llama-2, and as such are subject to the" Llama Community License.

The developer’s own claims
Learns new tasksneeds retrainingA

Developer claim: OpenVLA can be fine-tuned "for new tasks and robot setups given minimal demonstration data".

Works across robotsyesA

Developer claim: "It supports controlling multiple robots out-of-the-box".

Yes only if the developer itself says the model runs on different robot models, for example robots from two manufacturers or two of one manufacturer’s models. Several units of the same robot do not count.

Notes

  • The OpenVLA team describes OpenVLA 7B as an open vision-language-action model trained on 970K Open X-Embodiment robot manipulation episodes.
  • The OpenVLA repository lists the initial release as 13 June 2024.
  • The OpenVLA model card names its developers as the OpenVLA team of researchers from Stanford, UC Berkeley, Google Deepmind, and the Toyota Research Institute.
  • The Stanford University page gives its address as Stanford, California.

Sources

One line per source, not per figure: a source has one retrieval date. The letter beside a figure points to the line it came from.