University of California, BerkeleyUnited States2024
Octo
An AI model: a vision-language-action model (VLA) takes camera images and an instruction and turns them into movements for a robot. This page describes the software, not a robot.
Robotsnot stated
VersionsOcto-SmallOcto-Base
The developer has released this model in several versions. Each figure applies to the version its source names.
Key figures as stated by the developer
5 of 6 stated · 1 gap- Model type not stated
-
Controls A
arms and hands -
Inputs A
cameralanguage - Access A open download
- Parameters C 93,000,000
- Base model B T5-Base language encoder (language only)
How to read the figures
- 14kgA30.9lbA
- The letter in the grey square after a figure points to the source the figure came from. Robots with several sources carry several letters.
- 74.5lb
- Imperial figures are our own conversion unless the manufacturer states an imperial figure itself. The source letter always belongs to the metric figure — a conversion has no source of its own.
Versions and notes in the key figures (6)
Model type
Note The developer calls it something else and does not label it vision-language-action or world action model: "Octo models are transformer-based diffusion policies".
Controls
Developer: it "can control various robot arms".
Inputs
Developer: "Out of the box, Octo supports multiple RGB camera inputs, can control various robot arms, and can be instructed via language commands or goal images." Robot state only after fine-tuning: Octo is designed for "efficient finetuning to new robot setups, including robots with different actions and different combinations of cameras and proprioceptive information". Goal images have no value of their own in the closed list.
Access
README: "You can find pretrained Octo checkpoints" on Hugging Face.
Parameters
Two sizes: "Octo-Small (27M parameters) and Octo-Base (93M parameters)". Octo-Base is shown; Octo-Small is a variant.
Versions
- Octo-Small
- 27,000,000
- Octo-Base
- 93,000,000
Base model
Model card: "Language is tokenized by applying the T5 tokenizer, and then applying the T5-Base language encoder."
Full specifications
8 of 12 fields stated| Field | Value | Source and caveats |
|---|---|---|
| Input and output | ||
| Model type | not stated | Note The developer calls it something else and does not label it vision-language-action or world action model: "Octo models are transformer-based diffusion policies". A |
| Inputs | camera language | A Developer: "Out of the box, Octo supports multiple RGB camera inputs, can control various robot arms, and can be instructed via language commands or goal images." Robot state only after fine-tuning: Octo is designed for "efficient finetuning to new robot setups, including robots with different actions and different combinations of cameras and proprioceptive information". Goal images have no value of their own in the closed list. |
| Controls | arms and hands | A Developer: it "can control various robot arms". |
| Degrees of freedom controlled | not stated | Note Octo-Base is described as "predicting 7-dimensional actions 4 steps into the future using a diffusion policy"; the number of degrees of freedom controlled is not stated. B |
| Size and speed | ||
| Parameters | 93,000,000 | C Two sizes: "Octo-Small (27M parameters) and Octo-Base (93M parameters)". Octo-Base is shown; Octo-Small is a variant. Versions
|
| Control rate | not stated | |
| Where it runs and what it is built on | ||
| Runs where | not stated | |
| Base model | T5-Base language encoder (language only) | B Model card: "Language is tokenized by applying the T5 tokenizer, and then applying the T5-Base language encoder." |
| Openness | ||
| Access | open download | A README: "You can find pretrained Octo checkpoints" on Hugging Face. |
| License | MIT | B Model card: "license: mit". |
| The developer’s own claims | ||
| Learns new tasks | needs retraining | C Developer claim: the project page tests Octo's ability "to efficiently finetune to new tasks and environments with small target domain datasets". |
| Works across robots | yes | A Developer claim: it "can control various robot arms". Yes only if the developer itself says the model runs on different robot models, for example robots from two manufacturers or two of one manufacturer’s models. Several units of the same robot do not count. |
Notes
- The Octo repository describes Octo models as transformer-based diffusion policies trained on a diverse mix of 800k robot trajectories.
- The Octo paper was first submitted to arXiv on 20 May 2024.
- The UC Berkeley admissions page gives its address as Berkeley, CA.
Sources
-
A
Manufacturer product page. github.
com/ · Retrievedocto-models/ octo -
B
huggingface.
co/ · Retrievedrail-berkeley/ octo-base-1. 5 -
C
octo-models.
github. · Retrievedio/
One line per source, not per figure: a source has one retrieval date. The letter beside a figure points to the line it came from.