Tesla AI Day 2: Optimus Hardware and Software Design Details

Published in Tech on Oct 2, 2022

On September 30th, Tesla hosted its 2nd AI event. Although the event is mainly targeted at recruiting AI engineers, I've also been excited because I'm hoping it will help me gain insight about the future of the Optimus robot and Tesla. I'm watching it now and I'll take notes here:

Tesla Bot Demonstration

The screen parts and Tesla's Bumble C robot is revealed, standing there. It looks distinctly humanoid, but silvery metallic parts, with wires running everywhere. It looks like it was made in some eccentric nerd's garage, and likely to fall apart if someone looks at it the wrong way.

Bumble C looks at its hands, as if waking up in this body and examining them for the first time.

Bumble C steps out onto the stage with careful, measured steps.

Bumble C waves at the audience, acknowledging the crowd with its body language. It nods and turns as it waves to the span of the audience.

The Bumble C robot does a little dance.

Finally, the robot turns around and walks back behind the screen.

This demonstration was the first time robot was operated without a tether. They were apparently afraid it would fall and break, but they showed it because that was what they had working.

Someone comments that it uses the same FSD computer from the cars.

Next, they showed a few prerecorded demos:

  1. Bumble C robot delivers a package to somebody by placing it down on the desk next to them. Included is a visualization demonstrating what the robot sees in that situation.

  2. Bumble C robot picks up a watering can from a table, walks over to a planter, and waters the plant.

  3. Bumble C robot picks up a piece of metal from a factory workstation and moves it over to a side table.

My thoughts so far:

Overall, seems pretty basic. We've definitely seen far more advanced robotics from companies like Boston Dynamics. The package delivery looked good. A watering can has a smooth handle with a high margin of error for picking it up successfully. It wasn't clear how useful the robot's action was in the factory example, but that's obviously one of the more important/immediate practical applications of the technology if it is useful in that scenario. I would want to know more about the utility and challenges in the other tasks required at that workstation, that weren't included in the demo.

Optimus Tesla Hardware Prototype

Next, Tesla shows the Optimus robot, much closer to what the production version will look like. While its not as cool as the Optimus Concept from last year, its more real. It looks better than Bumble C, that's for sure. Its still humanoid, more metal, less rounded. This version can't actually walk yet, but its what they're planning on selling once they get it working.

The Optimus robot is designed for manufacturing, so it can be made high volume, low cost, high reliability. Prior humanoid robots are very expensive, lack a brain, and are made in low volume. In contrast, Tesla hopes to build thousands or millions of Optimus robots. And to get that done, they need to inspire talented engineers to join.

Words of Inspiration

The economy is productivity per capita. But what does economy mean once there's no labor shortage? To Tesla, Optimus represents a future of abundance, where people can have whatever products and services they want, and there is no poverty.

Without mentioning the arguments against AI, Elon addressed that elephant in the room by stressing that he believes Tesla is the best entity to create an AI robot.

If I go crazy you can fire me. This is important. ~Elon Musk

Tesla is publicly owned, has single class of shares. That's important that public can properly influence it. If they don't like what it's doing, buy shares and vote differently.

Self driving cars will improve productivity of transport buy 0.5-1+ order of magnitude. In contrast, the Tesla Optimus robot can potentially improve economic output on a global scale by 1-2 orders of magnitude minimum. We need to do it carefully and safely, make sure outcome is beneficial to civilization, but the opportunity is really huge.

If you're a talented mechanical, AI or software engineer, Tesla hopes you'll consider joining Tesla to help Tesla achieve those goals. At Tesla, we aspire to do the right thing. Then Elon made a comment about the road to hell being paved with mostly bad intentions. And Tesla will try not to pave a road to hell with their good intentions. Although he recognizes there is a risk there.

Optimus Trajectory

AI day 2021 - Concept

Feb 2022 - Development platform - bot that walked out on stage

Sept 2022 - Latest generation, Tesla-made-hardware

Optimus Technical Details

Optimus engineers gain a new appreciation of the human form.

Humans consume 100W power at rest, and 500W at a brisk walk. They weigh about 73kg.

I thought it clever that they present humans as the benchmark they're trying to improve on. This gives you a nice anchor to numbers that they can show improvement against.

For the robot, Optimus they're targeting minimal power consumption, certainly at rest, but even when active. And the structure and components are designed with this in mind.

Optimus features

2.3 kWh torso battery pack

Bot brain in torso as well

Wi-Fi, LTE, Audio

Hardware security features to protect robot and people around it

28 Structural Actuators. 11 degrees of freedom hands. This is significantly fewer than human degrees of freedom, but enough for specific useful tasks.

Designing Optimus Hardware

At this point the discussion really starts to get super technical. I gain an appreciation for what they're doing, but some details go over my head.

Tesla was able to leverage existing software, skills, and expertise from other businesses in this robot making endeavor.

For example, Tesla has crash modeling software, used same thing for Optimus.

They created a four-bar knee inspired by human biology. This requires much less force when knee is fully bent compared to a lever approach.

Joint Actuators

To evaluate an actuator for a single joint, Tesla ran it through a simulation, mapped the torque required over an actuator efficiency diagram, and puts a "point into the cloud". (I'm not sure what this cloud is.) Then they "parse the cloud" for each joint to determine the most efficient actuator design.

However, they want to minimize the number of unique actuator designs, in order to make it suitable for mass manufacturing. So they ran a "commonality study," searching for actuator designs that could accommodate the needs of multiple joints. They were able to come up with 6 different actuators that met all the performance requirements. Three rotary and three linear actuators.

After going into some technical details of actuators best appreciated by mechanical engineers, they show a video demonstration of one of the linear actuators lifting a half ton grand piano.

Hand Design

Human hands are awesome. They have many sensors and can grasp anything. The world is designed for hands, so using hand-inspired design for the robot allows it to be future-proof. Again some technical specs went over my head.

Optimus Software

Same software as FSD from a car, its just robot on legs instead of robot on wheels.

Training data had to be recollected for new use case.

Volumetric depth rendering - for understanding the world

Reused simulation software that was built for cars

Walking

Walking seems easier than it is. Requires balance, self awareness of your limbs, coordination. Energy efficient gait. Humans don't think much about it, but robotics engineers need to.

Optimus's Locomotion Planner generates reference trajectories, following the desired path.

Locomotion plans are great, but when you try to put it into practice it doesn't work - reality is not idealized. The robot takes a few steps and falls over. To resolve this, we use State Estimation.

State Estimation uses the understanding of reality to correct behavior as it goes.

Motion Control

Manipulation

Robot needs to know how to maniupulate objects, look natural, get there quickly.

Tesla created a library of natural motion references / demonstrations.

Trajectory Optimization - Adaptation of references to the real world, accounting for differences in reality from the simulation, and keeping in mind how the robot should balance as it does the task

What's Next for Optimus?

Really focusing on one factory use case, get everything needed for real world use solved. Examples include indoor navigation, graceful fall management, servicing

Need to get Optimus prototype up to par or beyond capabilities of the other prototype shown

"I'm pretty sure we can get this done within the next few months or years"

Takeaways from Part 2

Tesla cars already have had the hardware for full self driving for years. Tesla has been working on software to get to FSD. Last year FSD was on 2000 vehicles. This year 160,000. 75,778 models trained over the past year. That's one every eight minutes. 281 of the trained models were actually shipped to cars.

The FSD team goes into tremendous depth on the technical details of each of the many systems that go into it, which is beyond the scope of this article.

Takeaways

  • Tesla leverages its existing capabilities to work on Optimus.

  • Tesla believes in AI and admits there are risks but is full steam ahead with it.

  • Optimus isn't very useful yet, but seems like the team is doing great work.

  • Tesla currently uses 4000 GPUs for auto labeling and 10000 for training ML models.

  • FSD is extremely complicated, involving multiple neural networks, custom training infrastructure, training data, simulation, real world data, and custom AI Compiler and Inference engine.

    • New FSD scenarios are simulated. When the simulation fails, they "mine" the real world dataset for real world copies of that scenario, label them correctly, and then feed that back into the process, and repeat. This allows systematic resolution of errors.

  • Dojo supercomputer is coming along nicely.

  • Tesla is not a car company. It is a top tier technology company focusing on excellence in real world AI, hardware, and software.

  • I'm excited to see Tesla's impact over the next few years and beyond.