5 Cool FSD Beta Things We Learned After Elon Musk's Live V12 Demo

Automakers and technology companies are in a not-so-silent-anymore race to make the driver irrelevant. It's not just you, the casual driver, they want to replace, but the professional ones, too. Some corporations like Cruise even claimed that "humans are terrible drivers" in cheeky ads published in some printed newspapers. Soon after, the California Department of Motor Vehicles (DMV) requested the General Motors-owned entity to halve its fleet of robotaxis roaming San Francisco's streets.

Even though a lot of progress has been made in the autonomous driving department, it's clear that there's still much work left to be done. We can't trust software, hardware, and an Internet connection to act as our personal and entrusted drivers – yet. Some of us may like it; others may not. But what's certain is that self-driving cars are the future.

It may sound like a cliché, but keep in mind that's what people and even some incorporated car industry giants used to say about all-electric vehicles when Tesla started selling the battery-powered and Lotus Elise-derived Roadster. When the Model S came to fruition, looking like the premium version of a slightly redesigned Mazda6, naysayers were even more vocal.

However, here we are today: 20 years after its inception, Tesla is the world's most valuable automaker and the brand that manufactures the world's best-selling car (the Model Y). That is a premiere because it's the first time an all-electric car got the coveted title. It remains to be seen if it'll keep it until the end of 2023.

Participating in the AV race is easier with friends

The now Texas-based automaker is one of the leading players in the autonomous driving space. Even though the company's CEO made quite a lot of wild promises over the years and categorically overestimated the capabilities of their candidate for a self-driving system on numerous occasions, progress is undeniable.

With a lot of free help coming from paying customers who consider themselves early adopters or pioneers and many savvy engineers and data analysts, the Full Self-Driving Beta suite (that's not yet capable of what the name suggests) is a strong candidate for a computer that can replace the human sitting behind the steering wheel.

But for the time being, it's still just a Level 2 ADAS per the SAE J3016 driving automation standard. Others like Cruise, Waymo, Zoox, or even Mercedes-Benz have surpassed Tesla and either deployed robotaxis under strict remote supervision or have received certification for their autonomous driving system considered at least a Level 3 ADAS. That means the driver must intervene only when the suite requests it. Anything Level 4 or higher translates into the vehicle driving itself.

The latest FSD Beta released to "civilians" is v11.4.7. Since February, the automaker has also been delivering its next-generation computer (aka Hardware 4 or HW4) to some customers, but FSD Beta does not run on this system yet. Thus, the Model S used by Elon Musk is equipped with the previous-gen tech known as HW3.

Even though the risk of something going extremely bad was high, he decided to show everyone what FSD V12 could do. And did so live! Given what was hanging in the balance, it was not a wise decision, but nobody can say it wasn't a brave and confidence-inspiring move. Once the leader publicly backs such an intricate product, it becomes hard to say, "FSD Beta is vaporware."

Notice there was no "Beta" when referring to FSD V12? That's because the version put to work by the CEO is dubbed as the production-ready iteration of the suite. FSD Beta V11 made sure Autopilot and FSD can work together to enable the program to work on highways, and V12 should be the one that finally allows Teslas to become robotaxis. However, remember that from v11.4.7 to V12, there's a lot more room left for incremental improvements.

So, what did we learn?

FSD V12 is end-to-end artificial intelligence (AI)

While that may sound fancy because everything seems to be connected to or by AI these days, having Full Self-Driving become an end-to-end artificial intelligence means the system is trained on real-world data (video mostly) and takes decisions based on previously encountered scenarios. It can understand what happened before, replicate a similar action, and then adapt it instantaneously to that particular environment it finds itself in.

Things such as traffic lights or when and how to change lanes are not hardcoded. Moreover, it proves Cruise wasn't right to say humans are terrible drivers because Tesla's computers are training on human data. If it learns from us, it will most likely behave like us, minus the sporadic irregularities like when someone decides to break the law and drive under the influence or is too tired. That's where the labeling team comes in.

However, letting AI do everything may be a weakness in some scenarios. Suppose the car must deal with a situation where it must stop at an uncontrolled railway crossing for an oncoming train. If the FSD V12 can't correctly and timely determine what's happening, it could theoretically expose the occupants to a severe accident.

Such a shortcoming was highlighted in the live demo of FSD V12 (available below) at the 19-minute and 50-second mark when the vehicle attempted to run a red light once the left turn lanes' traffic light turned green.

Moreover, Elon Musk underlined more training data is needed to allow the system to learn how to correctly steer the EV in specific scenarios or when the weather isn't the friendliest.

But the neural network (an artificial system designed to "think" like a human brain) can constantly expand its capabilities. Learning never stops. And that's why Tesla employees are helping the company learn more about traffic in Japan, Thailand, and New Zealand. In these countries, people drive on the left side of the road in right-hand drive cars.

FSD V12 does not need an internet connection or maps

Some robotaxi companies like Cruise use a continuous internet connection to keep track of their vehicles. Waymo, on the other hand, only enables it sometimes out of the fear of having its units hacked. FSD V12 does not use one at all.

We believe that isn't a good idea. It works while a human in the driver's seat is ready to take control at a moment's notice. But if Tesla really wants to transform its EVs into robotaxis, it will need an internet connection. Otherwise, it won't be able to enable its self-driving cars to discover possible traffic changes, necessary route updates, and dangers shared by others in real time.

VW, for example, planned to create a car-to-X communication system that should have enabled vehicles to "warn" each other about potentially dangerous situations they encountered. In a way, that's what Waze is doing. Other drivers report potholes or unexpected traffic jams, and the app updates itself to let other road users know what they might encounter if they remain on the same route.

Tesla's FSD V12 also doesn't need constantly updated navigational maps. Elon Musk said the system just needs coordinates and will find the location by itself.

At the same time, however, the CEO said that when human intervention is registered (aka stopping the autonomous driving system), the event is recorded and sent back to Tesla for analysis. Afterward, the software gets more video training data to learn how to approach that scenario correctly. So, it'll still need an internet connection from time to time.

Enhanced parking capabilities and a neat robotaxi feature

The same live video published on Elon Musk's X, to which Tesla pays around $20,000 per year to have multiple verified accounts, showed the Model S maneuvering itself elegantly for a neat parallel parking job.

While sitting behind the steering wheel and apparently filming the whole thing himself, Elon Musk said the car arrived at that "random" spot by itself and knew it had to pull to the side and park. He again indicated that this was the predictable behavior the neural networks learned from the videos injected into it.

However, as the low-quality footage showed, the vehicle wasn't quite dead-on. It stopped a bit earlier than it should have. The engineer (Ashok Elluswamy) joining Musk for the ride explained that the EV would try to find a good spot to stop, even though it might not be able to take the passengers exactly where they wanted.

That is solid proof regarding neural net learning. The videos fed into the system probably showed that finding a good parking spot can be complicated and, in some instances, even impossible. So, it did the next best thing and parked instead of doing another maneuver or circling the block for a suitable space to stop.

But that's just one of the many things FSD V12 can do.

The executive pointed out that once the software is ready to be deployed to every paying customer (including corporations willing to license it) and qualifying employees, a robotaxi running FSD V12 would be able to look around for the person needing a ride, identify them, and stop as close as possible without disregarding safety.

Mountains of data and powerful computers

Tesla's FSD V12, an end-to-end AI (according to Elon Musk) without any specific scenarios being hardcoded, may feel like magic to those willing to dig a bit deeper and learn more about it or experience it before knowing anything about it. However, it raises the question of how far computational power can go: How many videos does it take before the camera-based software can behave like a predictable driver?

There are also particular instances where a human could react better. For example, someone drives in the wrong direction on a highway. Will FSD V12 understand what could happen and swerve out of the way without losing control of the car and prompting the person behind the wheel to take over? Keep in mind that we're talking about a Level 4 ADAS that functions properly in vetted environments.

Similarly, will Tesla be forced to constantly ask its customers for videos of them driving so it can further enhance FSD V12's training?

Expanding beyond California or other parts of the world is challenging because rules and driver behavior differ. Someone driving in Switzerland will obey all the traffic rules because the fines are enormous, and there's etiquette among traffic participants.

But in places like Eastern Europe, where traffic cameras aren't common, drivers may go over the speed limit regularly. How will the system blend in with other vehicles on the road? How will it figure out that you're supposed to drive fast on the Autobahn or keep to the right if you don't want to accelerate?

Moreover, if FSD V12 learns from humans, will it copy, combine, and behave like us?

Tesla's solution is to brute-force many videos into the system and let it figure out the best course of action. That may not work out in its favor in the short term. FSD could learn some not-so-neat tricks from us and apply them at the wrong moment.

Seven years ago, Microsoft tried to give the world the first ChatGPT. Tay was a chatbot that learned from humans. Sadly, it was exploited really fast and had to be shut down because it learned too many horrible things and ended up having extremist beliefs.

The good news, however, is that FSD V12 can manage calculations faster than FSD Beta V11, which uses both neural networks and C++ code. Hardware 3 cameras are locked at 36 frames per second (fps). Hardware 4 can go up to 40 fps. The more fps, the more information gets fed into the system. The more it knows, the faster it can act. So, the only way from here is up.

For Tesla, this is a massive win. After V12 is ready for wide release, Full Self-Driving does not depend on having programmers constantly improving little things anymore. It'll learn from those willing to share their video footage with the company and continuously improve. It'll become more human-like by itself. Hopefully, it'll do so without encompassing some of our known drawbacks. Feeling tired, bored, or losing focus are not good things to mimic.

Without sounding overly dramatic, it could one day become one of the autonomous driving solutions that will make future generations wonder why we ever needed a driving license.

The potential is enormous, but the journey to success… Will be exhausting!

Elon Musk said a month ago that FSD V12 would become a Level 4 autonomous driving system. If that turns out to be accurate, all the cars running this software should soon be capable of driving themselves in certain areas. Such software can take control of the vehicle when all the conditions for a safe operation are met. But you won't be able to turn on FSD and let the EV do the heavy lifting for a drive from Los Angeles, US, to Guadalajara, Mexico.

Meanwhile, Cruise and Waymo have driverless EVs roaming the streets of many American cities and plan to expand despite some recent shortcomings. The competition is fierce, and it'll only get worse for Tesla because when other players like Mercedes-Benz or BMW come to market with their solutions… They'll be completely ready and not rely on customers for corrections.

Returning to present times, we get to see the following:

• FSD Beta is still running on hardcoded info and only on Hardware 3 vehicles;
• FSD V12 might have just been demoed, but it's still nowhere ready for a public release yet;
• FSD Beta stays a Level 2 ADAS but is and will remain the basis for the accelerated neural networks-only V12 software;
• FSD Beta is expensive ($15,000) and might get even pricier in the future;
• a Coast-to-Coast robotaxi drive with a Tesla running FSD V12 will not be possible until the system learns the whole country from multiple points of view;
• the development of FSD was and remains pricey, while other players like Waymo want to play the same licensing-to-others card.

Finally, Tesla suggests it's close to cracking the true self-driving code that doesn't have tens of thousands of lines of human-written code. But it'll take a lot more time before we see the company's EVs capable of driving themselves everywhere without proper human supervision. Tesla won't take the steering wheel out of the car like Cruise or Zoox did anytime soon.

https://t.co/VzTxpktH1q

— Elon Musk (@elonmusk) August 26, 2023