00:02 → 00:04
Alright, I think we are now live
00:08 → 00:14
Great, and I think it's maybe better to go
00:17 → 00:20
We'll see you see more horizontal than vertical
00:21 → 00:25
All right. Hey guys
00:26 → 00:29
Let's see, and you can sort of read some of the comments there
00:31 → 00:36
All right, we're gonna let's see so here we are we're just
00:38 → 00:40
Leaving Tesla headquarters and
00:40 → 00:47
We just dropped a random pin at Stanford or whatever we can clear that though and I don't know we can pick it I'll just
00:50 → 00:52
AI end-to-end
00:52 → 00:54
Let's see how it does
01:05 → 01:12
Well, let's see, okay, so here we're kind of added to kind of random spot
01:14 → 01:16
That's gonna be pretty wobbly
01:18 → 01:21
It's it's really smooth sailing in the car itself
01:21 → 01:28
and here we're encountering some construction and
01:30 → 01:32
the car is just
01:32 → 01:36
Driving around the construction. So it has never seen this construction before
01:38 → 01:41
It is near the headquarters, but this construction is relatively new
01:43 → 01:55
Let's learn exactly the right lane, but now it's going over to the right lane. Okay
01:58 → 02:02
You can see the destination. Hopefully don't make me sister
02:04 → 02:15
Let's see, is this is this working this live stream working? Hello?
02:36 → 02:48
I mean, obviously it's a little boring because we're at a red light
02:50 → 02:52
And there's a lot of traffic
02:54 → 03:03
So I'm gonna bore people to death here, but we're just sitting at a red light in Palo Alto
03:06 → 03:16
Yeah, I think it like these videos like this are maybe more interesting if they're edited and sped up
03:18 → 03:25
Let's see, I'll try to answer comments if I see comments
03:55 → 03:57
Man, this is a long red
03:59 → 04:02
Well, the car is patiently waiting for the car light to change
04:03 → 04:26
Made a good left. It's kind of hard to tell maybe from the live stream, but
04:27 → 04:41
The car is driving very smoothly. I think I will rely on others to take this video
04:42 → 04:44
edit out the boring bits and
04:45 → 04:52
Speed it up. It's smooth right there
05:01 → 05:03
entirely AI and
05:03 → 05:08
Cameras just like our brain works, which is neural nets and eyes
05:12 → 05:19
Yep, just slow down for speed bump, which is cool to slow down for another speed bump
05:20 → 05:27
And we did not program in if there's no part. There's no line of code that says slow down for speed bumps
05:28 → 05:31
So it is doing this based entirely on video training
05:34 → 05:36
Yeah, and there was a we just saw
05:37 → 05:39
There's a bicyclist
05:40 → 05:42
again, there was no line of code that says
05:43 → 05:47
I give clearance to bicyclists. It is just doing what people do
05:48 → 05:57
It does it let it it can read science without ever being taught to read
06:17 → 06:24
Once again, there is no line of code that says stop at a stop a sign or wait for another car
06:27 → 06:34
Well who came there's not like wait X number of seconds nothing like that nothing is that this is all nets
06:36 → 06:37
Nothing, but nets
06:37 → 06:54
No, we're gonna get to our destination. We'll change the pin. It's like
06:55 → 07:01
Profit somewhere else random. Maybe we'll run into Zuckerberg and we can make a challenge him to a fight. That'd be fun. Let's mess it up
07:03 → 07:05
He's not traveling
07:06 → 07:17
Somewhere to his backyard
07:17 → 07:19
Here we are at a roundabout
07:19 → 07:21
So roundabouts were obviously pretty complicated
07:24 → 07:26
Yep, it just waited for those two cars to go
07:27 → 07:29
and then did the turn
07:31 → 07:35
Got a bit of being somewhat repetitive about this, but we have never programmed in the concept of a roundabout
07:36 → 07:39
We just showed a whole bunch of videos of roundabouts
07:43 → 07:46
Yeah, I mean for you definitely need a
07:47 → 07:52
Lot of training data to a lot of video training data in order to make this work
07:52 → 07:54
so it's and you need a
07:56 → 07:58
billions of dollars of
07:58 → 08:06
Training hardware and you need to how to run the neural net training hardware. So it's not like easy
08:08 → 08:14
The mind-blowing thing is that there are no there. There's there's no heuristics. There's no like
08:15 → 08:18
Lines of code like this. There's a guy on a scooter. It's never
08:19 → 08:22
It doesn't know what a scooter is. It doesn't know what paddles are
08:23 → 08:33
It's literally just been given a lot of video and it's doing all of this on hardware 3
08:34 → 08:37
With about a hundred watts of inference compute
08:38 → 08:41
So it's not like obviously it's not something like massive data center
08:41 → 08:44
And if if we were offline there would be no difference
08:44 → 08:50
This is locally all the inference that's happening is locally. It does not need an internet connection and
08:51 → 08:53
and obviously that's necessary because if you
08:53 → 08:57
You know lost your cellular internet connection the cat in the car needs to drive safely
08:59 → 09:02
But we could be somewhere that where there is no internet connection
09:06 → 09:10
And it's never seen the roads before it doesn't matter
09:14 → 09:21
Oh, yeah, we're running at the full frame rate, so it's taking eight cameras at 36 frames per second
09:24 → 09:31
The pure AI version runs better than it runs faster than the version that is a mixture of
09:32 → 09:34
normal software and AI
09:35 → 09:39
In fact if it would run it faster than 36
09:41 → 09:43
frames per second
09:43 → 09:46
Except the cameras are currently only capable of 36 FPS
09:49 → 09:51
You know back to the envelope
09:51 → 09:56
Frame number is we think it could probably run at 50 frames a second and
10:02 → 10:06
Reality of the roads are basically designed around 24 frames a second
10:09 → 10:17
Basically it's similar to I just I just forgot to go to our destination, so I'll just pick a new destination here
10:20 → 10:28
I'm like hello assassins if you want to get me now is your chance. You just need to be in Palo Alto
10:30 → 10:32
Like this the assassin count is low in Palo Alto
10:38 → 10:40
So we're just
10:41 → 10:43
Right, we're just gonna go to Palo Alto
10:44 → 10:46
Yeah, I think we're gonna go to Palo Alto
10:47 → 10:49
So we're just
10:55 → 10:59
Sure just dropped a sort of random, and we don't know where we're going really
11:01 → 11:04
Just somewhere in Palo Alto
11:06 → 11:11
But it's this also worth noting it and show you may want to actually sort of provide some come with some commentary as well
11:12 → 11:14
But we've got
11:18 → 11:22
FST 12 test drivers around the world
11:22 → 11:28
FST 12 test drivers around the world, so we've got we've got people in I think like New Zealand and like Thailand Norway
11:39 → 11:42
Right exactly just like a human
11:43 → 11:49
Can go travel to a country they've never been to before and rent a car and drive around to be you know
11:49 → 11:57
Maybe not quite as good as someone local, but you can still rent a car, and you know foreign country and drive around and
11:58 → 12:03
Then we just have like some students over there that one of them stumbled briefly into the road and the car
12:05 → 12:10
Drove around them
12:10 → 12:12
lovely day in California
12:13 → 12:20
the beautiful Stanford campus playing a little
12:21 → 12:34
Four seasons change the pretty soon we had also like you know instructions following so you could like say
12:35 → 12:37
Lane change to the left most lane. Yeah
12:37 → 12:42
Pull over here or something then the car should even respect those kinds of commands. Yeah exactly
12:44 → 12:46
Yeah, so here we are at a roundabout
12:50 → 12:51
It's you know
12:51 → 12:58
I've never I don't have been to this roundabout before and the car is not specifically trained on this roundabout
13:01 → 13:03
Yeah, that's exactly so
13:06 → 13:09
Correct for the correct amount of time drove smoothly around the roundabout
13:11 → 13:18
Again I will be somewhat repetitive that there yet there is no line of code that says whatever this is a roundabout
13:18 → 13:20
There is not nothing that says
13:21 → 13:24
you know X number of seconds, which is what we have in the
13:26 → 13:30
Explicit control stack that's the sort of version 11
13:31 → 13:34
there's over 300,000 lines of C++ in the
13:35 → 13:38
explicit controls control stack of version 11
13:38 → 13:42
And there's basically none of that in version 12
13:42 → 13:45
Just because there's no lines of code doesn't mean that it's not controllable
13:45 → 13:51
It's still like quite controllable on what you want by deciding data. Now you have to program with data instead of programming that
13:54 → 13:56
data curation
13:57 → 14:04
And then whenever we find that there's something so if the car doesn't perform perfectly
14:04 → 14:09
we give it more examples of what it should do in that situation and
14:11 → 14:14
You know updates the
14:14 → 14:16
the weights and
14:16 → 14:18
Then it works
14:18 → 14:25
Cleaning and making sure the data that goes in is like the good driving it and not like the bad drivers from there are bad drivers
14:25 → 14:29
Yeah, absolutely. It's very important. The quality of the data is very important. So
14:31 → 14:33
large amounts of
14:33 → 14:37
Mediocre data do not may improve driving
14:38 → 14:42
Yeah, it's quite the opposite actually. Yeah makes it worse. So
14:43 → 14:46
That's why the data curation is actually
14:47 → 14:51
Quite difficult and I should say that there is quite a bit of software around
14:52 → 14:56
What data you know, so selecting what data to train the system?
14:56 → 15:01
So software that runs in the car is minimal but the software in the back end to train is like much larger
15:01 → 15:05
Yeah, it's more sophisticated. Yeah, exactly. So we do we do use like
15:06 → 15:08
normal software for
15:08 → 15:11
you know C++ basically for the Python for
15:12 → 15:16
deciding what data to select from the fleet and
15:18 → 15:22
figuring out what what is the high quality data versus the
15:22 → 15:27
The pretty good data and once we have a model
15:27 → 15:32
We also ship those models in shadow mode to cars and every time it disagrees with what the user did
15:32 → 15:38
Yeah, exactly. I get the data back and then you know, that is more valuable than just collecting, you know, random data
15:39 → 15:41
Yeah, exactly. So
15:47 → 15:51
Yeah, so we feel good about actually having a
15:52 → 15:54
very rapid virtuous cycle
15:56 → 15:59
When there is an intervention in the fleet
16:00 → 16:06
That with that intervention automatically being uploaded to the to training
16:07 → 16:12
being integrated with training and then updating really just the the weights
16:14 → 16:16
the it's not the
16:17 → 16:22
It's not the binary that's that's changing. It's the weights. It's not the execution binary. It's the
16:22 → 16:25
Just really the weights
16:32 → 16:52
So I have not intervened
16:54 → 17:08
The drive has been butter smooth, you know again being being somewhat repetitive
17:09 → 17:11
repetitive about being repetitive in fact
17:12 → 17:18
But we have not programmed in the concept of traffic lights. So there's not like
17:18 → 17:23
This is a red light. This is a green light and this is the traffic light position
17:24 → 17:27
We have that in the normal stack, but we do not have that in
17:28 → 17:37
V12 this is just video video training. Like I said, nothing but neural nets
17:39 → 17:43
And yet it knows which light flies to it
17:44 → 17:47
And it stops at a red light
17:47 → 17:49
accelerates at a green light
17:49 → 17:54
now one of the sort of maybe slightly funny challenges we've had is that
17:55 → 18:02
Since the car is being trained on what humans do humans almost never stop fully at a stop stop Street
18:02 → 18:05
So when they get to a stop sign humans actually
18:06 → 18:13
Almost never go to zero miles an hour. They they may think they did but usually they they're doing at least a
18:14 → 18:16
few miles an hour
18:16 → 18:18
At a stop coming up to a stop Street
18:20 → 18:22
sometimes, you know people go faster than that, but
18:25 → 18:32
Regulators are somewhat they were quite insistent that we we go to a complete come to a complete stop at
18:33 → 18:35
at stop signs and
18:35 → 18:37
When we looked at the data
18:37 → 18:41
Only 5% of the time do humans actually stop fully
18:44 → 18:50
0.5% Wow. Okay. So basically people almost never fully stop at stop signs
18:51 → 18:53
So we had to
19:01 → 19:05
Yeah, they might like semi stop and then move a little bit and that kind of thing so
19:06 → 19:08
so we had to like
19:08 → 19:11
pull the fleet for rare examples that
19:12 → 19:16
less than 1% of the time when people actually come to a full stop and
19:16 → 19:22
Artificially train the system to stop at stop signs at the insistence of the regulators
19:43 → 19:46
like I said, this is a little slow because uh
19:47 → 19:50
We're driving around and basically rush hour. Oh
19:52 → 19:54
Intervention, sorry
19:54 → 20:00
Okay, so that's our first intervention because the car should be going straight
20:02 → 20:04
This model has small regression
20:07 → 20:11
Okay, but you know, that's why we've done releases the public yet
20:13 → 20:18
So an intervention at this traffic light of that that's the first intervention in the whole drive
20:21 → 21:11
Yeah, so just it just did a merge traffic merge super smooth so for that intervention that we just had
21:12 → 21:14
the solution is essentially
21:15 → 21:20
To feed the network a bunch more video of traffic lights
21:22 → 21:26
So that was a that was a controlled left inner controlled left turn
21:28 → 21:31
Where there was green light for the left hand, but not a green light to go straight
21:32 → 21:34
and so we'll feed it a bunch of video of
21:36 → 21:38
Control left turns and then it'll work
21:38 → 21:56
In the next two weeks we're going to release a shadow mode release where we're going to run this network in the background and then check
21:56 → 21:57
when for example in this case
21:57 → 22:00
We would have wanted to go but then the driver would not have gone
22:01 → 22:03
So we can just like check that. Okay, we
22:03 → 22:08
Then we can get the data back into the labellers and the labellers say who was correct. Yeah
22:10 → 22:15
We should not have gone so you don't have to even get the intervention
22:15 → 22:19
You can just like passively observe what we want to do and this is what happened in reality
22:21 → 22:43
Right, so here we have another controlled left
22:44 → 22:48
Traffic intersections, although it's good with greens. So it's kind of an easy one
22:59 → 23:01
There's the smoothness of the control
23:01 → 23:07
Like the car though just how smoothly the car is behaving is it's hard to convey I think on camera, but it's just
23:08 → 23:10
Super smooth. Yeah, you have to feel it really
23:11 → 23:14
So that's making a left turn
23:15 → 23:22
So it's gotten itself into the left turn lane
23:23 → 23:27
Again, we've never programmed in the notion of a turn lane or anything like that
23:30 → 23:37
Never there's no line that says I think that it has there's no line of code about traffic lanes at all
23:43 → 23:46
Internal to its mind. It might know this concept
23:49 → 23:55
Just not explicitly asked for it. Yeah, just like humans. Yes
23:55 → 24:08
Yeah arrived at this random pin
24:15 → 24:21
Yeah, so it cut so this is yes, this is pretty cool so the car just just pulled over to the side of the road and
24:24 → 24:30
Yeah and part so it would it knows at the end of its destination based on
24:31 → 24:36
The video that's received that at the end at the end of the destination you pull over to the side and park
24:36 → 24:41
So it gets the exact pin location in addition to the navigation route. So, you know, just
24:41 → 24:43
Pin is close. It's a good spot
24:44 → 24:48
Spot here puts over here, but in part big parking lots, but that might not be any map
24:48 → 24:52
Yeah, you should just go as close to the pin as possible, right?
24:53 → 24:57
Exactly. Even without any route or anything like that. Yeah, and in fact
24:58 → 25:00
It's a robo-taxi world. It would actually just
25:00 → 25:07
You know probably perhaps know what you look like and say and just literally look for you. Yeah
25:07 → 25:09
Yeah, if you have a picture or something you can yeah, exactly
25:09 → 25:12
It's like if you're signed up and it's like, you know
25:12 → 25:17
Just say you don't have to but if you want the car to literally find you
25:17 → 25:20
yeah, so you just have to send it a picture of you and it will
25:21 → 25:25
Will look for you and and and wait for you. Yeah, exactly
25:25 → 25:28
And then you can also say drop me off at the Starbucks or something
25:28 → 25:34
Yeah, you drop me off at the building's entrance. Yes as far back as possible as opposed to somewhere random. Yeah, exactly
25:36 → 25:38
Guess let's see. Well, I
25:39 → 25:42
Don't turn probably head back to HQ
25:43 → 25:45
Or it is a enduring HQ
25:49 → 25:51
Where does he live?
25:51 → 25:53
It's like somewhere around here
26:01 → 26:03
Mean, you know, we can knock on the door and whatever
26:05 → 26:09
Whatever Google says they say hi, I guess. Yeah, we'll say hi. We'll be friendly
26:11 → 26:17
Well, it's just a polite inquiry as to whether you would like to engage in hand-to-hand combat
26:19 → 26:22
You know if it's a
26:22 → 26:25
Yeah, you know not inconvenient
26:26 → 26:28
perhaps you would like to
26:32 → 26:34
Okay, so this is
26:34 → 26:38
We have no this is literally we just googled it. We don't know if this is where he actually lives or not
26:38 → 26:40
But we'll just go there
26:49 → 26:54
So now we're
26:55 → 26:57
We're at least going to where Google says
26:58 → 27:00
You know, it's like quick lives
27:01 → 27:05
You know, I don't think this is really, you know, it can't be really considered doxing if we just googled it
27:11 → 27:15
See that it's the drive correctly to where Google thinks you live
27:54 → 27:57
Doing a pretty good job of driving through the Palo Alto, so
27:58 → 28:12
For lovely even the speed assist everything is automatic, right? It's gonna stop. Yep. Okay good
28:13 → 28:15
Stopped at the red line
28:18 → 28:25
Pretty much what a person would do. Yeah, some cyclists over there
28:26 → 28:41
I'm Palo Alto really is a
28:41 → 28:45
Lovely town. It's it feels like a Truman Show
28:46 → 28:49
It's best for families. Yeah
28:49 → 28:52
It's like everything's perfect
28:53 → 29:10
I'll turn it to see the comments
29:11 → 29:14
Yeah, there's no lane drifting it's super smooth in the lane
29:15 → 29:24
And it doesn't confuse a bike lane with a real lane or anything like that
29:25 → 29:35
Yeah, this is the ride is super comfortable. As I see messages popping up. I'll try to answer them. I was a lot of messages actually
29:42 → 29:53
It's not it's gonna
29:54 → 30:06
Headed to like Edgewood Drive. I don't mean we don't know if this is actually probably not because it's like I would expect it'd be like
30:06 → 30:09
a lot of security and so
30:09 → 30:11
Yeah, well, this is
30:11 → 30:14
We are at the spot that's roughly where
30:15 → 30:22
But I don't think it just doesn't seem look probably realist because I have probably security and stuff
30:23 → 30:31
Anyway, um, but you know, it's sort of a little nice driving around Palo Alto
30:54 → 31:09
I mean Zuckerberg did say like name the date in bold letters or whatever on
31:10 → 31:15
You know, whatever platform he's on and so I'm like, hey, what about right now? So
31:19 → 31:21
Does now work
31:21 → 31:28
All right, well we can find him so we are heading back to Tesla headquarters
31:32 → 31:35
It's like overly cautious about the pedestrians
31:51 → 31:53
Yeah, of course
31:53 → 31:56
The car is very very polite with pedestrians
31:59 → 32:03
Stopped wait the for the couple to pass by and now it's continuing to drive
32:03 → 32:17
So here we are in Palo Alto driving on to pure AI
32:18 → 32:41
It's headed back to Tesla global engineering headquarters at Palo Alto
32:42 → 33:15
see how does react to low visibility conditions is one of the
33:17 → 33:22
Questions like at night and stuff or rain and snow
33:39 → 33:44
Yeah, and one of the reasons we definitely
33:45 → 33:51
Need training from all around the world. Is that the weather in California is amazing and it's a
33:52 → 33:58
Song goes like basically never rains in California. It's like sunny and nice almost all the time
34:00 → 34:04
The drivers are pretty nice to you. Yeah drivers here are very quite polite
34:11 → 34:13
Yeah, it's like we need
34:13 → 34:21
You know situations with like there's a parade or a crowded situation or there's a lot of pedestrians for whatever reason
34:22 → 34:27
Gonna be in a safe, but also confident and we compete in that. We don't want to be
34:28 → 34:30
Too skittish and I get the brakes all the time
34:33 → 34:35
Yeah, but it like it like so
34:37 → 34:44
Like it's winter basically in New Zealand and so we have like this how we can conditions there that we can train in
34:45 → 34:49
That's a dry
35:14 → 35:19
Yeah, it is very conservative with bicyclists and pedestrians
35:42 → 35:44
Yeah, okay, so this is a tricky one
35:45 → 35:51
So this is turning left onto middle field in Palo Alto with where visibility is
35:55 → 36:00
Sounds like the cars come from both sides at pretty high speeds
36:02 → 36:04
Did it yeah, no problem great
36:04 → 36:10
So I'm protected left on to a high-speed road fairly high-speed road
36:11 → 36:31
No problem. Yeah, so b12 will be I'd say actually smart Simon. Yes
36:44 → 36:46
I call it actually smart
36:51 → 36:53
It's like do you don't you want some air sets
36:54 → 36:56
Of course you do
36:56 → 37:11
Speed up is quite nice. Yeah, exactly like the
37:12 → 37:14
very very intuitive smooth
37:15 → 37:17
speed up and acceleration in turns
37:28 → 37:31
Yeah, exactly it's like currently set to 85 but it's
37:31 → 37:36
It's it's ignoring the set speed. It's driving at what would be intuitively the right
37:37 → 37:39
Speed for people to drive that
37:43 → 37:44
There's two lanes here
37:44 → 37:49
There's a lot more cars in that lane fewer cars in this lane and it's going straight
37:49 → 37:51
So it picked lane with the fewest cars
38:56 → 39:03
Yep, exactly, it's not no explicit distance that's programmed in for how close you should be behind a car. It's
39:04 → 39:06
Again just video training
39:06 → 39:09
Yeah, it's intuitive beats and right
39:11 → 39:13
Like what would humans generally do
39:14 → 39:20
And if it picks like a reasonable follow distance and does that? Yeah, the nice thing is, you know for bad weather conditions, for example
39:20 → 39:22
It automatically increases speed. Yep
39:23 → 39:25
or decreases
39:25 → 39:27
Increase the distance. Yeah distance. Yeah
39:29 → 39:42
This traffic takes a while. Oh, this is the
39:43 → 39:46
Yeah, I'll Camino and page mill a
39:47 → 39:49
classic Silicon Valley intersection
39:49 → 39:52
I've seen this intersection for
39:52 → 39:54
30 years basically
39:57 → 40:03
These are in quarters is HP used to be HP. Yeah. Yeah exactly the Tesla
40:03 → 40:09
Global engineering headquarters in Palo Alto are the former head Hewlett Packard headquarters
40:09 → 40:16
Yeah, exactly we're it's an honor to be for our global engineering headquarters to be the
40:16 → 40:18
kind of where the
40:19 → 40:21
birthplace of Silicon Valley was
40:22 → 40:25
As you can see lovely place
40:27 → 41:10
Well, actually, we'll see how it does in the parking lot. So because parking lots are complicated, especially the Tesla parking lot
41:10 → 41:12
Which is jam-packed
41:12 → 41:14
It's a probably pretty full even on a Friday
41:14 → 41:20
It's 7 o'clock on Friday, you know, this is a fairly tricky flight to get to
41:29 → 41:36
Controlled lefts and and to straight basically to two turn lanes and two straight lanes
41:40 → 41:42
After this turn, which is also interesting
41:53 → 41:59
It's like Dylan and Mars. Yeah, it's good. Exactly. It's gonna turn and merge
41:59 → 42:08
Real smooth mode now one of the interesting things about pure AI driving is that it actually doesn't need a map at all. So we could
42:32 → 42:54
Delete the navigation system simply give it a GPS point and say get to this GPS point somehow
42:54 → 43:01
We're not gonna tell you how it's you could say like you see that building in the distance go to the GPS point
43:01 → 43:03
And then you'll get to this GPS point and then you'll get to this GPS point
43:04 → 43:09
We're not gonna tell you how it's you could say like you see that building in the distance go there
43:10 → 43:16
And it would it would do that even with no it would just it might make some, you know, get to
43:18 → 43:20
Go down a road. That's a dead end and then have to reverse out
43:20 → 43:25
But it would basically be able to do what a human can do where if you said, please go to you know
43:28 → 43:32
Get a point to point to point at something and guys they go there so that's going into the parking lot
43:33 → 43:40
Yeah, there is no explicit map of the parking lot parking lot so now it is just just trying to get to a GPS point
44:25 → 44:30
Yeah, that's how it got to the point and that's it all right, so that was the
44:32 → 44:34
FSD 12 beta drive super smooth
44:36 → 44:39
One intervention which will fix with a bit more training data
44:44 → 44:46
I really have to like, you know, if this was a
44:47 → 44:53
Uber driver pretty much apart from that one intervention five-star. Yeah, so alright. Thanks everyone for tuning in