Tesla’s Model S: Key Observations About Autopilot & OTA

Tesla Model S.jpg

VSI recently rented a Tesla Model S to examine the functionality of Autopilot as well as gain a deeper understanding of the overall architecture of the vehicle.

The vehicle we had access to was a 2015 Model P90D and configured with Autopilot 1.0 V8 which is Level 2 automation.  As a research company, VSI has been examining the building blocks of automation for nearly three years and are very familiar with the technologies that are used in the Tesla Model S.

What makes the Tesla Model S so interesting?

  • The over-the-air (OTA) digital communications of the Tesla Model S is by far the most interesting element of this vehicle and is probably the most critical element of the vehicle architecture. 
  • This vehicle talks a lot to the network and most of it done over Wi-Fi as we found out. Within a 24-hour period this vehicle exchanged over 50MB of data with the Tesla’s Mothership, a virtual private network (VPN) that manages the data exchange. About 30% of that data is flowing out of the vehicle.  
  • There have been multiple updates to Autopilot over the past few months, particularly with regards to v8.0 (rev. 2.52.22) where vast improvements were made to the performance of the radar.  Further improvements have been made to enable fleet learning, and is likely the reason the volume of data exchange is so high.
  • v8.0 accesses more raw data from the front-facing radar and new Tesla software processes those inputs in enhanced ways.
  • Architecturally, the Tesla E/E systems rely a lot on the main media unit which manages all communications and HMI elements. The consolidation of so many functions into a single domain is remarkable.  Many of the Autopilot calculations are made on the main media unit plus another control ECU separate from the main media unit. The vehicle camera module have their own processing so they take some load off the main media unit.
  • We think the Model S is a proxy for future vehicle architectures, at least those with partial automation features. And again, we think the OTA capabilities of this vehicle is the most important element of the vehicle architecture. This becomes more obvious when you visit a Tesla vehicle center where service bays are less than traditional vehicles. Short of mechanical failures, this vehicle is repairable over the network!      
Tesla Cluster.jpg

Autopilot 1.0 (VSI Profile

Tesla’s Tech Package with Autopilot costs $4,250 and it enabled through an over-the-air update. The current system consists of a forward-looking camera, a radar, and (12) 360-degree ultrasonic sensors.

  • The camera-based sensor comes from Mobileye (camera & EyeQ3 SoC.) – this is a single monochromatic camera.  However, fallout from a May 7 fatal accident led to a split between Tesla and Mobileye.  Mobileye will not supply hardware/software to Tesla beyond the EyeQ3 or beyond the current production cycles.
  • Bosch supplies the radar sensor/module. Autopilot v8.0 will have access to six times as many radar objects with the same hardware with a lot more information per object. Radar captures data cycles at 10 times per second. By comparing several contiguous frames against vehicle velocity and expected path, the car can tell if something is real and assess the probability of collision. The radar also has the ability to look ahead of vehicles it is tracking and spot potential threat before the driver can.

Control Domain

  • Perception and control is enabled through the Nvidia Tegra X1 processor.
  • Tesla provided their own self-driving control algorithms and some of the software algorithms fusing radar and camera data. 

HMI Domain

The Model S works with two tracking mechanisms:

  • Locking onto the car ahead or sighting the lane marks. When there’s difficulty reading the road, a “Hold Steering Wheel” advisory appears. If lane keeping is interrupted, a black wheel gripped by red hands and a “Take Over Immediately” message appear on the dash. Failing to heed these suggestions cues chimes, and if you ignore all the audible and visible warnings, the Model S grinds to a halt and flashes its hazards. A heartbeat detector is not included.
     
  • A thin control stalk tucked behind the left steering-wheel commands the cruise-control speed (up or down clicks), the interval to the car ahead (twist of an end switch), and Autosteer (Beta) initiation (two quick pulls back). A chime signals activation, and the cluster displays various pieces of information: the car ahead, if it’s within radar range, and lane marks, illuminated when in use for guidance. A steering-wheel symbol glows blue when your steering input is no longer needed, and ­Tesla’s gauge cluster also displays the speed limit and your cruise-control setting. 

The Model S is considered Level 2 but will change lanes upon command via a flick of the turn signal stalk (Auto Lane Change). To move two lanes, you must signal that desire with two separate flicks of the stalk. This function also can be used on freeway entrance and exit ramp.

Autopilot Software v8.0 (rev. 2.52.22) will warn drivers if they're not engaged with their hand on the wheel (for 1 minute if not following a car, 3 minutes if following another car.)

If a driver ignores 3 audible warnings within an hour, Autopilot v8.0 will disengage until the car has been parked.

Autopilot 2.0 (VSI Profile)

Although not tested yet, it is important for us to explain Tesla’s newer Autopilot 2.0. Eventually we will test this once the software functionality is more complete. At the moment, Autopilot 2.0 is less capable than Autopilot 1.0 because data is being collected via shadow mode to validate the performance of the advanced features.  

Tesla’s new Autopilot 2.0 hardware suite ('Hardware 2' or 'HW2') consists of 8 cameras, 1 radar, ultrasonic sensors and a new Nvidia supercomputer to support its “Tesla Vision” Tesla’s new end-to-end image processing software and neural net.  Available today on Tesla Model S and X and will be available on Tesla Model 3, the new Autopilot consists of the following:

  • Cameras: Three forward facing cameras (Main, Wide, Narrow), 2 side cameras in the B-pillar, rear camera above the license plate, left-rear and right-rear facing cameras.
  • Processor: Nvidia Drive PX 2 capable of 12 trillion operations per second. This is 40 times the processing power of 1.0 Teslas. 
  • Sonar: 12 Ultrasonic Sensors capable of 8 meters
  • GPS and IMU
  • Radar: Forward Facing Radar
  • Software: Tesla Vision that uses a deep neural networks developed in-house by Tesla

Enhanced Autopilot - $5,000 at vehicle purchase, $6,000 later - The vehicle will match speed to traffic conditions, keep within a lane, automatically change lanes without requiring driver input, transition from one freeway to another, exit the freeway when your destination is near, self-park when near a parking spot and be summoned to and from your garage. Tesla’s Enhanced Autopilot software is expected to complete validation and be rolled out to your car via an over-the-air update in December 2016, subject to regulatory approval.

Full Self-Driving Capability - $8,000 at purchase or $10,000 later -  This doubles the number of active cameras from four to eight, enabling full self-driving in almost all circumstances, at what Tesla claims will be a probability of safety at least twice as good as the average human driver. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat. All the user needs to do is get in and tell their car where to go. The autopilot system will figure out the optimal route, navigate urban streets (even without lane markings), manage complex intersections with traffic lights, stop signs and roundabouts, and handle densely packed freeways with cars moving at high speed. This feature is expected to roll-out by the end of 2017.

Conclusion:

Tesla is by far the most important production car and is a proxy for future passenger cars.  The software enablement through over-the-air updating is the most striking differentiator in our opinion. The rate in which new features, updates and patches are deployed is astonishing. The volume of data is also a good indicator of what the requirements of cloud connected car should be.

Although not relevant so much for the purposes of VSI, it should be mentioned that the fit and finish of this vehicle is sub-par when compared to German vehicles. This is especially true for the interior components with the exception of the center stack, which is outstanding in quality and functionality. The same can be said about the instrument cluster, as would be expected in a digital vehicle like this.

The infotainment systems in this vehicle is enhanced by the large display but much more intuitive than most conventional vehicles. There is no switchgear in this vehicle at all exceptt for steering wheel stalks and controls.   

Performance is another key attribute of this vehicle. Power management and battery management are outstanding for this vehicle and can be attributed to the all-electronic powertrain as well as the ability to update the power management software via OTA.

Autopilot works very well and gets better all the time. Especially v8.0 where enhancements made to the sensor performance as well as the reduction of false positives are critical.  The self-learning capabilities are reflected in the amount of data that is now exchanged between the mothership and the car itself.

In normal driving modes the Tesla Model S is very tight and performance oriented. Handling is surprisingly good for a vehicle that weighs nearly 5,000 pounds. Acceleration is outstanding and rivals or exceeds most high-end performance (internal combustion) sedans. Braking is also very good, in part enhanced by the regenerative braking that feels like engine brake on conventional vehicles.   

Autopilot HW2 (v8.1) will undoubtedly continue the path that Tesla is on. We don’t have any reason to doubt Tesla’s abilities to realize full automation with the new hardware platform.