1. Attachments are working again! Check out this thread for more details and to report any other bugs.

Consumer Reports: Autopilot and Full Self Driving

Discussion in 'Tesla' started by bwilson4web, Oct 30, 2020.

  1. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,134
    15,394
    0
    Location:
    Huntsville AL
    Vehicle:
    2018 Tesla Model 3
    Model:
    Prime Plus
    First off, I'm no fan of Consumer Reports (CR) because of a decade of poor Prius reporting. Yet their annual subscriber surveys came back with a strong affirmation of the Prius. CR never adjusted their review criteria to match what the owners reported. However, I do have an online account and looked for Autopilot articles:
    • "Tesla's 'Full Self-Driving Capability' Falls Short of Its Name" - The $10,000 option doesn’t make the car self-driving, though it does offer a host of advanced features. CR evaluated all of them.
      By Mike Monticello Last updated: October 23, 2020
    • "Cadillac's Super Cruise Outperforms Other Driving Assistance Systems" - Other automakers close in on Tesla's Autopilot, now a distant second, in Consumer Reports' new ratings of 17 systems
      By Mike Monticello October 28, 2020
    "Tesla's 'Full Self-Driving Capability' Falls Short of Its Name" lists the features but the report is from "version 2020.24.6.4" which today is "2020.40.8" and significantly improved such as the GPS fix. So CR reviewed: Autopark; Auto Lane Change; Summon; Smart Summon; Navigate on Autopilot, and; Traffic Light and Stop Sign Control. What bothered me was failure to quantify the failures.

    Autopark does not always work but it would be useful to document how many times it worked, failed, or was attempted. A subjective 'score' by the CR staff does not hold weight. Reporting the occasional failures is OK provided the measured rate is shared.

    “It seems like Tesla is focused on being the automaker with the most features rather than ensuring that the features work well,” Funkhouser says. “Its time and energy could be better spent on developing a driver monitoring system for Autopilot to significantly improve the safety and usefulness of that system.”

    In this quote, I disagree. I would rather have as many features, even buggy, than just one that is perfect. But their fix, "a driver monitoring system" is not the answer.

    "Cadillac's Super Cruise Outperforms Other Driving Assistance Systems" starts out with a strong CR endorsement of driver in-the-loop monitoring, the Super Cruise approach:

    “The evidence is clear: If a car makes it easier for people to take their attention off the road, they’re going to do so—with potentially deadly consequences,” says William Wallace, manager of safety policy for Consumer Reports. “It’s critical for active driving assistance systems to come with safety features that actually verify drivers are paying attention and are ready to take action at all times. Otherwise, these systems’ safety risks could end up outweighing their benefits.”

    The rest of the CR article makes sure to restate their endorsement of the Super Cruise approach and neglect to mention what others, like Tesla does. This is a significant omission because different systems have different criteria. I have experience with BMW i3-REx (magic eye), Toyota TSS-P, and Tesla Autopilot, these were not quantified but just an arbitrary score applied without the backup data! Facts and data are needed and this 'trust us' does not work after a decade of CR Prius pissing.

    I'm not going to cut-and-paste the arbitrary scores assigned but list these were their evaluation criteria: Capability and Performance; Keeping the Driver Engaged; Ease of Use; Clear When Safe to Use, and; Unresponsive Driver. As mentioned before, recording the success and failures works, not an arbitrary score dreamed up by CR staffers.

    CONCLUSIONS

    Both articles endorse only Super Cruise monitoring of the driver. They did not even list what Tesla, Toyota, or other vendors do. But this is not the first time CR makes up their minds, don't inject facts and data, like they did the the Prius and the CR subscribers be d*mned.

    They omitted that Super Cruise is an extra cost, $2,500 to $6,150, but Autopilot is standard on all Tesla. Autopilot is getting more use and monitoring so Tesla reports, 2020 Q3:

    In the 3rd quarter, we registered one accident for every 4.59 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.42 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.79 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 479,000 miles.
    Source: Tesla Vehicle Safety Report | Tesla

    One wonders if anyone is evaluating Super Cruise performance in the real world ... or the sample size may be too small.

    The biggest CR omission is where Autopilot and Super Cruise can be used:
    upload_2020-10-30_5-33-48.png

    Supercruise is limited to certain roads. In contrast, Autopilot works every where. So which one provides more miles of service?

    In the past, I would send this report to Consumer Reports but experience has shown they really don't want another point of view. They've made up their mind so don't confuse them with the facts and data. I'm not bitter but interested in sharing an accurate synopsis.

    Bob Wilson

    ps. CR did not mention "phantom braking" with Autopilot which experienced Tesla owners fully know about.
     
    hill and 3PriusMike like this.
  2. hill

    hill High Fiber Member

    Joined:
    Jun 23, 2005
    19,674
    8,069
    54
    Location:
    Montana & Nashville, TN
    Vehicle:
    2018 Chevy Volt
    Model:
    Premium
    not just Tesla that experiences Phantom braking, as I've driven 4 different manufacturers safety versions where it occurs. Presumably the rationale is that better to Phantom break then not to break at all while a driver is inattentive.
    .
     
    #2 hill, Oct 30, 2020
    Last edited: Oct 30, 2020
  3. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,134
    15,394
    0
    Location:
    Huntsville AL
    Vehicle:
    2018 Tesla Model 3
    Model:
    Prime Plus
    Ahem "brake" but "break" still makes me smile.

    Bob Wilson
     
  4. mikefocke

    mikefocke Prius v Three 2012, Avalon 2011

    Joined:
    Nov 3, 2012
    3,641
    1,626
    0
    Location:
    Sanford, NC
    Vehicle:
    Other Hybrid
    Model:
    Limited
    Reading CR is like reading Motor Trend. Lots of prose, little in the way of facts. Almost as much fun and less accurate than political polls. I long ago abandoned CR. My wife wanted it, I've read two recent issues. I wouldn't waste my money. Just as I respect a poll that publishes its methodology, I want quantification so I can develop my own ranking based on my own criteria.

    TSS-P 2.n is good enough that in my first hour I was using it and on the second I was trusting it on an interstate. I have no interest in using a computer controlled drive assist to completely control the car in urban driving. In a dicey situation, I want to make the choice. I've done it wrong but also right thousands of times so I trust myself to be paying attention. So don't give me a limited route test, give me the results that occur when a deer jumps out, you see a runner on the road at dusk dressed in black, etc. If I'm forced to watch for those edge cases, then I will. If I think the computer and sensors are doing it I probably won't.

    Still the Tesla statistics are impressive. Not that I'd pay $10k more than the already $20+k more a Tesla would cost me.
     
  5. hill

    hill High Fiber Member

    Joined:
    Jun 23, 2005
    19,674
    8,069
    54
    Location:
    Montana & Nashville, TN
    Vehicle:
    2018 Chevy Volt
    Model:
    Premium
    seems that between the PC's speech translation & my failure to double check - it's bound to happen again. irony - i'd re-done the post, even prior to reading this one.
    :oops:
    .
     
    bwilson4web likes this.
  6. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    Yet it is always the same Tesla Fans that rises to the defense for Tesla out of all the members of Priuschat. Now I know I am the only one vocal for Mirai, but does it not make you wonder why so little interest in Tesla within such a large base of members??
     
  7. hill

    hill High Fiber Member

    Joined:
    Jun 23, 2005
    19,674
    8,069
    54
    Location:
    Montana & Nashville, TN
    Vehicle:
    2018 Chevy Volt
    Model:
    Premium
    not so, I've chided against Tesla's shortcomings on several threads here on PC, as well as TeslaMotorsClub. By the same token, if I got on the TMC site & started boasting about the electric Porsche - being greater, I would expect push back. So it shouldn't be any surprise with the hydrogen car shortcomings, that there would be pushed back as well. Surprised that's such a mystery.
     
    Trollbait likes this.
  8. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    But where are all the Tesla fans at on this site if Tesla is so great?!
     
  9. Prodigyplace

    Prodigyplace Senior Member

    Joined:
    Nov 1, 2016
    11,696
    11,317
    0
    Location:
    Central Virginia
    Vehicle:
    2017 Prius
    Model:
    Two
    Not braking could break the driver though. :rolleyes:
     
  10. hill

    hill High Fiber Member

    Joined:
    Jun 23, 2005
    19,674
    8,069
    54
    Location:
    Montana & Nashville, TN
    Vehicle:
    2018 Chevy Volt
    Model:
    Premium
    weather former prius owners keep in contact here or not is not relevant. One of the original members, Daniel, when he got his first Tesla, I don't think he ever came back here to PC. others like myself, or Bob, Etc, still share our experiences with the Prius noobs or with people inquiring how to solve varying problems with their cars. Why expect current or former Prius owners that now have tesla or ford or vw owners to be here - or not .... is that relevant?
    .
     
  11. Mark57

    Mark57 2021 Tesla Model 3 LR AWD

    Joined:
    Aug 14, 2009
    2,945
    2,735
    0
    Location:
    OK
    Vehicle:
    Other Electric Vehicle
    Model:
    N/A
    I honor my Prius roots, having owned 2. I would not be this far into EV's yet if it were not for my Prius and Prime experience. I am keenly aware that some here look down their noses at me for having sold my Prime and ordering a Tesla. That's their hangup, not mine. I think my path to Tesla was quite natural. I still hang around because I've been here since 2009 and still help when I can. No I don't reply to every post, but I read almost all of them, always did.

    PS, I also use the ignore button when obvious trolls arise. It works great.
     
    hill, Prodigyplace and bwilson4web like this.
  12. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    It’s relevant to the number of members on this site who don’t give a damn about Tesla.
     
  13. Trollbait

    Trollbait It's a D&D thing

    Joined:
    Feb 7, 2006
    21,736
    11,325
    0
    Location:
    eastern Pennsylvania
    Vehicle:
    Other Non-Hybrid
    Why would it matter to them?
     
  14. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    It doesn’t, that’s my point. Tesla is not king of the EV world as much as you would like it to be. The President of Toyota yesterday put it in prospective of how far Tesla still has to go to be anything like Toyota in scale. 500,000 annual vehicle sales is a drop in the bucket for Toyota.
     
  15. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    Have nothing against Tesla per say, just not my choice for several reasons. I do take issue against members on this site who misrepresent Toyota products and continue to bad mouth FCEV vehicles, as they have a vested interest in Tesla stock. No troll here - just voicing my opinions and keeping it real.
     
  16. Trollbait

    Trollbait It's a D&D thing

    Joined:
    Feb 7, 2006
    21,736
    11,325
    0
    Location:
    eastern Pennsylvania
    Vehicle:
    Other Non-Hybrid
    You were the one asking where all the Tesla fans are on Priuschat, and said it was relevant to the Tesla detractors here.

    I don't own Tesla stock, and was only interested in their cars when they were the only BEV with the range for my commute. I don't care for nearly everything being controlled through a touch screen, that they don't have shifter paddles for controlling regen, nor what their DIY repair policy looks like.

    I am against hydrogen for personal cars because building out the infrastructure for the US will cost billions, if not trillions, and those that would profit from selling hydrogen cars want someone else to pay for it.
     
  17. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    That’s my point
     
  18. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    Comment was not directed to you. You have always stated facts.
     
  19. hill

    hill High Fiber Member

    Joined:
    Jun 23, 2005
    19,674
    8,069
    54
    Location:
    Montana & Nashville, TN
    Vehicle:
    2018 Chevy Volt
    Model:
    Premium
    Maybe reconsider that line of think. How long ago was it that toyota FINALLY got to 500K vehicles in the U.S. ...... compared to the time line of millions? Might just want to take pause, thinking about the company that's already worth billions more than the maker of the Murai ? rather than continually bashing them, primarily because Musk (now the wealthiest person in the world) said current hydrogen cell cars are “mind-boggling stupid”
    .
     
  20. orenji

    orenji Senior Member

    Joined:
    Aug 17, 2013
    5,884
    3,486
    0
    Vehicle:
    2010 Prius
    Model:
    Three
    Hydrogen is here and growing worldwide nothing stupid about it. And for Musk his Top of the world could be short lived with competitors like NIO.