Relax,

This report suffers a seriously flawed methodology:

What they calculated was the distribution of types of accidents for different classes of vehicles, not a rate proportional to the actual number of vehicles in the targeted states. What this means is we can not tell the relative risk of HEV to non-HEVs because they did not use the population of all vehicles.

Let's start with their numbers:
HEV accidents |ICE vehicle accidents|total|HEV %|ICE %
8,387|559,703|568,090|1.4%|98.5%.
Now let's add the absolute numbers of HEVs versus ICE vehicles:
HEV vehicles|ICE vehicles| total | HEV %| ICE %
?? | ??| ?? | ?? | ??.
Wait a minute ... what is the ratio of HEVs, actually their miles traveled to ICE vehicles? ... It is missing!

Without the population of hybrids and ICE vehicles, no one knows if perhaps ... just perhaps ... the HEVs are having half the number of recorded accidents as the ICE vehicles. That data is not in this report, is it?

If you have a small population compared to a large population, this is comparing the ratio of nose and ear sizes within each population without determining how many of each group are causing the problem, accidents. The ratio of types of accidents means the distributions may be different but if the worst relative case, is half the rate of the other vehicles, it doesn't matter. Until we have the rates relative to the total numbers of HEV and ICE vehicles, this tells us nothing about the relative risks of each type of vehicle.

The other obvious question is the distribution of these incidents by vehicle body-style. The reason this is important is some body styles have less visibility than others and are more likely to obscure pedestrians and bicyclists. The "backover" special investigation report was specific in pointing out the problem of poor visibility, especially to the rear, of some car styles ... having nothing to do with noise.

Then there is the problem of small numbers in pedestrian accidents (table 6a):
Vehicle Maneuver| Pedestrian count - HEVs|Incidence rate of pedestrian crashes - HEVs|Pedestrian count - ICE vehicles|Incidence rate of pedestrian crashes - ICE vehicles
Going straight| 33| 0.9%| 2,069| 0.8%
Making a turn (*)| 19| 1.8%| 698| 1.0%
Slowing/stopping| 6| 0.5%| 148| 0.2%
Backing| 7| 5.3%| 261| 2.9%
Entering/leaving parking space/driveway| 1| 1.2%| 55| 0.9%
Starting in traffic| 3| 2.9%| 50| 1.2%
Other| 6| 0.3%| 238| 0.2% .
Does anyone notice the number of turning HEV count is larger than the number of backing incidents but the percentages are reversed? How do "19" Pedestrian incidents become a larger percentage than the "7" backing incidents. I have no idea how they came up with these numbers. Equally bad, these are laughably small numbers to draw any conclusions ... especially "backing up". Sorry but my eyes see a really bad problem here.

Just for grins, I did a Bob Wilson style of table 6a:

Vehicle Maneuver| Pedestrian count - HEVs|Incidence rate of pedestrian crashes - HEVs|Pedestrian count - ICE vehicles|Incidence rate of pedestrian crashes - ICE vehicles
Going straight|33|44.0%|2069|58.8%
Making a turn (*)|19|25.3%|698|19.8%
Slowing/stopping|6|8.0%|148|4.2%
Backing|7|9.3%|261|7.4%
Entering/leaving parking space/driveway|1|1.3%|55|1.6%
Starting in traffic|3|4.0%|50|1.4%
Other|6|8.0%|238|6.8%.

Before sack cloth and ashes, read the report and see the real numbers, the absolute numbers, and realize they have not counted the rates of accidents versus the population of vehicles.

Fortunately, we can file a Freedom of Information Act request and ask for the missing data. ... And share our concerns that this is a half-baked report missing critical data. ... More to follow after I get a chance to read it in more detail later this evening. <grins>

There is a wide spread problem of too many folks not understanding math and numbers
because it means they can be baffled by numerical nonsense. The truth is in the tables and just a cursory glance shows this report to be at best, incomplete, and possibly, misleading.

Bob Wilson

We've been waiting to for this, but they did some odd things.

First, they excluded the Insight from the study because it can't move without running the ICE. But they included the HCH. I thought the HCH couldn't move without running the ICE.

Second, they list the Toyota Corolla as part of the HEV group. There was never a Corolla hybrid, was there?

Third, they didn't actually calculate the crash rate (crashes per vehicle mile or crashes per vehicle). What they actually calculated was the fraction of crashes the involved pedestrians and bikes. That's just ... odd. I can't quite get my mind around that except that it's expedient.

Fourth, they controlled for weather and lighting, but not urbanicity. My understanding is that low-speed pedestrian crashes are far more likely in urban areas. But looking at their GESS database, it looks like information on urbanicity is missing for better than a third of the cases. So it's possible they couldn't correct for that (though you think they'd have the ZIP of the accident location in most cases).

The "statistical significance" of the results appears to be controlled almost entirely by the number of cases. When the split the data along various dimensions, the small cells are unlikely to be "statistically significant". The only significant exception is for impacts while turning, which appears for pedestrians but not for bikes.

In particular, I don't think you can make anything of the under-35-mph/over-35-mph split, as there were just too few accidents in the over-35 category to give any results at all.

That said, they found the effect where they expected to find it. If I were handed the task, I'm not sure I'd have done anything that differently, other than for the possible technical error with the HCH.

The passage summarizing the cars involved is here:

"The HEVs (case group) selected were the Toyota Corolla, Toyota Camry, Toyota Prius, Honda Civic, and Honda Accord. The ICE vehicles (control group)
selected are the Toyota Corolla, Toyota Camry, Honda Civic, and Honda Accord. The analysis is limited to vehicles of model year 2000 and later. The Honda Insight was considered but excluded from the analysis due to the fact that the earlier model years of Honda Insight always operate using their internal combustion engines, even at low speed."

Actually, now that I read that, it's not even weighted. So on a weighted-average basis, what you are seeing is, I think, essentially a comparison between Prius (most popular HEV) and whatever the most popular ICE car on that list is.

I think urbanicity is important, and if I can find the data on urbanicity and pedestrian accidents I'll post it here.

EDIT: Bob, I don't think your point about the numbers is correct, although it's admittedly difficult to get your hands around the numbers. Those rates are the ratio of pedestrian accidents to all accidents of that type. The denominator is different for every cell in the table. What they are saying on the line that looks odd to you is that, of all incidents involving backing up, for the HEVs, 5.3% involved a pedestrian.

So, yes, the numbers are small. I'm sure they normed this to all accidents rather than all cars as a matter of expediency. (Not sure I'd have done it differently, given the work involved in estimating the car fleet by state). The apparent technical errors in the car lists gives me pause -- but they could be a typo, for all I know. The "matching" gives me pause -- looks like they didn't actually have a matched sample of HEV and non-HEV, looks like they had two pools of cars, and whatever showed up in this pools is what they analyzed. So the the HEV list is (by N of observations) heavily weighted to the Prius, while the non-HEV list would have to be mostly Camrys, I'd guess. Not clear that makes any real difference.

And, yeah, your point about these rates not being per car or per vehicle mile is well taken. In their, they results might be from Prii having relatively few total accidents in a class (small denominator) instead of relatively many pedestrian accidents (large numerator). Yet, that seems fairly implausible to me. That the stars would align to create these results.

OK, one more esoteric statistical issue, in that this is just 12 states. Really, this is a clustered sample. The hybrid list has to be dominated by the Prius, and Prius ownership rates (particularly at that time) were highly geographically concentrated. To the list of things that might be true, the results might be an artifact of geographic location. But to tell you the truth, aside from fatalities, I'm not sure there's any comprehensive data on pedestrian accident rates by state (not even sure if you could look at that).

But the bottom line is that they saw the effect right where they expected to see it. Yeah, it's not the strongest result, but ... having done my share of studies with thin data, I wouldn't dismiss it out of hand. If this is all the information available at the time the Congress has to make the decision, then the slender weight of the evidence suggests that it is an issue.

In terms of what's feasible, for the ICE cars at least, I'd at least like to have seen the spread of pedestrian-as-%-of-total across the makes of cars. I mean, basically, given the time period, this is essentially a comparison of Prius versus other cars (weighted average basis). What's the likelihood that you'd see some statistically significant results (given the fairly large N of tests performed), if you just took two of the ICE cars in the ICE pool and did the same comparison?

I'll keep thinking about it and share any further insights. I mean, it's not implausible. The numbers are suggestive. The report makes it clear that this is not the last word. If the Congress acts soon, this is moot, but if there is going to be another round, with possibly another study, it might do some good to offer constructive criticism. But given the small numbers, there's only so much you can do with this.

FURTHER EDIT: I was just quietly corrected on the HCH thing -- they can move at low speeds without the engine firing, which therefore puts them in the same low-noise class as the Prius at low speeds.

BW's arguments likely do not hold up. The 'relative' in "relative incidence" looks like a normalization of vehicle type on the road.

A couple of comments occur to me:
1. What is the relative risk of HEV's to SUV's ? Or trucks ? How about cars where the driver sits so low in the cabin that only the driver's forehead reaches the window level ? If the public is interested in vehicle traits that endanger pedestrians, lets talk bumpers and grills. Or ICE/vehicle weight. In short, they should be setting up a multiple regression analysis, not this hack job.
2. Prius does not have a valid ICE only comparison car. How do the numbers look without it ? or compared to split-wing hatchbacks ?
3. Any attempt to normalize driver demographics ?

.. Unrelated to the weaknesses of this study -- I have to admit I always thought that reverse beeping *inside* the cabin was a bit silly.

I tried that approach and it really goes 'off the scale:'
Vehicle Maneuver| Pedestrian count - HEVs|Incidence rate of pedestrian crashes - HEVs|Pedestrian count - ICE vehicles|Incidence rate of pedestrian crashes - ICE vehicles|total within group
Going straight|33|1.57%|2069|98.43%|2102
Making a turn (*)|19|2.65%|698|97.35%|717
Slowing/stopping|6|3.90%|148|96.10%|154
Backing|7|2.61%|261|97.39%|268
Entering/leaving parking space/driveway|1|1.79%|55|98.21%|56
Starting in traffic|3|5.66%|50|94.34%|53
Other|6|2.46%|238|97.54%|244.
When I sum the number of accidents for HEV and ICE, we get a nice large number for the denominator but the calculated percentages, incidents/total count of incidents, really throws the percentages off. It becomes even more impossible.

I'm open to any suggestions on how the 'percentages' in table 6a were generated. I have no problem with the raw data but the calculated percentages in table 6a to my eyes are catty-wampus. I still don't see where the denominator(s) come from. For example,

2,069 - Pedestrian count - ICE vehicles going straight
0.8% - Incident rate
258,625 - the denominator to give 0.8%

132 - Pedestrian count - ICE backing
2.9% - Incident rate
9,000 - the denominator to give 2.9%

I just don't see where these denominators are coming from.

Bob Wilson

I'm good with that if there were a table in the report showing these numbers but all I can find is this quote standing alone:

This is only HEVs and does not include ICE registrations and it is only a rate of increase, not an absolute number of vehicles registered. So it tells us nothing about "normalization of vehicle type on the road."

If you can explain the calculations in table 6a, please do. I don't see where those percentages came from and those are the ones used for the claims of higher turning and backing rates.

Bob Wilson

ps. Please don't take my questions as anything but curiosity. I'm willing to learn but right now, the numbers in table 6a, the percentages, don't make a lick of sense.

It's worse than silly, it is distracting/irritating to the driver and as a result likely to cause more incidents than it prevents. I disabled it for that reason. Might as well have a telephone ringing inside the car while you back, it is that stupid of a distraction. Toyota needs to figure out who championed this feature and give them a permanent vacation.

No problem, happy to help you get a clue: They are using ~ 70:1 ICE:HEV on the road. Rounding errors from only reporting one place after the decimal makes the arithmetic messy.

Very valid [sic] point. Similar statistical miscues are rife in the medical literature, like HLA disease associations. Check 20, get one positive association for free

Nope, not what we wanted to hear. Forgive me for not being an astute statistician, but I'm not sure how to interpret the conflicting reports. Sam says one thing and Bob says another, and I don't want to have to choose sides.

I'd be interesting to see what the results look like distributed according to driver age, given that hybrids tend to be driven by older drivers.

That doesn't make sense for how the NHTS converted these raw counts:


became these percentages:


There is no 70:1 ratio that converts the raw counts into an equivalent percentage. All I'm looking for is some function:

rate% = F( incident_count, ... )

I don't see the relationship. There is no predictable denominator that makes the conversion from incidents to %rate work. But I notice the report makes this claim:

In this text they are using "daytime" as the selection criteria. The problem is table 6A gives total incident counts without any qualifiers such as "daytime" used to calculate the percentage. This implies the percentage calculation is:

%rate = SUM_OF( F_daylight(HEV,ICE) + F_rain(HEV,ICE) . . . )

This would provide a non-linear relationship to generate a %rate that is disconnected from the raw incident count. Clever but it turns out table 3d is the 'denominators' table.

Looking on pp. 12-13, we find:

So what they are doing is calculating a ratio of pedestrian involvement by maneuver from table 3a that were involved in an accident. Now the numbers of table 6a make sense ... but we still don't know the ratio relative to vehicle miles. This means even a higher relative rate for pedestrian involvement could still be a lower rate per vehicle miles for HEV relative to ICE.

Bob Wilson

ps. Please don't think of this discussion as 'choosing sides' as much as a stream of consciousness ... a collaborative effort. Being 'right' is less important than 'getting it right.' <grins>

Raw_HEV*70*ICE_% / HEV_% = Raw_ICE

If you still cannot see it, try rearranging terms:

(70)(Raw_HEV/HEV%) = Raw_ICE/ICE_%

---

Table 3d provides the denominators.

Bob Wilson

Ok, now the numbers begin to make sense as well as the flaw in their conclusion:

• Table 3d - is the distribution of maneuvers
• Table 6a - is the rate of pedestrians involved in these maneuvers
• "Making a turn" is 13% of all accidents both HEV and ICE
• "Backing" is 2% of all accidents both HEV and ICE
• Only 15% of all accidents show a difference in pedestrian involvement between HEV and ICE accidents

The NHTSA conclusions are looking at pedestrian and bicyclist involvement in 15% of the accidents. Worse, there was no effort to rate the absolute accident rates as function to fleet inventories and more importantly, by vehicle miles, in the study area. Then there is the Pinto memo risk.

Someone in Ford wrote a memo that it was cheaper to deal with the results of an 'exploding gas tank' than to fix it. That is different from asking what the relative risk per mile or vehicle do the turning or backups pose relative to ICE vehicles. If the relative risks of these 15% are equal to the same risk per vehicle mile of ordinary vehicles, it would imply that the other 85% maneuvers for HEVs are even lower!

OPINION

If someone accepts the NHTSA study, it suggests turn signals that emit an audio alarm (13%) and backup lights (2%) should emit an external noise as well as light. It turns out there are after market products in this area already:

Backup Warning Device Features

• Uses Echolocation Detection (Similar as the Military Sonar Systems)
• Instantaneously Alerts Driver to the distance of any objects that is behind their vehicle from
• eight (8) inches up to 15 feet away.
• Detects both stationary and moving objects.
• 8 bit Micro Processor with Patented “Intelligent Control Software”.
• Clear VOICE Identification of Objects made directly to driver automatically.
• Accurate Distance Detection of + / – 2 inches
• Works in all Types or Weather Conditions.
• Works at Night.

Also, Consumer Reports referenced a backup light that emits noise:

DESIGN TECH BACK UP WARNING ALARM & HALOGEN LIGHT 20100 HOPKINS MANUFACTURING
3156 Back-Up Warning Alarm Halogen Light



Something like this mounted in the fog-light positions and wired with the turn signals might work for audio turn signals. There is a simple, differential wiring setup that might allow one 'fog-light' (and fog-horn?) to make noise only when turning and when both lights are on, remain silent.

NOTE: I have no commercial interests in either product but this box:


Bob Wilson

Partial retraction:
The 70:1 ICE:HEV ratio is present, but is not based on car registrations; it is ICE_accidents:HEV accidents. It happens (and not just by coincidence) that the ratio probably mirrors the registrations ratio too. Moreover, it is very noteworthy that the distribution of accident type unrelated to whether pedestrians or bicyclists are involved are so similar between HEV's and ICE vehicles. Reality has normalized the data, so the author's goofy methods have inadvertently come up with ballpark relative risks.

It doesn't matter to me though. The authors groups are so different that they can lay NO reasonable claim to having cancelled out competing factors. More GIGO. Prius Vs (Toyota Corolla + Camry + Honda Accord ) ? Give me a f'g break.

Shall we start a lottery what the Prius Vs large_SUVs data will look like ? Prius Vs teenagers ? Prius Vs rednecks ? Prius Vs diesel trucks ? Prius Vs octagenarians ?

I can think of much more productive witchhunts.

Without knowing how many vehicle miles were travelled by the HEVs in the city, and how many vehicle miles were travelled by the ICE vehicles in the city, the report is nearly meaningless.

Since ICE vehicles have very poor efficiency in the city, I strongly suspect that consumers who must drive in the city very much prefer HEVs.

If ICE vehicles are driven less in the city, then their relative incidence of accidents in the city could be less, even if the accident rate per vehicle mile were the same as HEVs.

One could just as well compare interstate greyhound busses with city busses. Obviously, the city busses would have an enormously higher relative incidence of city accidents, but not because they are inherently unsafer, but because they travel much more in the city.

Knowing that HEV is the technology of preference for city driving, the slightly lower relative incidence of city accidents for ICE vehicles does not allow me to draw any conclusions about the safety of HEVs vs. ICE vehicles if they were to be driven equal distances under equal conditions.

Upon reflection, given that I do this type of work for a living, I'll give you what I think is my final take on it.

First, any inference about over-35-mph versus under-35-mph is not meaningful. There are too few events over 35 mph. You don't have enough data there to be able to reject the null hypothesis at any reasonable excess accident rate. (Or, in English, given the small numbers over 35 MPH, you were pretty much bound to find no statistically significant difference).

Second, that said, they found a pretty consistent "effect size" (ratio of HEV to non-HEV accident rate) right through Table 6a. Only one of those, alone, pops up as statistically significant. They also separately pool all the guaranteed-low-speed maneuvers and come up with a statistically significant difference.

At first, I thought the turning effect might be due to the large front pillars on the Prius. (Keep in mind that this is, on a weighted average basis, a comparison of the Prius to roughly similarly sized ICE vehicles). But in fact, they get the effect everywhere, it just tops the criterion for statistical significance in turning.

Third, the basic hypothesis is plausible. Inattention plus lack of noise leads to slightly higher pedestrian accidents than inattention alone.

Fourth, the estimated level of excess risk appears plausible - 40% greater likelihood of pedestrian collision. (Not pedestrian deaths, pedestrian injuries.)

Fifth, the arguments about other vehicles are only tangentially relevant. You want to talk about the cost/benefit analysis for backup beepers for SUVs relative to noisemakers for HEVs, OK. But the mere fact of excess risk in other segments of the market, that may or may not be fixable, that's not really what's on the table.

Sixth, could this be spurious? The people at NHTSA would have a far better feeling the likelihood of that than I could, since they actually work with the data. The fact that it's (weighted average) essentially a contrast between Prius and other vehicles does not thrill me. (But because Prius is only HEV, you can't directly adjust for models using "fixed effects" approaches.) Lack of data on urbanicity does not thrill me. (But judging from their GESS data, they may not have that coded, or have it missing in a large number of cases). Lack of reference to actual vehicle registrations ditto. (But I can see where that would be a lot of work to get to that.) Lack of adjustment for other geography (state) does not thrill me. (That one, they could do in a regression framework).

And, really, I understand how the advocates will use this (i.e., as proof), but the NHTSA analysts don't overstate the results. They were asked to look at this, the early results suggest that its an issue.

From this, we can do a rough cost-benefit analysis to see what noisemakers would buy you. I'm just going to do it crudely. There are about 300M cars registered. There are about 70,000 pedestrian accidents per year (per NHTSA). That's .00023 accidents per vehicle. If the excess accident rate is 40%, that rounds to .0001, or 1 accident avoided for every 10,000 HEVs fitted with a noisemaker. A calculation like this doesn't deserve any better than rounding.

So, every 10,000 vehicles will avoid 1 non-lethal pedestrian accident per year. Say vehicles last 20 years. That's 20 accidents per 10,000 vehicles, or 1 accident avoided (per vehicle lifetime) for every 500 vehicles.

Upshot: multiply the cost of the proposed noisemaker by 500 to get the expected cost per (largely non-lethal) pedestrian collision avoided.

If the noisemaker costs $100, then that's $50,000 per avoided collision. All assuming the NHTSA analysis is correct.

My judgment may be colored by working in health care, but that doesn't strike me as too outrageous. It's more than the average medical cost of the accidents, for sure. It would be nice to see similar data for (e.g.) airbags or something. But at face value, this doesn't appear grossly unreasonable.

Yeah, I would have liked to have seen some better study. I think I understand why the study was done as it was (lack of data, and expediency). Even with the caveats, I think it makes at least a plausible case for considering this issue further.

Ok, this is still a 'work in progress' and subject to change. Heck, I don't even have a change log, yet, and the HTML is primitive. Regardless, this is my 2001-2007 Prius fatality report:

Prius Fatalities 2001-2007

I take all comments and criticisms in the spirit they are given and I am not the least bit offended if you help identify a problem early.

Thanks,
Bob Wilson

Maybe Priuses are driven in areas where there are more bikers. If no one bikes in Topeka Kansas, does that mean all the cars there are bike safe?

Hi Bob,

The problem with mounting a beeper in a fog light is that the usefulness to signal somebody that a driver cannot see them will be lost. In other words, it will be a false-warning syndrome device. Eventually, pedestrians will just ignore the beeping, because cars driving down the road, no danger to them, will be beeping. And when a true back-up beeper comes on, it will also be ignored. And indeed, the back-up beepers on large trucks will be ignored, and pedestrian death improvement due to those devices will be lost.

I have had a pedestrian start to run in front of me while making a turn. I was following a white (my car is blue) pickup around a curve. The light had just gone green and we both had our turn signals on, doing about 20 mph. I was 3 car lengths behind the truck as we approached the turn. The pedestrian was watching for left turning traffic over his right shoulder away from the direction I was coming from. As the pickup truck passed he began to run the last two lanes across a four lane road. I did not see him until the pickup cab cleared the turn. I went full brakes quickly, got the Prius "WHAM!" assisted brake action, and there was a slight squeel of the tires. After stoping , I imediately checked if I was going to be run into from behind, turned my head back forward, and only then did the pedestrian turn his head. He never heard the tire squel or banging sounds from the brakes. At this time he was 5 foot from my car, and I was dead stopped. Why? Because of the loud engine noise from the Pickup as it did a full throttle acceleration out of the corner and my tires were sound blocked by the bed of the pickup. It happened so quick I did not have time to hit the horn. No car idling at a stop is going to be able to be heard above a pickup truck focusing its sound energy backwards 30 feet away. This pedestrian almost ran into my stopped car because of that. When he did stop he just kinda grined confusedly. Its as if my car materialised, stopped in front of him.

What this points out, is to be effective in this scenario, the noise maker will need to be on all cars, and so much louder than the exhaust of vehicles in front of the noise maker that it can be heard through the major body assemblies of the lead vehicles. And that the device can only be activated when a danger situation occurs. Otherwise, its a false alarm. Such a device will be in the 10's of thousands of dollars, due to the sensor to activate it only when a danger is present. And the sensor will need to be higher than any lead vehicle, which will be very costly in fuel if applied to the whole fleet of vehicles.

Imagine sitting on a street corner, waiting for a light to change, and having every car making a right turn in front of you emitting a 100 dBa noise?! This is what it would take.