1. Attachments are working again! Check out this thread for more details and to report any other bugs.

The battery fires at ECU sense connector thread

Discussion in 'Generation 1 Prius Discussion' started by ChapmanF, Mar 19, 2015.

  1. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I may well have detected this problem at an early stage. I have a 2008 NWH20, about 160,000 miles, and it has been parked up for a few weeks (still using it a little but not much). I have a Nissan Leaf which I have been using for my commute - work has been so busy I don't need to drive any further (for now).

    I fired up the Prius - red triangle of death - no big deal (still original 12v battery). However it was a bit different; the ICE stayed on, it kept charging the HV battery which went from full, to 1-bar, while charging. It had been fairly well charged when I last used it - and 12v battery was disconnected.

    The error code P0AFA came up; looking at everything in more detail - the cells are well balanced, but 'blocks' 1 and 2 (out of 14, so basically 1,2,3,4 of the 28) keep switching back and forth between positive and negative approx 16v (low SoC). They also charge more slowly. The car worked but now, the 'ready' light flickers on and off then stays off, and it can only go to neutral.

    I am planning to see if I can remove the upper case of the battery 'in situ' and see what the state of the voltage sensors are. The car has been fine and no HV battery problems have every occured before now. Hopefully it doesn't catch fire before I can figure out what is going on!
     
  2. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,068
    15,372
    0
    Location:
    Huntsville AL
    Vehicle:
    2017 Prius Prime
    Model:
    Prime Plus
    Now if each sense wire had a 1-10k ohm resistor at each battery tap end, there would be no fire hazard. There would not be a low resistance path to short out the huge currents a module-to-module short can provide to 'let the smoke out.'

    Bob Wilson
     
    Travis Sanders and Adebrettski like this.
  3. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    My thoughts exactly!

    Am about to get to the HV ECU. Looking at photos of the circuit board, each pin has a 1kOhm resistor on the circuit board. If my connector isn’t destroyed, I plan to get a new basic connector of some type, splice it in, and basically have a way of ‘inserting’ another connector between the two. Then, I plan to remove all the 1k ohm resistors on the HV ECU, and instead put 1k ohm resistors into this new loom which inserts between the sensor harness and sensor plug. Labelling it up and labelling up the ECU as ‘warning - DO NOT USE without special loom’ will do for DIY safety.

    Have also thought of a fuse for each line. 100mA fuses may do the job but may put the voltage reading off by up to 1% (or 0.1%??). These could go into a special loom too - would prevent that massive plasma arcing in some of the earlier photos.

    Anyway, off to check this ECU - maybe it isn’t what I think at all!

    I’ve had a few ideas about the reason for why these pins fail - will examine the motherboard once it is out - it may be there is a way to fix this problem for good!
     
    Travis Sanders likes this.
  4. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    Edit : thinking about it - the short circuit must be ‘on the ECU’ side (be it the pins inside or outside - eg bridged by corrosion). As the connector keeps its pins apart (until it melts of course). The corrosion must be the short circuit issue - will do resistance testing at some point. I will look at the pins in great detail as there may well be another issue that could initiate unexpected plasma short circuits and then be gone after the event!
     
  5. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    Okay Houston, we’ve had a problem here.
     

    Attached Files:

  6. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    The white dot inside the connector is light shining through where the connector has burnt through. Am inside sitting in front of air con (New Zealand summer). It appears that basically the NWH/XW20 and earlier with prismatic batteries have a major problem. My mum’s car which I will fix in a month probably has the same thing (NWH11 Prius). I’ve bought a second hand battery pack in my city, not inspected, $600US - it will be worth it for spares or forensic investigation. I will now enter forensic mode! And open the ECU.....

    The $600US battery has an ECU so I am going to inspect and swap, then the orIginal battery and ECU in this car (New Zealand brand new from dealer, no third party mods to battery ever.....) can be inspected and evaluated in detail!
     
  7. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    PS the pin in the corner is still present, just very charred - and is the main player it seems. This car never had any battery faults until 2-3 days ago, last power up was 2 weeks prior, drove it a bit, full charge cycle (on the road). So this is of concern.
     
    dpower likes this.
  8. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    23,073
    14,982
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    Hey, that's just what I was thinking back in 2008!

    Wish I'd sold Toyota my idea. :)

    That sounds like it might be more complicated than necessary, and at the same time still not be the optimal placement for the resistors.

    Since Toyota hasn't told us what the input impedance of their voltage-measuring circuits is, we're not yet in a position say how much the readings would be dropped by adding a given resistance in series. (We know the impedance is at least 1 kΩ, from seeing the 1 kΩ series resistors sitting there, but it may be much higher than that; voltmeter inputs usually are.)

    A little experimenting could be in order ... build up a battery where, in place of one block (or more than one), there's a stable 14.4-ish voltage source, and see how much the ECU readings change when adding a few different values of resistance in series.

    It could turn out that adding 1 kΩ in series (even without drastic mods like removing the resistors in the ECU) would make a difference so small it wouldn't matter.

    My favorite design for a harness would be to have the resistors right out at the battery ends of the wires. End of wire spliced to resistor, ring terminal spliced onto resistor, ring terminal attached to battery.

    That would protect from damage anywhere along the harness. I don't see the need for a fuse, because just having the resistors there would limit the worst-case fault current to about a fifth of an amp, in a short between the worst choice of two wires.
     
  9. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I agree. I am going to repair this HV ECU!!!

    Question - does the NWH11 ECU swap with NWH20? I am getting a spare NWH20 Ecu. It doesn’t matter if your guess is wrong but I am going to take the spare ECU back home anyway!

    Also, the pins that have experienced such high voltage that they have literally melted are not limited to the pin in the corner.
     
  10. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    On further exploration, my ECU has a pin melted internally in the middle. In fact about 3-4 pins are melted on the actual motherboard; will do more research as the connector on NWH20 may be different to NWH11 or 10. But probably not much different!!!
     
  11. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    On even further exploration, it can be seen that the final ‘catastrophic’ shorting occurs inside the ECU (on the thick metal pins that link the voltage sense connector to the motherboard). This shorting does not seem to be a fault of Toyota, and if it is what I think it is, it is of concern to many of us!
     
  12. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    On even more detailed visual exploration, the culprit is obvious. It is not Toyota’s fault but technically might require replacement of all ECUs up till 2008 (NWH20). I need to do more research but - if you get sudden anomalous cell readings, fluctuating readings, please get a brand new ECU from Toyota. If you are a DIY person who knows your way to avoid death by electrocution from the battery, you may wish to consider disconnecting the ECU immediately, then consider your options.

    An open-source ECU seems to be required. It appears the same ECU was used in version 11 / 20 priuses, probably also ‘prius C’ / aqua. Maybe more. If anyone knows how the HV Battery ECU communicates over the CANBUS ‘ethernet’ please get in touch (is it paired or encrypted?).
     
  13. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I will be away for a bit, but, these two photos show two things; the first shows something extreme happened - an extreme thermal event, yet the HV ECU kept ‘working’; the second shows that the connectors were not overheated in the underneath of the motherboard........
     

    Attached Files:

    dpower likes this.
  14. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I've made a thread about the diagnostic stuff I've done so far; to avoid spamming this thread. I would be interested to know your thoughts (particularly the implications of how pin 5 has totally burnt up on mine which, given the lack of the usual pin 22 corrosion, may allow further investigation as to the root cause of all this!)

    Troubleshooting HV ECU burnt connector (causes P0AFA, red triangle, 'replace hybrid battery' etc | PriusChat

    Would be good to hear your thoughts on this! (I think, for one, putting some resistors before the voltage wires enter the ECU makes a lot of sense, maybe even just some fuses!)
     
  15. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    23,073
    14,982
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    I'd go with resistors. Really more predictable than fuses ... you know all the voltages involved, so knowing the resistance, you can calculate the maximum current that could flow in any fault, and keep it well below the temperature limits of the wires and connectors ... as opposed to having a higher current flow and counting on the fuse to then cut it off.

    Resistors are probably cheaper, too, and come in a nice form factor for just crimping to the ring terminals. (Though some non-replaceable small fuses do come in a similar axially-leaded package. Those are probably not the kind of fuses capable of interrupting this battery's available fault current, though.)

    I would watch with interest any effort to experimentally determine the ECU inputs' actual impedance, as suggested in #88.
     
  16. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I agree re: resistors; basically I2r, and be obsessive about where they go / electrical isolation / working out max temp. I do like the idea of fuses, but it would require a dismantling of the car to get to the HV ECU. But I agree - I doubt they could tolerate the currents involved (given the thick connectors inside the ECU got vaporised, likely in a plasma arc, a fuse will likely do an even better job at keeping the arc running).

    I should add : I personally believe the cause is something which is unpredictable, which worsens over time, worsens with thermal cycling, worsens with mechanical stress, worsens with humidity, worsens with exposure to some chemicals. Specifically ‘Tin Whiskers’. That is the only thing I can thing of for my pin 5 short. Just my opinion for now though :) But, reading some NASA papers where they take actual tin whiskers, silver-solder them to ‘Jacobs ladder’ test rigs (like our connectors), then the plasma that is created can be created well below the voltage between sets of pins, and well below the current.

    I want to recoat the whole connector in lead solder, repair the burnt sections, repair the area where the circuitboard arched back to a pin, and see if it works. But - I will wait for my secondhand ECU to arrive; I want to use a microscope to look for tin whiskers on this ECU (note - if they didn’t use tin, then zinc whiskers etc!)

    Once my car is back in the road I will do some impedance tests on this ECU!
     
  17. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    A possible further clue; I took the a good battery computer from another battery and plugged it
    In; the ECU didn’t burnt out despite the damage to the connector on my battery; in fact it (and the battery) worked fine!
     
  18. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    I took a closer look at my damaged ECU. The pin in the middle that melted looks, in close-up, as follows:
    melted_pin_circle.jpg
    The pin to the left of the melted one is a bit charred at the base, but fine otherwise. The pin to the right has (circled in red) an area that looks like it was arcing with the melted pin. The short circuit was initiated somehow, but not actually via the ECU circuitboard (given that there are 1000 ohm resistors). Something caused a plasma arc to initiate and it kept going until it randomly stopped (at least that is my impression).

    Looking at the classic pins that fail (11, 22 etc), I saw this:
    small_whiskers.jpg
    Circled in red, three small metal whiskers. Given how hot these pins would have got, I am surprised any whiskers remained (given how fine they are, they would melt / vaporise even if not directly involved in the arcing).

    And finally, the same image as above, but focused differently, reveals this:
    longer_whisker.jpg
    Circled in red, is a longer whisker. If it was just a little longer, and if there was another one that happened to be coming from another pin and going towards it, it would likely initiate an arc that would burn out with a high current, vaporising the whisker in the process.

    So, it seems that whiskers grow, even inside the ECU in some cases, causing metal vapour arcs. If this is what is happening, then it makes sense that it goes together with corrosion, because corrosion accelerates whisker growth.

    The connectors in Denso ECUs of this era have tin coating on the connectors - as discussed in this paper : https://www.autosafety.org/wp-content/uploads/import/Tin%20whisker%20analysis%20of%20an%20automotive%20engine%20control%20unit%20(3).pdf

    Tin is also able to act as a sacrificial anode for a number of metals, including copper. Given the corrosion that happens to the copper bus bars, I wonder if the corrosion at the far end (battery end) of the voltage sense connector is resulting in increased corrosion by cathodic protection at the battery 'ECU' end of the connection?.

    I've inspected my 'new' secondhand battery ECU with the microscope - no corrosion or whiskers of any kind; I'm going to swap out the battery and, soon (next week or two maybe) properly inspect the battery currently in my car (the one that shorted) and will post then. I expect to find corrosion!
     
    Travis Sanders and SFO like this.
  19. landspeed

    landspeed Active Member

    Joined:
    Feb 25, 2007
    351
    198
    0
    While doing a bit of research on the NWH10 this Sunday afternoon, I came across a thread where an Australian Prius owner is converting an NWH10 to a plug-in Hybrid. It sounds like a good plan since the NWH10 is a good car except for the original batteries / BMS etc.

    I know this thread is dedicated to the NWH11, a totally different car, which is much more similar to the NWH20,NWH30 etc. However, something interesting from the Australian owner's thread:

    User Antiscab (also has an account on Priuschat); link to thread:
    Antiscab's NWH10 Prius plugin Lithium conversion - Page 2 - AEVA Forums
    On page 2, among many cool photos of work in progress, is a picture of the voltage sensor connector;
    Antiscab NWH10 connector burn DSC_0048.JPG
    So... this photo was taken in 2014, so 5 years have passed, but I will PM him anyway. However, it would appear that this actually an NWH10, with a very different battery pack design, that again shows the burnt HV connector problem starting. This could suggest the problem is even more widespread (I guess there is no reason that the Gen 1 Insight wouldn't suffer from this, and the Nissan Leaf certainly doesn't do anything different in the voltage sense lines as far as I can see from my spare Leaf BMS.
     
  20. Angry Andy

    Angry Andy Junior Member

    Joined:
    Sep 30, 2017
    6
    8
    0
    Location:
    Pleasant Hill, CA
    Vehicle:
    2003 Prius
    Model:
    I
    Well now you can add myself to this list.....happened a few weeks ago on my wife's '03 with 107k.

    backstory:

    First I screwed up and jumped it with reverse cables on the jumper battery, I fried 3 or 4 micro fuses and the PITA 100A bolt in. Read up on the repair here, got the parts and had it running in a couple hours.

    I *THINK* I had the triangle warning light at the time, but I had recently "fixed" triangle of death light a couple weeks beforehand (late Feb) when I had the inverter pump coolant level too low. So I didn't bother getting out my SLLLOOWWW laptop with techstream to pull any codes, I wanted to get it running and drive it a bit before pulling the code(s) after fixing the fuse issues.

    So after fixing the reverse jump issue, I drove it around the block a bunch of times, and expanded the block to a mile or two round trip, did this drive at least 6 times over 2 days. She seemed fine...

    Next morning I drove her to the freeway, and when I was getting up to speed a felt a sputter and smelled burnt wiring, and the car had a decent amount of smoke inside.

    The car sputtered along and I put my emergency flashers on and crept along about 40MPH in the exit only lane off the freeway...limped it home freaking out that I was already on fire or about to catch fire any second. Luckily I made it home and backed it into the driveway and let it sit as my wife was out of town visiting her mom in the hospital in Mexico.

    A few weeks go by and I get motivated to look into it, the car starts up fine and runs fine in the driveway....again I take it around the block thinking maybe the inverter had fried or some 'other' wiring.....drove it a few miles took it up to 50mph and all appeared fine.

    Techstream disagreed, battery block voltages were ok on blocks 1-8 but zero on all the rest....so I began the battery removal procedure. As soon as I got the cover off I saw the carnage.

    I got a few pics, but don't think I can upload them as of yet....

    Not sure where I'm going from here, all my cells are reading 7.88v (but I do have five dead cells). Im guessing get 5 replacement cells, get a hobby store charger and recharge them while waiting for a new sensor harness kit and used ECU.
     
    Fred_H likes this.