2008 Prius Battery Doesn't Charge Right Away

Okay, so I've read about batteries not charging, and batteries having various issues, but not found this specific issue anywhere on the forums.
So I turn on the car and drive, and the battery just drains down steadily to one bar (or even to empty once) before I get up to 55 MPH. It stays like that until I've driven for 5 miles, or waited 5 minutes or so, then it quickly charges up again. But until then, it runs solely on the engine, and the OnScreen Display shows it is "charging" the batteries the whole time, but the batteries don't go up at all till approx. 5 mins./miles.
It happens after leaving the car sitting for a while (over 3 hours or so), and not just overnight, but if I leave it off at work my whole 8 hour shift during the day, when I leave to go home, it does the same thing (even in the summer). If I turn it off and start it back up 2 hours later, it acts normally, but otherwise, it takes some time to get going and charge normally.
My guess is the ECM, but not sure if anyone else has seen this before. It started doing this about a year ago, and has only gotten worse since then. The car has over 200,000 miles and is 10 years old now, and I'm worried it will shorten the already stressed lifespan of the engine.
I appreciate any advice.

 

i've hit 3 deer in the past decade, but i've dealt with all the damage caused by them, and replaced several parts over the years.
the HV battery, do you mean the battery array under the back seat? i have tested every battery and they are all a steady voltage when full. within 1% of each other. again, after driving a while, they act normally. it is just on a cold start they take a while before working properly...

 

As i said earlier, i've hit a few deer since i bought the car. the first time, they had to replace the engine, and of course, my cheap insurance co. put in a used replacement. i'm over 200,000 miles, and i've known people who HAVEN'T hit any deer whose cars gave in to issues on less than that. i am pretty good about getting oil changes done (professionally), but beyond that, i've not really had anything done to the vehicle until problems arise. so i am always trying to keep my car problem free, and this is one problem that i'm not sure what the problem IS. so i thought i'd ask the community here, as these forums have always been helpful.

i seldom let the car warm up, except when there is snow outside.

i am not very auto-savvy, though i have taken care of a few issues myself when possible. as such, i could be wrong about this, but doesn't leveling drain and charge the battery array so they are all at the same voltage? i did a voltage test on every single one of the cells in my HV battery, and they were ALL at pretty much the same voltage (according to my multimeter). is there some other way of testing this?

thank you for your help.