Dead or Alive.

The first thing to check is the 12v battery voltage when the car is 'dead'. If you are lucky it is just the battery and you can get a new one.

The 12v system is powered from a DC-DC converter, from the HV battery, which is like the alternator in a normal car (as it is powered from the Hybrid battery, the engine can shut down and the 12v power system keeps running). So, as long as the Prius is powered up, the 12v 'supply' will be 12 volts. If the 12v battery is dead, however, the car would behave just like what is happening with your car. The resting voltage should be above 11 volts (that would be a bit worn); I can say that at about 10 volts things start to get a bit dodgy trying to get the car powered up!

 

Gas engine/regenerative braking recharges HV Battery.

HV Battery supplies all electrical demands for vehicle and recharges 12V Battery

12V Battery does NOT crank the car like a traditonal gas vehicle.
12V Battery just powers the ECUs when you start the Prius (READY mode).
Dead 12V Battery, can NOT start the Prius w/o a jump. Recharge dead 12V battery, or replace 12V battery.​

Your 12V battery was completely dead again after your jump then subsequent drive, b/c the duration of the car being in READY was too short, to recharge the 12V battery. You would have to go on a 12+hr road trip, OR leave the car in READY mode overnight, to hope to recharge the 12V battery. Do one of these:

1) Recharge it w/ a smart AGM compatible 12V battery charger.
2) Leave the car in READY mode for 12+hrs (24hr should do it), in a non-enclosed space that is not in a high crime area, with doors locked (take the small metal key w/ you so you can unlock the door).
3) Go on a 10+ hour road trip.
4) If the battery is more than 5 years old, consider just replacing now.​

PepBoys has a cheap and GOOD, 12V AGM battery for the Prius. $146.24 Bosch.
Direct fit. JIS battery posts. Vent tube hole. 4year straight warranty (no proration). 45Amp hour battery.
AVOID the YellowTop Optimas b/c it is only 38Amp hour. I believe 2004 Prius have NO smart key. All 45Amp hr batteries will require you to change the battery clamp, so the larger battery will fit. Read this so you can see how I did it and the part required (new from Toyota or get from a junkyard off a SKS Prius in the yard): 12V Battery Upgrade: non-Smart Key (35A h) to Smart Key (45A h) | PriusChat

Is my inverter coolant pump slowly failing? | PriusChat

Read Post #5 (how to apply a load for a more meaningful measurement of 12V battery health)
Read Post #8 (Battery SOC (State of Charge) chart and 12V AGM compatible smart chargers to consider buying).​

 

^ what this person said....

You need to boot the car up before it can do anything, and unlike the old cast-iron V-8 powered cars this doesn't happen by sending 650 amperes from a ginormous battery into a huge electric motor to break all of those pistons loose.

You're literally closing a circuit and sending voltage into a microprocessor, and it's a voltage thing not a current thing.

That means that there's no warning.
There's no dimming of lights....no clacking of a solenoid, or the pitiful grunt from an engine that wants to keep slumbering.
You either start
Then...you get nothing.

You can check the OCV (open circuit voltage) of your battery if you have a meter, or you can take it to a place that gets paid to tell you that you have a dead battery and let THEM....ah...."test" it for you.

Good Luck!

 

Because the big battery charges the little battery. Once the car is started the little battery does nothing.

Buy a new 12v battery is the answer.