Having trouble finding part names/part numbers

No. You can not find or buy just the acrylic facia portion, even though they can be separated. "Grille Sub-Assembly" is the name of the part and it consists of both the black bumper-cover and acrylic facia part. The parts name maybe now changed to simply "Grille" for newer model years Part Number: 5310147150. And this is without front bumper sonar sensors for your XLE trim. There are currently three other listed Part Number: 5310147041 or Part Number: 5310147160 or Part Number: 5310147052. I think those three are all for Limited (or Advanced) with the front bumper sensors but I am not exactly sure what the difference between those parts is.
@Elektroingenieur may be able to help distinguish different part numbers for specific year models and trims.

See this thread: 17-18 Prius Prime grille question | PriusChat

The parts in question "Grille Sub-Assembly Rad - Toyota (53101-47051)" contains clear acrylic part around the emblem and just below. (inside of red marked area in the diagram below) The remaining part is always black in all models of PRIME. There are others, including me, posted complaints about the clear acrylic part getting chips and cracks so easily. Toyota does not seem to sell the clear acrylic part separately from the rest of the larger black grill part. I don't have an answer to your question, but the acrylic part must come off, so painting the black part to any color you wish has to be possible. But I don't think it comes painted black, to begin with. Mine does not look like any paint on the black part. Just virgin plastic with matte texture finish.

 

If you look at the diagram you attached, notice that the parts you're interested in have the note "refer to fig 53-01".

Fig 53-01 is here (if you look at the URL, you'll see it ends in 5301). And on that page you will find the part numbers you want.