Welcome!!
This is a hard one and I've been planning to get either an IS 500 or LC 500 in the next ~2 years, so I've considered the same question.
Here is my .02: For a performance car, I would want a color that makes more of a statement than a gray, black or silver, which is why I lean towards Infrared. It's a classic, dynamic, eye catching Lexus color that I think suits the IS very well.
The IS 500 I drove for media reviews was Cloudburst Gray and it was... fine? A pretty low profile color if I'm honest. If that's what you want, I think you'd be happy with it but I found it a little bit dull overall.
I want to like the red interior, but I wish it was only the seat material and door inserts that were red because IMO in person, the full lower red door panel and lower dash are a little bit "much" and I'm not sure they'll age very well. That leads me to the black interior, and while I sometimes think black interiors can be a bit too monochrome and dull, I like it more than the red for aforementioned reasons.
This (older IS) looks so much better balanced to me with tasteful red accents:
...than this:
That leads me to Caviar, and while I said above that I don't really love black on a special performance car, Caviar is so gorgeous that you can probably get away with it. I love the color so much. I also think it suits the IS 500's character very well, being a Lexus at heart but a performance car when you push it.
For me, I have "mentally" landed on:
Infrared exterior, black interior
Atomic Silver exterior, black interior
Caviar exterior, black interior -- POSSIBLY the white/black interior.
I have seen a few IS 500s in the USB 2.0 color, and while I don't usually love blue... sort of like Infrared, it's a classic, dynamic, beautiful Lexus color and I'm finding myself more drawn to it.
Anyway, let us know what you choose and post pics!