I'd be curious if there are any tests out there running these M1 Pro and Max MBPs while also driving an external display. It would be interesting to see if driving an XDR or maybe a 5K LG is a noticeable performance or render drag.
I've got a 4K display hooked up to this M1 Max right now, but I don't have a 5K or 6K display handy. I suspect it won't change much, but it could depend on the tests you're running.
With system memory and GPU memory being shared, I suspect you're unlikely to run short due to using extra displays, and it's going to be the hardware encoding units doing most of the heavy lifting most of the time.