hardware-decoded HEIC is around 10x slower than JPEG.

software-decoded HEIC is around 100x slower than JPEG.

WebP is something like 20-50% slower than JPEG, so still solidly an order of magnitude faster than best case hardware-accelerated HEIC decoding.

HEVC intra is definitely not 10-20x slower than decoding VP8 intra, so something is way off with your numbers. That's on the order of comparing HM at -O0 against an actual VP8 decoder.

How slow are we talking though? 10x slower than instant and not noticeable for the average use case? Dog slow and everyone will always notice?

well that depends on the size of the image. But a modern cell phone is shooting 50mp+, and at that size you're talking 50-200ms for a jpeg vs. a couple seconds for HEIC. Which is borderline instant-feeling vs. extremely not instant.