Yes, and the reason why pass@k exists is because of self-consistency. There is no guarantee for right answer to be selected or for the LLM to correct itself. While I agree pass@1 is a useful metric, I'd be more interested to know pass@5 so I can better compare the results.