I've had no trouble so far. I'd have no problem with ephemeral build artifacts going either at the root, or in src.

edit: I have had one pain point: devs/managers ask me why I do that, and that I stop. I refuse.