HPC jobs generally don't stream data to disk in the first place. They write out (huge) snapshots periodically. So mount a network filesystem and be done with it. I don't see the issue.