I've been there and spent hours if not days shaping builds and using workarounds/hacks to fit the esoteric manner in which CI services need your builds to run (setting up the build pipeline as a series of containers but without easily accessible persistent storage between the containers seems to be the current popular model - which is the case because it's very easy for a service provider to provide and manage the compute this way but pushes all the work onto you instead to get it working).

This problem is what I'm trying to solve with https://boxci.dev for myself and hopefully others too. The idea is it'll do all the management stuff that a CI service should do, but let you run the actual builds however and wherever you like, on your own machines, using an open-source cli. Check it out, perhaps you'll find it useful. It's launching very soon!