Just a warning, this isn't a magic bullet to replace all A/B testing. This is great for code that has instant feedback and/or the user will only see once, but for things where the feedback loop is longer or the change is more obvious or longer lasting (like a totally different UI experience), it doesn't work so well.

For example, if your metric of success is that someone retains their monthly membership to your site, it will take a month before you start getting any data at all. At that point, in theory almost all of your users should already be allocated to a test because hopefully they visited (and used) their monthly subscription at least once. So it would be a really bad experience to suddenly reallocate them to another test each month.