I'm looking for an ML engineering gig. I can help you gather data, preprocess it, design models, train models, etc etc. My ideal job would be doing generative AI stuff with images/audio/video, but I'm open to anywhere there's gradients to descend. Recently I've been working on a project building a text-to-image model that learns solely with unlabeled image data, relying on CLIP for the link between captions and images[1]. I think it's a) cool and b) demonstrative of strong abilities. At a higher level of abstraction you can think of this as embedding guided content synthesis. The model learns to generate images conditioned on their CLIP embedding being within an input spherical cap. If you center the cap on the CLIP embedding of some text you get images that look like they'd have that caption, if you center it on the embedding of another image you get semantically similar images. The radius of the cap determines how similar the outputs are.
I'm looking for an ML engineering gig. I can help you gather data, preprocess it, design models, train models, etc etc. My ideal job would be doing generative AI stuff with images/audio/video, but I'm open to anywhere there's gradients to descend. Recently I've been working on a project building a text-to-image model that learns solely with unlabeled image data, relying on CLIP for the link between captions and images[1]. I think it's a) cool and b) demonstrative of strong abilities. At a higher level of abstraction you can think of this as embedding guided content synthesis. The model learns to generate images conditioned on their CLIP embedding being within an input spherical cap. If you center the cap on the CLIP embedding of some text you get images that look like they'd have that caption, if you center it on the embedding of another image you get semantically similar images. The radius of the cap determines how similar the outputs are.
[1]: https://www.echonolan.net/posts/2024-03-09-is-it-possible-to... new model that generates better samples coming soon