If you want to play with "this A.I. stuff" and you have a half-way modern-ish graphic card in your PC or laptop (or even a somewhat modern-ish phone) there's a fair few ways to install and run smaller(ish) models locally on your own hardware, and a host of fancy graphical interfaces to them. I personally use ChatBox and Ollama with the Qwen2.5 models, IBM's Granite series models, and the Gemma models fairly successfully on a reasonably decent (couple years old now) consumer-class gaming rig with an NVIDIA GeForce GTX 1660 Ti and 64 gig of RAM. There's also code editors like Zed or VSCode that can connect to Ollama and other local model runners, and if you wanna get really "nerdy" about it, you can pretty easily interface with all that fun junk from Python and script up your own interfaces and tools.
Except your toy model or toy version of the model will barely work to talk to you, much less write code.
I've done this experiment with a much beefier set of GPUs (3080 10 GB + 3060 12 GB) allowing me to run one step up bigger model.
It's not even comparable to free tiers. I have no idea how big the machines or clusters running that are, but they must be huge.
I was very unimpressed with the local models I could run.
I dunno. Maybe you're expecting too much of them? They're obviously not gonna be like those massive data-center LLMs, but I've had some pretty good "brainstorming" sessions about code and documentation with Qwen and Gemma, and the latest vision-capable Qwen models do a really decent job of extracting data and text out of images (often more accurately than typical OCR engines) and even describing or captioning images, and for writing code, I've had good success with asking it to write me docstrings and type hints, and common "boilerplate" code, and even some pretty solid function requests that often come out exactly as I expected it to (as long as it's fed good "context" to work with, like some basic code-style rules, and access to library documentation and existing codebase).
This seems like becoming an expert at hand-cranking engines or writing your own Forth compiler back when compilers were just getting started.
My point of view, I guess, is that we might want to wait until the field is developed to the point where chauffeurs or C programmers (in this analogy) become a thing.
