I'm telling you, with all the cost and problems with cloud AI, local is where it's going to be.

I'm genuinely shocked people are driving around in Atlas right now, showing OpenAI how to click buttons and how to login to their bank accounts.

This feels like an oversimplification of a difficult problem. But agree local LLMs are the future!

We're seeing the same thing for many companies, even in the US. Exposing your entire codebase to an unreliable third party is not exactly SOC / ISO compliant. This is one of the core things that motivated us to develop cortex.build so we could put the model on the developer's machine and completely isolate the code without complicated model deployments and maintenance.

It’s convenience - it’s far easier to call an API than deploy a model to a VPC and configure networking, etc.

Given how often new models come out, it’s also easier to update an API call than constantly deploying model upgrades.

But in the long run, I hope open source wins out.

Since each chat is virtually independent there’s no switching cost. I’ve moved between Claude and ChatGPT with no cares.

It’s not like Facebook where all my friends stay behind

> Since each chat is virtually independent

That hasn't been true for a while though. Open a new chat tab in ChatGPT and ask it "What do you know about me" to see it in action.

You can turn that off. If you're using LLMs for technical or real world questions, it's nicer for each chat to be a blank slate.

You can also use Temporary Chats for that.

We are working on a fully local coding assistant with auto complete and agentic modes. We created a novel post training pipeline to optimize an 80b param model to run on a standard laptop (16gb RAM) so we can offer truly unlimited and private AI coding.

Sign up for our beta https://cortex.build

I would've considered signing up if scrolling on your website didn't make my modern flagship phone drop frames.

I was interested but looks like it's only available for Macos.

We developed a novel optimization pipeline for LLMs so large models can run on a standard laptop.

Our first prototype optimized an 80B model to run at full 256k context at 40 tokens/s while only taking up 14gb of RAM.

We are currently leveraging this tech to build https://cortex.build a terminal AI coding assistant.

Active params are the future

My instinct has been that decline in birthrate is directly connected to the distribution of wealth, at least in the US.

Boomers could afford a house and 6 kids on one person’s salary.

Today both parents need to work just to afford the basics, let alone child care and food. That economic pressure has to have some affect on birthdate.

Salaries were probably equal to or better. Far, far less stuff to spend money on. Houses were (way) smaller and less featureful. Kids were self-amusing and out of the house in packs all day.

Price per sqft has risen 60% and median housing price has dramatically outpaced median income.

https://brilliantmaps.com/us-houses-prices-1950-2024/

I feel like there’s a million tools for this like t3 chat