> my case this is caused by the training data having
I think it's caused by you not having a strong enough system prompt. Once you've built up a slightly reusable system prompt for coding or for infra work, where you bit by bit build it up while using a specific model (since different models respond differently to prompts), you end up getting better and better responses.
So if you notice it putting plaintext credentials in the code, add to the system prompt to not do that. With LLMs you really get what you ask for, and if you miss to specify anything, the LLM will do whatever the probabilities tells it to, but you can steer this by being more specific.
Imagine you're talking to a very literal and pedantic engineer who argues a lot on HN and having to be very precise with your words, and you're like 80% of the way there :)
Seems like this is the sort of thing the tooling should do for you. Maybe have a few preset prompts for different contexts. I shouldn't have to type in a magic phrase to get it to write code that follows basic professional practice.
> type in a magic phrase to get it to write code that follows basic professional practice
The thing is, there is no agreed upon "basic professional practice" that could be encoded. Ask 10 programmers what "basic professional practice" is and you're gonna get 10 different answers.
For me, simplicity, avoiding over-engineering and being deliberately slow are the best ways to program. But lots of people disagree with this, especially the last part, so why have a different model for every preference, when you can use system prompts to have one model that could follow all of it?
Yes, you are definitely right on that. I still find it a concerning failure mode. That said, maybe it's no worse than a junior copying from online examples without reading all the text some the code which of course has been very common also.
I think it's caused by you not having a strong enough system prompt. Once you've built up a slightly reusable system prompt for coding or for infra work, where you bit by bit build it up while using a specific model (since different models respond differently to prompts), you end up getting better and better responses.
So if you notice it putting plaintext credentials in the code, add to the system prompt to not do that. With LLMs you really get what you ask for, and if you miss to specify anything, the LLM will do whatever the probabilities tells it to, but you can steer this by being more specific.
Imagine you're talking to a very literal and pedantic engineer who argues a lot on HN and having to be very precise with your words, and you're like 80% of the way there :)