When reviewing the changes made from agent mode, I don’t know why the model made the change or whether the model even made the change versus a tool call making the change. It’s a pain to go fish out the reason from a long response the model gives in the chat.
Example: I recently asked a model to set up shadcn for a project, but while trying to debug why things looked pretty broken, I had sift through a bunch of changes that looked like nasty hallucinations and separate those from actual command line changes that came from shadcn's CLI the model called. Ended up having to just do things the old fashioned way to set things up, reading the fine manual and using my brain (I almost forgot I had one)
It would be nice if above every line of code, there’s a clear indication of whether it came from a model and why the model made the change. Like a code comment, but without littering the code with actual comments
That's still not what they're asking for. If it's a commit message, it's already saved to disk. This is at the point where the LLM has written code but the diff to the file hasn't been saved yet.
Even if you were willing to deal with your agent making commits and having to fiddle with git to break those up or undo them or modify them, it still doesn't solve OP's need. They want to know what bits of diff apply to which thinking tokens. The prompt and the output are necessary, but there's no mapping between the final diff and the stream of tokens that came out of the LLM (especially in a commit). But that information is known by the tooling already! When generating the diff, the LLM had to output code changes. So you can map those changes to where they originated from. It's not just "what did I tell the LLM to get this change" it's "what was the thought process that the LLM went through after getting my prompt to end up with this line change"
When reviewing the changes made from agent mode, I don’t know why the model made the change or whether the model even made the change versus a tool call making the change. It’s a pain to go fish out the reason from a long response the model gives in the chat.
Example: I recently asked a model to set up shadcn for a project, but while trying to debug why things looked pretty broken, I had sift through a bunch of changes that looked like nasty hallucinations and separate those from actual command line changes that came from shadcn's CLI the model called. Ended up having to just do things the old fashioned way to set things up, reading the fine manual and using my brain (I almost forgot I had one)
It would be nice if above every line of code, there’s a clear indication of whether it came from a model and why the model made the change. Like a code comment, but without littering the code with actual comments