I gave it a try by asking it a question relating to my own work, essentially "how can I do mock patching in Delphi".
A basic correct answer is that "not easily, as the mocking frameworks available for the language don't support it". If you gave a programmer this question and a bit of time to solve it they might discover the Detours library which could be used in conjunction with the existing mocking frameworks to do it. This kind of insight seems to be beyond LLMs, at least for the moment, but I was hoping for at least the basic understanding of the question required to get the "no can do". answer.
But instead, it waffled on and gave me examples of creating mock objects explicitly in the two mocking frameworks that are available, all of which was quite impressively presented but not answering the question I asked.
If somebody can figure out how to stop LLMs BSing at length they'll be a lot more useful.
The holy grail would of course be composing an example showing them being used together, but the expert mode answer was very, very useful and way better than existing search tools were able to do.
A basic correct answer is that "not easily, as the mocking frameworks available for the language don't support it". If you gave a programmer this question and a bit of time to solve it they might discover the Detours library which could be used in conjunction with the existing mocking frameworks to do it. This kind of insight seems to be beyond LLMs, at least for the moment, but I was hoping for at least the basic understanding of the question required to get the "no can do". answer.
But instead, it waffled on and gave me examples of creating mock objects explicitly in the two mocking frameworks that are available, all of which was quite impressively presented but not answering the question I asked.
If somebody can figure out how to stop LLMs BSing at length they'll be a lot more useful.