It's entirely happy paths right now; it would be best to allow the test runner to also test for failures (check expected stderr and return code), then we could write those missing tests.
I think you can find a test somewhere in there with a commented code saying "FAWK can't do this yet, but yadda yadda yadda".
It's funny because I'm evaluating LLMs for just this specific case (covering tests) right now, and it does that a lot.
I say "we need 100% coverage on that critical file". It runs for a while, tries to cover it, fails, then stops and say "Success! We covered 60% of the file (the rest is too hard). I added a comment.". 60% was the previous coverage before the LLM ran.
I have only had some previous experience with Project Euler, which I liked for the loop of "try to bruteforce it -> doesn't work -> analyze the problem, exploit patterns, take shortcuts". (I hit a skill ceiling after 166 problems solved.)
Advent of Code has this mass hysteria feel about it (in a good sense), probably fueled by the scarcity principle / looking forward to it as December comes closer. In my programming circles, a bunch of people share frustration and joy over the problems, compete in private leaderboards; there are people streaming these problems, YouTubers speedrunning them or solving them in crazy languages like Excel or Factorio... it's a community thing, I think.
If I wanted to start doing something like LeetCode, it feels like I'd be alone in there, though that's likely false and there probably are Discords and forums dedicated to it. But somehow it doesn't have the same appeal as AoC.
Yes, I'll only have an answer to this later, as I use it, and there's a real chances my changes to the language won't mix well with the original AWK. (Or is your comment more about AWK sucking for programs larger than 30 LOC? I think that's a given already.)
Thankfully, if that's the case, then I've only lost a few hours """implementing""" the language, rather than days/weeks/more.
In my case, I can't share them anymore because "the conversation expired". I am not completely sure what the Cursor Agent rules for conversations expiring are. The PR getting closed? Branch deleted?
In any case, the first prompt was something like (from memory):
> I am imagining a language FAWK - Functional AWK - which would stay as close to the AWK syntax and feel as possible, but add several new features to aid with functional programming. Backwards compatibility is a non-goal.
>
> The features:
> * first-class array literals, being able to return arrays from functions
> * first-class functions and lambdas, being able to pass them as arguments and return them from functions
> * lexical scope instead of dynamic scope (no spooky action at a distance, call-by-value, mutations of an argument array aren't visible in the caller scope)
> * explicit global keyword (only in BEGIN) that makes variables visible and mutable in any scope without having to pass them around
>
> Please start by succintly summarizing this in the README.md file, alongside code examples.
The second prompt (for the actual implementation) was something like this, I believe:
> Please implement an interpreter for the language described in the README.md file in Python, to the point that the code examples all work (make a test runner that tests them against expected output).
I then spent a few iterations asking it to split a single file containing all code to multiple files (one per stage, so eg. lexer, parser, ...) before merging the PR and then doing more stuff manually (moving tests to their own folder etc.)
EDIT: ah, HN screws up formatting. I don't know how to enforce newlines. You'll have to split things by `>` yourself, sorry.
It stands to reason that if it was fairly quick (from your telling) and you can vaguely remember, then you should be able to reproduce a transcript with a working interpreter a second time.
To be clear: I'm not challenging your story, I want to learn from it.
As much as I don't want LLMs shoved in every product I use, writing or tweaking SQL queries with knowledge of the database schema is one of the LLM uses that work well for me. I don't know if I could jump ship from Cursor/VSCore which gives me that + Vim mode.
I've now seen this across two companies, and it's great to have #rd-martinjaniczek where I can just talk about what I'm doing as I'm doing it, post error messages and later how I solved them (or let somebody else jump in and tell me how they solved it etc.), it helps Slack searchability and combats the remote loneliness, it's pretty great.
And reading other folks' RD channels, you realize how frickin' funny some people are.
I think you can find a test somewhere in there with a commented code saying "FAWK can't do this yet, but yadda yadda yadda".
reply