I am really trying to get AI to do software engineering and be genuinely helpful at the higher end of coding. Still working on it 🙂 but the Observable toolchain I co-produced with AI is a step up from where I was a year ago. The technique that helped was feeding the test suite back into the LLM context so we would co-evolve the suite and implementation at the same time. Furthermore I had extra context tools, so if I wanted it to concentrate on a specific test case I had a UI to select that case. I don't think it could make the whole thing alone yet, I definitely wrote a good chunk of the Observable decompiler/compiler pair, but it was very helpful at

• Writing semanticallt broken but executing and close first draft of code
• Choosing which bug to fix next and coming up with a suggestion
• Using new libraries idiomatically (e.g. acorn-walk)

observablehq.com/@tomlarkworthy/observablejs-toolchain

I've been wanting a decompiler for observable for ages, and I don't think I could have done it alone either.