Cool stuff. Reminds me of github.com/ggerganov/llama.cpp?tab=readme-ov-file#constrained-output-with-grammars . I've used it to generate some scenarios for RPG games with friends. One issue that I've encountered (ignoring overabundance of tropes) was that the model was sometimes very uncooperative and refused to progress through the given template. Instead it got stuck in some place where it was free to generate arbitrary text and just happily stuttered around, never reaching the end. Have you encountered this? If so - what's your solution?

Thanks and interesting use-case! Yes, there are several solutions for structured/constrained/guided generation, but all of the ones I've seen base the structure validation either on regex or context-free grammars. I can do context-sensitive grammars and semantic rules. Say you want to generate RPG character attribute values between 4-18, but the sum should be based on the level of the character. I don't think the solutions I've seen offer good support for something like that, but it would be fairly trivial with Levlo.

I haven't really tried structures with free arbitrary text beyond just generated names, but I would imagine either prompting the LLM to only generate arbitrary text of certain length (as in "a paragraph", "couple sentences") or using appropriate length in samples would work. Have you tried? I suppose a potential challenge could be to make the LLM understand which part of the structure you mean with the length instructions in prompt, but I would be surprised if a dozen or so samples wouldn't work. Although, maybe the samples would end up directing the content of the arbitrary text too much as well..

Also, I'm very interested in hearing about use-cases people have had for structured generation and the challenges you've had with the current solutions. So thanks for sharing @Marek Rogalski!

Actually some better prompting might have been the way to go... I was rewriting the whole scenario with LLM over many iterations (leaving the PC overnight) and sometimes it was randomly looping somewhere in the middle of text. I guess that shorter, maybe section-by-section prompts could prevent that... I've tried the grammar-based approach to limit the length but the issue there was that the model sometimes planned out the text for more sentences than the grammar permitted. This resulted in truncated descriptions.

but the issue there was that the model sometimes planned out the text for more sentences than the grammar permitted

Yeah I can see this being a problem

I'm not sure if a pair of x/y array's should be passed together on a single wire, or if they should be passed separately for the most flexibility

For some reason this made me think of twisted-pair wiring.

@John Christensen, you and Tom Lieber should talk. See alltom.com/pages/ppl-lab-notebook

Ivan Reese Twisted pair wires sounds like a nice visual metaphor! And a fun svg rendering puzzle to figure out

Kartik Agaram Tom Lieber ooh those are some really unique/elegant visuals

I really like the aesthetics you’ve picked for the interface. Everything has a tasteful, low-key, LaTeX-adjacent vibe. I don’t think I’ve seen that done!

(I guess PANE is sorta in that direction? But not in the same way.)

Joshua Horowitz thanks! I'm trying to go for an interface somewhere between a physicist's white board, and an immaculate TikZ figure.

I appreciate the comparison to PANE, I hadn't come across it yet. I'm always interested in finding better ways to make data visible by default, and PANE has some really cool ideas to draw from!