I think that this might be interesting to FoC’ers because: it demonstrates various techniques (t2t, 0D, feedback with queues, etc.) that aren’t common in Functional Programming. These techniques might inspire ideas of what is possible in the FoC realm, beyond the usual more-programming-using-only-the-function-based-paradigm. I, also, believe that VPLs and DPLs are ignored for at least 2 reasons: (1) it’s believed to be very hard to parse diagrams, and, (2) text says it all, what more could there be???. This progress report quietly attacks both mis-beliefs. (1) Parsing diagrams - in 2024 - is sooo easy that even someone like me can do it. (2) Feedback loops and structured message-passing produce efficient ways to express certain idioms that can’t be expressed - easily - in textual form, hence, are roundly avoided, hence, causing inadequate simplicity of design in certain problem domains.

One thing that I keep forgetting to mention, is that even though I appear to be drawing node-and-arrow diagrams, nodes contain input and output ports. Possibly many of each. In contrast, functions in FP, have exactly 1 input port and exactly 1 output port (modulo the wart called “exceptions”. So-called “multiple parameters” to functions are but an optical illusion supported by destructuring operations (it’s all just one block of data that all comes at the same time, regardless of how you want to slice-and-dice the block)). Arrows can only be drawn from port to port, not from node to node. This simple concept of ports provides high levels of encapsulation of data /and/ encapsulation of control flow. Nodes are akin to Actors, except that Container nodes can be recursively defined to contain other Container nodes. Akin to the idea of Lisp lists containing atoms or, recursively, lists.

It's been on my mind to come up with a schema for this. You've mentioned channel -> post -> comment before, and cerca has post -> comment. I feel we need something slightly richer that allows wiki-like workflows as well. Revisions. The ability to summarize a thread in a wiki page while maintaining provenance to the original thread (so another kind of revision?) The ability to create multiple views of a thing, qua Glamorous Toolkit. Maybe that's also another kind of revision?

One challenge in combining markdown with an easy build script: markdown is notoriously hard to parse. Do you have thoughts on how to slice that tension? My thought so far has been to just restrict ourselves to a subset of html. Many forums and blog platforms allow html in comments but only a short list of permitted tags they can just mention below every textarea. That feels like one reasonable approach.

(I'm so sorry to hear about your basement!)

I'm thinking we can get pretty far with a subset of markdown, parsed using a series of not-awful regexes. Here's what I currently have — cobbled together from a few different sources online, not well tested yet.

FWIW: I strongly favour the use of OhmJS (PEG) over the use of REGEXs... REGEXs fool you into thinking that they work, until you hit some edge-case that doesn't work. Unlike CFGs, PEGs can be used for simple parsing tasks instead of mega-project full-blown compilers. I would immediately jump to using OhmJS for parsing something like markdown.

djot.net is mostly markdown but is significantly easier to parse. You probably don't want all of djot, but studying what they changed to make markdown easier to parse seems worthwhile.

Paul Tarvydas I'm generally inclined to agree. Humblebrag: I work on a three-person team at my job, and Alex Warth is one of those three people. But in this case, I value the code being dependency-free and trivial to rewrite in other languages, so Ohm is probably out. So if we figured out a way to parse this subset of md syntax that satisfied the goals of this project, that'd be awesome. Would be happy to switch to that once it's figured out.

dependency-free and trivial to rewrite in other languages

Translation: only languages with built-in regex support are allowed, and let's hope that their notions of regex are sufficiently similar (probably not an issue unless you need advanced regexes, but... we will see).

Just to point out the can of worms hidden behind any idea in the space of "let's keep data transformations simple".

This is strengthening my preference for "html (subset) all the way". Consistently across both wiki and life raft.

OhmJS (unlike Ohm?) is portable to every browser of note.

Reminder that the build script is a sideshow. The main attraction is authoring content for the wiki. That experience should be made as good as possible for as many folks as possible. This goal is in tension with the goal of making the build script stupid-simple and portable (in several senses — don't want to be tied to any particular host platform either).

To the extent that we have a build script at all, the reason to have multiple impls is to allow people to run it locally without prescribing a single ecosystem (eg: npm). But that's not the same goal as "support every language in existence".

Yeah, fair. I can live with a markdown subset. We just need to be able to clearly communicate what subset it is, that'll be part of the authoring experience.

Paul Tarvydas I meant OhmJS, sorry.

Ivan Reese In your defence, a bird in the hand is worth two in the bush. And, my opinion is highly polluted by the notion that most existing 3GLs (Python, Haskell, WASM, etc.) are just assemblers and that OhmJS is the vehicle for making zillions of little DSLs using these assemblers.

We are re-enacting "worse is better" here: go for a simple solution that does 90% of the job and leaves the remaining 10% to the user to deal with? Or go for the Right Thing with a proper parser and proper error messages, at the cost of more code and some dependencies?

The relevant question then is: will our users (present and future community members) be able to deal with the 10%, or will they give up when they can't get their contribution to render correctly?

Konrad Hinsen But what different set of tradeoffs would you make? Can you find one that might better satisfy more people?

Not to ignore your question, though it seemed a bit rhetorical: I don't think the regex will produce an unsatisfying result 10% of the time — not to take the idiom literally, but we're engineers, so let's try to work with realistic numbers. I think it'd be more like 0.01% or 0.001%, and the failure mode would be rendered text that visibly doesn't match your intent, not a silent failure. The ultimate fallback in that case is likely "just write that bit in html". My hunch is that people will generally prefer writing in the subset of md, and would more often hit a case where plain html fails them (they miss a closing tag, say) than where the md regex fails them.

Shipping a bigger, more complex build script will make it harder to port and maintain the build script in multiple ecosystems. That'll mean fewer people use the build script locally, relying on the build step that runs on push. That's a much longer feedback loop.

I think we have something similar to Zookoo's triangle: out of three desirable characteristics, we can pick any two. The three are:

• Simple build system
• User-friendly input syntax (Markdown)
• Well-specficied build system (meaning an input format specification that actually corresponds to what the parser takes)

If we give up 1, we use a Markdown-to-HTML compiler with a parser for something well-defined (Commonmark, djot, etc.).

If we give up 2, we use HTML as our direct input format.

If we give up 3, we use some regexp-munging for transforming something Markdown-like to something HTML-like.

My personal order of preference is 2, 1, 3 (the number being the feature I am willing to give up). I am not going to give any arguments, because I can easily give arguments for any of the three choices. It really comes down to weighing them against each other.

If everyone agrees that these are the trade-offs, we could do a poll to see what we mostly care about.

I appreciate the effort to find a framing that helps us think about this issue. But I disagree with the premise behind this particular framing. I think all 3 are achievable, and are already sufficiently achieved for the immediate needs of this project. The parsing worries expressed in this thread are missing the point of markdown. (For those who know their markdown history — I side solidly with Gruber, and solidly against Atwood.)

I'm going to go ahead and ship what I've got. That's the spirit I'd like to engender with this project — just get something out there, it doesn't have to be perfect. If people try to use it and find that something about the setup impedes their desire or ability to contribute to the wiki, we can work on that concrete problem. I don't expect that what I have will work perfectly, especially not at first. That's a non-goal. The goal is to get people writing wiki entries.

(And because I love a framing, even one I disagree with: 3, … … … … … … … 1, 2)

My point 3 indeed assumes Atwood's perspective on Markdown ("Markdown is a formal language"). I have a lot of sympathy for Gruber's view ("Markdown is a notation") but judging from the Markdown use cases I have experience with, it has failed in practice. It could perhaps be implemented robustly using AI, but there isn't much money to be made there so it won't happen.

Side note: Markdown history deserves being studied in detail because it showcases many general issues with human-computer interfaces.

That said, please go on with your project. "We believe in rough consensus and running code."