I’ve come to believe that we’re missing out on approximately 90% of what’s possible because of how we program. It’s not just about the syntax, it goes much deeper - it’s rooted in culture, religion, dogmatism, and closed-mindedness. I think the word “function” has been overused, and the emphasis on efficiency has gone too far. Here are a few thoughts… YMMV. What do you think? How Did We Get Here? The Best Programming Language
2025-02-07
📝 The Best Programming Language
2025-03-07
I’m curious about your thoughts on this idea: Glamorous Toolkit and Live Programming environments seem to essentially be REPLs that use modernized input and output technologies.
Back in the 1950s, REPLs used the best I/O and interface tech they had back then - mostly text-based command lines because computers couldn’t do much else. But now, environments like Glamorous Toolkit make output look better with graphics instead of just printing text, and Live Programming environments have GUIs with interactive stuff like sliders to change things while the program is running.
The only tricky part in designing modern REPLs seems to be figuring out how to keep things flexible while still working with strongly-typed, functional programming languages.
Do you think I’m oversimplifying these modern programming environments? Am I missing something important about how they differ from traditional REPLs?
Sometimes pieces of the environment have an event loop of their own with which the coder can interact in a variety of ways. This is quite different from a 50s/60s-style REPL where there’s a single thread of execution.
Actually, I'm still missing something. How is the modern main-loop different from the 50s/60s idea? In the 50s/60s REPL, the code runs a main-loop, and serially, runs a little GUI each time through. In the modern stand-beside Live thing, the main-loop serially runs a little GUI each time through. The GUI code is embedded in the main-loop in the 50s/60s version, but is stand-beside in the modern version. The effect is the same, just the implementation details differ (slightly). In the end, either approach invokes a function(s) to change something at runtime. Only the presentation of the GUI is different. And the speed at which the main-loop comes around is faster today. ? Jack Rusher
In the Smalltalk case, there are objects that can have their own threads of execution independent of the GUI loop. In the Lisp case, you can have running threads that are independent of the REPL loop. These threads can be doing various computations, passing messages to each other, &c, with the REPL allowing you to “talk” to them. Most music livecoders have multiple such threads of execution producing different streams of audio at any given time, for example.
I'd say the overall idea of these environments is the same as with the REPL of the 1950s: better feedback loops. As Jack Rusher points out, this has literally led to more loops that users interact with. Glamorous Toolkit has a lot more concurrency than a Lisp REPL. It has even added quite a bit support for asynchronous execution to Pharo, the Smalltalk that it builds on.
How much that matters depends a lot on what you are doing. For me, it's only a convenience because most of my work can be summarized as automated mathematics, meaning that timing doesn't matter except when it exceeds my patience level. But if you have interactive processes, or need to supervise concurrent execution, the difference really matters.
The only tricky part in designing modern REPLs seems to be figuring out how to keep things flexible while still working with strongly-typed, functional programming languages
Can you expand on this point? What do you want a strongly typed language to give you in a repl environment?
I can try to expand on this point :-). Please ask some more, if interested. programmingsimplicity.substack.com/p/on-repls?r=1egdky.
📝 On REPLs
2025-03-15
Paul Tarvydas I agree with you that type checkers ought to be add-on tools rather than part of a language. And I'd want my type checker to be moldable as well. But I have only once seen this approach implemented, in Shen, which is far from a mainstream language. Are you aware of other attempts to do this?
Shen Programming Language
Konrad Hinsen No, I haven't found any languages that satisfactorily split type-checking from implementation. I have seen many hints that it is a reasonable goal, e.g. lint, Ohm (not types, but locality-of-reference), Statecharts (not types, but locality-of-reference), Agda (which looks to me like it contains 2 syntaxes, types and implementation), having played with Prolog for inferencing semantics from diagrams, having worked on Y2K stuff which involved weekend-long inferencing runs to dig out semantic information beyond what is commonly referred to as "types", ML, etc.
Paul Tarvydas There indeed many verification and validation systems that are similar to static type checkers. But there is also one important difference: static type checking relies on syntactic features of the underlying language, so it requires some co-design of language and type checker even if the implementations are separate.
Konrad Hinsen Yes, but that is not a good reason to sully the readability of code by conflating it with type annotations and v.v.. IMO implementation and typing are orthogonal concerns, hence, should be written in separate places.
Denotational Semantics hangs actions off of syntax. There is no reason to force humans to DRY syntax patterns. Syntax is cheap, push this concept further and let the machine figure out how to join things. ... [example to follow] ...
Paul Tarvydas OK, we agree. Now let's convince language designers and implementers!
Konrad Hinsen example, finally (Note that I shamelessly stole your phrase "Data Model" 🙂) [I seem to be batting 0.000 for explaining clearly what I mean, but, I would be glad to entertain further questions and requests for clarification]
2025-03-16
I'm not really sold on ZUIs (zoomable user interfaces) but I became a bit more enticed once I realised that you could make ZUIs for graphs instead of trees. Where you can zoom into things and eventually get back to where you started... I feel like (aside from the potential motion sickness) it induces this visceral sense of how software and data topologies are governed by laws that we made up... And that, so long as we don't break any actual physical laws we are free to make up the semantic rules as we please...
Anyway, made this zoomable hypertext thing the other day to figure out how you would do infinite zoom in an infinite coordinate space that has this weird graph topology, where you can have A --inside-> B --inside-> A
Ended up with what I called a Shifting-Origin Graph which has a series of matrix transforms that can traverse through the graph with you, keeping floating point precision at bay and keeping memory usage fixed.
I would be curious to hear takes on ZUIs here, I feel like there's a more general view on them that would be more compelling to me, I got a glimpse of that as I made this experiment, realising that it would open up some interesting semantics that aren't really to do with ZUIs, and more to do with the ability to warp space in new ways.
interesting, software systems that support circular imports have this structure, and you do want to traverse to dependencies sometimes
amazing ... def checking it out
youtube.com/watch?v=En_2T7KH6RA
checkout this demo of Xanadu ... I think ZUI was a part of it
It's very weird but I can't think of any natural analogue for infinite zoom. I mean this in evolutionary / cognitive sense - an object that could exist at arbitrary scale, or an experience where "infinite zoom effect" would happen naturally. We perceive the world around us at several different scales - from a couple kilometers to ~100s of micrometers. Different scales carry different significance for us. Objects around us help us navigate while small things can be acted upon. Infinite zooming mixes those two categories. This can be used to make interfaces that smoothly switch between being an object in our hands and our surroundings. It's really cool and can be slightly confusing but it's probably still way more intuitive than "page by page" navigation.
I also have a feeling that the infinite zoom UI is using the same neural pathways as 3D navigation actually. So arranging the "doors" horizontally might reduce the confusion and make it easier to navigate.
I'm a "leverage human familiarity with physical 3D reality" kind of person. So for me the only way ZUIs could be natural and not disorienting is if they were designed to look and work like 3D perspective.
@Marek Rogalski could it be “as above , so below“ the infinite recurring pattern in nature , fractals , Fibonacci etc…?
Oh yeah - fractals appear in plants! That could make a nice metaphor for ZUI.
But still, as I say, humans don't normally spend a significant proportion of their everyday productive waking existence zooming into and out of Romanesco broccoli with a magnifying glass, or spotting Fibonacci patterns in their shopping lists..!
Lua is 1,250x larger than Sector Lisp. Rust is 750,000x larger than Sector Lisp.
Are my numbers and data correct? programmingsimplicity.substack.com/p/programming-language-sizes?r=1egdky
2025-03-12
@Alex Bender It works fine for me when I click on it. Hmm. I don't know how to diagnose / fix this. The bulk of the article is a PDF. I can get it to you some other way, if interested.
We should talk.
Data-First Principles:
apologies for the logo - it was an AI experiment that I dropped into the article and now it's all over the place!
I mentioned this elsewhere but I am skeptical that multiplayer is an important concept for data first, personal computing apps. Even in big b2b companies actual honest concurrent editing and realtime is quite rare, that only an important concept in instant messaging so I don't think you should over index on it.
(the problem with multiplayer is it is directly at odds with other things and mandates an intermediate server)
Rather than multiplayer as one of the principles of data-first, I would prefer to have “write only”(/append only/historical). You can build multiplayer on top of that.
The idea here is that if you have your one-person write-only local store, and you ingest values from someone else’s local store, where you’re wanting to present a multiplayer store built out of some parts of both, then you have all the information you need to present the multiplayer store; it is left an exercise for the reader.
Also, you might want to do this in different ways. If you separate the multiplayer concern from having the data, you can support different syncing strategies, presentation of history etc.
Rather than multiplayer as one of the principles of data-first, I would prefer to have “write only”(/append only/historical). You can build multiplayer on top of that.
The idea here is that if you have your one-person write-only local store, and you ingest values from someone else’s local store, where you’re wanting to present a multiplayer store built out of some parts of both, then you have all the information you need to present the multiplayer store; it is left an exercise for the reader.
Also, you might want to do this in different ways. If you separate the multiplayer concern from having the data, you can support different syncing strategies, presentation of history etc.
Tom Larkworthy i’m not sold that we should throw away multiplayer because of that. It’s almost more motivation to figure out how multiplayer fits into a decentralized, local-first vision.
We have a couple of POCs for folk canvas exploring what infastructureless WebRTC looks like. “Infastructureless” in the sense of there being no need to setup your own servers for webrtc signaling, using physical, local protocols (e.g. sound, qrcodes) or piggybacking off open networks and protocols (e.g. gundb, MQTT, bittorrent trackers). There are still question about reliability, but feels promising so far.
folk.systems/canvas/#network (still really early on)
I love multiplayer. But I think we could accept single player data first apps as legitimate if they have the other things.
For me, the point is to escape the poor local maximum that SQL gave us. To have a world where we can wield the relational model freely. Multiplayer is lovely and should be more common. But I think it’s a separate concern.
I would like to think my single file local-first unbundled Observable notebook thing with integrated dataviz and reactive spreadsheet coding would qualify as data first. Its literally a local-first port of a reactive dataviz platform, it ticks every box except live multiplayer. I do not intend to add live multiplayer, I think its not so useful in comparison to async git style merging which I would like to have.
Exactly! I think for some purposes, having a single-player only-mine store with explicit syncing would be a nice choice to have, rather than must-have-multiplayer.
Users of git should likely agree with me.
I'm still processing all of this, but for now I'd like to say that the primary, or indeed sole, purpose of the article was to stimulate discussion, not to get all abstract and definitive about what is or isn't Official. So ... "achievement unlocked"! I've been having some extremely intense and stimulating email conversations with many of the leads of the projects I listed, and would love for the insights we're digging up to be made more public. This article is by far the most popular I've written, so it's hit a chord (or a nerve!) with quite a number of people. I'm really excited at the possibility of more open discussion (i.e. not in Closed Proprietary Slack or my closed off email threads!) A public forum of some kind would be an obvious outlet for all of this energy.
Obviously, many of the leads are right here: @Omar Rizwan @Alexander Obenauer @Andrés Cuervo Jonathan Edwards @Guyren Howe @Federico Pereiro Clemens Klokmose Christopher Shank Orion Reed Peter van Hardenberg Joshua Horowitz Paul Sonnentag Geoffrey Litt - so it's a common ground amongst many future-of-computing folk. In other words, the community and the realisation that so many have such similar drivers and motivations is more important than the details of whether it's this or that in detail on the network. In fact, why not simply settle for "something that feels like Local-First" as having a common "vibe" in that department?
Tom Larkworthy I must admit I'm still struggling with the idea of non-collab subsets of a Data-First philosophy... so you have use-cases where you yourself are literally the only and sole and forever-only consumer of your own work? This is hard for me to grasp. But maybe that's just me.
And git as an alternative to Local-First is something I really need to process on. I'm finding all this, well, surprising!
Chris Gebhardt has got me thinking about immutable content on the net, which seems to be related to @Guyren Howe’s comments above. The Noosphere project had something along these lines also. Again, I'm processing.... 🤔
Christopher Shank I've moved Folk Canvas up with the others in the article in the light of the network stuff you linked to. I'm really excited to see what you come up with in that area!
I think economic incentives in the development of technology have strongly favoured big business. We want to write large apps with lots of users that can run efficiently on AWS.
But computing is staggeringly cheap. If we are willing to entertain “inefficiency”, we can make small business and individual user software in very different ways to what we do now.
This and other aspects of the economics of all this lead me to believe that our default when storing data is that it is a write-only store. The “current” version of a row is the one with the latest timestamp.
It is easy enough to roll event sourcing into this. We already “store” incoming requests — in the stupid text log file, if nowhere else. If instead, we store full, structured inputs to each request coming into a system in a database table (because we’re all about relations), then we arrive at the results of an input to a system are the results of triggers on those inputs.
Step back, and consider the larger picture: every state the system was ever in can be reviewed. Every input to the system is recorded, and every state transition.
Now, we circle back to small business software, and to my other bugbear: we don’t make software for non-developers to solve their own problems.
But if you put a FileMaker-like interface in front of that write-only store, and you think about augmenting that UI with tools to explore its history, I think you really have something.
Future of programming? Give me this system, with a Datalog query interface, and I can replace most of what I do in a traditional programming language with queries. Traditional programming is relegated to side-effecting or efficiency-concerned stuff.
I must admit I'm still struggling with the idea of non-collab subsets of a Data-First philosophy... so you have use-cases where you yourself are literally the only and sole and forever-only consumer of your own work?
Sharing is to me is extremely important too. You build insights with data, you take data backed actions. Some actions affect other people, its good to construct and share data based arguments. Its one way of clear communication . The web/email is great for sharing. But, I don't personally see a need for live collaboration for the construction of data based arguments.
I do think that if you receive a read only data based argument, e.g. a blog post with attached computation, then it would be nice to be able to construct a retort by forking that blog post and adding your own data/computation. That then becomes distributed async collaboration. I don't think data-first would need to insist thats a principal, thats just my personal taste of ideal collaboration.
Anyway, I think collaboration is an orthogonal concern, there are many useful and different ways of collaborating and it doesn't make sense to be prescriptive. I only bring it up because the original blog has it as the litmus test to being "data-first". I love the overall idea especially about empowering people to derive data insights.
Multiplayer gets into authorisation issues, if nothing else. Sometimes, you want a different sort of sharing.
I suppose I should weigh in here since I triggered Duncan with my call for Data-first software in my LIVE keynote talk.
I defined Data-first software as:
I suggested a research agenda:
There is some discussion on this thread about multiplayer collab. My opinion is that this may be a necessary feature, but it would actually be better to have a version-control-like async collaboration model.
A great way to further this conversation would be to submit demos of your Data-first or Substrate ideas to the LIVE Programming Workshop, which will be online this year.
📝 Workshop on Live Programming (LIVE)
The 11th Workshop on Live Programming (LIVE 2025) will take place online. LIVE invites submissions of ideas for improving the immediacy, usability, and learnability of programming.
This!
It is natural for “data first” to suggest having lots of “data about your data”. Its history is the obvious starting point — hence, my suggestion of append-only stores.
Another natural thing is diffing and merging.
Once you have that, it will be natural to want branching, as in version control.
Properly presented, branching and merging will be natural for non-developers to use.
Really, you’re talking about supporting multiple models. Meaning, first-class models.
Imagine an accounting tool that supported this. A user could look at their entire system, subject to different assumptions, and run all their models. What if sales had been higher last quarter?
Coming super late to the party, but I'm very excited by what's being spoken about on this thread. Although we're coming at it from different angles, it feels like we want to converge onto the same thing.
That would be nice, Federico! My goal was only to get a conversation going, not to get all divided!
why ~is~ does it feel like the world of "what people and businesses are paying for today to lower the barrier to entry for computational needs" is an almost non-intersecting set with "what excites us about lowering the barrier to entry for computational needs"?
The main mistake they’re all making is to treat the user as too dumb.
What the user wants is an end-user programming tool about as complex (and sophisticated!) as Excel.
Not these paint-by-numbers monstrosities that look they’re intended for a 10 year old audience.
Nothing wrong with a 10 year old audience, of course. But business users can handle more. And a really successful tool will provide it.
@Arvind Thyagarajan I'll widen your question to "why are so many bad decisions taken in the world?", with "bad" in some semi-objective sense (so not just "opposite to my opinion").
My answer is: hubris. There are too many people who believe to know the Right Answer to complex questions and don't bother to check.
A quote that sums up the straitjacket feeling of so many of those tools:
“Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.” — Doug Gwyn
You should definitely submit a demo to the LIVE Programming Workshop this year! It is being held entirely online so no excuses. Submit by July 21
[is this the right place to post this?]
📝 Workshop on Live Programming (LIVE)
The 11th Workshop on Live Programming (LIVE 2025) will take place online. LIVE invites submissions of ideas for improving the immediacy, usability, and learnability of programming.
Rhetorical question?: how would your programming workflow change if you deeply believed that creating SCNs was cheap and easy? Perl leaned heavily on REGEX. REGEX was once thought to be hoary and problematic. Perl made it accessible. New kinds of things were invented when Perl started being widely used. T2t is “better” than REGEX, because t2t makes it easy to deal with patterns containing recursive nesting. CFGs, like YACC, make it possible to parse recursively nested text, but, CFGs need painful, full specification of too much detail. If you had a way to specify pattern-matching using tiny DSLs that didn’t cost a lot of time to build, how would your workflow change? ['t2t' leans heavily on PEG and esp. OhmJS] [SCN === Solution Centric Notation, essentially a nano-DSL, I think of Richard Feynman breaking away from mathematical/Gutenberg notation and inventing and using Feynman diagrams. Language affects thought, notation affects thought].
I do actually believe that creating SCNs is cheap and easy, but so far I express this belief mostly by creating s-expression-based SCNs in Lisp. I am aware of the limitations of that technique, but so far it has been a good compromise for me. The step to PEG gives more power but also comes with a higher cost, in particular if you consider editor support for your DSL, which with s-expressions basically comes for free.
What I'd love to see explored is other wide-spectrum syntax frameworks than s-expressions. See this blog post by Shriram Krishnamurthi for a detailed discussion of s-expression as a framework for building DSLs (the label "framework" is mine though). In particular for non-textual syntax, which remains largely unchartered territory.
Assuming we get to the point of people widely believing in SCNs being possible and desirable, the next interesting question is software system architecture design with SCN, i.e. putting them to good use with minimal undesirable side effects.
Parenthetically Speaking: Articles by Shriram Krishnamurthi
The goal should be to produce notations so concise that whole programs fit in one eye-full, so that traditional concerns, like editor support, don't apply any more. Writing "bigger" programs should mean writing less code in islands and layers. I first wrote t2t using diagrams (using the non-programming editor draw.io) based on the idea of pipelines and circuits that used OhmJS multiple times. I've since reduced it to a single js program to ease carrying it around. In my mind, the technique is powerful enough to build compilers with and to do non-traditional things like parsing diagrams and building notations that fit in less space on the screen. I've written a lot of Lisp macros, now PEG/OhmJS/t2t give me the power to do this with non-lispy languages. I was taught to build compilers as pipelines connected by little, custom SCNs for each pass - I found this to be mentally liberating allowing me to concentrate deeply on one task at a time. Parsing is easy, the other stuff is hard, I toss custom parsers around liberally.
Paul Tarvydas For the use cases I have in mind, there will always be incompressible long code because of the amount of embedded data. That's why I want editor support.
Jack Rusher That's very nice indeed, as long as you stay inside the Racket ecosystem. What I like about Paul Tarvydas t2t approach is that it can target many compilers and runtimes, so you can use what you know and/or fits your needs. Plus you can easily inspect the t2t output, whereas inspecting fully macro-expanded Racket can be quite challenging (trying to put it nicely).
Ruby lends itself to DSLs and this pardigm is widely used in Ruby code.
![[SPLASH'24] Pacific A - LIVE (Oct 21th)](https://i.ytimg.com/vi/4GOeYylCMJI/hqdefault.jpg)