Hi everyone, I’ve been thinking about using hypertext instead of text files as the medium to represent programs and their executions .
Programs contain a lot of cross references that are resolved only after the parser/runtime have had a go. IDEs also redo this parsing work to simulate the same cross references for easy editing. The idea is to embed these references in the medium itself. So, get rid of import statements, file boundaries and such. Each textual reference - function name, type name, any identifier really - would be link to the object it references. Unlike strongly structured environments, this allows some flexibility in representing partial and possibly invalid structures. I’m interested in implementations you know of that match this idea.
About “and their executions”: after execution of a metaprogram, and during execution of the program, many references are resolved - perhaps this can also be represented in the same medium. The “generated program” or “execution trace” could form new hypertext objects that reference the original or other new hypertext objects.
Your idea reminds me of Jonathan Edwards' Subtext system: subtext-lang.org
Ah yes thanks for bringing that up. I’m familiar with Subtext and in fact using the same structure for both the execution trace and the program is influenced by Reifying Programming. The difference here is using hypertext objects as the underlying structure. Subtext tends to have more strict tree like editors.
I assume hyperlinks here would be just a mechanism to construct trees?
I've long thought there should be a dual of link
called inline
: you don't need to click, you just see the contents of the destination in place. A combination of link
and inline
might subsume lots of ideas like Subtext and Brief (youtube.com/watch?v=R3MNcA2dpts)
Thanks for the video - will check it out.
Hyperlinks are not mechanisms just to construct trees. Even in basic cases references form DAGs and even cycles. eg here’s a trivial DAG:
class T: ...
def f1(a: T): ...
def f2(b: T):
f1(b)
The T
annotation in both function definitions link to the class definition. The f1()
call links to the f1
function definition.
Any mutually recursive definition will form cycles.
Note also the hypertext objects here are individual definitions and dont impose any language semantics.
Not sure I understood the inline
idea. Are you talking about transclusion?
Indeed I think to inline
any refererence makes sense. Any hard link should be inline-able by the reader. I don’t think the writer has to decide if the link should be inlined or not.
There's advantages both ways. Writer deciding can construct experiences.
I mostly care about when writer == reader, so the question is moot.
Shalabh Chaturvedi One ingredient of your idea is extreme late binding. That's one of the principles of Smalltalk, and Smalltalk IDEs implement some of what is being discussed here, though only within the universe of the image.
Example: Glamorous Toolkit lets you see the definitions of methods you call inline, which is very convenient. Your outline of metaprograms defining programs which again contain dynamically resolved references is pretty much what people call metaprogramming in Smalltalk.
Pharo's relatively recent slot mechanism would allow implementing more general types of links, but I haven't seen this done.
Smalltalk users wouldn't call their code "hypertext" but rather "object graph", as much of the code structure never gets expressed as text. But that seems a minor technical detail to me.
I very much like this idea, it's so intuitive that I just thought, geez why isn't it done like that already
Konrad Hinsen interestingly I was thinking one aspect of code-as-hypertext is earlier than usual binding. References get bound to objects when written. This is even earlier than in plain text compiled languages (identifiers in a text file are not bound at all). However the bindings-as-written are not the final bindings. During the course of meta-program and program execution they get rebound. Eg a reference to an abstract type may get re-bound to a concrete version of that type. Evolution of these bindings can be represented in the same structure.
Smalltalk / Newspeak etc are a key influence behind the above idea. In Smalltalk we write classes and their methods directly, not worrying about file boundaries. I was thinking why the method body is text and not hypertext? Maybe new versions of GT/Pharo do this better. GT still seems layered on rather than native.
Where do you imagine the hyperlinks going to, other definitions? Many of my apps have that ability, Teliva and some of the Freewheeling stuff, and I'm wondering if the ability to click on hyperlinks is all we're discussing.
Regarding the runtime execution of a program, the bottleneck is performance which feels orthogonal..
Where do you imagine the hyperlinks going to, other definitions?
Yes and also derived definitions (eg objects created via meta programming) and even runtime traces can link back to the definition objects.
I’m wondering if the ability to click on hyperlinks is all we’re discussing.
Good question. Clickable links between source definitions exist in many environments. Even vscode lets me do that with most of my programs. However this is typically implemented as a layer on top of the underlying media (text files). The links may not match always actual semantics, but more importantly they do not navigate to derived definitions or connect runtime objects with definitions. In fact derived definitions are not typically available in the dev environment at all (smalltalk envs are the exception).
The idea is to use hypertext as a unifying interaction model between the person and the system - for hand written definitions, derived definitions as well as runtime traces. The hypertext ux would include clickable links and any other features we want to include eg inline content.
I think I understand now. The runtime traces is hard to do performantly, but for everything else, yes this sounds like a nice reasonable goal. My stuff isn't it yet; you can't click on any function to go to its call, as a very basic example. The metaprogramming, I gather you want to be able to click on a macro call and see its expanded output. All this seems doable. I think I'm headed in a similar direction but wasn't aimed quite here. Now I might be. 🤔
FWIW: this idea fires a set of neurons in me, labelled “GOTO”.
Synopsis
GOTO v1 - assembler GOTO
GOTO v2 - message passing
GOTO v3 - CPS - Continuation Passing Style
GOTO v4 - URL
All versions of GOTO suffer from the problem of so-called “structured”-ness.
Random Notes
- Kinopio is GOTO v4
- Message Passing is my current sweet spot.
- most current programming languages are based on the Synchronous Pattern, i.e. they handle GOTO v1 and GOTO v3 but fumble GOTO v2 and GOTO v4.
- “come from” is just GOTO in reverse, i.e. backlinks
Lessons From Org Charts
Further Thoughts
📝 2023-07-01-GOTO - Obsidian Publish
GOTO GOTO v1 - assembler GOTO GOTO v2 - message passing GOTO v3 - CPS - Continuation Passing Style GOTO v4 - URL All versions of GOTO suffer from the problem of so-called "structured"-ness. I am s…
The runtime traces is hard to do performantly
Agree. This could be opt-in via a debugging / “capture trace” mode. Hypertext that links back to definitions seems better than explicit print style debugging.
The metaprogramming, I gather you want to be able to click on a macro call and see its expanded output
Yes. I also think it would be nice to have the generated definitions available in searches. Derived definitions may also be produced from external schemas or import time execution.
Paul - thanks for sharing. My initial thought is that what you are calling GOTOs I usually call forms of coupling. It’s the essential quality of computers - specifically the couplings in the system simulate couplings in the real world or other models that we are trying to represent.
I’m not sure how URLs are v4 - I assume you mean some kind off identifier that represents a semantic concept (like in RDF?)
I was thinking one aspect of code-as-hypertext is earlier than usual binding.
and then:
However the bindings-as-written are not the final bindings.
I am not sure I understand your bindings. Are they mutable or not? Or maybe mutable initially, then frozen?
As for GT, yes, it's a new tooling layer on top of a standard Pharo object graph. The bindings presented in the UI thus reflect the current state of the system, but everything can change at any time.
I was thinking why the method body is text and not hypertext?
Names would then point to what? There is nothing but names and objects in the Smalltalk object graph (if you see literals as special names).
In fact derived definitions are not typically available in the dev environment at all (smalltalk envs are the exception).
And the big practical issue in Smalltalk is that you get only the derived definitions, with no provenance tracking unless the metaprogramming framework has its own mechanism for that. But then, that's not a fundamental issue, it's part of the tooling layer.
I am not sure I understand your bindings. Are they mutable or not? Or maybe mutable initially, then frozen?
This would be PL dependent but typically I imagine no in-place rebinding. However a form of rebinding happens by creating new versions of the hypertext objects. Eg the hand edited source objects could only be rebound by the user as part of editing. In the metaprogramming stage the system would create derived versions of these objects where some of the bindings have changed. For instance a binding to an abstract type in a source object may be resolved to a concrete subtype in the derived objects. So the PL semantics would also restrict what kinds of “rebindings” happen. Eg “constants” would be bindings that don’t change in any derived objects.
I think you might run into situations where you want a thing to hyperlink to multiple possible targets. The current binding yes, but sometimes also the previous binding, or the previous edit. Or the edit history.
Does your point against inline
apply also to the very idea of hypertext? Why does the writer need to create hyperlinks, can the reader just have tools for all these transforms? That's basically the world I live in today. My IDE takes me to a word's definition with F12, all its references with S-F12, all its callsites with C-k C-t. Perhaps we just need to do more of this?
Now I want to link to an old thread of Lisp Machine screenshots, but it's sadly stuck in Twitter.
I think you might run into situations where you want a thing to hyperlink to multiple possible targets.
Yes. I imagine viewing the source and various derived objects as distinct hypertext objects where you are able to go back and forth and follow the chain of provenance.
Does your point against
inline
apply also to the very idea of hypertext? Why does the writer need to create hyperlinks, can the reader just have tools for all these transforms? That’s basically the world I live in today.
Good question. While it does bother me that the tooling overlaid links are an approximation (and poorer for dynamic langs generally), the bigger issue is the lack uniform navigation across derived definitions and source definitions. I want to see the rendered macro, the concretized generic method and so on. I want to deftly bounce between those and the original source definitions.
BTW your earlier counterpoint of writer designed experience via inline
is valid. I think the reader should still have the ability to customize it further (eg collapse an inline
).
A related point is that reader doesn’t get to customize the text in text-file-as-code. For example if the writer chose to write import pandas as pd
, the reader has no choice but to see pd.foo
scattered across the file. If this was hypertext-as-code, it’s easier for tools to offer customized rendering, eg “show the fully qualified name everywhere”, or “show the short name, but blue for pandas
” and so on. In theory this is possible with text-file-as-code but more challenging, given that tooling is trying to simulate the PL semantics to extract high fidelity references.