You are viewing archived messages.
Go here to search the history.

Shalabh 2023-07-01 01:59:44

Hi everyone, I’ve been thinking about using hypertext instead of text files as the medium to represent programs and their executions .

Programs contain a lot of cross references that are resolved only after the parser/runtime have had a go. IDEs also redo this parsing work to simulate the same cross references for easy editing. The idea is to embed these references in the medium itself. So, get rid of import statements, file boundaries and such. Each textual reference - function name, type name, any identifier really - would be link to the object it references. Unlike strongly structured environments, this allows some flexibility in representing partial and possibly invalid structures. I’m interested in implementations you know of that match this idea.

Shalabh 2023-07-01 02:03:26

About “and their executions”: after execution of a metaprogram, and during execution of the program, many references are resolved - perhaps this can also be represented in the same medium. The “generated program” or “execution trace” could form new hypertext objects that reference the original or other new hypertext objects.

Kevin Greer 2023-07-01 02:10:31

Your idea reminds me of Jonathan Edwards' Subtext system: subtext-lang.org

Shalabh 2023-07-01 02:14:55

Ah yes thanks for bringing that up. I’m familiar with Subtext and in fact using the same structure for both the execution trace and the program is influenced by Reifying Programming. The difference here is using hypertext objects as the underlying structure. Subtext tends to have more strict tree like editors.

Kartik Agaram 2023-07-01 02:41:05

I assume hyperlinks here would be just a mechanism to construct trees?

I've long thought there should be a dual of link called inline : you don't need to click, you just see the contents of the destination in place. A combination of link and inline might subsume lots of ideas like Subtext and Brief (youtube.com/watch?v=R3MNcA2dpts)

Shalabh 2023-07-01 04:09:05

Thanks for the video - will check it out.

Hyperlinks are not mechanisms just to construct trees. Even in basic cases references form DAGs and even cycles. eg here’s a trivial DAG:

class T: ...

def f1(a: T): ...

def f2(b: T):

  f1(b)

The T annotation in both function definitions link to the class definition. The f1() call links to the f1 function definition.

Any mutually recursive definition will form cycles.

Note also the hypertext objects here are individual definitions and dont impose any language semantics.

Not sure I understood the inline idea. Are you talking about transclusion?

Kartik Agaram 2023-07-01 04:11:41

Yeah, everything is in Xanadu.

Shalabh 2023-07-01 04:13:59

Indeed I think to inline any refererence makes sense. Any hard link should be inline-able by the reader. I don’t think the writer has to decide if the link should be inlined or not.

Kartik Agaram 2023-07-01 04:15:52

There's advantages both ways. Writer deciding can construct experiences.

I mostly care about when writer == reader, so the question is moot.

Konrad Hinsen 2023-07-01 06:44:43

Shalabh Chaturvedi One ingredient of your idea is extreme late binding. That's one of the principles of Smalltalk, and Smalltalk IDEs implement some of what is being discussed here, though only within the universe of the image.

Example: Glamorous Toolkit lets you see the definitions of methods you call inline, which is very convenient. Your outline of metaprograms defining programs which again contain dynamically resolved references is pretty much what people call metaprogramming in Smalltalk.

Pharo's relatively recent slot mechanism would allow implementing more general types of links, but I haven't seen this done.

Smalltalk users wouldn't call their code "hypertext" but rather "object graph", as much of the code structure never gets expressed as text. But that seems a minor technical detail to me.

Kongwei Ying 2023-07-01 11:45:48

I very much like this idea, it's so intuitive that I just thought, geez why isn't it done like that already

Shalabh 2023-07-01 22:10:25

Konrad Hinsen interestingly I was thinking one aspect of code-as-hypertext is earlier than usual binding. References get bound to objects when written. This is even earlier than in plain text compiled languages (identifiers in a text file are not bound at all). However the bindings-as-written are not the final bindings. During the course of meta-program and program execution they get rebound. Eg a reference to an abstract type may get re-bound to a concrete version of that type. Evolution of these bindings can be represented in the same structure.

Smalltalk / Newspeak etc are a key influence behind the above idea. In Smalltalk we write classes and their methods directly, not worrying about file boundaries. I was thinking why the method body is text and not hypertext? Maybe new versions of GT/Pharo do this better. GT still seems layered on rather than native.

Kartik Agaram 2023-07-01 23:16:48

Where do you imagine the hyperlinks going to, other definitions? Many of my apps have that ability, Teliva and some of the Freewheeling stuff, and I'm wondering if the ability to click on hyperlinks is all we're discussing.

Kartik Agaram 2023-07-01 23:24:06

Regarding the runtime execution of a program, the bottleneck is performance which feels orthogonal..

Shalabh 2023-07-02 00:45:29

Where do you imagine the hyperlinks going to, other definitions?

Yes and also derived definitions (eg objects created via meta programming) and even runtime traces can link back to the definition objects.

I’m wondering if the ability to click on hyperlinks is all we’re discussing.

Good question. Clickable links between source definitions exist in many environments. Even vscode lets me do that with most of my programs. However this is typically implemented as a layer on top of the underlying media (text files). The links may not match always actual semantics, but more importantly they do not navigate to derived definitions or connect runtime objects with definitions. In fact derived definitions are not typically available in the dev environment at all (smalltalk envs are the exception).

The idea is to use hypertext as a unifying interaction model between the person and the system - for hand written definitions, derived definitions as well as runtime traces. The hypertext ux would include clickable links and any other features we want to include eg inline content.

Kartik Agaram 2023-07-02 01:14:34

I think I understand now. The runtime traces is hard to do performantly, but for everything else, yes this sounds like a nice reasonable goal. My stuff isn't it yet; you can't click on any function to go to its call, as a very basic example. The metaprogramming, I gather you want to be able to click on a macro call and see its expanded output. All this seems doable. I think I'm headed in a similar direction but wasn't aimed quite here. Now I might be. 🤔

Paul Tarvydas 2023-07-02 01:27:12

FWIW: this idea fires a set of neurons in me, labelled “GOTO”.

Synopsis

GOTO v1 - assembler GOTO

GOTO v2 - message passing

GOTO v3 - CPS - Continuation Passing Style

GOTO v4 - URL

All versions of GOTO suffer from the problem of so-called “structured”-ness.

Random Notes

  • Kinopio is GOTO v4
  • Message Passing is my current sweet spot.
  • most current programming languages are based on the Synchronous Pattern, i.e. they handle GOTO v1 and GOTO v3 but fumble GOTO v2 and GOTO v4.
  • “come from” is just GOTO in reverse, i.e. backlinks

Lessons From Org Charts

Further Thoughts

publish.obsidian.md/programmingsimplicity/2023-07-01-GOTO

📝 2023-07-01-GOTO - Obsidian Publish

GOTO GOTO v1 - assembler GOTO GOTO v2 - message passing GOTO v3 - CPS - Continuation Passing Style GOTO v4 - URL All versions of GOTO suffer from the problem of so-called "structured"-ness. I am s…

Shalabh 2023-07-02 03:08:26

The runtime traces is hard to do performantly

Agree. This could be opt-in via a debugging / “capture trace” mode. Hypertext that links back to definitions seems better than explicit print style debugging.

The metaprogramming, I gather you want to be able to click on a macro call and see its expanded output

Yes. I also think it would be nice to have the generated definitions available in searches. Derived definitions may also be produced from external schemas or import time execution.

Shalabh 2023-07-02 03:24:35

Paul - thanks for sharing. My initial thought is that what you are calling GOTOs I usually call forms of coupling. It’s the essential quality of computers - specifically the couplings in the system simulate couplings in the real world or other models that we are trying to represent.

I’m not sure how URLs are v4 - I assume you mean some kind off identifier that represents a semantic concept (like in RDF?)

Konrad Hinsen 2023-07-02 07:19:52

I was thinking one aspect of code-as-hypertext is earlier than usual binding.

and then:

However the bindings-as-written are not the final bindings.

I am not sure I understand your bindings. Are they mutable or not? Or maybe mutable initially, then frozen?

As for GT, yes, it's a new tooling layer on top of a standard Pharo object graph. The bindings presented in the UI thus reflect the current state of the system, but everything can change at any time.

I was thinking why the method body is text and not hypertext?

Names would then point to what? There is nothing but names and objects in the Smalltalk object graph (if you see literals as special names).

In fact derived definitions are not typically available in the dev environment at all (smalltalk envs are the exception).

And the big practical issue in Smalltalk is that you get only the derived definitions, with no provenance tracking unless the metaprogramming framework has its own mechanism for that. But then, that's not a fundamental issue, it's part of the tooling layer.

Shalabh 2023-07-02 17:24:28

I am not sure I understand your bindings. Are they mutable or not? Or maybe mutable initially, then frozen?

This would be PL dependent but typically I imagine no in-place rebinding. However a form of rebinding happens by creating new versions of the hypertext objects. Eg the hand edited source objects could only be rebound by the user as part of editing. In the metaprogramming stage the system would create derived versions of these objects where some of the bindings have changed. For instance a binding to an abstract type in a source object may be resolved to a concrete subtype in the derived objects. So the PL semantics would also restrict what kinds of “rebindings” happen. Eg “constants” would be bindings that don’t change in any derived objects.

Kartik Agaram 2023-07-02 17:42:21

I think you might run into situations where you want a thing to hyperlink to multiple possible targets. The current binding yes, but sometimes also the previous binding, or the previous edit. Or the edit history.

Does your point against inline apply also to the very idea of hypertext? Why does the writer need to create hyperlinks, can the reader just have tools for all these transforms? That's basically the world I live in today. My IDE takes me to a word's definition with F12, all its references with S-F12, all its callsites with C-k C-t. Perhaps we just need to do more of this?

Now I want to link to an old thread of Lisp Machine screenshots, but it's sadly stuck in Twitter.

Shalabh 2023-07-02 19:12:51

I think you might run into situations where you want a thing to hyperlink to multiple possible targets.

Yes. I imagine viewing the source and various derived objects as distinct hypertext objects where you are able to go back and forth and follow the chain of provenance.

Does your point against inline apply also to the very idea of hypertext? Why does the writer need to create hyperlinks, can the reader just have tools for all these transforms? That’s basically the world I live in today.

Good question. While it does bother me that the tooling overlaid links are an approximation (and poorer for dynamic langs generally), the bigger issue is the lack uniform navigation across derived definitions and source definitions. I want to see the rendered macro, the concretized generic method and so on. I want to deftly bounce between those and the original source definitions.

BTW your earlier counterpoint of writer designed experience via inline is valid. I think the reader should still have the ability to customize it further (eg collapse an inline ).

A related point is that reader doesn’t get to customize the text in text-file-as-code. For example if the writer chose to write import pandas as pd , the reader has no choice but to see pd.foo scattered across the file. If this was hypertext-as-code, it’s easier for tools to offer customized rendering, eg “show the fully qualified name everywhere”, or “show the short name, but blue for pandas ” and so on. In theory this is possible with text-file-as-code but more challenging, given that tooling is trying to simulate the PL semantics to extract high fidelity references.

Jan Ruzicka 2023-07-02 09:36:06

Recently, I realised that despite “vaguely knowing” what VPRI was doing (and trying to do), I never bothered to thoroughly read their reports and papers. Now that I took the time to read them, I feel like overall it’s been time well spent (I got inspiration and encouragement for some projects).

I’d like to know if you know about the VPRI undertaking, and if you do, whether you read (some of) their papers/reports - see poll below. I’m also interested in similar experiences you had with other projects (thinking you know what they do, then having this understanding deepened by looking at their work more carefully).

⚪ I didn’t know about VPRI.

🔵 I know about them, but don’t really know what they did.

🔴 I know what they did, but read nothing from their writings.

🎯 I read some of their reports.

✅ I read some of their reports and specific papers.

Joe Nash 2023-07-02 09:51:50

I’m not familiar, and vpri also appears to be an overloaded acronym. Do you have a link or the full name of the project you’re referring to?

Stefan Lesser 2023-07-02 10:30:44

It used to be vpri.org, but this one is probably more helpful at this point: tinlizzie.org/IA/index.php/Papers_from_Viewpoints_Research_Institute

Jan Ruzicka 2023-07-02 10:31:13

Joe Nash Sorry, didn’t realize that. Updated the post with links.

Stefan Lesser 2023-07-02 10:39:46

I especially admire Ian Piumarta's work on taking late binding and dynamic object systems to the extreme with his combined lambda/object architecture, and Alessandro Warths' work on fast programming language iteration, which gave birth to OMeta and then Ohm.

But there is so much more. I dream of starting a STEPS reboot that changes the context for building a complete environment in 20k LOCs from multimedia PC to today's mobile computing platforms. What would this look like if we assumed touch interaction and all the sensory input our mobile phones support was the baseline?

Jan Ruzicka 2023-07-02 11:44:29

Stefan Lesser I agree... These two worked on what seems to be the key to VPRI's (partial) success.