You are viewing archived messages.
Go here to search the history.

Dennis Hansen 2024-05-22 21:33:18

Hello! So after much obsession with Propagator Networks as discussed on this slack šŸ’¬ #thinking-together@2024-03-21T19:26:33.915Z, i decided to make a little propagator network simulator on top of tldraw.

In short: Propagator networks enabling bi-directional computation via independently operating nodes- in the gif you can see one for converting temperatures. Propagators (squares) listen to changing inputs, run code, and update connected outputs.

You can make your own here.

  • Put your variables in circles

  • Put your JS in squares (you can write a return or not)

  • Draw arrows from circles to squares with text that matches the variables

  • Draw arrows from squares to the circles to be updated.

There's lot of awesome stuff than can be made with these (maybe everything?). If you want to have fun with it, try to make a conditional or a switch. Its fun šŸ™‚

Cheers!

ezgif-2-f1371da04b.gif

Elliot 2024-05-22 22:06:58

Awesome! I tried doing some array stuff! I wasn't able to use anything but the first entry of the array in a JS square. I tried a few different methods including wrapping the array in another array. Is there a way around this? I also tried wrapping the entries in an object instead.

image.png

Dennis Hansen 2024-05-22 22:14:35

This thing is so raw, haha let me get in there and troubleshoot this.

Dennis Hansen 2024-05-22 22:18:23

Ok, im stringifying cell output, should work properly now after a refresh

Screenshot 2024-05-22 at 6.16.54ā€ÆPM.png

Dennis Hansen 2024-05-22 22:19:00

Thanks for the feedback!

Elliot 2024-05-22 22:20:24

Sweet!

image.png

Elliot 2024-05-22 22:21:32

I'm excited about propagator networks and I've been hacking together my own extension to them; but I've been doing a lot of working things out on paper. Its super cool to have this interactive playground!!

Dennis Hansen 2024-05-22 22:25:07

Hell yes! So glad i can help. Im using it for experiments too, so if you have more trouble with this let me know. Planning on getting code error messages in there somehow

Elliot 2024-05-22 22:29:13

Here's two arrays that are synchronized but have their elements swapped!

image.png

Elliot 2024-05-22 22:29:20

So cool!

Dennis Hansen 2024-05-22 22:30:15

Yoooo thats so sick!

Dennis Hansen 2024-05-22 22:30:52

There's something magical about these things. It feels like we discovered some law of nature haha

Elliot 2024-05-22 22:47:22

Does it propagate based upon if changes are detected with string comparison?

Dennis Hansen 2024-05-23 00:24:26

Yep. After stringifying and removing any quotes it does a strict inequality via !== operator in JS.

Screenshot 2024-05-22 at 8.22.17ā€ÆPM.png

Elliot 2024-05-23 00:47:24

Big TY!

Elliot 2024-05-23 00:49:35

I've been trying to make a little network that keeps track of a cell along with its previous value, but I don't know if its possible.

The attempt in the attached screenshot doesn't work because the value instantly loops around and always results in an array with the same two entries equal to the cell on the left.

image.png

Dennis Hansen 2024-05-23 00:59:25

Ah, like a memory cell. That seems.. important... im going to play around with this.

Elliot 2024-05-23 01:15:00

ah! I got it

image.png

Dennis Hansen 2024-05-23 01:35:04

Brilliant!

Dennis Hansen 2024-05-23 01:36:49

I figured it needed some conditional in there haha

Dennis Hansen 2024-05-23 01:41:48

This is wonderful. Sets the groundwork for storing variable history

Screenshot 2024-05-22 at 9.41.32ā€ÆPM.png

Elliot 2024-05-23 03:19:01

Here's one so that you can selectively record changes to a when the c flag is truthy.

image.png

Jack Rusher 2024-05-23 08:22:54

This is good stuff! I recommend adding an interval node of some kind so you can feed the network over time. There are some examples of this kind of thing in this Maria document:

maria.cloud/cells

Jack Rusher 2024-05-23 08:23:37

(Also some examples of how useful a good sequence API is in this context.)

avon 2024-05-23 15:14:42

This is amazing! Thanks so much for sharing. I wonder if it would be possible to create ā€œsub-canvasesā€ which would be ā€œnormalā€ TLDraw canvases that expose their geometric properties and mouse/keyboard events to the propagator simulator. This could give us a a sort of Fabrik-like environment where itā€™s possible to create bidirectional relationships between computer graphics and propagators.

Dennis Hansen 2024-05-23 19:01:03

Thank y'all so much!

Jack Rusher Really interesting, and very cool examples. Cells in that context are almost like the merging of a cell and propagator in propagator model context, but rather updating 'upstream' variables via reference rather than a reversal function (if i surmise correctly). In any case, intervals would be epic, and are necessary- basically need a way for propagators to wake themself up in the future ha. Gotta stew on that one.

@avon I think this is an excellent idea and i think your seeing a lot of the vision that i am. Essentially propagators can be unidirectional or composed into bidirectional connections that can connect together geometric objects on a canvas. This can be used to create constraint systems and logic. I think the only key difference with my current thinking is... how do i get it all the logic to live right on the canvas rather then having things plug into a canvas like a data source. Effectively merging the inner 'code' world with the visual world (hiding 'implementation' details away when they aren't helpful, of course). Here's a graphic i made that has given me a hint of how propagator networks map to a geometric system like this (and an algebraic one too).

shapes at 24-05-15 15.54.59.png

Lu Wilson 2024-05-23 19:25:53

omg nice will share this with the rest of the tldraw team

Dennis Hansen 2024-05-23 19:28:39

Lu Wilson Ayy, heck yeah! Code is here šŸ™‚ github.com/dennishansen/propagator-draw

avon 2024-05-23 21:09:07

Effectively merging the inner 'code' world with the visual world (hiding 'implementation' details away when they aren't helpful, of course).

Really cool! This is the kinda stuff thatā€™s been bouncing around my head for the past few months. I donā€™t have any great approaches yet, but there are some attempts at this in the Fabrik paper I linked earlier, which you might find useful.

Pure Data, also has some interesting attempts at this too, both in its ā€œcanvasesā€ (pd-tutorial.com/english/ch05s02.html) and also with its ā€œdata structuresā€. For a cool look at PDā€™s data structures at work check out this clip where geometric shapes are passing messages to a DSP graph thatā€™s running behind the scene: youtu.be/ZOf4v-0U1I4

Computer music is where I spend a bunch of my time so I figure itā€™s also worth mentioning PDā€™s ~_*delay*_~ object which takes an arbitrary message and schedules it some time in the future (this is how intervals are set up in PD from what I understand) thereā€™s also similar constructs in the ChucK programming language where you can ā€œchuckā€ your program forward in time, but I donā€™t have a ton of ChucK experience.

for kicks hereā€™s my attempt at a ring buffer memory propagator:

single-value-history-propagator

Dennis Hansen 2024-05-23 21:28:40

@avon Wow ring buffer is freakin sick. Secondly, im excited to read the rest of this Fabrik paper as well as the PD stuff. In the meantime to address your delay/ChucK ideas- as well as Jack Rusherā€™s interval idea..

If propagators are implemented like they are intended: as independent, continuously running machines, i suppose waking up and doing something is straightforward. Less so in the simulator since functions are run reactively. With that being said...

I did find a workaround. Creating a loop where a propagator outputs time and hooking that up to itself allows us to at least get a time variable into the system, which enables making.. well.. āœØ interactivity āœØ . Here's a counter. Notice i reset it by switching the 0 to a 1.

0a7664b21a99bd9cf361e11113d187a4.gif

Lu Wilson 2024-05-23 23:19:51

cc Orion Reed get in here

avon 2024-05-24 22:35:25

Been having some fun experimenting with propagators reacting to time! Hereā€™s my attempt at a network that continuously calculates the rolling average of a cell's contents over time:

IMG_2150

rolling-avg-propagator.svg

avon 2024-05-24 22:36:36

(I shared the svg which might allow others to just open it in their browser and extend my implementation? Not sure)

Dennis Hansen 2024-05-24 23:05:37

Listen my mind can only be so blown. Unreal. haha. The SVG us just an image but if you export as JSON i believe that should do it! I can import that

avon 2024-05-25 00:21:04

ah my bad let's see if this works:

Jacob Zimmerman 2024-05-25 20:51:27

This is awesome! Iā€™ve been hacking on some propagator stuff in C too. How are you handling the scheduling?

Kartik Agaram 2024-05-23 16:45:02

šŸ§µ for Cole Lawrence's demo today.

Kartik Agaram 2024-05-23 16:45:23

Q: At one point you typed in "what is webp?" into the plugin. Is that question also just scanning one's own browser history? So it would need one to have previously asked that question or have some history with the term "webp"?

Cole Lawrence 2024-05-23 16:58:04

Here is the screenshot of that interface.

The knowledge of WebP is informed by the ā€œContextā€ seen above the userā€™s message.

That context is a series of visited webpages.

When we visit a webpage, we store a scrape of that webpage in our database, which is being used to answer the question.

2024-05-23 Local 5377 at 01.56PM@2x.png

Kartik Agaram 2024-05-23 16:59:51

I see, so you're basically building "what I would know if I had perfect comprehension and memory of everything I ever read" šŸ˜‚ ā¤

Cole Lawrence 2024-05-23 17:02:02

Yeah, that, and I like to call the long-term goal ā€œthe cure for context switching,ā€ which includes a few more proactive features.

Ivan Reese 2024-05-23 19:04:58

Yeah, that "cure for context switching" framing is especially interesting as a point of comparison to Microsoft's Recall, Rewind.ai, and the old concept of LifeStreams, which seem more about aiding memory. Would love to hear more about what you have in mind here.

Cole Lawrence 2024-05-23 19:19:28

Yeah, I havenā€™t found memory to be nearly as painful as my experience of ā€œcontext switching.ā€ For example, communicating event announcements or invoicing clients (each of which happens every few weeks) is incredibly painful for me because of the activation energy required to start the task and resume the context that I was in several weeks ago.

So, I want to basically ask ā€œinvoiceā€, get a suggestion for ā€œprepare a new invoiceā€ and get a list of bookmarks and summaries of the last invoice activity I had.

Separately, Iā€™d like to say ā€œmeetā€ (or to simply visit calendar.google.com) and have it see an upcoming meeting and suggest ā€œprepare for meetingā€, and so on.

Cole Lawrence 2024-05-23 19:23:28

Similar energy as goblin.tools - but with your personal context and temporary interactions.

Mariano Guerra 2024-05-23 17:16:41

šŸ§µ for Maikel van de Lisdonk's demo today.

Mariano Guerra 2024-05-23 17:17:04

thinks out loud

Mariano Guerra 2024-05-23 17:17:06

Is nodes and wires a "death by thousand cuts" problem?

We all agree on the main idea, concepts, how it looks and in general how it works, but to make it nice/easy/fast to use and make it scale to real world problem it requires a huge number of small features.

Mariano Guerra 2024-05-23 17:44:29

Maikel van de Lisdonk feel free to share your nodes and wires hot takes here šŸ˜„

Maikel van de Lisdonk 2024-05-23 17:44:57

Yes, I will do later today/evening šŸ˜Š

Kartik Agaram 2024-05-23 17:51:03

My bias: text is unbeatable in the small. But there's tons of room for visual approaches to take over the in-the-large macro view of a codebase.

A crucial question I've been thinking about in this vein is how a visual representation might be grounded in text when you zoom in enough.

Arvind Thyagarajan 2024-05-23 18:07:31

it requires a huge number of small features

agree! and my hot take here is that it's worth it šŸ™‚

Ivan Reese 2024-05-23 19:10:00

My bias: text is unbeatable in the small

I used to agree with this position, but I've moved slightly.

I think text, blocks, nodes, formulas, spreadsheets, etcā€¦ they're all good for representing and manipulating things that fit a certain structure or pattern. Sometimes small things fit that pattern. Sometimes large things do.

For instance, nodes are great in the small for graphs . Describing graphs in text, even for tiny graphs, kinda sucks.

Math notation is another one. I'd love a rich GUI for building math equations, that leveraged the keyboard but not merely for ASCII input.

Maikel van de Lisdonk 2024-05-23 19:19:06

I think that visual programming has a lot of potential , especially if its interactive and it can be used to "play with the code" in combination with visualizations and timelines which can be scrubbed to see either the moment in time what the state of a program was and/or the impact of changes. It gives a different perspective and can help to visually see what a program does or behaves. However... the tooling needs to really help and not work against you (which my own tool still does). A lot of small UX and UI features are needed to make it easy to work with nodes and wires .. some things are obvious and explored a lot already: tldraw does a great job on the white boarding infinite canvas space.. but thats not enough I think.. also in the structural programming side : quickly insert nodes into a connection or replace a node with another , reuse flows in various ways (I am exploring these things) are some things.

Maikel van de Lisdonk 2024-05-23 19:20:35

Not needing a mouse to build a visual graph can also help

Maikel van de Lisdonk 2024-05-23 19:29:15

I also have some use cases for which I could use it: for building complex conditional multi step form applications (thats something I've done that a lot in the past using code and json structures sometimes with the help of a cms) including the api pipeline and interaction with that. Visually seeing what you do would help there. My current tool can't do this yet but my previous attempt to building a vpl some years ago could.

Another use-case in which I want to build a specific vpl for on top of my system is to be able to visually play with geometric calculations for something that can be used in the vpl tooling itself (specifically to visual program how I can create perfect arrows instead of via textual code)

Maikel van de Lisdonk 2024-05-23 19:42:41

codeflowcanvas.io is the link to my project

Paul Tarvydas 2024-05-23 20:01:10

What is the first thing that a non-programmer (aka "end-user") draws on a whiteboard?

Maikel van de Lisdonk 2024-05-23 20:06:16

I hate to say it, but "text"? Although I would love to say "boxes and arrows"

Arvind Thyagarajan 2024-05-23 20:08:30

I was actually going to say a box. Then they label the box. Then another box. Then a line between them. Then they explain what that line means. A nicely self justifying story I'm going to stick to!

J. Ryan Stinnett 2024-05-24 12:22:04

Maikel van de Lisdonk Curious about the "Fibonacci" example in your demo... It seemed like there was an expression node ( list[-1] + list[-2] ) reading the list variable by name without reading the value via wires, am I understanding that correctly?

Maikel van de Lisdonk 2024-05-24 12:30:44

yes, thats correct, the flow-engine has the concept of variables (an array node has a variableName property, although I just noticed a bug that I can currently not edit that. There are also variable node-types which can have a data-type and which can be observed) and these variables are registered in the engine and can be referenced by other nodes like the expression-node. Variables can be setup as global or scoped. The scoped variables are used in the quicksort example, because that is implemented using recursion (flow's can also have functions which can be called recursively which get their own scope-context which is used for storing the variables that are scoped).

J. Ryan Stinnett 2024-05-24 12:34:55

Interesting, I don't know if I've seen this kind of "hybrid" of wires as well as referencing by text before. It's maybe a nice affordance, allowing some "implicit" wires via referencing variables that don't need to be literally displayed on screen.

Maikel van de Lisdonk 2024-05-24 12:42:06

These references can be shown by pressing the arrow button in the right side menu-bar (I havent found a better icon yet). The dark gray dotted line is the reference ("list" is read by the expression node)

image.png

J. Ryan Stinnett 2024-05-24 12:53:42

Ah okay, that's helpful! šŸ™‚ I couldn't work out what any of the right side buttons did, perhaps consider adding simple tooltips to explain on hover via title attributes or something.

Maikel van de Lisdonk 2024-05-24 12:55:04

yes, for sure that's needed šŸ™‚ . The UI/UX needs still needs a lot of workšŸ˜Š

J. Ryan Stinnett 2024-05-24 12:58:45

Totally understandable and expected, it's at an early stage. šŸ™‚

Kartik Agaram 2024-05-24 13:05:41

Ivan Reese

For instance, nodes are great in the small for graphs . Describing graphs in text, even for tiny graphs, kinda sucks.

Math notation is another one. I'd love a rich GUI for building math equations, that leveraged the keyboard but not merely for ASCII input.

This is fair, but strikes me as orthogonal. Like, you could decompose "future of software" into two pieces:

  • Lots of little improvements to the UX, each doable in isolation in any context.
  • A radical piece that you'll never get to by business as usual in the present of software.

When I normally think of visual programming I tend to associate it with the latter.

A wysiwyg editor for math equations or graphs in the small belongs in the first category IMO. Easy to bolt on to any IDE today, just a "small matter of programming." I'm not convinced it helps with the foundations. What is it about large codebases that causes lots of little improvements to gradually get more and more expensive until they're no longer economic?

I suppose this is just a vanilla disagreement. We have a bunch of pain points today, and a blind men and elephants situation when it comes to prioritizing pain points.

Mariano Guerra 2024-05-23 18:16:22

forgot to share this in the demo

gloodata-inception.gif

Ivan Reese 2024-05-23 19:12:04

haaaaaaaaaaa

Konrad Hinsen 2024-05-24 17:34:33

Unrelated to this but related to Mariano Guerraā€™s demo yesterday: I saw someone mention Tako today and at first sight it looks quite similar.

šŸ“ Tako

Democratizing Data Access