After lots of reading (books and other people's code), watching video's, thinking, asking chatgpt for help (it needed help itself), experimenting and frustration, I finally managed to get a neural network working including the training using backward propagation, in my visual programming system!! You can see a small video where I demo it here: youtu.be/xOKzC77NgRU
The results are far from perfect and neither is the visual flow itself. Currently I only used a subset of mnist to train (9000 images) and on every training iteration the weights are updated per training image. Also the network can for sure be improvement by using a different layer setup and configuration parameters (different learning rate and activation / cost functions). From a visual programming perspective there are also lots of things to improve: currently a lot of the needed functionality is inside the nodes instead of being visible in the flow. So the neural and training nodes are black boxes as well as the node that loads the mnist dataset and handles some of the logic in the training proces. I want to change this in the near future though. You currently can't even see the resulting weights and biases.
Hopefully it is clear from the neural node types how the neural network is structured: it shows the number of nodes in the layer and an icon illustrating whether a node represents an input, hidden or output layer.
In the video I show a slow and fast run of the training proces: by putting the speed slider to the right you can run the flow without animations otherwise it takes too long.
There's also a custom node type that can be used to draw a digit manually and provide the digit which the drawing represents for purpose of calculating the error cost/loss.
Anyway, for now I am happy with the result. More to follow in the future :-)
At about 00:25 I see feedback and fan-out (to the gray boxes on the right plus to the feedback loop). Can you comment about that? For example, is there some sort of queuing and propagation delay going on?
There's no queuing but when a node triggers multiple nodes, these run in parallel independently of each other
How is "run in parallel" implemented? Do you spawn several threads (using a thread library, for example)? For fan-out, do you make copies of the data for each target node?
Yeah sorry, I should have said "parallel". Because of the animation system it only appears that the nodes runs in parallel. The nodes are normal functions that can either run in sync or async when it returns a promise to the flow-engine. When the animation system is disables (by putting the speed slider in the top left at max) , the flow runs normally and nodes have to wait on each other. I probably could use a pool of web workers to improve this
Input send to each node is currently not copied/cloned. The input data is in most cases just a simple string but can be an object or array. And depending on the exact implementation of a node, the input data is passed through or not. In the passed I've build a different system which cloned the input payload to a node and a node could add or change data of that payload and it was passed on. But it was quite heavy on memory. That's why I am trying a different approach in this implementation
over on the FBP slack (Flow-Based Programming) someone suggested using copy-on-write instead of copy ; this suggestion lingers in my mind but I haven't tried it yet ; aside: in general I've just stopped worrying about "efficiency", machines are so fast today that I don't need to care about implementation details, knowing that whatever I do can be production-engineered later to be a bit faster, "getting it right" is my foremost concern
I'm thinking on "... either run in sync or async when it returns a promise to the flow-engine. ...". From my perspective, the goal is to reach into a bowl full of Arduinos and put the code for each rectangle on the diagram onto its own Arduino. It's not clear (by this I mean: I don't know) whether a promise-based approach is realistic ... pondering ... I'd be interested in your thoughts ...
My first thought is that in that scenario you always want to return a promise because you're (probably) dealing with hardware communication and want to return from the node when the needed communication is done. In my scenario I used it when a node needs to do fetches for example (the node that initiates the training in the neural network example fetches the mnist dataset).
... how do you send a promise across a thin wire from one node to another? ...
I meant the internal code implementation of the node. The information that is send over the wire should be serializable to json (in my case at least)
Have you thought about what the information sent across a wire needs to be? (I don't mean to derail you, if you haven't thought about it, then there is no need to spend lots of time trying to answer)
Currently it varies per node/node port, and I can specify some constraints so that only node-ports that can receive that data can connect to each other. That implementation is still limited and I want to expand on it further in the future