You are viewing archived messages.
Go here to search the history.

Kartik Agaram 2024-10-08 00:33:39

A text editor that supports doodling.

Some previous ways to have text coexist with a canvas:

Here I'm instead attaching doodles to a pivot character that they move with in the presence of edits. When you start drawing you're presented with a set of pivot options, one grey square per character on screen. After you pick one, it turns pink and the other options disappear. Now you can start drawing.

Displacements to the pivot are preserved in font-derived units, so it looks "reasonable" as you resize the font.

Deleting a character deletes all drawings pivoted on it. (But there's undo.)

This took 200 lines, so not too much though it was more than I'd initially expected.

Inspired by this thread.

Joshua Horowitz 2024-10-08 03:47:41

Neat. Less radically, have you considered sketches in the margins, linked to characters? Or sketches that appear in boxes, with text wrapping around them? I ask cuz “sketch directly on top of text” doesn’t seem as useful as those. Aside from, say, editing marks. Maybe increasing line spacing would make sketches literally anchored to characters more useful?

Joshua Horowitz 2024-10-08 03:48:15

(Or if you do think sketches directly on top of text could be useful or fun, I’d love to see some examples like that.)

Kartik Agaram 2024-10-08 04:21:48

It's a good question. This particular configuration came up in conversation, I think, as something that looks and feels like paper. I'm not sure either that it's useful as is.

Konrad Hinsen 2024-10-08 05:21:54

As a heavy user of pencil-on-tablet annotations of scientific papers, my vote is for annotations attached to a place in the text but shown/drawn on the margins. Plus manual text markup: highlight/underline, strike through.

Kartik Agaram 2024-10-10 16:02:05

(Where's everyone else? I'm feeling lonely in this channel.)

I hadn't tried this until today, but it turns out to work: I can create equations forwards and back, and trigger either selectively based on what blanks I fill in.

The fine print: to switch directions I have to fill in the right blank, clear the old blank, and then type in something outside the old blank (to indicate I'm not going to type further into the old blank).

Works better if I clear the old query first, but who can remember that?

Perhaps I should just recompute on every mouse click. Then the third step could be clicking outside any blanks. More wasteful, because most of the time a mouse click won't require recomputation. It's Tadi, but is it permacomputing? 🦋

Scott 2024-10-10 16:26:37

Hm was the "indicate I'm not going to type further into the old blank" interaction for switching direction intentional? It's kind of surprising that going the first direction the blank updates for every keystroke but it would require additional interactions to get it to recompute the other way..

Kartik Agaram 2024-10-10 17:52:54

There's basically 2 scenarios here:

  • I'm clearing a field before typing something else in.
  • I'm clearing a field so the computer can fill it in.

If I optimized for 2, then I end up with a situation where a blank has say 10, I click into it to change it to 20, but when I backspace to nothing it immediately goes back to 10.

So currently the update as you type only triggers for fields that are blank when cursor is outside them.

It does seem sub-optimal, so suggestions appreciated.

Scott 2024-10-10 18:01:22

Ah I think I see what you're saying - they're kind of "linked" to each other and always want to stay in linked even during edit, which gets you that first interaction where celsius updates instantly, but you needed to add that additional interaction to go back the other way...

Scott 2024-10-10 18:02:53

It may not be what you're going for, but in my mind I'd kind of expect more of a leader/follower model for these links - if I start editing field A, anything linked to it should be updated until I unfocus that field

Scott 2024-10-10 18:03:27

but nothing should really trigger an update on A until I unfocus that field

Kartik Agaram 2024-10-10 18:08:12

That's certainly an approach. The drawback there is that you might change a complex sheet in unanticipated ways because you forgot about the presence of some formulas. Things change outside your field of view. Right now that only happens for fields you've not typed anything into. So no data loss. I tend to prefer UIs with a sense of stability that aren't constantly moving what I put into them..

Scott 2024-10-10 18:28:33

Ahh right...yeah wasn't thinking about the more complex sheets with things out of view

Scott 2024-10-10 18:33:58

I imagine I'm probably going to run into a similar problem pretty soon with these event-based agents haha

Scott 2024-10-10 16:38:04

(I'll join in on the fun!)

I've been experimenting with using generative AI inside Github Actions lately and have been really intrigued by this async/event driven interaction model with code generation.

In this repo we have a Github Action that runs every morning and has Gemini, Claude, and GPT-4 generate a PR adding additional functionality. Then on PR merge a release message is sent to our discord with example use cases and each of them riff on what was merged and try to come up with additional related functionality.

I've already merged a few PRs and some of the ideas they've come up with are pretty interesting, especially the riffs - I merged a PR about querying a notion database, and Gemini came up with taking a row in a Notion database and creating an Asana task.

Some of the things they've come up with so far:

image.png

Kartik Agaram 2024-10-10 17:56:25

Do the prompts aim in the direction of some rough goal or domain, or are they fairly untargeted, getting the AIs to add anything (that compiles?) to the codebase?

Scott 2024-10-10 18:05:43

For the daily run its pretty untargeted...just "here's what Sublayer Actions are used for, here's a repository of ones we've found useful in the past, come up with some new ideas for things we might find useful and generate them"

Scott 2024-10-10 18:07:26

but once something is merged it becomes a bit more targeted: "we just added an action to do X on this service, what other things might people want to do on that service?"

Scott 2024-10-10 18:08:15

for this example its kind of anticipatory code gen

Kartik Agaram 2024-10-10 18:11:53

I see. So the act of humans merging something is guiding the computer to push more in that direction?

This is cool. It reminds me of Doug Lenat's "Automated Mathematician" which would go off and search for interesting relationships between numbers and try to generalize from them. Once it defined some predicate or function over numbers, a human could go in and rename it (say if it was a well-known concept, like 'odd'). One of the heuristics it used was to pay more attention to definitions if a human went in and renamed them.

Scott 2024-10-10 18:22:03

Yeah exactly! The main thing keeping it from growing more is that since these are external services I need to do a lot of work to verify them by hand, test API keys and stuff....working to come up with something with a little less manual review...

But that Automated Mathematician looks really interesting, I hadn't heard of that before! It seems like I've been pulling on similar threads to that for a while now though with programming concepts instead of mathematical ones...

Scott 2024-10-11 20:27:11

Merged a couple more exploring this async AI pattern in some new directions and starting to feel a bit like the sorcerer's apprentice with every PR I merge resulting in 3 new PRs 😆

image.png