Eli Mellen 2023-08-02 14:03:01 You know that thing when playing Pokemon games where after you throw a Pokeball you frantically tap the action button to ... do something? It is never made clear if this helps or even does anything at all.
What are other ghost-mechanics like this in games or software that you know of, where folks often take actions that aren’t clearly signposted to actually do anything?
The most immediate other example that comes to mind is swiping apps away in the iOS app switcher, but that isn’t exactly the same thing, since, even if it doesn’t free up memory, it is a direct interaction that alters state (removes options from the app switcher).
Jimmy Miller 2023-08-02 14:18:47 First thing that comes to mind is leaning in racing games. We also do it for bowling.
Jimmy Miller 2023-08-02 16:27:24 I will also say, if you watch any non-technical computer user, you will find this behavior. Things like copying files to make sure they are included in a powerpoint presentation. They will tell you X won't work unless I do abc.
Walker Griggs 2023-08-02 16:45:32 Saving multiple times, ritualistically, in hopes that it'll really be saved. Same goes for Ctrl-C and similar terminal signal handlers
Jimmy Miller 2023-08-02 16:46:37 If you watch chess players playing online they also often move the piece and click several times to make sure the piece is really there.
Walker Griggs 2023-08-02 16:50:31 The last few examples seem to highlight a distrust for systems that work their way into muscle memory. I've had a file not save so I lost my progress when my laptop battery died. I've mouse-slipped playing chess more time than I can count. As such, I spam the save button on repeat move actions out of distrust, but now they just feel like ritual
Denny Vrandečić 2023-08-02 16:54:32 pressing the close door button in elevators. I heard that in most cases they are entirely non-functional.
Jimmy Miller 2023-08-02 16:56:33 Many cross walk buttons are that way. But one in my home town was the opposite. The second you pressed it, no matter how long the light had been green, it would go to yellow. You could do it back to back. I thought it would be hilarious to setup a little microcontroller to press the button over and over again and just watch traffic be ruined. (Obviously I wouldn’t actually do this)
Ivan Reese 2023-08-02 18:15:09 When I first learned the Mac as a kid, I was taught that you ~had to~ do a copy before each paste. Even when cutting! So you'd do a cut, then copy, then paste.
Sort of like people who double-click links in web browsers: there's no negative consequence ~most of the time~ , so it's a hard behaviour to unlearn.
Kartik Agaram 2023-08-02 18:51:07 Clicking on the trackpad when a light touch will suffice. I think I got this habit after moving away from Mac. I imagine it happened because sometimes it isn't enough, but I'm not sure.
abeyer 2023-08-03 01:07:01 Imho a light touch is always too much or not enough but never what you actually meant... Disabling this and making sure scrolling goes the one true direction are the first two configs needed on every new trackpad 😛
abeyer 2023-08-03 01:09:25 As to the original question: I think the early Metroid games made me do all sorts of little rituals around the mostly undocumented special movements
Josh Cho 2023-08-03 13:25:48 Retrofitting of Medium
I tend to think every time a new medium is introduced, all the contents of the previous medium are ported over, without regard for fully understanding the new medium . For instance, films matured when it stopped porting over books, games matured when it stopped porting over movies, etc. An excellent example of this is Outer Wilds, a space-exploration time-traveling game which is simply impossible to recreate in any other medium. It fulfills the medium.
Given that, I think two new forms of medium are being introduced in the next 5 years: LLMs and VR. If we accept the supposition that there will be a lot of retrofitting of the medium, i.e. chat into LLMs, games into VR, what are the content that will truly fulfill the mediums of LLMs and VR?
Josh Cho 2023-08-03 13:30:31 I think this is a hard, open, and very exciting question. I will take a stab at it:
LLMs: Chat with PDFs is retrofitting. Perhaps a different kind of UX to deeply interact with difficult words on paper? LLMs can create a tree of concepts, so for difficult words (e.g. homotopy), selecting them would display a tree of related concepts (like homeomorphism), right within the PDFs. Other good ideas here and here.
VR: Ideas as physical objects? Maybe a 3d notetaking app, but instead of notes being rectangles, they can be any shapes.
Eli Mellen 2023-08-03 13:32:49 I think there is an interesting distinction to be teased out in your question between a few things
- a new UX
- what that new UX is used to do
With your example of LLMs and PDFs — is there something distinctly LLM-ish about that UX solution? Could that same UX be achieved without an LLM?
Eli Mellen 2023-08-03 13:33:42 (also, while not exactly directly related to your ask, I’d push back on that Nielsen piece about this being the first new UI paradigm in 60 years…and then push back further to question their use of the term “paradigm” at all)
Josh Cho 2023-08-03 13:35:26 I think one can potentially do the tree of concepts idea with an extensive knowledge graph. But at least feasibility-wise/commercially it's difficult. You need a flexible KG usually, especially if you want to do some link prediction based on the PDF at hand.
Re: Nielsen piece, fully agreed. I think it's an interesting new medium.
Eli Mellen 2023-08-03 13:41:14 I think another way to approach your question is to explore something like vocabulary or that moment when a new medium establishes its own norms for communicating through itself.
So, for film, that would be building up a toolkit of specifically filmic story telling techniques that are born from film, rather than another medium (but may then find their way into other mediums)
With video games, these techniques are often game mechanics — but also can overlap with animation, etc.
By attributing “maturity” to this sort of technique, I think you can start to decouple the medium and the message a little.
Josh Cho 2023-08-03 13:51:57 That rings true. Words have power, a la call a demon's name to dispel it, express the psychological problem to solve it, i.e. Focusing. (Perhaps vocabulary doesn't have to be linguistic, but linguistic vocabulary has the highest virality)
A tangent, but then what's the best medium for crafting a vocabulary around a new medium? Like could we build a meta-tool to help people understand the latent structure of another tool?
Eli Mellen 2023-08-03 13:55:52 oh! I dig that question so hard!
well, my academic background is in theory and art history — there, new vocabulary comes in through a few different processes, but, the easiest to trace is often borrowing , e.g. there exists a body of work that talks about painting, BOOM! Photography explodes into the art scene, so, at first, folks write about photography by borrowing terms from painting, often times focusing on the differences and similarities between the 2 mediums…over time, those terms transform, new terms/vocabulary are invented to begin to wrestle with that new stuff photography is bringing to the plate
Josh Cho 2023-08-03 14:12:10 That inspires a lot of thoughts. It reads to me like there is an isomorphism with scientists trying to explain a new phenomenon with existing theories, e.g. Newtonian mechanics not being enough to explain phenomena near light speed. I understand the "new stuff" you mention as different affordances/spaces/possibliites that must be wrestled with.
I guess new vocabulary is created in the chemical reaction/wrestling/resistance between the borrowed terms and the new phenomena. So perhaps operationally,
- Don't settle for the borrowed terms. Keep looking for new phenomena and "react" with borrowed terms.
- Borrow not from the direct parent of the new medium, but somewhere else. Maybe vocabulary in games (e.g. juice) is useful for understanding LLMs. Determination of which arbitrary medium to borrow from would also be interesting.
I am also thinking, which field has the most experience dealing with this phenomenon of creating new vocabulary? They would have the most meta-models regarding new vocabulary. The top candidate seems to be art, right? The ultimate medium of mediums. Tools for thought can also be rendered as tools for expression.
Josh Cho 2023-08-03 14:15:58 But also, there must be something different when applying Kuhnian paradigm shift to the shift of mediums, right? (in a truly meta sense)
Josh Cho 2023-08-03 14:17:01 I can't think of any on top of my head, but also I am not too familiar with Kuhn
Eli Mellen 2023-08-03 14:17:31 the book is interesting, but I think the big points to take away can be pretty quickly surmised (I think Jimmy Miller did it on a recent episode) as something like:
- there is an existing understanding of a thing, with an existing model
- someone pokes a hole in that understanding that throws it into question (triggering a crisis)
- to deal with the crisis a new model is brought into being (paradigm shift)
- that new thing becomes the next model until another crisis comes up
Josh Cho 2023-08-03 14:18:11 I feel like this is an on-paper thing that is really different when it's real and in your face.
Eli Mellen 2023-08-03 14:18:12 arguably, Isaac Asimov’s Foundation series is about this, too, but a lot longer to read than Kuhn’s book 😆
Josh Cho 2023-08-03 14:18:58 There is something truly psychologically difficult about people with different sights
Eli Mellen 2023-08-03 14:19:31
But also, there must be something different when applying Kuhnian paradigm shift to the shift of mediums, right? (in a truly meta sense)
I am not sure!
Josh Cho 2023-08-03 14:20:07 I am wondering, in a similar history of science way, whether there is a way to incorporate Feyerabend into this discussion?
Josh Cho 2023-08-03 14:20:31 But perhaps study of mediums is not methodological enough for there to be an "Against Method"...
Josh Cho 2023-08-03 14:22:50 maybe doing more art will help one naturally understand this. math made more attuned to epistemic mediums (e.g. some subfields of math are more "precise" than others), but maybe exploration of medium shifts through art can be fruitful.
greg kavanagh 2023-08-03 16:57:38 The big change is that computers can be communicated with in a similar fashion to the way we communicate with each other so words contain more instructional information for a computer than before. So the UI is text right? Or even with current speech recognition the obvious UI is answer and response. That’s the most efficient way we communicate, but what we say rarely is what’s received but our internal error checking irons out most misunderstandings quickly. Also if the person you’re speaking to gives a puzzled expression that’s a huge amount of information for you and you’ll reformat your speech. LLMs have none of those abilities which makes them pretty inefficient when they misunderstand. LLMs are a flawed model of human communication. I hope we don’t bend ourselves out of shape to compensate. But the medium is not new IMHO.
Eli Mellen 2023-08-03 21:21:24 both natural language and VR are interesting because they’re not new — I think they’re trying to converge on meat-space reality
Josh Cho 2023-08-04 02:04:02 I have to inquire about your definition of medium. In addition, I am curious about the effects of the definition, as in the utility of it. I am getting the sense that LLMs and VR are viewed as flawed versions of mediums that already exist in reality, thus not new medium. That feels like retrofitting LLM and VR to reality. Surely they look similar to reality rn (or try to), but they will mature.
A weak example is electronic music, which developed into a field of its own. Or texting, which is different from phone calls, which is different from physical comversation. Messaging apps nowadays have reactions, which allow you to juggle talking about multiple things at the same time.
All of this discussion (personally) is for developing interesting pieces of technology within VR, or through LLMs. I would imagine thinking of them as flawed rendition of reality would not help in furthering what can be done. I think we can go further than imitating human communication (which already is cool).
greg kavanagh 2023-08-04 05:52:16 If I was to be rigorous about investigating it I’d make clear boundaries between the examples. Instructing a computer is tool using whereas and music (electronic or otherwise) is art. Computers are tools. Using that tool to make music does not change the nature of music. I don’t think theories of art can illuminate the effect of a new tool.
Denny Vrandečić 2023-08-04 03:13:22 Is there a reasonably decent JavaScript library that allows me to have a reasonably terminal-y experience in the browser, so I can implement a shell? Ideally with history, autocomplete, colors, etc. - and then with the ability to add DOM nodes too, so I can click, show images, etc. Preferably no backend necessary.
Denny Vrandečić 2023-08-04 03:36:48 Interesting. It seems to be really close to, well XTerm, i.e. it also uses \033[1 instead of just <b>. I wonder if there is something slightly more webb-y in addition to being a shell
abeyer 2023-08-04 05:13:45
I wonder if there is something slightly more webb-y
Sounds like a bug 😝
Denny Vrandečić 2023-08-04 06:10:11 It seems like it might be easier to implement it from scratch than to fight with all the xterm / ansi compatibility
Kartik Agaram 2023-08-04 18:40:43 Personally I try to avoid terminals these days after investing 20 years in them.
On the other hand I also try to avoid the web 😅
Denny Vrandečić 2023-08-04 21:39:54 how else would you do a CLI? (Lemme guess, the answer is you'd avoid CLIs)
Kartik Agaram 2023-08-04 22:00:25 Depends on how you define CLI. We tend to think of CLIs (and even TUIs) as requiring a terminal. But if we notice that modern terminals are all implemented atop a canvas of pixels, then arguably everything I build is a CLI. Some examples:
It seems like you can get all the behaviors we tend to look for in CLIs with very little code if we give up various intermediate levels of abstraction and just go back to drawing text on a canvas of pixels.