You are viewing archived messages.
Go here to search the history.

Walker Griggs 2023-07-10 19:23:24

I've been thinking a lot about apprenticeships over the last year and specifically the lack of apprenticeships in tech.

My mental model of apprenticeships is primary formed around how they apply to the arts -- people learning a craft from a single practitioner who has worked their life to develop a personal style or system. But apprenticeships also exist in trade, maybe even more so. Fundamentally, apprenticeships facilitate knowledge transfer -- all industries can benefit from that, no? So, I wonder why apprenticeships are far less common in software?

  • Is it because so many industry vets were themselves self taught and so that mindset has lingered?
  • Is it that so much of our industry is focused on access, democratization, and open sharing of information?
  • Could it be that we need to train more software engineers than we have hours in the day, and that need for more qualified minds has pushed us beyond the 1-on-1 learning paradigm and towards bootcamps etc?
  • Maybe so much of software and the knowledge around it is already accessible, so apprenticeships are just overkill? That said, there are still pockets of software where subject matter expertise and knowledge silos outweigh what is either 1) commonly taught in schools or 2) generally available.
  • Software doesn't have the same requirements around professional certifications -- could that be at play?

Generally, do you all think there's value in apprenticeships or should we (continue to) move in the other direction entirely -- teach the masses? In that sense, I could see the argument that apprenticeships border on nepotism and are actually a form of gatekeeping. Have any of you been part of a formal apprenticeship (on either end)? If so, what conditions made that possible?

Eli Mellen 2023-07-10 19:44:33

Generally, do you all think there’s value in apprenticeships or should we (continue to) move in the other direction entirely -- teach the masses?

Why not both?

I think they meet different needs and communities. They’re not 2 ways of approaching the same thing.

Eli Mellen 2023-07-10 19:44:57

there are also historical examples of one phasing into the other

Lu Wilson 2023-07-10 19:49:20

We have a person starting an apprenticeship with us soon! It's through a local link here in London. Seems like a great scheme. Previous years have been very successful, but unfortunately - this year, we're one of the only companies taking on board an apprentice from them. I think it's one of the first things to drop with budget cuts. It's a shame! A big part of it is introducing tech to a wider range of demographics. By appealing to the masses we run the risk of attracting more of the same. I hope the scheme can find some more industry links for the next cohort...

Gregg Irwin 2023-07-10 22:35:24

I think there's enormous value in mentoring, formal or not. I've had many mentors, in different areas, and value them highly. I try to give back as well, mentoring as best I can.

Why don't we have it in software? My personal opinion, 100% biased and anecdotal, having done this professionally for a bit over 30 years now is... devs don't want to be mentored. Some young people, fresh to learning the craft are open to it, but not all. And while skepticism is important, ego is the main barrier.

As an industry, we also don't ask about it when grooming new recruits. We don't chat socially and say "Who is your favorite coder or designer from the 60s, 70s,..." etc. "What UI from the past do you love and why?" "What are the most important, timeless values in software, to you?" We don't learn from other great artists, stand on the shoulders of giants, and carry things forward to the extent artists do. It's one of our major failings.

Greg Bylenok 2023-07-11 03:08:46

Dave Hoover and Corey Haines have a lot to say on this subject. Their two organizations (Obtiva and 8th Light) experimented with apprenticeships quite heavily around 2010, and Dave literally wrote the book on software apprenticeships ("Apprenticeship Patterns"). Happy to put you in touch if interested.

Jack Rusher 2023-07-11 07:08:19

💯 Mentorship/apprenticeship is very powerful and under-utilized in tech culture.

Irvin Hwang 2023-07-11 12:28:37

I think it's an inertia thing where it's easier to do things the way most people have been doing things. For a new company that is probably focused on just surviving, trying a "new" model of training like apprenticeships might seem unnecessarily risky. For a mature company the existing way of doing things could be too ingrained and the incentives for any individual to change things might not be there. Also in tech there's a perception (and reality?) people change jobs more frequently so putting a lot of investment into an individual's initial education would probably not be worth it unless that person had to stay at the company for a certain period of time.

Marcelle Rusu (they/them) 2023-07-11 15:46:38

I really do think so, especially with people starting their first programming job remotely its tough.

There's another aspect I encountered is I don't know how to mentor people properly & I realized I don't think many people do. There's no gold standard I can point to & say, we should do it this way.

More so, when I've gone and done this myself, I essentially got told to stop from my managers, but never got provided tools/ideas to do it better.

Especially important for those in under represented groups (even more with low confidence). My biggest concern is capable people getting completely turned off tech due to cultural problems. I've experienced it, and I know many who have.

Walker Griggs 2023-07-11 16:42:54

Lots to respond to

there are also historical examples of one phasing into the other

@Eli Mellen I'd love more context here if you have some.

We have a person starting an apprenticeship with us soon!

Lu Wilson in your opinion, where's the line between internship and apprenticeship? I ask only because you used the first person plural here, so I'd imagine your apprentice will learn from a few folks and receive instruction from a team.

devs don't want to be mentored. Some young people, fresh to learning the craft are open to it, but not all. And while skepticism is important, ego is the main barrier.

This is an interesting callout @Gregg Irwin. I hadn't considered the ego part. I try to keep a beginners mindset at all times but probably fail at it more often than not. I'll have to think more on this -- I think you're digging at a cultural issue that I've noticed but never formally identified

Dave Hoover and Corey Haines have a lot to say on this subject.... Happy to put you in touch if interested

I haven't read this but certainly will put it on the reading list. Might even reach out for that conversation if I have follow-ups -- thanks!

Walker Griggs 2023-07-11 16:47:24

Also in tech there's a perception (and reality?) people change jobs more frequently so putting a lot of investment into an individual's initial education

I recently moved into a fairly specialized space (video encoders and infra) and found that more folks have either worked with each other in past or at least know about each others work.

From the few folks I've talked to about knowledge transfer, there seems to be a pretty real fear that 'young people' aren't investing the time necessary to learn the industry.

In that sense, people are willing to teach you (even if you do leave) because it benefits the industry as whole -- plus you might boomarang back at some point.

In a way I feel this connects to the earlier point about ego. The "I don't need to be an apprentice when I can bounce between companies every 2.5 and maximize profit"

Walker Griggs 2023-07-11 16:54:20

@Marcelle Rusu (they/them) so many good points to unpack

I don't know how to mentor people properly & I realized I don't think many people do.

Did you have a mentor yourself? I wonder if this is a 'chicken and egg' problem. Potential mentors themselves need a good example to emulate. What in your mind does a "proper" mentorship look like?

More so, when I've gone and done this myself, I essentially got told to stop from my managers

😞 ugh, I'm sorry. Mentorship programs don't start and end between mentor and mentee, there has to be a support system to facilitate that relationship.

My biggest concern is capable people getting completely turned off tech due to cultural problems

I've noticed most mentorship or apprentice program in software only start when you're in the door. On the other hand, trade apprenticeships are paired with school and take you from square 0 to 100. Do you think there's room for industry vets to reach outside of the industry to students considering CS or broadly 'people interesting in software'.

Lu Wilson 2023-07-11 17:01:14

It's an "apprenticeship scheme", which is a national government thing here. I don't know what the difference is between that and 'internship', as I suspect it might mean different things in different English-speaking countries. 'Apprenticeship' just means that it's officially recognised here. Whether it's officially led by one person or many, I'd certainly expect them to learn from multiple people! Different people have different skills and backgrounds, and being exposed to that variety of people is a huge benefit of schemes like this. In my original career (not tech), I did a similar thing.

Marcelle Rusu (they/them) 2023-07-11 17:27:23

@Walker Griggs

Did you have a mentor yourself? I wonder if this is a 'chicken and egg' problem. Potential mentors themselves need a good example to emulate. What in your mind does a "proper" mentorship look like?

I didn't, and I completely agree. I'm still trying to figure out what a "proper" mentor looks like, but I'd think it'd have to incorporate the following with some frequency.

  • work with the mentor on something - pair programming (?)

  • talk through problems mentee is facing & get encouragement + advice

I've noticed most mentorship or apprentice program in software only start when you're in the door. On the other hand, trade apprenticeships are paired with school and take you from square 0 to 100. Do you think there's room for industry vets to reach outside of the industry to students considering CS or broadly 'people interesting in software'.

Would love to do this and I think many would, but I wouldn't know how.

On another note: my sister is in planning and after you start the job, you need to go through a mentorship to get to a certain title. It's mostly just talking + encouragement from what I know, but I think the encouragement is at least as important as technical if not more.

Gregg Irwin 2023-07-12 02:29:01

@Walker Griggs I've been fortunate to work with may types of people, around the need/love of building software. Where my ego comment comes from leans heavily toward those (broad brush here) who are off-the-charts smart as engineers, but struggle on the social/empathy/human side. I think I'm somewhere in the middle. :^) People in this group are interested in constructive collaboration, learning together. That's why I feel safe posting things that may be flat-out wrong, or make me look ignorant.

We're all somewhere on this journey, via different paths. Some of my favorite memories are working with someone who was way smarter than me on the math/code side, but also able to meet me in the middle on the need to make things accessible to "normal" devs. Many times, though, working with people so much smarter than me on the machine side is tough, and even at my advanced age I can doubt the value of my experience and design values. I've also been on the other side, where maybe I wasn't so great at being mentored, because I thought I was right. :^)

A direct analog for me is theatre. When I direct, it's very intuitive and comes from my passion for it, just like software. And, just like software, sometimes the most talented actor just won't take direction. Sometimes it still works out, but it can also make the whole show less than it could be, because the vision doesn't hang together when one or more pieces don't mesh (this is true for the tech aspects like lighting and set design as well).

As for how to mentor, I can only say I've had some great ones who probably taught me by example and osmosis. Don't try to mentor everyone. Find the right mentees, who have a spark, and just share your love for your craft with them. Be the student as much as the teacher. I probably learn more than I teach, because seeing through new eyes make you question and try to understand more.

Máté Hidvégi 2023-07-13 11:23:59

When I started out as a junior developer I was yearning to be mentored. But I received the message from my leads that I do a great job, and I should strive to be more independent, do things my own way. At my next job I worked in a team where I was the only one with a programming degree and experience so again there could not have been any mentoring.

And at my last job I was working with great people and my leads were open to mentoring me. But here I experienced that they did not have a talent for it yet, they did not really listen to my problems, just giving me general good advice. For me that resembles more like watching youtube videos about programming then an actual two way discussion.

When I tried to teach people younger then me I also noticed the ego problems that @Gregg Irwin mentioned. A lot of times young developers just want to play around with their own solutions, and they don't want to deal with learning something. Also I met people who openly said they are doing this for the money, and they don't care about learning if they don't need to. And of course developers mainly care about technologies, libraries, tools and programming tricks. Its very hard to have discussions about the human side of the job, communication skills, psychological safety, etc, although these can have a lot more impact on day to day work and these are the things that are not taught in programming courses.

Jack Rusher 2023-07-13 15:47:52

Also I met people who openly said they are doing this for the money, and they don’t care about learning if they don’t need to.

While I understand the drive to maximize the rate at which one can trade labor for capital, I do not enjoy working with people who approach their work (their lives!) from this perspective. 😞

Walker Griggs 2023-07-13 16:46:09

I noticed pretty early on in undergrad that my peers could be roughly broken down into categories.

  • those who were in it purely for the money
  • those who were interested but not convinced that software was their calling
  • those who ate, drank, bled, and obsessed over software

I also found that people's success and happiness in their career was strongly correlated. Nothing groundbreaking about that observation and it can probably be applied to most fields of study. BUT I also noticed that ego in that last category could run rampant -- in myself even. I think mentors actually serve to reduce that ego -- to show their mentees just how big the pond really is.

You don't know what you don't know. But once you know how much there is you don't, you're forced to either adopt that beginners mindset (good) or dig your heels in (bad)

Walker Griggs 2023-07-13 16:47:32

It's an "apprenticeship scheme", which is a national government thing here. Apprenticeship' just means that it's officially recognised here.

Lu Wilson I really like this. I'll give it a read through. I didn't know that is was govt supported

Gregg Irwin 2023-07-13 17:45:45

For me, a related area to mentoring is community leadership. I've seen many people and manners of interaction. It helped me see that someone (I'll call out the especially smart ones; the elite) didn't just resist me or my ideas, but almost anyone. They could also be hard on those of us not as gifted or adept. Encouraging empathy here, and trying to get them to do the mentoring has helped at times. Not always, but I love, love, love, to see someone else get that big Thank You for their help and seeing it affect their future interactions. We can all fall back on habits of course, but it's a journey, right? And this empathy has to go both ways, because we get some very special people in the modern view of neurodiversity, and dealing with that is hard as well. We all need each other.

Some of us are deeply connected this craft, but we're still social creatures with all that history in play as well.

Gregg Irwin 2023-07-13 17:50:41

Living my own example, as a re-newbie here, I'm flooding thoughts out that I haven't had a chance to get out otherwise, so I appreciate everyone's patience with that. :^)

Jack Rusher 2023-07-14 15:30:51

A nearly ten year old artifact of mentoring Paul Ford (in this case, in Clojure):

gist.github.com/ftrain/8655399

Gregg Irwin 2023-07-14 18:54:46

I'm reminded of Beautiful Code and Beautiful Architecture, which I really enjoy and think the idea should be much more widespread. Exercism is a site that works along the lines of mentoring, at the small scale of functions and problems.

Gregg Irwin 2023-07-14 19:00:06

Here's an old gist I made, which falls somewhere in two venn circles of mentoring and literate programming: gist.github.com/greggirwin/38883ca5109175a60896d2f406ee49f6

Gregg Irwin 2023-07-14 19:18:31

Gist updated for a recent change in Red, for anyone who wants to run it, and also updated the code-only version at gist.github.com/greggirwin/6745b96d0531ee88f5755d6e62515a9d

Daniel Buckmaster 2023-07-11 23:30:27

What we talk about when we talk about expressivity

I enjoyed the little tangent on expressivity in the latest episode. I've had thoughts on this recently and it prompted me to skim Felleisen's paper. I really resonated with the hosts' reframing of expressivity as being the part of the language that's oriented towards the programmer, not towards the machine.

I currently think that most programmers, when talking about "expressivity", actually mean essentially two things:

  • I can use my own words
  • I am not restricted by grammar

These things are strongly tied to writing, as that's still how we do most of our coding.

"Using my own words" is literally that - in any given chunk of source text, how many of the words were chosen by the programmer (e.g. to be particular to their domain or their theory of the program) and how many were specified by the language or environment? Punctuation, I think, also counts as words the programmer didn't get to choose. Random examples:

  • In Ruby, one can create little DSLs where almost all words in a specific part of the code are "my own words"
  • In assembly languages, the programmer can choose almost none of the words (except labels?)
  • Being able to rename imported symbols lets the programmer choose their own words in specific contexts (same goes for type aliases, etc.)
  • Languages with few keywords should tend to have more words chosen by the programmer... or at least, by the authors of the standard library?

I equate being "unrestricted by grammar" roughly to whether a language is statement-oriented or expression-oriented. The Austral spec has a great section on why it chose to be statement-oriented, and concludes that "a statement-oriented syntax is less simple, but it forces code to be structurally simple": austral-lang.org/spec/spec.html#stmt-orientation

In Austral, it's an ideological choice to force programs into a certain shape. But in general, it seems to me that languages with less "grammatical restrictions" in their parser are described as more expressive. Maybe this is just correlation with other features of those languages.

I'd love to know how everyone else understands "expressivitiy" when we talk about programming.

Gregg Irwin 2023-07-12 02:08:50

There's a risk I'll go on for a long time. :^)

I work on Red (red-lang.org), a descendant of Rebol, which was inspired by Lisp/Forth/Logo. A key point in Carl Sassenrath's design of Rebol was that it was first a messaging language; a data format. Secondarily, you could interpret it to make stuff happen.

To the first point, everything is data, so the number of lexical forms is very high compared to most langs. But the number of keywords is zero. Yes, in a standard environment there will be a lot of words you expect to be there, and work in a certain way. But you can break or enhance the system by altering them.

Another key feature in standard Red is the parse function, which makes it relatively easy to write BNF-like PEG grammars to build dialects (eDSLs). Not only at the string/char level, but at the value level, because everything is data once loaded from text.

All those lexical forms (and also standard types that don't have a literal syntax) can be used in your dialects. This makes for a very flexible and powerful "language construction toolkit", which is how I sometimes describe Red. And to bring it full meta-circle, parse and other features are dialects, as is the low level Red/System language, a C-level language that shares syntax, but not semantics, with Red and is used to write the runtime for performance reasons (it compiles to machine code). e.g. it's static and lacks most high level datatypes. Red also has a GUI dialect.

Now, for how expressive this all is...I'm biased, but there is an aspect that isn't very flexible: lexical forms (literal syntax for values). You have to hack the lexer and add new types for that, which is by design. Why not make it easy? Back to Red being a messaging language. In the context of data exchange, you need to agree on the basic language elements.

Formatting is another facet of expressivity. Consider poetry, free verse versus something like a pantoum or sestina, which are very rigid forms. This relates to both form and function for PLs. Whitespace, line breaks, statement-ending punctuation, all effect how you write for your readers, and perhaps how susceptible things are to (un)intentional corruption. e.g., you can't strip spaces and lines from a Python program. On the function side, we veer into static/dynamic aspects.

The less rules you have, the more expressive you can be. Consider art. What you gain in expressive freedom, you may give up in appreciation. Constraints are important, but also contextual. Can you have an effective fully expressive GPL? Or to get that (ASM) do you give up too much "appreciation", because we also have to share, communicate about, and understand these works we craft.

Personally, I think we have to balance things based on context. Be as flexible and expressive as possible, in the target domain, to the point where any more leads to less benefit and more cost for target users. Please some of the people, most of the time.

Gregg Irwin 2023-07-12 02:09:47

I warned you. :^)

Duncan Cragg 2023-07-12 07:51:59

I'll let everyone read both above long posts before adding my own take, but briefly, I'm a secret fan of Red/Rebol as my own lang shares much with it/them. I have type=grammar for both inline small types and whole objects.

Konrad Hinsen 2023-07-12 08:41:44

Rebol and descendents are something I have been looking at as well every now and then. Red seems to be the most active project in this space right now. Another one that seems stuck in an early design phase is altscript.com. But there is nothing I see as good enough for actually playing with at this time.

Jimmy Miller 2023-07-12 16:15:46

@Daniel Buckmaster Glad you enjoyed the discussion. I definitely think you've hit on some really important aspects of expressivity. I do think it is a complicated subject and hard to pin down exactly what we want to say it is.

On one hand expressivity is how we say something, your "Using my own words" on the other hand it is also about what can be said. A DSL would then be more expressive in the "Using my own words" sense, but less expressive in what can be said.

To me an expressive language works on both of these aspects. It allows me to express things the way I might want them to, but also allows me to express more kinds of things. Depending on my interests and contexts those kinds of things might change. So I don't think we can make a total order out of expressivity in languages.

For example, if we compare Haskell and Idris, I can express types in Idris that I can't (without contortions) in Haskell. I am able to represent the type of printf in the language itself in a very clear and concise way.

But when we compare lisp and Idris things are a bit confusing. Lisp does not have the kind of type system does. So for example, I can't express things things like "this function is total" in the way I can in Idris, but I also can express complex macros and have fewer limits on the kinds of things I can express because I lack that type system.

C feels less expressive than Java say, but I can express things about memory layout and allocation that I can't (easily) in Java.

I think this is what make expressivity so hard. Gaining to the ability to express something can restrict some other aspects.

Jimmy Miller 2023-07-12 16:16:30

@Gregg Irwin Super interested in Red/Rebol. Though I haven't written much/any. It's definitely a tradition that doesn't get much attention.Any good texts on it that we should read?

Andrew F 2023-07-12 17:11:46

My best shot at defining expressivity is in terms of some abstract space of logical relations/structures. The elements to be related include types, machine details, phase distinctions, etc, as well as elements of your application domain.

A more expressive language is one that can span more of this space. To the extent that languages cover (roughly) the same space, you can rank them by (roughly average) concision. We can recover notions like C being more expressive for machine details, or a DSL being more expressive for a given domain, by restricting the logical space we're considering.

It gets tricky, though, because you're probably talking about Turing complete languages that can all simulate each other, and just ranking by concision doesn't quite capture the notion that writing a simulator doesn't "count" as expressing it; even if you could write a logical inference engine in five lines of C, that doesn't change the intuitive notion that Prolog is more expressive for certain problems. You want a notion of "direct expression" that's not covered by concision. Maybe something to do with (cyclomatic?) complexity of the code to express a given thing?

Gregg Irwin 2023-07-12 19:44:49

The old Rebol core guide (rebol.com/docs/core23/rebolcore.html) is still a seminal reference for the language. rebol.com/docs.html links to some other primer bits. Red's reference docs are here but not in User Guide form for learning. A Red user has written helpin.red. You can also scan the old blog entries at red-lang.org for Red-specific features.

There are a few other langs out there with the same heritage, which we sometimes call Redbol as a "genre". :^)

📝 REBOL Documentation

Links to all documentation, including tutorials, examples, manuals, and references.

Jimmy Miller 2023-07-12 19:47:38

Thanks definitely helpful. I'm mostly interesting in something a bit more meta/philosophical on the perspective. Like a paper giving the why of Rebol like languages.

Gregg Irwin 2023-07-12 19:49:36

Konrad Hinsen we don't know if Carl will continue with AltScript or not. He tends to disappear into his cave for long stretches. :^) Rebol2 (R2) was closed source, but Rebol3 (R3) was done quickly by Carl and a few others to make a FOSS version. It's still alive as well. Where Rebol is written in C at the low level, Red was bootstrapped on R2 and is designed to be self-hosted, hence the need for Red/System.

Gregg Irwin 2023-07-12 20:04:36

If we go Rich Hickey on this: expressive - Effectively conveying thought or feeling.

To Jimmy Miller’s point, a language might be expressive in terms of type systems, but not at all able to express a GUI.

We also have limits, which tooling can help us overcome. For example, in a low level language, you may want/need to express multiple integer types by size and signing ability, but no strings. In an HLL, you just need number but have many types of strings (plain, filename, email address, url, tag, etc.). Maybe you justify a Cartesian coordinate type, but not UTM coordinates for mapping projections. How do we keep from being overwhelmed, while being able to express things (thinking text here) as we do with natural language?

Gregg Irwin 2023-07-12 20:26:34

The why: Rebol was designed for the semantic exchange of information between people and machines.

That's why there are so many datatypes with literal forms. Because we need them to talk about things easily, both on the human and the machine side.

On the philosophy of Redbol, that's quite unfortunately strewn throughout time and space. I can say that Carl spent 20 years designing Rebol, after deep study of denotational semantics, before building and releasing it. The first version was done in Scheme, but was too slow. He had built things using many paradigms, and rejected OO as the answer after trying it. I can't speak for him, but his designs say he values simplicity and will give up other things for that. He also values the human side which is why while Rebol is a totally wacky language internally, when compared to how other langs work, there is this lovely, simple façade that lets you wade in comfortably for a while (or forever) before you fall off the edge and into the deep.

For this chat, Red is expressive in different ways to different people. To a high level user, it's a single EXE with a built-in GUI system, compiles standalone EXEs without any external tools, and is easy to learn. To a PL enthusiast, it's Logo (Lisp without parens) that uses definitional scoping, f-exprs, and free ranging evaluation. Each word is bound to a context and there is really no such thing as "code". There is only data that is evaluated. See: this

I could say it's a contradiction in terms, but it's probably just as accurate to say "Old School". :^)

David Alan Hjelle 2023-07-13 01:11:29

@Gregg Irwin What's the best way to try Red on Apple Silicon? I tried the macOS download on the website, but I got "“red-view-12jul23-aea09888d.app” needs to be updated.". Maybe I should just pull out a Raspberry Pi…

Gregg Irwin 2023-07-13 05:18:18

Red is currently 32-bit only, so no latest MacOS, and also no Apple silicon support. The downside of being your own toolchain.

David Alan Hjelle 2023-07-13 12:43:45

Darn. But makes sense. Thanks!

Duncan Cragg 2023-07-13 22:19:51

I enjoyed the little tangent on expressivity in the latest episode. I've had thoughts on this recently and it prompted me to skim Felleisen's paper. I really resonated with the hosts' reframing of expressivity as being the part of the language that's oriented towards the programmer, not towards the machine.

Many decades ago I coined the acronym "DTIL" or Domain and Target Independent Language, to clarify what I was seeking in my Perfect Programming Language. Meaning: a language that's not in any way constrained by the machine (the Target) OR the Domain of application. A pure language of thought. A language that allows expression by humans in their most intuitive way of what they wanted the computer to manifest for them. So a cognitive-oriented programming language, but I was only allowing a formalism or symbolic mechanism, rather than in any way a natural language, which I saw as redundancy-heavy and fuzzy.

Declarative languages immediately stood out and wiped out Imperative languages for me, as these had too much Target (machine) orientation. The Declarative mantra "What not How" gives it away: just say What you want, don't tell the machine laboriously How to do it.

I currently think that most programmers, when talking about "expressivity", actually mean essentially two things:

  • I can use my own words
  • I am not restricted by grammar

These things are strongly tied to writing, as that's still how we do most of our coding.

Hmm, then that's "just" asking ChatGPT!

"Using my own words" is literally that - in any given chunk of source text, how many of the words were chosen by the programmer (e.g. to be particular to their domain or their theory of the program) and how many were specified by the language or environment? Punctuation, I think, also counts as words the programmer didn't get to choose. Random examples:

  • In Ruby, one can create little DSLs where almost all words in a specific part of the code are "my own words"
  • In assembly languages, the programmer can choose almost none of the words (except labels?)
  • Being able to rename imported symbols lets the programmer choose their own words in specific contexts (same goes for type aliases, etc.)
  • Languages with few keywords should tend to have more words chosen by the programmer... or at least, by the authors of the standard library?

It's important in a DTIL that the mechanisms available are pure and singular: there should be only one language representation of each cognitive entity and nothing that biases the language to either a machine or a domain. Ideally the whole base syntax should be just half a dozen unique things (like symbol, sequence, structure, consequence).

An important aspect of this whole conception is that data should be simply strings or text and structures of that. If you have a "double" - rather than a "float" or "int"- you're immediately binding yourself to machine concepts, as provided to you by the FPU. In reality, humans don't think like that, we simply write our "data" in text. So spreadsheets and Awk have some precedent there.

This leads to the (apparently radical) concept of type being simply syntax: if you parse a string or structure in a way that's meaningful to you, you've made your own type matcher and thus your own type. You don't have to be bound by the types in the mind of the originator of some data, or by the types the machine supports best.

I equate being "unrestricted by grammar" roughly to whether a language is statement-oriented or expression-oriented. The Austral spec has a great section on why it chose to be statement-oriented, and concludes that "a statement-oriented syntax is less simple, but it forces code to be structurally simple": austral-lang.org/spec/spec.html#stmt-orientation

In Austral, it's an ideological choice to force programs into a certain shape. But in general, it seems to me that languages with less "grammatical restrictions" in their parser are described as more expressive. Maybe this is just correlation with other features of those languages.

Not sure about the statement- vs expression-oriented thing, but again, simplicity and power are key to maximising human expression of virtual stuff and their desired behaviours.

Duncan Cragg 2023-07-13 22:22:53

Grand Plans, but over 4 decades later I'm still working on it! 😄

Duncan Cragg 2023-07-13 22:24:24

It was kinda spooky when the phrase "DSL" appeared, and of course I immediately knew it would be anything I wanted!

Konrad Hinsen 2023-07-14 09:27:38

Thanks @Gregg Irwin for the links to and the background of Rebol/Red!

How do we keep from being overwhelmed, while being able to express things (thinking text here) as we do with natural language?

The comparison with natural language is difficult. Natural language serves for informal, i.e. context-dependent, reasoning. It's OK to have the same terms refer to different meanings in different contexts. In a formal language, everything needs to be explicit and non-ambiguous. So I guess different but similar-in-spirit and interoperable languages are probably our best bet. That's something I think Red got right. As did Racket (although it lacks the system layer for now).

Marcelle Rusu (they/them) 2023-07-14 16:20:31

Im starting to think that expressiveness is not a solvable problem at least directly

Any representation will have downsides, so we need multiple representations.

When im glancing over my code, id prefer to look at the lines of regex in text because its terse and can be readable enough to know “this is a phone number validator”

But when im writing regex, or testing it, i want a UI - i use regex101.com

I think there are many things like this, where if we try to tackle the expressiveness problem directly with text we may fall into problems of performance, optimization, terse vs readable.

If we have ways to easily swap out parts of our code with different representations, it may make the language & the specific expression less important.

Libraries can expose their API as a verbose yet clear data format, while providing plugins to swap between representations of it. - calendar plugin allows you to manipulate a calendar to set hardcoded holidays for your system, etc.

Andrew F 2023-07-14 17:52:46

I think regex is a good example of why grammar/representation are a red herring. No matter the syntax, the underlying formalism of regex cannot express a language with matching brackets, or HTML. This is, if not the only, then the most important aspect of expressivity.

Within the domain of regular languages, or perhaps the extended domain of PCRE, I would suggest that grades of expressivity come in the form of primitives and compositions that let the abstract structure of your "program" correspond more directly to the structure of the problem in your mind. Maybe combinator-based APIs are nicer, for instance, and I'm sure we could define less-expressive languages with the same power if we wanted...

Looking back at what I've written, I wonder if I'm stretching the meaning of "expressive" too far to include formal power. But I do feel it's part of the intuitive idea of "expressivity", and I stand by the idea that trivially isomorphic representations of the same formalism can only have minor differences in expressivity.

Gregg Irwin 2023-07-14 18:34:28

Natural language serves for informal, i.e. context-dependent, reasoning. It's OK to have the same terms refer to different meanings in different contexts. In a formal language, everything needs to be explicit and non-ambiguous.

Konrad Hinsen context is the very thing I meant. The question is, can we have both context and unambiguous use in a proglang? Red tries. For some people it's a reason to never consider using it, and they run screaming in terror. For others it's "I can use that to do this really wacky thing.". For most I believe it's "Oh, I have no idea what's going on under the hood. It just works." Finally, a few deep divers will say "It makes this edge/exception case impossible to handle in all cases, so it's a bad design." Maybe prompt-based development will lead us to new approaches where, like with human dialogue, the system can say "Did you mean A or B here?"

Konrad Hinsen 2023-07-15 13:17:29

@Gregg Irwin Making context explicit is indeed one way to handle this problem, and if Red is working towards that goal, that's a reason for me to take a closer look. There is some tradition of this kind in Lisp/Scheme, but it's a bit fringe.

Prompt-based development is perhaps a solution at the IDE-level, but not at the language level. The IDE must store the developer's answers in some way that makes the code usable without prompts later and elsewhere. That some way would then have to be part of the language.

Duncan Cragg 2023-07-13 20:42:06

I think this topic, based as it is on personal experience, deserves better visibility, so hopefully Rachel won't mind my reposting it here! 🤗

[July 13th, 2023 9:38 PM] rachel: hey both! thanks for replying here.. as per your question Joe, I would consider someone a software engineer if they know how to code - whether they have a CS degree, or did a bootcamp, or learned on their own. Someone who can code and build applications. I have no formal training in coding. However, I am excited about the potential of AI to help me make an app by generating code for me.

To answer your question Duncan - I've mostly been using a combination of asking chatGPT for the steps and code, and also using an app I am working on with a cofounder called Sublayer. We noticed that the chat interface is not really the best interface for building (and changing) complex apps, but we're still exploring the right workflow for it. Definitely eager to learn from this group how others are getting the best code generation out of the existing LLMs.

Duncan Cragg 2023-07-13 21:15:18

.

Duncan Cragg 2023-07-15 16:20:33

OK, just me then! 😁

Sorry, Rachel, perhaps we're over the AI hype curve in this community! 🤓

Scott 2023-07-14 21:10:45

I've been thinking a lot about the patterns and architectures we're going to see start to emerge that lend themselves well to being written by generative AI and came across this technique being used by a library called Marvin (github.com/PrefectHQ/marvin) where they limit the tokens the LLM can respond with to just a single token corresponding to a value in an enum. They then have it respond with the value as the response to a natural language query. This is extra interesting because responding with a single token is relatively fast and cheap.

The example they give is using it in routing:

    USER_PROFILE = "/user-profile"

    SEARCH = "/search"

    NOTIFICATIONS = "/notifications"

    SETTINGS = "/settings"

    HELP = "/help"

    CHAT = "/chat"

    DOCS = "/docs"

    PROJECTS = "/projects"

    WORKSPACES = "/workspaces"





AppRoute("update my name")

# AppRoute.USER_PROFILE

But I feel like there's a seed of an idea here that points to what a piece of an LLM-core architecture may look like. I experimented with the idea a bit in chatgpt earlier today (screenshots attached) and I'd love to know if anyone finds this interesting or has any thoughts/opinions.

📷 image.png

📷 image.png

Alex J. 2023-07-15 02:53:06

Just for clarification - is the idea that the LLM can be used as a sort of router?

Scott 2023-07-15 13:48:24

Yeah, so they're using it as a natural language router which is super interesting in itself - that "update my name" could also be the output of another LLM (ex - as a result of a user action in a UI...user fills in a "name" field and clicks the "save" button, router POSTs that message to the API).

You can take that further on it's own - can you have an LLM route messages in a smalltalk/ruby/objective-c message-passing style language?

But where my head went is - given a specific context in a prompt, we can encode a bunch of information (in binary, hex?, emoji?) and enable new patterns that give us cost/speed benefits from LLMs while also not having to explicitly write as much code. I need to do a bigger experiment, but in my screenshots, I'm showing it actually being able to encode and decode the information, at least from a simple binary encoding.

A while back I followed a tutorial for building a chess engine in C, and they used a similar technique but with U64s and a lot of bitwise operations, bit masks, etc and I'm thinking there's a way to take inspiration from those techniques and use it with an LLM...