I'm like a third of the way through this and loving it! Fantastically written and interesting.

First reaction: Yes!

Second reaction: Wouldn't we be better off with a P2P architecture, rather than client-server? Put differently, isn't a user-respecting server a well-meant step into a fundamentally wrong direction? The very terms "client" and "server" imply an asymmetry that's perhaps not good to have as a default.

@Daniel Buckmaster thanks! I’d love to hear what you think once you finish.

Konrad Hinsen “Aim high but make sure you shoot”. It is a peer to peer architecture. That’s what it’s enabling. But people have many devices, and there’s thorny realities about network traffic. I vehemently disagree it is in the “wrong direction”. Unless you plan to go back and re-invent everything from the IP layer up. I believe Server User-Agents can restore the missing sovereignty now.

There is nothing to say that a single device or piece of software could not fill the responsibilities and definitions of both client and server, but reality is often more complex. People own multiple devices. Personal devices are not always on or always connected. Some devices like a watch are incredibly low power, low bandwidth. Some situations need high compute, high bandwidth, high storage etc. I think it would be oversimplifying to try and remove the terms client and server from our vocabulary. The shift is that we have always assumed users only have clients, and servers are in someone else's control. I'm agreeing with you that a single end user, should have sovereignty over a stack that can combine roles, but I do not agree that it would be appropriate to flatten everything (oversimplification) or to remove the terms from our vocabulary (starting over on the internet)

@When Leggett I agree that there are situations where you want a device to be only client or only server. So maybe our only difference is in jargon. I prefer to start from P2P as the norm and single-role devices as exceptions. But technically, that doesn't make much of a difference.

I think that there should be a goal of being able to write our software in a local-first and peer-to-peer orientation. I think we need infrastructure in server-land to accomplish this in a general way

I think that it will sometimes be necessary for your SUA to facilitate that role (even as a stun/turn server), but there would also be scenarios like p2p messaging where the peers relationship might have to fall back to an SUA if the client is disconnected, for example

It'll take me a while to have a formed opinion on this. Thanks for writing this, I find it really stimulating!

Gut reaction, since I'm already familiar, is that elements of this remind me of what Sandstorm was trying to do with its fine grained objects and capabilities. It is one step away from just "self host apps" and towards something more Solid-like, I think. sandstorm.org/how-it-works#grains

Yeah, Sandstorm is cool. And definitely has a lot of value to learn from.

@Daniel Buckmaster Sandstom has been on my radar for a while, so I wonder why talk about it in past tense. Has the project been abandoned?

No, but the contributor base is sadly very small, even if tenacious. I have high hopes for an upcoming rewrite to increase maintainability, but it's a lot of work for a small volunteer group. I don't have the time or expertise to contribute to the core. (Sandstorm might be worthy of a separate discussion thread! Which I may start sometime when I organize some thoughts.)

tiny example of some of my rules

  // Security: Secrets and credentials (Score: 0.9, Red highlight)

  secrets: {

    score: 0.9,

    highlightClass: 'secret',

    patterns: [

      { regex: /(['"][a-zA-Z0-9]{32,}['"])/g, desc: 'Long alphanumeric strings' },

      { regex: /\b(api[_-]?key|secret|password|token|auth)\s*[:=]\s*['"][^'"]+['"]/gi, desc: 'Credential assignments' },

      { regex: /\b(sk-[a-zA-Z0-9]{32,}|pk-[a-zA-Z0-9]{32,})\b/g, desc: 'API keys (OpenAI, Stripe, etc)' },

      { regex: /\b([0-9a-f]{40,64})\b/g, desc: 'Hex keys (SHA hashes, tokens)' },

      { regex: /-----BEGIN (PRIVATE|RSA|OPENSSH) KEY-----/g, desc: 'Private keys' },

      { regex: /\b(ghp_|gho_|ghu_|ghs_|ghr_)[a-zA-Z0-9]{36,}\b/g, desc: 'GitHub tokens' },

      { regex: /\b(AKIA[0-9A-Z]{16})\b/g, desc: 'AWS access keys' }

    ]

  },

  

  // Security: Dangerous functions (Score: 0.85, Orange highlight)

  dangerous_functions: {

    score: 0.85,

    highlightClass: 'danger',

    patterns: [

      { regex: /\b(eval|exec|execFile|spawn)\s*\(/g, desc: 'Code execution', languages: ['javascript', 'typescript', 'python', 'ruby'] },

      { regex: /\b(innerHTML|outerHTML|document\.write|execScript)\b/g, desc: 'DOM injection', languages: ['javascript', 'typescript'] },

      { regex: /\b(system|shell_exec|passthru|proc_open)\s*\(/g, desc: 'Shell execution', languages: ['php'] },

      { regex: /\b(__import__|compile|globals|locals)\s*\(/g, desc: 'Dynamic imports', languages: ['python'] },

      { regex: /\b(eval|instance_eval|class_eval|module_eval)\b/g, desc: 'Dynamic evaluation', languages: ['ruby'] },

      { regex: /\b(Runtime\.getRuntime|ProcessBuilder)\b/g, desc: 'Process execution', languages: ['java'] },

      { regex: /\bsystem\s*\(/g, desc: 'System calls', languages: ['c', 'cpp', 'rust'] }

    ]

  },

  

  // Security: Weak cryptography (Score: 0.75, Yellow highlight)

  weak_crypto: {

    score: 0.75,

    highlightClass: 'warning',

    patterns: [

      { regex: /\b(md5|sha1|des|rc4)\b/gi, desc: 'Weak hash algorithms' },

      { regex: /Math\.random\(\)/g, desc: 'Insecure random', languages: ['javascript', 'typescript'] },

      { regex: /\brandom\(\)/g, desc: 'Insecure random', languages: ['php', 'python'] },

      { regex: /\bnew\s+Random\(\)/g, desc: 'Insecure random', languages: ['java', 'csharp'] }

    ]

  },

also, I swear that I'm not a monster who keeps their browser window smooshed up against the top while leaving gutter to the bottom and sides...my OBS is wonkily aligned since upgrading macOS...it is Apple's fault.

another approach to the rules that I've considered but haven't implemented yet, is to use AST parsing or something like tree sitter to help to build or apply the rules. So far, haven't been super convinced that'll be deeply better than what I'm doing beyond that having knowledge of the AST could allow me to approach whole logical blocks more easily than I can with regex rules.

This is awesome! Neat project. How are you approaching the definition of what makes for a good rule, especially without "intelligence" backing your ruleset, or is that outside of your concerns? ("Gnarly logic" seems like a tough one to do with pure regex 🙃)

I got curious about what the inspiring project uses as its rules and funnily enough, from what it looks like, they leave that almost entirely up to gpt to decide :)

@Matt Curtis yeah, I found it kinda unsettling that the other system doesn't really have rules, it is rather trusting the model to do it all.

Right now, rules are mostly me trying to think through what I know to be "fragile" or potentially hairy changes that require more attention. My goal isn't for this to be a linter or code quality tool, but instead a way to draw attention to places that could be borked up, and that need human eyes to linger on them.

If you do happen to look at other parsing technologies, do look at ohmjs. OhmJS is better than PEG, PEG is better than LR. REGEX is unsuitable for anything with structure beyond line-at-a-time.

In addition, I like Prolog for defining and exhaustively searching rules. The easiest-to-understand Prolog that I’ve encountered is Nils Holm’s Prolog control in 6 slides. I ported it to javascript, then didn’t bother using it when I found out how simple and fast it was to shell out to SWIPL. (I also ported it to common lisp). At one point, I used OhmJS to give myself a quikie DSL that allowed me to express rules in Prolog and to format results in JS, and, produced a bash script to choreograph the whole thing. It ran in the blink of an eye - I wouldn’t use it in production code, but, I did use it my development workflow.

(Aside: we used backtracking TXL to create date-finding rules and to process bank code before Y2K).

oh! Thanks for the call out of ohmjs! This looks interesting, and maybe, might break me of my unwillingness to use other people's code in a thing -- Prolog was actually my first thought, or a constraints systems, but I couldn't quickly come up with a way for how to write anything like reusable logic for various languages in something shaped like prolog.

FYI - there’s a discord for OhmJS (Ohmland) and several experts are probably reachable in this slack (Patrick Dubroy Mariano Guerra). Further FYI - I consider that my programming language is “UNIX pipes”. This lets me use multiple languages to solve any problem in multiple paradigms. I find this to be more productive than simply trying to force-fit just one language onto my problem(s). OhmJS makes it even easier to glue stuff like this together (and, I’ve gone way down that rabbit hole creating my own little DSLs and using OhmJS to parse and transmogrify diagrams saved out as XML, graphML, SVG, …)

@Eli Mellen A nice case of Moldable Development!