You are viewing archived messages.
Go here to search the history.

Ivan Reese 2026-02-12 20:23:25

i'd like to add some more vaguely "private" pages to my website. private in the "obscure" sense, not "secure". like, i'm going to leave breadcrumbs so interested people can find them, and i'm legit fine with anyone finding them, but i don't want them to be out in the open.

the site source is going to be on indexed code-hosting social networks (like github), but i don't want shit coming up in search results. i'm thinking of doing, like, rot13 in a git hook? that might be fine?

curious what ya'll would suggest i do instead.

Mariano Guerra 2026-02-12 20:24:47

robots.txt?

Ivan Reese 2026-02-12 20:38:41

Oh yeah, right. I don't want AIs training on this shit. Hmmmm well there goes that idea.

Eli Mellen 2026-02-12 21:05:49

robots.txt has successfully kept my ENORMOUS blog out of search indexes.

AI crawlers are defo, though, 100% ignoring robots.txt

Eli Mellen 2026-02-12 21:06:26

Here is my robots.txt for ref.

It specifically allows and disallows a bunch of different things

Marek Rogalski 2026-02-13 20:04:25

You can save stuff as images. Robots don't like that.