Referer Reality

50 points by tobr 7 days ago | 21 comments

I didn’t imagine that I would stir up quite so much interest when I decided to ban query strings!

When I said, “if it isn’t there, it’s probably for a good reason”, I was fully aware of native apps, and decided to gloss over them, because I doubt (on no evidence) they’re so significant for me, and things like email and feed readers may well be web apps anyway. And because I’ve tended to fine nuance in the past, and am trying being concise instead.

While thinking about it—is it possible for a browser to know which app sent it the link, e.g. androidapp://com.example.app? Because that’d probably be a perfectly reasonable referrer, if a browser wanted to send it.

> Note that a handful of sites do choke on unexpected query strings, including YouTube (!)

I want to learn more about this.

randallsquared 5 days ago | |

> Because that’d probably be a perfectly reasonable referrer

Well, it might be better than no referrer, but if the app is a mail app or feed reader or something, it doesn't really mean much more than "Safari" or "Chrome".

big_toast 4 days ago | |

I love Robin's newsletters (and the books etc) and love "opinionated operator decisions make the internet go round".

Maybe I'm missing something but it seems like if i go gmail -> via a Robin Sloane Newsletter -> webpage, it seems generally useful to know Robin sent me. Like a backlink that's been liberated?

Is something different happening?

minebreaker 5 days ago |

I strongly disagree with a lot of claims here.

>> if it isn’t there, it’s probably for a good reason. > which isn’t really true anymore. For most websites, the majority — not just the plurality, but the majority — of visitors arrive by following a link inside an email or an app

I don't think the statement is factually backed up. At least I hate native apps.

> Even so, my custom query string is, in my calculation, an expression of digital etiquette: rather than dump a load of anonymous traffic on your doorstep, I reveal who’s linking, so a website or online shop operator can trace it back and get in touch, if wanted or needed

Anonymity considered harmful these days?

> a wave of new subscriptions and weren’t sure if they were legitimate; a brief email correspondence assured them that yes,

It's not legitimate unless it's signed (and if anybody gives a shit to verify it).

> Note that a handful of sites do choke on unexpected query strings, including YouTube (!),

This is a good habit IMHO.

Anyway, I'm thankful to the original post because it was a good reminder to re-review my browser settings.

Honestly I don't understand why the EU focused on the stupid cookie law instead of referers which are clearly privacy-violating.

If you use Firefox I recommend you make sure `network.http.referer.XOriginPolicy` set to 1.

intronic 5 days ago | |

I also dont get why adding 'utm_source=Robin_Sloan_sent_me' somehow gels with his calculation it is 'an expression of digital etiquette: rather than dump a load of anonymous traffic on your doorstep, I reveal who’s linking'. Theres no actual link to the referring website just some bunch of characters which look like someones name and some of which are in the actual referring URL.

And yeah I'm also thankful also to see that firefox setting.

randallsquared 5 days ago | | |

I am totally sympathetic to the "this isn't a legible link for machines". After all, this utm_source implies that some human is looking at the logs manually (or at least, trying to decipher aggregated utm sources manually). However, we are well into a transition to a world where this isn't true, and 'utm_source=Robin_Sloan_sent_me' is probably just as good as 'utm_source=robinsloan.com' for LLMs of 2026+.

jampekka 5 days ago | |

> Honestly I don't understand why the EU focused on the stupid cookie law instead of referers which are clearly privacy-violating.

Neither the ePrivacy directive (commonly called the "cookie law") nor the later GDPR focus on cookies. They are "technology neutral", applying to e.g. URL parameters and HTTP headers too, but just widely misunderstood and badly enforced.

extraduder_ire 4 days ago | | |

It doesn't help that the most visible and best SEO'd sites purporting to explain the GDPR are made by advertising/tracking companies, or firms representing/selling services to them.

I'd guess that the average person doesn't know that the GDPR applies even when you're taking details from people by hand with a pen on paper.

IdiotSavage 5 days ago | |

Or set network.http.referer.spoofSource to true.

bji9jhff 5 days ago |

For the privacy of the user, nowadays the referer header are either shortened or voided by the browser. You reintroducing the string by a side-channel could be seen by users as a backstab, even if you see it as "etiquette".

extraduder_ire 4 days ago | |

At the very least, people seem to have settled on one or two args to use for this. e.g. "utm_source".

I tried entering ?utm_source=foo on Chris's site when it was posted the other day, and was surprised it didn't trigger the page. Pretty sure one or more of my firefox extensions remove that specific one from URLs before making the request.

Facebook combining tracking and IDs into an opaque UUID in URLs so they can't be removed is a level of user hostility beyond this.

chrismorgan 4 days ago | | |

Firefox’s Enhanced Tracking Protection removes a few such parameters.

Now I have the stupid contrary idea, require a specific query string parameter to be present…

kijin 5 days ago |

If there's a query parameter that you have a legitimate use for, like `q` for searching, obviously you should configure your web server to let it through.

Even in that case, you might want to block unexpected values as early as possible in your stack. For example, if you have a legitimate use for a certain set of `utm_source` values, but someone sends you bobby tables, you probably shouldn't log it blindly.

Ditto for the Referer header -- there's a lot of spam, and some of those strings might even be dangerous. You can't trust any of them anymore.

akersten 5 days ago |

i'm confused, why would you go out of your way to add this tracking info to external links voluntarily? it doesn't benefit you, it just helps other websites segment their traffic?

chrisdinn 5 days ago | |

When other website owners realize how much traffic you send them that is good for you. Maybe they want to sponsor that newsletter if it’s driving a lot of business.

simon84 5 days ago |

Opinionated: I dont get the issue about people (incl. bots) visiting your website, regardless of where they originate from and what url they use.

Publishing something online means I want it to be seen.

Relying or enforcing Referer or query string basically contradicts the intent. Wanting to know where your users come from could be legit but then you basically distort the "public web" aspect to a "public trap" where you try to sneak on visitors.

Not my vibe

jstanley 5 days ago |

> which isn’t really true anymore. For most websites, the majority — not just the plurality, but the majority — of visitors arrive by following a link inside an email or an app

I highly doubt this is true, unless your definition of "app" includes "web browser".

swiftcoder 5 days ago | |

If your traffic is mostly links from social media, it may well be accurate - up to orders of magnitude more folks are using Facebook/Twitter/BlueSky/LinkedIn from the mobile apps than from desktop browsers

thayne 5 days ago |

I get adding query strings on links to your own domain, but from the original post, it sounds like the author was seeing third parties add query strings to links to their domain and I don't see what the point in doing that is, unless the entity adding those quey parameters is able to somehow intercept them between the browser and the website (which is possible if they control the browser or a cdn).