> How can any business be OK with this??

First step is probably "don't look too closely" (:

I agree though — I would not want this thing doing who-knows-what-unverifiable-fiddling with my data and credentials.

We appreciate the good luck! We're definitely going to need it. I spoke to the security side in a previous comment, but here's some thoughts on the other points you mentioned:

* A/B testing: We're grabbing the browser session data, which includes things like cookies and local storage. In our experience, this tends to be where A/B testing group identifiers are stored. But, if it was on something like the IP, then, yeah, it is pretty hard for us to get around.

* Extensions: Yeah, this is a really tricky one. We've resorted to asking some of our more active customers, who've had these issues with content blockers, to use a separate browser profile. One thing we do for this (which also can help solve the first point) is what we call "smart capture", in which we capture based on a DOM selector, instead of x/y coordinates. That way, no matter the page layout, we get it right. The hard part of this, though, is this is not reliable on every site, and we've only enabled it on sites that we know it works well on.

* Networking/auth issues: This is a really fun one! I mentioned it in another comment, but we're doing some logic around detecting logged out state, with an image classification model, and checking the HTTP status code for resource requests. When we can detect it, we send the user an email, and give them a flow where they log back in, and allow our extension to capture the state data again. On networking, we can't get around things like corporate intranets of course, but these seem to be rare for our customers. We also will just retry the capture if there's a network error, and we notify people when there are errors, so they can take action accordingly.

I had the same thought, It's a novel idea but the security / practicality of it all is a nightmare to think about.

> What if your clients are using Chrome extensions that affect how the page is rendered? How can you replicate that?

You should not rely on anything on the client side, just accept the fact you don't control the code that is run there.

If you care about a few cases like adblockers, just test them in a headless browser.

Tangentially related question, what are the most successful products built around browser extensions? I sometime have ideas that would require browser extensions, but it feels like the target market must be computer literate for any chance or success (no mobile support for example).

Minor thing: I can't watch the video because of the tracking system and my adblock. but cool project!

Any Googlers in the team ?

Came here to say the same thing. The easel feature in Arc can do this.

Arc browser https://arc.net/

I love arcane tools like this. I'm definitely going to share this with our team. Thanks!

This is awesome. Can't believe I've never seen this before.

(I'm an EM at Plus) It's a bit complex, and not quite perfect, but I'm pretty happy with what we've done so far. The first method is by looking at the HTTP status codes. Since we're running a full browser on our side, we can tell if the status codes that returned are different than the initial capture. We also have been training an image classification model on pictures of log-in screens — this has worked surprisingly well, honestly. We've started expanding it to other types of "incorrect" screenshot scenarios as well, like loading screens, and we're seeing some cool early results.

This is covered in the documentation. You have to log back in, and Plus prompts you. Hence it's discouraged for sites that log you out a lot, like financial sites.

(I'm a designer at Plus) It wasn't our intention to look specifically like Gumroad, but some of the visual patterns are definitely similar. This aesthetic is making the rounds right now: https://bootcamp.uxdesign.cc/the-neubrutalism-or-neo-brutali...

We were looking for something that felt approachable and fun, magical but not mysterious. After trying a bunch of different things, this is where we landed.

They look very different to me. Maybe a similar-ish color palette is what you're seeing?

I agree, looks a lot like Gumroad

It feels like a call back to 90s print, like the magazines I remember as a kid. 3-2-1 contact and the like.

I assume they mean by default, employees do not have access to the KMS key necessary to decrypt the sensitive data. (They mention using an individual KMS key per customer)

I suspect there must be a handful of SREs who, who could access it if they really wanted to - though that access would still be logged in cloudtrail.

Lots of our customers use https://VisualSitemaps.com just for that. It's great for also keeping track of Competitors/Markets, besides internal QA.

something like https://playwright.dev/ might fit your needs.

This is similar for Jupyter + Robot Framework + Selenium: https://marketsquare.github.io/robotframework-seleniumscreen...

What’s your usecase? Can you define “more complex”?

Yes. This is what we need too

I was thinking through similar scenarios. I think there will be tons of fringe cases that this won’t work well with, but for the core use cases (Google Analytics, Amplitude, etc.) it will work more predictably. An analytics tool should have minimal announcements, pop-ups, A/B tests, etc. This isn’t the right tool for perpetually updating screenshots of a specific region of CNN’s homepage.

This is not remotely close to "Plus"

Might be useful when an image url is embedded in another page (like notion for instance), that it become a two-step workflow, so you have to go back to your Plus dashboard and allowlist that specific page that is the requestor.

That way, if someone takes the url of the image and shares it, it doesn't work without the owner allowing it again.

Of course, this isn't meant as a security measure as it would be trivially simple to circumvent, but more of a way of keeping track of the general surface area of how widely shared and image and putting the power in the users hands to reign it in.

As someone in IT, but as a lowly technician, medium and large companies are leaning more and more towards a locked down Microsoft, Google, or Apple platform with managed browsers with extension whitelists, and whitelist allowed OS apps only via MDM.

It's both a tech and management thing. Management likes the control and less risk, and having one console to login to as opposed to 4 makes things easier for IT.

Just a general comment.

Do you support apps that do proper session invalidation (auth cookie can't be reused after the user signs out for example)?

The cases I'm thinking about would more be internal data protections.

I'll give an example:

We have wikis that are internally and externally accessible, with permission systems for internal users and external partner users that carry different restrictions (e.g., VPN concentrator address range restrictions). If someone tries to access a page in the wiki that they don't have access to, the result is the same as if the page doesn't exist. This reduces leakage from link-guessing (I bet there's still a timing side-channel attack). Additionally, if someone builds a page that uses excerpts from pages that they don't have access to, the excerpt will appear blank. This has led to plenty of funny meetings where one party was talking about a status or readout and the rest of the room was deeply confused (due to a lack of access).

This particular wiki is one of dozens of internal tools with similar (but not identical) compartmentalization protections that I use weekly. Unless Plus can safely and securely account for such restrictions, it's going to be a tough sale for us, and limited coverage areas from partial integration would likely leave the tool with usage start-up issues. To some extent this is a classic uncrackable nut, as the most natural approach (integrate with services and systems) isn't entirely under the control of one party. The next left turn is to integrate with popular software/service providers, something they'll resist due to the natural incentive to avoid disintermediation and the high risk of incorporation of other access models.

Maybe in 10 years Plus will have been the source of a comprehensive delclarative permissions modeling system replete with formally verified macro system composition (boil the ocean style), or maybe I'm missing a clever simplifier to address these and other headwinds stemming from business model and tech architecture intersections. Either way, the explainability of the feature and the end-customer simplicity leave me hoping that things work out. It'll definitely be an interesting ride.

An idea, that you’ve probably thought of, but is it possible to do this without allowing others to update the screenshots? Updating would require the user(s) who produced the document to be logged in to the service.

Guess, adding source string at the bottom on Plus screenshots might be at least a partial solution:

> Source: <URL/WebsiteName>, <AccessDate>

Yes?

When you take a screenshot, its contents are known and fixed.

When you regularly and automatically refresh a screenshot of a particular x/y coordinate part of a specific screen on a website, what happens when "positive customer interactions" gets moved, and "internal notes on this customer, not for sharing with them" gets put in that spot?

My guess is this works by uploading your browser's data (cookies, local storage, etc.) to Plus so that it can retake the screenshot with a headless browser even if the web page requires authentication. So if someone hacks Plus or if some employee goes rogue, they can potentially access any web app you are taking screenshots of.

In other words, if Plus becomes popular, its database will become a prime target for hackers and three letter agencies.

That screenshot possibly becomes outdated the second after it’s snapped. This seems to be able to store session data from the user that took the screenshot, and will happily allow anyone access to view it.

The example video shows him taking a screenshot of an Amplitude dashboard using a browser plug-in. Thinking about how that’s possible, it must be skimming the session cookie off the browser, and using it to request the same page on Plus’s side to generate an image. You can imagine how that might be compromising.

Edit: you log in within the plus web app itself, which feels a little better. Still no totally secure way to do this but seems really useful.

A lot of orgs lock down access to cloud storage (eg only company managed OneDrive on company managed devices) to prevent these kinds of data leakage. So they’d block access to this tool too.

Well managed companies will use one approved cloud tool like Onedrive or Gdrive and turn that off in permissions if they want. Just like this site will and should be blocked if it gets popular enough.

One manually taken screenshot vs dozens of screenshot taken automatically