Show HN: GPT-4-Vision UX audit for your landing page (relaunch)

Show HN: GPT-4-Vision UX audit for your landing page (relaunch)(flawless.is)

15 points by liorgrossman 2 years ago | 15 comments

Last week, I got seriously fascinated by the new GPT-4 Vision API introduced on OpenAI DevDay, so I decided to build a UX Audit tool with this API.

It's pretty simple to use - just go to https://flawless.is and enter your homepage or landing page URL.

I'm using the urlbox API to get a screenshot of your page, then I utilize GPT-4V to analyze the screenshot, find potential usability and conversion issues, suggest fixes, and highlight the relevant areas in the screenshot (although the position is sometimes inaccurate).

Last week, I launched it on Hacker News (https://news.ycombinator.com/item?id=38207892), but I wasn't ready for the 20,000 visitors that came next... Unfortunately, GPT-4V has a rate limit of 100 events daily, so almost no one could use the service - EPIC FAIL.

To manage demand, I now priced the service at $1.99 (still less than a cup of coffee), which should keep the demand at bay, and also cover the costs of GPT-4V, the screenshot API, and hosting.

As a proof of concept, there are some demo audits on the homepage you can click and check out.

Please let me know your thoughts and suggestions on how to improve it!

imranhou 2 years ago |

The feedback for Slack is a hit and miss, for example it considers the signup buttons too small, or customer logos too small - is there an issue with its perception or mine? Additionally down below it says button color matches background which isn't true, but other sister buttons may be, which is misleading. At the end of this, I think some parts are useful, but not sure by how much when I have to sift through and ignore the other parts....

Is it possible to adjust the prompts to address some of this? or have a second to run validate the output?

liorgrossman 2 years ago | |

All great points, thank you!

I think GPT-4V perception of size might be distorted, since it downsizes everything to be 512px wide before scanning it. I wonder how can this be overcome, I'll try to play with the prompts to instruct it specifically about sizes.

Regarding the "Meet Slack for Enterprise" - you're right, it is weird, looks like it's hallucinating. I'll try and dig deeper.

artur_makly 2 years ago |

Any Senior UX folks here that can review the quality recommendations shown on the audit results posted on this site? A quick glance shows some dubious suggestions. Still better than no UX audit.

potatosalad21 2 years ago | |

UX here with 10 years experience. It's... not great. Mostly suggestions to fix things that are fine with made up reasons, but some advice that's actively bad. For example:

Problem:The FAQ section requires users to click to view answers, which may reduce engagement

Suggestion:Use an accordion with the first item expanded by default to encourage interaction

liorgrossman 2 years ago | |

Yeah, happy to get it roasted by real (human) UX folks for some sanity check.

Broiling3167 2 years ago |

It is not clear what aspect of the payment is "one time", please clarify this!

Is it per-domain or per-email?

Rerunning the same domain takes me to checkout again, and I didn't receive an email so I'm lost.

liorgrossman 2 years ago | |

Thanks for giving it a try.

One time meaning you are charged $1.99 one time, and you can audit one URL, the one you've chosen on the main page.

After you pay, it takes you to a unique URL with a token. Might be a good idea to send users email with this URL...

There's no concept of logging in or logging out right now, that's why that URL is important. It looks something like this: https://flawless.is/audit?url=https%3A%2F%2Fslack.com%2F&t=1...

If you can't find this URL in your browser history, please ping me at lior at lifehacklabs.org, and I'll restore this URL and send it to you!

Broiling3167 2 years ago | | |

Thank you, I did find that url now.

I’m still confused though, let’s say I “fix” all the problems it gave me, and afterwards if I want to run it again with those fixes, I will need to pay again?

user_named 2 years ago |

I paid for this and tried it on one of our main pages.

It did identify one point that is correct and that I have brought up with the team previously, and we are currently fixing it.

So either points like fine print being too small I think we're not correct and it doesn't understand the use of such a section.

Also the screenshot is of the page not loaded correctly.

liorgrossman 2 years ago | |

Thanks for being one of our early adopters!

Glad to hear it did find something that you can correct.

Regarding the points about fine print (and the screenshot not loaded correctly) - would you kindly reply with the URL of the website you audited here, or email it to me to feedback at flawless.is ?

I'd love to take a look and see if these issues could be fixed or improved.

11217mackem 2 years ago |

is the prompt any more sophisticated than? "do a ux audit on this page with the format "problem" and "solution" with tags messaging, cro, usability, marketing, etc"

or rather, significantly more?

this seems super, super simple.

liorgrossman 2 years ago | |

Well, it started off this way, but I went through several iterations with the audit prompt and the system prompt to improve the issue detection, the solutions suggested, tag them, reduce hallucinations, provide approximate image coordinates, provide examples to the model, etc. It's currently at over 20 lines of prompting and I guess this will grow over time.