Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website. |
Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website. |
The result contains headers like content-length and similar.
Also it would be nice if it could factor out common schemas.
I mean, and I'm asking as a backend dev, if you have to integrate with some API, you use the provided docs/swagger ui.
Why/when would you care to rely on an API integration when it's interface is not publicly shared?
What would be particularly useful is if it saved token values and then (through search) joined them on the response of the auth call to get the initial token.
That way you could easily determine what auth call was needed to get you a token to use the endpoint.
For instance, I had GPT help me write browser JS that groks literally thousands of IP addresses in an open security tool that shall not be named. I can vacuum much of their entire database in seconds by making hundreds of async calls. While they do have bot protection on the website, they appear to have no protection at all on their browser APIs once the user has been given a cookie… I suspect this is common.
I used https://vite-plugin-web-extension.aklinker1.io/guide/ before to have cross browser extension support. If you don't mind I could take a look to add firefox support (no guarantee)
So if you can pass the equivalent of that in Firefox/other browsers to the insert method and switch things up a bit, it should be relatively straightforward. I will think about pulling out the core logic into its own lib.
https://developer.chrome.com/docs/extensions/reference/devto...
https://developer.chrome.com/docs/extensions/reference/devto...
https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
Their type definition for HAR request isn't exported https://github.com/DefinitelyTyped/DefinitelyTyped/blob/mast...
So I can't drop in replace the type on https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
Would love a "copy to python requests" button that
grabs the headers
generates a boilerplate python snippet including the headers and the URL:
import requests
import json
url = '<endpoint>'
headers = {
'User-Agent': 'Mozilla/5.0 ...',
...
}
data = {
"page": 5,
"size": 28
...
}
response = requests.post(url, headers=headers, data=json.dumps(data))
if response.status_code == 200:
print(response.json())
else:
print(f"Error {response.status_code}: {response.text}")https://github.com/alufers/mitmproxy2swagger
However, having the capability delivered in a browser extension is extremely handy!
Thanks for sharing this, I suspect this is going to be super useful for my work
Let me know if you have any questions or feature request
> Akita makes monitoring and observing system behavior accessible for every developer. Quickly discover all your endpoints, see which are slowest, and learn which have errors
Translation: Install a Docker extension that intercepts and inspects your network requests to infer the shape of your API.
I feel like when you're targeting developers, you should quickly explain what it is you actually do.
I'll second/third the feature request for auto-including auth headers/calls (as many of the sites I'm trying to understand/use APIs from use persistent keys, and scraping these separately is just unnecessary extra time).
On that same note, I'd greatly appreciate keeping the initial request as a "sample request" within the spec.
I'd also greatly appreciate an option to attempt to automatically scrape for required fields (e.g. try removing each query variable one at a time, look for errors, document them).
Thanks for this :)
Dreams never die and what is old will be new again.
1. Ability to filter response properties.
2. Ability to work with non-JSON (web scraping) by defining a mapping of CSS selectors to response properties.
3. Cross-reference host names of captured requests with publicly documented APIs.
4. If auth headers are found, prompt user for credentials that can then be stored locally.
5. "Repeater" similarly found in Burp Suite.
6. Generate clients on the fly based on the generated OpenAPI spec.
- Integration with some kind of web crawler to allow automatically walking a web site and extract a database of specifications
Edit: Hmm, it seems that genson-js[1] was used to merge schemas.
The idea for a crawler is a good one. The core logic that handles spec generation is decoupled from everything else, so it can be extracted into its own library.
But there are approaches that exist for this already, such as har-to-openapi.
8. Optionally publish generated OpenAPI specs to a central site or open PR to a GH repo, "awesome-openapi-devtools"?
It was always my step 1 towards Xxx. Keen to know what directions you were thinking?
I’d love to see more remixing on top of API’s websites typically only expose for their own use.
It's amazing to see a tool that simplifies the process of generating OpenAPI spec. this is the best showHN this year.
Suppose I have a site that runs a search that I want to be able to automate. However, instead of sending the search term in the URL, it updates live (presumably via some API call).
Now suppose I need a one-click solution to be able to open that page and run a specific search.
Is there another Chrome plugin that would allow me to use this API data to make that happen?
Sometimes I click on a path parameter and it doesn't "create" it, even though there are several other examples in the list. Not sure if it's a bug, or something I'm doing wrong.
Overall, this is an absolutely wonderful tool and I've wanted something like this for a long time. Incredibly useful, thank you!!
I have a lot of ideas in this space (some PoCs), and I've been starting to scope out a company around them. Would love to chat to see if there's any shared opportunity for both of us!
The spec produced will be incomplete (missing paths, methods, response variants, statuses). For that you should use a framework like Fastify, NestJS, tsoa, FastAPI, which have built-in OpenAPI support.
Can be very valuable for reverse-engineering though :)
so I can be typing in the URL bar for any website I have landed on in the past and tab through all the available routes?
e.g.
- news.ycombinator.com_
- news.ycombinator.com/new
- news.ycombinator.com/submit
- news.ycombinator.com/show
etc.
I assume it monitors all XHR requests as you browse a website, and if the request/response matches [some criteria (e.g. is JSON?)] it will assume it's an API request and log it?
Is that correct?
If so, it will only work on websites where the frontend is implemented like a PWA, with lots of AJAX calls to fetch data, etc. For sites whose pages are all generated server-side, the extension won't generate any API schema, right?
Edit: Also how does it differentiate "API requests" with regular AJAX content fetching? If a website fetches some arbitrary content via an AJAX request (e.g. some lazy-loaded HTML), that's not an API request. That's just part of a website's layout.
"Using" could mean navigating between pages, submitting data via forms, etc.
> Instantly generate an OpenAPI 3.1 specification for any website or application just by using it
I want to know what this extension does that's different than me looking at the browser's Dev Tools > Network tab.
(You can also use it to generate automated tests)
If you're interested: mish@stepci.com
I wanted to draw your attention to "normal" POST application/x-www-form-urlencoded <https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/vers...> and its multipart/form-data friend <https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/vers...>
The latter is likely problematic, but the former is in wide use still, including, strangely enough, the AWS API, although some of their newer services do have an application/json protocol
I know that's a lot of words, but the tl;dr would be that if you want your extension to be application/json only, then changing the description to say "OpenAPI specifications for application/json handshakes" would help the consumer be on the same page with your goals
- open the network console
- right click on the request
- click "copy as curl"
- visit https://curlconverter.com/ to convert to Python/Node/any language
I was just trying to save a few clicks
2. requests can dump to JSON for you by using json=, so you don't need a separate module. It'll even set the content-type header to application/json for you.
import requests
url = '<endpoint>'
headers = {
'User-Agent': 'Mozilla/5.0 ...',
...
}
session = requests.Session()
session.headers.update(headers)
data = {
"page": 5,
"size": 28
...
}
response = session.post(url, json=data)
if response.status_code == 200:
print(response.json())
else:
print(f"Error {response.status_code}: {response.text}")awesome-test-automation/python-test-automation.md lists a number of ways to wrap selenium/webdriver and also playwright: https://github.com/atinfo/awesome-test-automation/blob/maste...
vcr.py, playback, and rr do [HTTP,] test recording and playback. httprunner can record and replay HAR. DevTools can save http requests and responses to HAR files.
awesome-web-archiving lists a number of tools that work with WARC; but only har2warc: https://github.com/iipc/awesome-web-archiving/blob/main/READ...
https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
https://github.com/curlconverter/curlconverter/issues/64#iss...
This is why feature bloat is a thing
My favorite is when they think they're keeping it short and to the point, with no bull. So, they'll have a hero section with copy like "Sharpen capacity. Scale across segments. Nuff said." No, not enough said, say more!
This seems to appeal to purchasing teams. When you write what the app actually does suddenly it’s technical and the team doesn’t understand what is written any more.