Convert Curl Syntax to Python Requests

Convert Curl Syntax to Python Requests(curl.trillworks.com)

166 points by kapkapkap 6 years ago | 45 comments

NickC_dev 6 years ago |

Author, here. curlconverter gets about 17k unique users per month. The code generated by this tool is probably in production all over the world.

But the codebase needs maintenance! I hired a junior developer (cf512) to do about 40 hours of dev work last month, but that contract has ended and I'd appreciate more volunteers. All the code is open source.

I see a few bug reports down thread. Please open tickets.

A big thanks to the top contributors, csells and jgroom33. Many others have pitched in over the last five years. Also, big thanks to Daniel Stenberg for writing curl in the first place.

amrrs 6 years ago | |

Thank you for making this. Wouldn't hacktoberfest tagging help some visibility for contribution?

098799 6 years ago | |

This tool is amazing. I use it daily. Please don't let it disappear.

kapkapkap 6 years ago |

I'm not the author of this tool, but I submitted it to HN because I just stumbled upon it after unsuccessfully trying to transpose a complicated cURL from Chrome devtools into python requests for a good 20 min. This worked instantly. Props to the author - https://twitter.com/nickc_dev

Wheaties466 6 years ago | |

I've been looking for something like this for a while but I never wanted to spend the time actually searching when I could just spend the time converting it manually. Thanks for submitting this.

vesche 6 years ago |

There's a bit of a hidden feature in Postman that can do this and way more, hit the code button in the right corner and you can convert your web requests to something like a dozen languages: https://i.imgur.com/0qUV8b9.png

whitehouse3 6 years ago | |

Insomnia REST Client converts cURL commands into request sessions too.

girvo 6 years ago | | |

Paw, too

rgovostes 6 years ago |

I've also written a tool that converts curl commands to Python Requests or JavaScript XMLHttpRequest code. It is tested with curl commands generated by Safari, Burp Suite, and Charles Proxy. It can also translate raw HTTP requests, so it can be used when you have a packet capture.

The design is modular and separates the frontend (curl) from the backend (Python), so more input and output formats can be added. It tries to be smart about generating "clean" code, so it will, for example, remove the Content-Length header when it can be recomputed from the request content.

https://ryan.govost.es/http-translator/

It is also open source. Feedback or pull requests are welcome. https://github.com/rgov/http-translator

mycodebreaks 6 years ago |

Is it all client side?

I wonder how many cookies/secret headers will be pasted by unaware and unknowing users.

ken 6 years ago |

This is neat, but also seems like a code smell. Do people really need to convert between HTTP request formats/languages much? When I see many independent implementations of an N-to-M mapping, it looks like the perfect use case for a standardized interface, like a DSL.

In fact, can't HAR do this? I'd love to be able to just pass a HAR string to any HTTP client library, and have it execute that. Or call "dump_har()" (or "--dump-har") on any client, and have it spit out HAR that I can take it to any other client.

eat_veggies 6 years ago | |

When reverse engineering or automating an API my workflow will often look like this:

1. Open browser inspector, watch API requests for one I'm interested in

2. Right click, "Copy as cURL"

3. Paste into a text editor, remove some unnecessary headers, re-running the curl command to verify that the headers I removed aren't important

4. Convert into python requests so that I can play with the parameters

ShakataGaNai 6 years ago | | |

100% this. I do a lot of small automations against various API's (or sometimes against systems that don't API's at all) and have used this same workflow many times.

Also handy because even well documented API's don't have examples in every language, but most do have examples in curl. So you can easily take those example curl's and dump them out to something programatic in your language.

ken 6 years ago | | |

Right, and Firefox and Chrome both have "Copy as HAR". That should save a step, right?

Plus, you probably have a text editor that knows about JSON, so it's easy to (say) remove keys from a JSON file, or even just keep the keys on separate lines and add/remove entire lines. I haven't seen a text editor (even Emacs!) that was as good at editing shell command parameters. It's a lot easier to quote special characters for JSON, too, compared to quoting special characters for my shell.

dmurray 6 years ago | |

HTTP is the standardized interface. If you wrapped it in a prettier DSL, people would still have to take raw requests and figure out how to make them in the DSL.

hkchad 6 years ago |

Postman also has the ability to import a curl command (from chrome dev tool) and then export to many programming languages.

northisup 6 years ago |

I made a similar library! https://github.com/northisup/curlit (sorry for the poor documentation!)

40four 6 years ago |

OK, I'm not going to lie. I didn't even know you could copy requests from the network tab into curl (or fetch/ powershell/ etc.) like that. Embarrassing! :) Seems pretty useful!

Falling3 6 years ago | |

One of my favorite parts is it grabs the cookie as well.

fireattack 6 years ago |

This is especially handy on Windows, where Chrome failed to even produce proper cURL commands in CMD format [1].

[1]: https://bugs.chromium.org/p/chromium/issues/detail?id=658956

[2]: https://bugs.chromium.org/p/chromium/issues/detail?id=798498

_wldu 6 years ago |

Here is a similar site that translates curl to Go:

https://mholt.github.io/curl-to-go/

symmitchry 6 years ago |

Paw, which many people love, has a neat and similar feature: you can get extensions that convert the request you build in the app to any format: curl, python, etc.

It's very handy.

https://paw.cloud/docs/getting-started/code-generator

scotteh 6 years ago |

The "uncurl" library[1] does something similar and can be used in the terminal. I've found it extremely useful.

[1]: https://github.com/spulec/uncurl

m463 6 years ago |

I wrote a subroutine macos python to do the opposite.

On macos, requests is not included (although it can be installed with a little effort), but curl was there.

Note that you can easily automate interacting with a web page using the developer menus in safari and firefox.

monitor the network requests, then for the request you're interested in, use "copy as cURL'. You can copy/paste that to invoke curl in the same way the web page used it.

(sometime I have to use --cookie-jar /tmp/cook --cookie /tmp/cook)

jchw 6 years ago |

Oh wow, not just Python - quire a wide array of languages are supported. Very nice, this will be handy for prototyping things.

keymone 6 years ago |

reminds me of a little clojure macro i wrote to demonstrate "power" of macros in lisp: https://gist.github.com/keymone/d7725767dc0425a7bcf9, obviously not feature-complete but sufficed for the demonstration

daolf 6 years ago |

For those interested in integrating such a feature on their website.

An helper for building API request, for example, there is this great open source project: https://github.com/Kong/apiembed/

It supports a dozen of language.

sikhnerd 6 years ago |

The capability to convert a network request from one format to another is the main reason I use Postman, which has a great implementation of this and a ton of tools and languages it can do it in. This looks great as well

faizshah 6 years ago |

This tool is incredibly useful, I use it at least once a week. Its incredibly useful to explore a site with network in devtools then be able to quickly turn it into a python prototype! Amazing work :)

pramttl 6 years ago |

Great tool, would be nice to also have a chrome extension for the same that augments dev-tools to reduce one extra step of copy/paste.

SEJeff 6 years ago |

Failed entirely to work on my first attempt with:

    curl -XHEAD https://google.com

It is just flat out wrong on:

    curl -X HEAD https://google.com

In that it does a GET request:

    import requests
    
    response = requests.get('https://google.com/')

Clever idea, but my first totally valid example failed (it does OPTIONS, GET, POST, PUT, PATCH successfully)

OJFord 6 years ago | |

`curl --head` (short form: `-I`) is converted correctly though, and `curl -X HEAD` (or `-XHEAD`) does generate warnings telling you to use `--head`/`-I`.

duckmysick 6 years ago | |

In case the creator of this tool doesn't read this thread, you can report the issues on GitHub: https://github.com/NickCarneiro/curlconverter/issues

greglindahl 6 years ago | | |

Even if the creator of a tool does read the thread, discussions like these are not the place to report bugs.

kissgyorgy 6 years ago | |

https://github.com/NickCarneiro/curlconverter/issues/new

agordhandas 6 years ago |

I independently used this tool a couple days ago. Helped save me a lot of debug time!

danielhlockard 6 years ago |

oh hey, I wrote the `Go` support for this tool.

NickC_dev 6 years ago | |

How could I forget, PR #100.

Fyi, Go received about 1200 conversions last month. Thanks!

runnr_az 6 years ago |

hey... that's super handy. thx!

mlevental 6 years ago |

this is one of the most useful little tools on good's green earth. I used to work for a scraping company and I used it all day every day - load a page with inspector open, find your request, right click copy as curl, paste into trillworks and boom you have the request you want to automate (modulo fiddling with cookies). A+ bang/line of code

tzury 6 years ago |

Failed for this one

    Curl -vLkXPOST https://www.google.com

kissgyorgy 6 years ago | |

Because that's not a "curl" command. It's a "Curl" command.

danielhlockard 6 years ago | | |

Also it works if you split out the flags.

empath75 6 years ago |

I feel like going the other way would be more useful.

winstonewert 6 years ago | |

I thought the same thing at first, but the site actually gives an indication of why this would be useful. The developer tools in at least some browsers can export a request as a curl command. So I can thus easily get the code to reproduce any request I see in my browser. That would have been useful a few times in the past, and more useful than the inverse.