Show HN: Sultan – Pythonic interface to your shell

rekwah 8 years ago |

I wrote a pretty significant process wrapper in python at $PREVIOUS_JOB. The problem I have with these loose wrappers around subprocess is that they're, imho, solving the wrong problem. Or, maybe better put, not solving _enough_ of the problem.

Imagine writing a wrapper for "grep". It may work in one environment, but if you need portability, you're going to quickly realize that there are nuances in implementation and the actual calling interface. Now you're handling branching logic in your application code based on platform, version, etc.

Interfacing with a command line tool shouldn't be thought of much different than a remote HTTP/ReST API talking text/plain instead of application/json. You're looking for your "client wrapper" to handle argument validation, parsing, versioning, etc.

marmaduke 8 years ago | |

> much different than a remote HTTP/ReST API

does that make subprocess module like urllib, and these wrappers like the ubiquitous requests package?

I think it's harder than you say: a given tool like grep has an API the size of an entire website, and then depending on usage, has different usage patterns which different optimizations.

Also, a REST API user is consuming an API, whereas a CLI wrapper is consuming an API to implement another. Much bigger fish to fry there, I think.

aeroxis 8 years ago | |

You make a very good point. Sultan, at the end of the day, is a syntactic sugar for Subprocess. It has a couple of doo-dads for remote SSH execution, and things like that.

I originally wanted to have Sultan to do exactly what you're saying, but when I started diving into it, it seemed like a task that was very tough to solve. For example, grep's syntax remains similar between OS, but it has some API changes between versions. Trying to figure out what version has what syntax seems a bit daunting, and perhaps impossible to do with all commands.

The usecase that Sultan really caters to is the developer who writes their code on one OS, and keeps that code in the same OS across different environments. Not everyone wants their code to run across different OS and Distributions. I felt like it was a special case to do this, and decided to keep Sultan as generic as possible and let developers build the logic for different OS and Distributions on top of it.

The problem Sultan does solve is code maintainability, reusability within their own project (which assumes the project runs on 1 OS), and testability (unit and integration tests can be written, while you can't do that in Bash easily).

hultner 8 years ago |

Very nice! I've been using somewhat similar sh[1] for the same purpose. It's nice seeing more alternatives.

[1]https://amoffat.github.io/sh/

git-pull 8 years ago | |

Two more resources on top of sh:

Invoke: http://www.pyinvoke.org/ wraps subprocess commands into Makefile-like tasks

Fabric: http://www.fabfile.org/ Same as invoke, but catered to remote subprocess commands over SSH

zwischenzug 8 years ago | | |

This was featured on HN the other day:

https://learnxinyminutes.com/docs/shutit/

https://news.ycombinator.com/item?id=14526567

hultner 8 years ago | | |

Very nice, as a side note I must say that I as an avid tmux-user much appreciated your "The Tao of tmux" piece!

onyb 8 years ago |

This seems to be a simple wrapper around subprocess, but I'm afraid it is not changing things dramatically. Overall, I don't agree that Sultan's syntax is any more Pythonic than subprocess itself.

+1 for https://sultan.readthedocs.io/en/latest/sultan-examples.html...

I think it would be interesting if you could iterate on the results of "ls -l", where each row is represented by an object.

aeroxis 8 years ago | |

Representing each row as an object works, but it ends up having us write custom objects for different result types. When you run Sultan().ls('-l'), you will get back a list of Strings, which you can freely use in Python for your specific needs (like use a Regex to fetch date/times or file size). I felt like this offers developers the best flexibility, rather than trying to customize each output (which will definitely make the codebase very confusing)

friendzis 8 years ago | |

And then I remember that Windows PowerShell does (attempts to, to be fair) exactly that

stinos 8 years ago | | |

attempts to, to be fair

Output of ls in PS is an Array of objects (DirectoryInfo or FileInfo instances). How is that merely an attempt?

friendzis 8 years ago | | |

While PS-native stuff yield objects, not all pre-PS utilities/builtins are overriden or have interface for PS and output old fashioned text blob, which is not automagically parsed in any way by PS. My comment was system-level, not utility-level.

asdfgadsfgasfdg 8 years ago |

How does

      s(command_from_potentially_unsafe_source).run()

compare with

      os.system(command_from_potentially_unsafe_source)

or

      subprocess.call(command_from_potentially_unsafe_source, shell=True)

securitywise? I'm assuming all three are equally bad?

AkshatM 8 years ago | |

I looked at the source code. Under the hood, it's just a thin wrapper around `subprocess`.

asdfgadsfgasfdg 8 years ago | | |

Cool subprocess is a little verbose. But did you happen to notice if it executes with shell=True or shell=False?

mintplant 8 years ago | | |

Do you include Python 3.5's subprocess.run in that assessment? I find it much cleaner for the common cases than the rest of the API:

https://docs.python.org/3/library/subprocess.html#using-the-...

asdfgadsfgasfdg 8 years ago | | |

I hadn't actually noticed that! It looks like .run() allows input and output to be piped and the return code captured. That is a serious improvement over subprocess.call().

icebraining 8 years ago | | |

Doesn't seem that different from call()

asdfgadsfgasfdg 8 years ago | | |

It looks like it allows stdin, stdout, stderr and return code to be handled properly which call() doesn't. It's still syntactic sugar over Popen() and communicate() but will cut down boilerplate for calling external processes.

Cynddl 8 years ago |

There is also pyinfra [1], which provides the same sort of features for both local and remote devices, yet designed for consistent deployments and state diffing.

[1] https://github.com/Fizzadar/pyinfra

gerdesj 8 years ago |

When I dabble with Python to get a job done, I invariably end up using subprocess at some stage and going through some contortions. I like the look of Sultan, it seems well thought out and has loads of well written docs.

Anyone know how the name came about?

aeroxis 8 years ago | |

Since I was creating a tool that works with Commands, I thought of the tag line "Command and Rule Your Shell", and I thought of different types of rulers (Kings, Shahs, Czars, Sultans, etc.). The Sultan name seemed to fit the best.

I was also trying to grow a Handlebar mustache at the time, and I wanted my logo to have a Handlebar Mustache, so that's why the logo has a Handlebar Mustache :-)

abdulkareemsn 8 years ago | |

Sultan means king If you see the tag line for project

Eridrus 8 years ago |

Since we're throwing out random shell related tools, I found pexpect recently: https://pexpect.readthedocs.io/en/stable/api/replwrap.html

Which was super useful for controlling some utilities that wanted to be run in an interactive bash shell.

fermigier 8 years ago |

I've been using 'tentakel' for remote server admin for years, 'fabric' for lightweight deployment, and more recently 'invoke' for local invocations (all are Python projects).

This project looks interesting too, will give it a try.

BTW: Has anyone news on Fabric 2 ?

niothiel 8 years ago | |

Would love to know this as well.

vram22 8 years ago |

Here's a comment I made some time ago on HN about similar / related Python libs / tools:

https://news.ycombinator.com/item?id=8735892#8737512

Edit: That comment was part of an HN thead about this topic, which might be interesting in its own right:

Streem – a new programming language from Matz (github.com)

https://news.ycombinator.com/item?id=8735892

santiagobasulto 8 years ago |

How's this different from sh or fabric?

brett40324 8 years ago |

Ive written numerous python wrappers around shell commands for all sorts of one off utils, but this is nice and flexible and generic. Sultan looks well thought out. Open to contributors?

aeroxis 8 years ago | |

Sultan is open for contribution. Feel free to put your thoughts in the Issues for features you'd like, and submit any Pull Requests for any issues that already exist.

llccbb 8 years ago |

Seems like there is lots of talk here about how this is thin wrapper over `subprocess`, but I like the idea. What I don't like is that the logo looks like a penis with a mustache.

aeroxis 8 years ago | |

Haha! It does, doesn't it! Once you see it, you can't unsee it!

StavrosK 8 years ago | |

It's a turban, jeez.

jklehm 8 years ago |

Seems similar to the package sh/pbs. [0]

[0] https://amoffat.github.io/sh/

aeroxis 8 years ago | |

Yes it is. I looked at SH before writing Sultan. I was not very crazy about the syntax. I wanted the code to be easy to read, because one of my primary goals was the code maintainability, and that requires the code to be as readable as possible.

rntksi 8 years ago |

Trying very hard to ignore it, but to be honest... The logo looks like a smiling phallus.

falsedan 8 years ago |

Why not plumbum?

(I know why not to use plumbum)

codewithcheese 8 years ago | |

Thanks didn't know about plumbum.

What do you know?

falsedan 8 years ago | | |

bash is the best tool to write bash in, and writing a shell in python doesn't automatically make it pythonic.

singularity2001 8 years ago |

No example even if you click on FAQ … what is sultan.

majewsky 8 years ago | |

Huh? It has examples on the first page, the one that's linked in the submission.

kvajjha 8 years ago |

>Bash, while it seems arcaine, actually is quiet powerful!

This line triggered me.

aeroxis 8 years ago | |

:-) I'm glad! I love Bash, and I think it is a great tool, but I always end up looking up the Syntax, which is what inspired me to write Sultan.

pknerd 8 years ago |

how is it different from Fabric?

Nice stuff though

aeroxis 8 years ago | |

It is actually inspired by Fabric, but I was not very happy just using Strings. I also wanted better context management.

Fabric is great for DevOps, but there are applications that we build that aren't catered for DevOps work, and that is why I created Sultan.

fnj 8 years ago |

Xonsh is how to do this.

noobiemcfoob 8 years ago | |

Xonsh is a shell that integrates Python at the command line. This is a library to make calls to the command line.

fnj 8 years ago | | |

And your point is??

gerdesj 8 years ago | | |

At the risk of simply putting words into noobiemcfoob's mouth:

"Xonsh is " NOT " how to do this."

fnj 8 years ago | | |

OK. I thought this would be so obvious that people would see it. Sultan says "Sultan was created to help with scripts that we create with Bash, that tend to get complex." If one has an open mind, one should be asking "why would I use bash for this at all, when there exists a far superior command shell, xonsh, which incorporates all of bash's capability, with all of python, in one piece?"

Why are we trying to interface with bash to run commands from python in the first place? Cut out the middleman, bash, which is just a nuisance in the way.

michaelmrose 8 years ago | | |

You don't need to interface with bash necessarily you need to interface with binaries on the system like apt for example

jjuel 8 years ago | | |

The point would be that Xonsh is a Python command line. This is a Python library for running commands. So they definitely have two separate use cases.