Show HN: ZFS Implementation in Python(github.com) |
Show HN: ZFS Implementation in Python(github.com) |
1: I say 1.5 because there is only one false value, but it has 2 idiomatic meanings: nil (equivalent to python's None) and the empty list.
Except, it seems this is BSD-licensed, so I'm not sure how that would work in the kernel (which is GPLv2).
But I'm surprised this is possible without a specification - how can you test a filesystem through hexdumps? The effects of some operations are going to pretty far-reaching, surely?
And thanks for the Borges callout.
Original comment: I could swear this was actually the standard practice for writing an implementation of an unknown file format or interface without infringing on copyright. But I don't remember the term for it.
EDIT: Ninjad because I left the reply in a tab without posting.
That PDF says ”Unless otherwise licensed, use of this software is authorized pursuant to the terms of the license found at: http://developers.sun.com/berkeley_license.html”*. That link is broken, but it seems that’s Berkeley license (whatever that means for a specification, and for which variant?)
According to http://open-zfs.org/wiki/Developer_resources, its outdated, but still useful.
I think I would use that, rather than spend months diffing disk images.
This is the greatest thing ever. I wish I could just write code for the fun of it. Every time I wonder whether people will use it and give up before I even get started.
https://github.com/jaysoffian/eap_proxy
This was a really simple project but tickled all my fancies: Python, low-level, networking, reverse engineering, system administration.
Just do it! Who cares if people use it?
Alternatively, contribute something to some open source project you use. I’ve done that too. Just small stuff here and there but that’ll guarantee someone uses your code if that’s what’s important to you. It only takes 39 commits to get on this page:
https://github.com/git/git/graphs/contributors
:-)
The readme literally answers this..
Edit: of course not. This is actually just it’s just a ZFS user-facing ”front-end”, not a ZFS implementation.
This project is cool not because you're going to run the Python in your kernel today, but because someone can use it as a documented reference implementation of all of the data structures and transactions that is not covered by the CDDL, so another implementation based on this can live in the Linux kernel without problem.
Not taking anything away from the work that the author has done though. It's a nice project. I just think a little pragmatism is needed before we get carried away with the ZFS GPL comments.
[1] https://blogs.oracle.com/solaris/zfs-under-gplv2-already-exi...
note that the issues are entirely different from those with a kernel implementation since you aren't having to think about page cache et all.
I know you from ICV - we used to hang out online on the forums and IRC.
Nice work on this project, I'm looking forward to diving into the codebase!
ICV was the name of the forum for the book, now defunct (icodeviruses.com)
https://www.amazon.com/1337-h4x0r-h4ndb00k-tapeworm/dp/06723...
Thanks for sharing though. Maybe could be useful in making a suite of zfs inspection tools?
Is the OP here? How difficult would it be creat a zfs reshaping tool, allowing for the offline expansion of a vdev?
No, but then you run into Go's GC and green threads. File systems fit squarely in the realms of "systems programming" (old definition [1], not new). Languages like Ada, Pascal, C/C++, Rust and D (without GC).
[1] - https://en.wikipedia.org/wiki/System_programming_language
GIL is for safety and correctness, not speed.
Python is only slow if you use it wrong:
A good filesystem implementation requires tight memory management and good control of what happens at the OS level. I am not saying it can't be be done in python, but it clearly isn't the right tool for the job.
I meant that for a production implementation. Python is perfectly fine for a proof of concept, in fact, it may be better than jumping straight down to C. But keeping it for production is foolish IMHO.
Python's global interpreter lock was added for single threaded speed and c library integrations, which often can't be used multithreaded
There was some talk about removing it recentlish to improve pythons multithreaded performance and Guido said something along the lines of
> "I'll remove it as long as single threaded performance doesn't suffer"
Which nobody succeeded in
To be fair, the GC can be disabled. But it's only safe to do so when you know there are no cycles, and even when such guarantee can be had for your own code, I've never seen a library guarantee that to API clients.
This is not true. A FUSE implementation is wanted though: https://github.com/zfsonlinux/zfs/issues/8
Curiously, IronPython did better than anything (but still slow). Haven't tried Jython.
Compiling the whole thing with Cython was less effective than PyPy.
It is a specification with multiple implementations, some of them are even bootstrapped in Java.