Coreutils for Windows(github.com) |
Coreutils for Windows(github.com) |
Remove batch, VBS and switch from powershell 5 to 7, add bash
Replace DSC in favour of Ansible
Remove windows registries
Switch to systemd for services
Rename folders like Users to home, programdata to etc, program files to opt, store/winget apps /usr
and call it microsoft linux server hybrid
https://www.gnu.org/software/coreutils/manual/html_node/dir-...
winget install uutils.coreutils
It’s annoying enough to support the differences between BSD and Linux, and now Linux has GNU and uutils, and now we’re gonna need Windows variant of uutils…ugh.
It seems to be working as intended for uutils.
Microsoft "loves" Linux for years and the entire point was to bring the Linux userspace on the Windows Desktop.
At best, we can make the guess: They chose the license that best fits their interests.
In powershell everything is much better than cmd, but it's just not enough.
WSL is generally great, but there are annoying downsides. I often get "catastrophic" crashes and the zone identifier files drive me nuts. Plus it takes so much longer to start VSCode when connecting with WSL, and now you've got two file systems. WSL1 was in many ways better than WSL2 for these reasons.
"UTF-16 is used by the Windows API, and by many programming environments such as Java and Qt. The variable-length character of UTF-16, combined with the fact that most characters are not variable-length (so variable length is rarely tested), has led to many bugs in software, including in Windows itself.
"UTF-16 is the only encoding (still) allowed on the web that is incompatible with 8-bit ASCII. It has never gained popularity on the web, where it is declared by under 0.004% of public web pages (and even then, the web pages are most likely also using UTF-8). UTF-8, by comparison, gained dominance years ago and accounted for 99% of all web pages by 2025."
You can use UTF-8 on a per-application basis, within limits.
https://learn.microsoft.com/en-us/windows/apps/design/global...
Conversely, UEFI is UTF-16 only, thanks to Windows.
UTF-8 only would be an ABI breaking change, so that's not going to happen. We don't want the NT kernel to end up like Linux, after all :-)
Worse are the byte order marks required to support both endians that end up in files.
Web browsers use UTF-16 internally. So Windows isn't even largest "platform" that uses UTF-16.
an interesting tidbit, some Windows kernel developer realized that most registry keys are ascii anyways so they could save up to 50% space simply by storing the name as ascii. The flag is called "compressed name" and they will pad with 0x00 when reading the name to make a proper utf-16 string.
You cannot ditch CRLF, Microsoft isn't Apple.
Windows accepts backslashes and forward slashes, only old applications that manually search for one of them get it wrong.
Really not possible as most of POSIX semantics arise naturally from the kernel (or are enforced/executed at the kernel level). Windows technically provides some of them (or semantic equivalents) so you could make something work, but in order to do a full port you'd need to strip out too many concepts for it to be worthwhile. For instance the idea that "everything is a file" or the single root filesystem layout (which iirc is segmented deeply at the kernel level).
The kernels undoubtedly take different approaches. But there's nothing in NT that strictly prohibits POSIX compatibility layers. As we see with the many compatibility layers that have existed for Windows over the years.
> For instance the idea that "everything is a file"
POSIX doesn't have a concept of "everything is a file". That's Plan 9. UNIX and POSIX actually made numerous concessions here and there are plenty of constructs that are not exposed as a file.
Windows does already abstract a few primitives as files too. In fact even DOS has the concept of device files, though in typical Microsoft fashion, it's implementation was a clusterfuck that took MS 40 years to fix.
> or the single root filesystem layout (which iirc is segmented deeply at the kernel level).
NT uses an object system for filesystem objects, but it still has a root. As you can see in the WinObj (eg screenshot below)
https://learn.microsoft.com/en-us/sysinternals/downloads/med...
The C:\ convention is really more there for compatibility with DOS-lineage software but the underlying filesystem APIs work fine with NT objects.
For example, C:\foo.txt might be equivalent to \Device\HarddiskVolume3\foo.txt
In this regard, it's similar to how POSIX has a database of inodes and the filesystem hierarchy is actually just an abstraction that sits on top of that
Interix[0] did a pretty good job of this, but MSFT killed it. I was compiling GNU tools w/ GCC and running bash under Interix back in in 2000 under Windows 2000. It was grand.
I only know it post acquisition.
The only places that still forced CRLF were batch files and clipboard.
That has been my experience as well. I can't remember the last time I had an issue related to CRLF.
All software accumulates warts over time. Linux is overflowing with horrible warts and tech debt. As is any software that has successfully served customers for decades.
But multiple line endings are quite possibly the easiest most trivial thing to support and there is absolutely no negative cost of any kind in doing so. Linux ecosystem chooses to be stubborn and provide a strictly worse user experience out of pure spite and for zero user benefit. It’s very irritating.
The Linux ecosystem handles it fine (by using a single standard). Windows doesn't. That's its problem.
Hahahahaha. That's hillarious.
Oh god, you're serious?
Do you have any idea how much of Windows, and user software would break? Any idea at all?
You really want MS, who has built backwards compatibility as a core feature of Windows, to break countless thousands of pieces of software that run on it?
I'm sure there's some idealized fantasy in which that change gets wrapped in a neat little abstraction that prevents anything from breaking. I promise you, there is no way of encapsulating or abstracting that change that works for everyone.
If I could wave a magic wand and make it so without breaking it, I would. But it's a fantasy.
I'd estimate millions
{To the extent such stuff is pragmatic:} I think we should switch to Pascal-style strings everywhere, and then have no need for having special-purpose characters like path separators (a path now being a list of strings).
powershell is good. its much better than unix's everything piped is Text idea. godawfull that. outputs being objects is a really solid take.
WSL is trash.
besides that, lf vs. crlf is silly as you mention but crlf is more logical considering what its implementing. that being said the notion of these control chars is already based on outdated and limited ideas.
if you want a consistent system to do things with dont pick a system which tries to be two systems.
Linux has wine. Windows has WSL.
I'd recommend BSD. any flavor will do.
might take some adjustments but you will have a more 'rational' system if that is what you desire.
(otherwise, embrace the madness!)
Glad I'm not alone here ha!
Being able to go someoutput | Format-Table | Select ColumnName,ColumnName,CloumnName is great. Beats memorizing the output format of any specific command and trying to wrangle it with awk.
Windows needs to ditch itself.
Well this is not very satisfying, what about proving a way where it actually works without us having to guess where the failure root cause happens to be?
So ls in many systems will match the behavior of dir, and only accept the flags for dir. But if you use a system with the newer coreutils release here, ls will expect ls flags!
So ls would actually match the behavior and accept the flags for Get-ChildItem, not dir.
Start a terminal session where they come first in the PATH?
That way one knows where they are getting into.
That was the most plausible reason to even mention it, that I could think of.
The reason seems to be a few windows specific fixes (https://github.com/uutils/coreutils/compare/main...microsoft...) which can probably be upstreamed into the main repo.
Otherwise just don't do it, if it is going to be a mess to work with.
When on Windows, the models default to bash / coreutils conventions until they realize it doesn't work / not available unless explicitly instructed otherwise.
Even on Mac, they tend to default to bash instead of running things in zsh.
I know I could use Powershell for those kinds of tasks, and I certainly do make a lot of use of Powershell, but the familiarity of those simple tools and the decades-old "muscle memory" of using them on various Unix, Linux, and Windows boxes makes them hard to ditch.
The project includes all of those. Or were you talking about the past?
A gigantic idiot. Sorry.
"Note: Any command not mentioned is included in this suite. "
which I found quite confusing. It's a very large set, potentially infinite depending on what the universe of all commands is :)
You should link to the list of all the commands in your package.
More project information: https://gitforwindows.org/
Official download: https://git-scm.com/install/windows
I explained the differences here: https://news.ycombinator.com/item?id=48375716
winget install -e --id frippery.busybox-w32
I didn’t see less or a decent pager. MS needs their analytics on WSL and implement the top 50 commands on powershell
Powershell is very good, but lacks brevity and convenience of coreutils so this should be a big win.
There's a "%SystemRoot%\System32\find.exe" on every Windows NT-derived OS. That's absolutely a conflict.
Also, the "find" command from "findutils" is in no way functionally similar to the "original DOS command" (which is for finding text in files).
Aside: Eschew "find.exe" on Windows for "findstr.exe". The latter is vastly more efficient. I discovered that by happenstance once and have trained my hands to type "findstr" when I mean "find" on Windows.
This is one of the few features that Linux file systems do not have.
WSL2 is great, but native POSIX is even better. Of course it’s a big undertaking, but it makes Windows a first-class dev platform for those who need POSIX in production.
And that would suck.
However, I'm not keen on using yet another Unix clone as well. At least Windows NT brought the world into 90s in the OS state-of-the-art where Unix clones are stuck in 80s and each of them patch around the deficiencies of POSIX. Native asynchronous APIs and truly object-oriented system call infrastructure is nowhere to be found in POSIX.
On Windows it is possible of course, WSL, msys, what not, but it is cumbersome. And I hate the default compiler on windows. So if coreutils on windows helps simplify all the base toolchain, I am all in favour of it. Windows really needs to make compiling stuff a LOT easier by default. I don't want to download some x GB of stuff I don't really need.
The shred command relies on a crucial assumption: that the file system and hardware overwrite data in place.
...
many modern file system designs do not satisfy this assumption. Exceptions include:
...
Log-structured or journaled file systems, such as.
...
NTFS.
If you told me during the Windows 7 era the Windows CLI would not only be getting nice but getting pretty comfortable I would never have believed it.
If they just kicked them out and left the Windows div alone it'd be a decent OS. All the bones are there.
Also, MS does absolutely support EoL versions of their OS - it took MSVC's STL until 2024 to finally drop support for Windows 7 (whose final ESU update was 2023): https://github.com/microsoft/STL/issues/4858 - it isn't unlikely that they (STL maintainers) are not going to get approval for dropping Windows 10 support before 2029.
https://unxutils.sourceforge.net/
Busybox's shell is ash, but the above set includes an old zsh IIRC.
Note also that the Frippery Windows busybox is available as a 64-bit version, in case 2gb is not enough (easy with some big awk associative arrays).
Wine and other compatibility layers show that non-trivial software doesn't work if even one of the many layers uses something unsupported.
https://www.gnu.org/software/coreutils/manual/html_node/ln-i...
If anyone from MS is reading this can we please also get an equivalents (or even alias) for the thing that shows IP address? The windows equivalent of "ip a" is some convoluted PS command that I can never remember
> gip
You could also make your own alias if you specifically want to type "ip a" just add a powershell function to your $PROFILE. function ip { param($argument)...." etc. have it call Get-NetIPAddress, else fallback to ipconfig.
It can be verbose, but the object approach, and having .NET, DLLs and COM as first class primitives is quite flexible.
Naturally you can have a similar experience on UNIX, e.g. fish, but very few do it.
Just do `someoutput | ft ColumnName,ColumnName,CloumnName`
But the rest are in there:
https://github.com/microsoft/coreutils/blob/3fa7aaf832ffc81d...
uutils enables fully Windows native commands in the subset of system calls and disk structure Windows supports.
The project will deny it, but this is clearly an attack against the GPL
But then I realized that the point of this project is to make it easy to write scripts that can be used on multiple OSs... and that is going to make fully-qualified paths possibly a nightmare. Anyone know if these get put at `/bin/`?
Due to Windows's execution model it is not possible to natively implement fork. Cygwin implements those via a executable backdoor hack. Basically the executable starts executing from the usual start point and it receives where to jump via a named pipe. Since MSYS2 is a fork it uses the same implementation.
Unlike Cygwin, MSYS2's goal isn't to be a complete system but ship just enough tooling to enable development with Mingw-w64. Mingw-w64 is the toolchain that ships GCC and it also defines its own ABI where the C ABI is the same as MSVC / Win32 ABI except that Mingw-ABI comes with its own threading and structured exception handling infrastructure. For C++ Mingw-w32 uses Itanium ABI instead of MSVC. If you use GCC to compile the debugging symbols will be DWARF with Mingw-32.
You can also use Clang environments where you can generate PDB debug symbols and use native Win32 threading.
Except for the differences above, the executables targetting Mingw-w64 ABI are normal native Windows executables. They don't have access to most of the unistd.h and they have to use native Windows system calls (kernel32.dll user32.dll, ucrt etc.)
You can target MSYS2 / Cygwin ABI too (just like Bash). In this ABI the entire system works like a POSIX system. Unlike Mingw-w64 backslashes are not native. In MSYS2-ABI programs execute very slowly compared to purely Mingw-w64 executables since they always have to pass the POSIX emulation layer.
I also don't quite get why one would want such a setup - why not just use MSYS2 or WSL? As it is, it's just a mishmash of CMD builtins, Windows utils, Powershell, and these Coreutils. Will one have to use CMD-style (%var%) variables or will it be the POSIX way ($var)? Also just keeping in mind when to use /s or -s style switches, which version gets invoked depending on the PATH, PS aliases, etc. is just a lot of mental overhead for seemingly little advantage over WSL.
Windows NT shipped with three personalities, OS/2, POSIX and Win32.
OS/2 was more of a compatibility thing than anything else, given the OS/2 history between IBM and Microsoft.
POSIX subsystem was half backed implementation, only enough to check some boxes in US government contracts, naturally it never took off. In fact, many people including myself, only never considered GNU/Linux, if Windows NT POSIX support was at the same level as any other UNIX, like it happens on mainframe and micros OS environments, e.g. PASE on z/OS.
Then as Windows actually started to take over UNIX workstations, with Win32 as main subsystem, there was MKS Toolkit as commercial product, Microsoft felt the complaints about the POSIX subsystem, then we had Interix acquisition, which evolved into Windows Services for UNIX, that you mention. Still not without quirks.
WSL took off, because given the experience on OS X, where devs buy Apple expecting GNU/Linux instead of POSIX, Microsoft took the right decision to offer GNU/Linux right from the start instead of yet another POSIX attempt.
By the time WSL came to be, it was already a standard practice to use Virtual Box or VMWare workstation instead of mingw/cygwin, so even better not having to install them.
Ironically I learnt UNIX on Xenix, when it was still sold by Microsoft, maybe they should have kept it.
Because LLMs are trained on UNIX-style shell usage and, in general, much more competent and efficient at using that over PowerShell or Batch/CMD. They want agents to run natively on Windows out of the box instead of through some compatibility layer.
That'd be my guess, at least.
"Yo make some UNIX stuff to show at BUILD as developer tools".
I think you're making a joke, but it still doesn't make sense. Linux does avoid breaking changes to its userspace ABI
You encounter encoding requirements not in kernel system calls but in more user-facing sides of the OS. So your comparison should include all the userspace components of a Linux system (e.g. gettext), if you're considering the places where you encounter actual Unicode string-based operations where UTF-16 comes into play in Windows.
ALL OF THE equivalent Windows libraries (user32.dll, kernel32.dll, ws2_32.dll, shellex.dll ...) to the Linux ones I counted above and more are ABI and API stable.
> We don't want the NT kernel to end up like Linux, after all
(Emphasis mine)
And that's... The one part of the OS where Linux is ABI stable.
I used to support a group of completely tech illiterate users in construction & manufacturing. Them figuring out T:\ was hard enough, ask them to type in a UNC path into the address bar in explorer and you get "Wtf is file explorer? Wtf is an address bar? Where is the backslash key??"
Get the 64-bit version: "there's some advantage in using the 64-bit executable busybox64.exe. In particular, it can be quite a bit faster."
... but they failed to provide a port of Bash - so how exactly do they expect someone to run a bash script in Windows "without translation"? If the answer is WSL, then there's no need to port the coreutils over because WSL distros already include them. If the answer is to port the scripts over to PowerShell, then you wouldn't want to call Windows coreutils in your PowerShell scripts and run into unexpected behaviour (and also lose out on the object manipulation advantages of PowerShell).
And finally, they failed to port over commands that would actually be useful - like dd, for writing ISOs to a flash drive or backing up drives. chroot could've taken advantage of Windows' new sandbox feature to switch to a virtual C: drive. chown could've been an easier alternative to takeown/icacls. chmod could be used to remove the annoying network file blocks and also change file attributes and so on.
This whole project seems like a half-assed attempt at nothing.
A better option is a pure POSIX shell, the best-known of which in Linux is dash, but there is an existing Ada port to Windows here:
https://github.com/AdaCore/gsh
If non-POSIX features are required, the Android decision for mksh might suggest oksh for Windows.
Maybe someone out there mixmashes PowerShell, bash, sh and cmd scripts from different platforms in one session - but usually it's one, quite straightforward 'flow' which requires a quite specific environment.
OSes and modern programming languages have APIs for file navigation for a reason.
On Windows this implies old applications from another era, good old command window, or people that don't know better.
Get-ChildItem /w
And it was never _fully_ implemented, as my post said. The NT kernel doesn't support certain POSIX semantics (fork).
> As an example, the Linux fork() syscall has no direct equivalent call documented for Windows. When a fork system call is made to the Windows Subsystem for Linux, lxcore.sys does some of the initial work to prepare for copying the process. It then calls internal Windows NT kernel APIs to create the process with the correct semantics, and completes copying additional data for the new process.
The MSFT driver prepped something-like-fork and then called the native NtCreateProcess, which does not implement anything like fork().
The SSD controller does not usually keep a history of where older versions of a block of data were stored, so it's not practical to erase an individual file and catch any partial older versions that may not yet have been garbage collected.
It's always funny to see how the fanbois treat their way as the one and only 'True Way'.
Multi-platform is very easy and a solved problem if you try juuuust a tiny amount.
For example the Rust stdlib iterator for lines() handles both conventions. It just works. Very easy.
I live in a cross-platform world. Line endings in text files should not be a breaking problem because some CLI tool refuses to support both. That’s just plain bad software engineering.
I expect Unix tools that process text files to be capable of supporting text files that have different conventions. This is very easy. Refer to previous comments on stubbornness out of spite.
Heck many mistake UNIX with GNU/Linux even, and then complain when given a UNIX that isn't a Linux distro.
cough notepad cough
Just because YOU don't have access to the functionality used for fork, it doesn't mean that there are no internal API calls that can do a fork or do something very similar that it is indistinguishable. A ring-0 driver (like lxcore.sys) that can access a non-standard special process model (pico processes) and is allowed to register its own syscall entry/exit points can also access lower level functionality like accessing the page table of a pico process, modifying it and responding to non-Win32 syscalls.
When you run fork in a WSL1 program now, its effects are the same as doing a fork natively on a Linux kernel: the memory space view is cloned as CoW, a new stack is allocated and the execution is resumed in both processes. If it forks like a duck, it is a duck.
Yes standard Win32 cannot do forks. However, Win32 isn't the only identity NT can expose and NT provides functionality to do forks. Just not to the normal programmers.
Actually a further research reveals that it is very much possible to trigger forking behavior even from the user space. Here is the proof: https://github.com/mobdk/CloneProcess https://stackoverflow.com/questions/985281/what-is-the-close... . You just need to use ZwCreateProcess.
You're right, this is a complete nonissue.
