SCIM: Ncurses based, Vim-like spreadsheet(github.com) |
SCIM: Ncurses based, Vim-like spreadsheet(github.com) |
It's saved my ass on multiple occasions for data wrangling and munging and highly recommend people use it in their own toolkit.
I am not fond of the the usual spreadsheet data model. "It's a big bag of cell" does not fill me with joy. And upon a bit of reflection I think it is the rows, I really hate how easy it is for rows to get out of sync. All I really want is row security.* And this is what visidata brings to the table.
* Relational databases provide this in spades. and in truth most of my spreadsheets have been replaced by them(I maintain a local postgres server on my desktop for all the small random prototype junk you would usually do in a spreadsheet using visidata as a pager) And while the database is great for analysis, random data entry sort of sucks. There are some great tools out there for this. I don't tend to use them, mainly because psql is always there, so I just sort of grumble when I have to do random entry without trying to fix anything. Why postgres instead of sqlite? I like the types and functions better.
For handling structured data, spreadsheets can do it, but they're not always going to be the best tool for the job, as they do a lot of other things, too, which means if you're not careful, your data will lose its appropriate structure.
Submitted it here in case people want to comment about the video:
It has support for the Scheme programming language, MySQL, sending email (was jwz on this project?), and being part of a whole office suite.
SIAG Office:
(I'm kinda surprised. I remember when this was just that odd little program that shipped with Damn Small Linux.)
It can even load multiple file formats, including LaTeX, troff, and HTML tables:
The terminal tools have gotten so much better in the last few years. There's a real Renaissance happening.
Now we are doing the same thing to the terminal.
I'd rather have a poor UI that works than a flashy one that breaks, which is what we'll get in 10 years. Just like with regular UIs.
https://www.google.com/search?q=640kb+ought+to+be+enough+for...
(They patented an innovation called hardware attached to software, instead of the traditional software attached to hardware.)
That should easily suffice for many instances of bloatware, er, Electron apps running in tmux tabs.
Hybrid apps FTW!
Because really, do you want all of vim in sheets, or just navigation (`i/h/j/k/gg/G/^u/^d`) and selection (`v/V`)? It has some other basic stuff, like `dd` and `o/O`, but otherwise conflicts with browser and Google functionality keep me away.
If you’re looking just for spreadsheets, Travis Ormandy somehow managed to get Lotus 1-2-3 to run on Linux a few years ago[1]. It’s a neat comparison point.
[0] https://www.visidata.org/ [1] https://lock.cmpxchg8b.com/linux123.html
PS: Jeez, so much ads in Byte Magazine! Or is it because I haven't open a magazine in a couple of decades and forgot how cluttered they were??
In a spreadsheet, I'm used to being able to move around with arrow keys and start typing immediately. Using SCIM, it feels like I'm constantly hitting a wall.
Despite that, I think the idea of a spreadsheet as a TUI is really great.
It would be fine if I'm in insert/edit mode and I can move around entering values in several cells and then press escape to exit that mode.
The reason I think TUIs are attractive to use is because they're more efficient to use. But this one doesn't feel more efficient to use than its GUI counterparts.
> sc-im is based on sc, whose original authors are James Gosling and Mark Weiser, and mods were later added by Chuck Martin.
Plain and simple C, etc. I would have liked a one compilation unit with proper preprocessor namespaces/name mangling, that to be picky.
A far cry from the world of the GUI, but a welcome world
In my opinion: if you can use vim, you can probably code, or at least figure it out without too much trouble. If you can code, then you don't need a spreadsheet. You can just write a program to crunch the numbers, or produce a report etc.
Excel is so popular, because it is a way for non-coders to crunch a bunch of numbers in a relatively easy way. And the best way to get the answers that they are getting out of the spreadsheet is to write code. But because they can't code, they have to use a spreadsheet.
If there is a use case for spreadsheets that is not better served by some real code, I'm interested to hear what it is.
You could also make the "speed" argument (just a quick calculation) for spreadsheets, but in that case, I find something like a python REPL just as quick, and still better anyway.
But this is pretty cool
One nice thing is that Emacs has calc-mode built in that gives you all kinds of advanced math capability you can use. The tables in org-mode support this directly, but since calc has a Lisp interface you can use it pretty much anywhere.
It is more keyboard driven than Excel and felt a bit more "productive", far less focus on designing the spreadsheet.
You use Excel because you have to, as part of your job. But I used this for some smaller things and it worked reasonably well.
open /Applications/Microsoft Excel.app/I think it's neat for terminal usage, though.
Other problem: I can vim but I prefer the emacs keybindings.
And supports plugins in a much nicer language than dBase.
Now if it only supported easy form and grid-based UI builder, it would also be comparable to Clipper :)
Might affect searchability.
that, and there aren't any "ui/ux designers" specialising in cursor-addressing uis.
Depends where you look.
https://davideellis.wordpress.com/2012/08/31/ibm-tivoli-moni...
It sounds like "scim" is to "sc" vaguely like "vim" is to "vi": new program with more features cloning/imitating ancient program.
"vi" was written by Bill Joy in 1979.
"sc" by James Gosling in 1981.
sc-im claims to be based on "sc".
It's a direct lineage unrelated to GUI spreadsheets.
For those who don't know, James Gosling invented a popular VM-based "write once, test everywhere" programming language named after a tree. Then named after a coffee.
https://forum.winworldpc.com/discussion/7590/software-spotli...
Better yet, make all user interfaces the same as a toaster. Everyone can use a toaster. Bread goes in, push the lever. One universal way of thinking for everyone and everything. No domination by the tyranny of choice.
And if that requires any tradeoffs like it did for GUIs (no hover, no small elements) it'll end up getting dumbed down for mobile like GUIs did.
While vi might be a code editor first and foremost, not all vi users are coders. There are copy writers, academic and literature, having a need for fast and focussed touch typing (George R. R. Martin comes to mind as prominent WordStar user). The entire point of SGML/XML/HTML markup is to be able to create rich text documents without binary formats and special editors; this is also the case with Wiki syntaxes like markdown, which have been around since long before John Gruber's original Markdown.PL and are directly supported as a shortref customisation in SGML, from 1986, BTW.
Conversely, even if you are a coder, classic spreadsheets are extremely useful for any type of ad-hoc reproducible calculation (such as for taxes or other personal or business finance stuff). You really should check out spreadsheets if you haven't already; the point is that you can cross-reference cell values and copy/paste with relative cell positions to create large calculation tables/matrices, then update base values and perform "what-if" analyses, etc. etc. Using cell formulas is more like a logical programming language environment. I've used it for all kinds of reports apart from financials (benchmarks, construction/project planning, even a Tic Tac Toe game in school out of boredom).
> Conversely, even if you are a coder, classic spreadsheets are extremely useful for any type of ad-hoc reproducible calculation
Again, I still feel like code is the ideal solution to "ad-hoc reproducible calculations"
> the point is that you can cross-reference cell values and copy/paste with relative cell positions to create large calculation tables/matrices, then update base values and perform "what-if" analyses, etc. etc.
I still don't see how you can't do that with code, nor what the spreadsheet is doing that code can't.
> I've used it for all kinds of reports apart from financials (benchmarks, construction/project planning, even a Tic Tac Toe game in school out of boredom).
Sure. It has been proven that excel is Turing complete. But I'd rather use a programming language (a tool that was literally designed for the purpose of writing code), than a clunky spreadsheet. Both can get the job done, I never denied that. But I still don't see the value that the spreadsheet brings over custom code written to solve the exact same problem.
Multi-country taxes are too much fun. Every dollar/GBP amount needs to be converted to the other currency for taxes in that country.
I originally did this in libre office but I got annoyed at it and wrote a markdown pipeline to produce PDFs for my accountants.
I would do data entry in CSV and wrote a CSV to markdown converter. Along the way I wrote a simple CSV formula language with just a couple of functions that would do column level operations e.g. =MUL(C, E) to multiply columns C and E.
This worked pretty well, and I could make a small directory of sorted markdown files to assemble, and a Makefile to transform the CSV+formula files to flat CSVs.
But CSV input was kind of annoying and my formula language wasn't easy to extend, or very nice. So I jumped at sc-im which can directly output markdown tables.
Anyway, I highly recommend sc-im, the .sc files are a fine replacement for my custom solution and I haven't looked back (and taxes are coming again soon!)
> If there is a use case for spreadsheets that is not better served by some real code, I'm interested to hear what it is.
Sharing information with non-coders could be an obvious one. I could have done a database with my wife's sewing patterns collection, of which she has ~300. Instead, I did a spreadsheet in google docs, which she's very familiar with. Told her how to enter data there (4 fields, name, file, tags and picture) and there's a couple tabs that allow her to filter things out, etc. Then she can do whatever she wants with it, like using conditional formatting to make dress patterns red. It was done in 2-3 hours, and she got exactly what she wanted.
I worked in a place where both the input and output was spreadsheets, and the users were spreadsheet users. We did implement a database for this particular one for the heavy algorithmic part, but a lot of the business logic (e.g. initial data validation, final presentation of the results) lived on the spreadsheets themselves.
Another interesting one is data-to-visual time. Unless you happen to be proficient in a particular area of programming (e.g. front programming with proficiency in something like D3, or R programmer) getting decent graphs out of data is a chore when going the programming route. With spreadsheets, you put the columns and get the graphs essentially for free.
To me spreadsheets are just another tool in the toolbox; they are appropriate for some tasks. They can definitely be misused and abused. Knowing which occasion is which is where experience comes in.
You can do the exact same thing with code. You can still output reports and diagrams, and literally anything the "customer/user" wants.
> I could have done a database with my wife's sewing patterns collection...
Your example is a good one. I didn't think of images.
> It was done in 2-3 hours, and she got exactly what she wanted.
Fair enough. A spreadsheet was probably the right tool for the job in this case. But if she ever wanted more features, and the complexity increased, I don't know if it still would be.
> I worked in a place where both the input and output was spreadsheets, and the users were spreadsheet users.
Okay, but should they have been? Would custom software not have been the better solution?
> a lot of the business logic (e.g. initial data validation, final presentation of the results) lived on the spreadsheets themselves.
When I think of "business logic", "data validation", and "final presentation", a spreadsheet is one of the last tools I'd reach for.
> Another interesting one is data-to-visual time. Unless you happen to be proficient in a particular area of programming (e.g. front programming with proficiency in something like D3, or R programmer) getting decent graphs out of data is a chore when going the programming route. With spreadsheets, you put the columns and get the graphs essentially for free.
I also disagree with this. I am very much a back-end developer. But using something like GNUplot or a library like matplotlib is pretty easy for outputting a nice looking graph from tabular data.
> To me spreadsheets are just another tool in the toolbox; they are appropriate for some tasks. They can definitely be misused and abused. Knowing which occasion is which is where experience comes in.
I agree with this. But I guess the difference is I can think of almost no circumstances where it's the better tool for a coder.
And, btw, if you search for "distraction free writing",you will find that even some non-coders use vim.
"Scientists rename human genes to stop Microsoft Excel from misreading them as dates" (2023) <https://www.theverge.com/2020/8/6/21355674/human-genes-renam...>
Or anyone else for whom Excel's automatic data-munging "features" are a concern.
I've never really had a problem entering data into CSV file or database, but I will concede that it is even easier to enter the data via a spreadsheet.
However, I still think custom software (with proper UI for input if necessary) is the better way of dealing with the actual solution that a spreadsheet is ultimately trying to solve.
Think of spreadsheets as a convention-over-configuration low-code environment with a tightly coupled GUI for spinning up run-almost-anywhere apps that non-coders can modify. One of the few low-code environments that I actually like.
I'm a solopreneur whose SaaS product I coded from scratch. I love GPPLs, but I also create spreadsheets all the time.
Sometimes that's because of the stage of an idea: There's a new process you've discovered you need, but you don't have time to invest in building/buying/learning a tailor-made solution.
A case in point is a business overview dashboard I built to keep an eye on the metrics I care about. It pulls data from disparate sources using Power Query, which is built into Excel and can pull from databases, APIs, CSVs, etc. There's no hosting infra and no new monthly fee as I already had the Excel license.
Another nice thing about spreadsheets is that today, they're multi-user and real-time collaborative. You can send someone a link and both edit an Excel or Google Docs spreadsheet in the browser with very low ceremony. And if they're on a power user, they can modify the formulas themselves. That's a bad thing for some use cases but a truly great thing for others.
The ubiquity, the local-first option, the tightly coupled GUI, the widely known syntax...those things make spreadsheets very attractive for certain types of projects.
For others, building an app is the right answer. They both have a place.
While you can solve a lot of problems with code, code might not be the best way to solve some problems.
I want to type in some data, mix it up, explore, maybe make a quick graph, get some stats, decide if I need to make more calculations. By the time you decide on what tools to use and run your pip install or whatever, I'd be long done.
Conversely, I have seen spreadsheets used for a lot of things that they shouldn't be.
but while this is not for me (no interest in learning vim), i'm pretty sure many other people will find this useful
This is a poor man's excuse for not liking excel.
I know. Most people don't know how to code, thus they are forced to use spreadsheets as I already mentioned.
> spreadsheets have a lower threshold to pure data
I'm not sure what you mean here. That it is easier for the average human to work with data using a spreadsheet compared to code?
Sure, as already mentioned. Most people don't know how to code.
But what I'm getting at is that a spreadsheet is the poor mans code.
And if you audience is coders (not the whole world), then why go for the worse option?
Lotus was also text based.
open -b com.Microsoft.Excel
Disadvantage: autocomplete doesn’t work on that in the terminal.
The files /Applications/Microsoft and /Users/lsllc/Excel.app do not exist.
/s
[0] To escape the space, or you could quote the entire argument to `open`
Ace: a syntax-driven C preprocessor: https://swtch.com/gosling89ace.pdf
In 1981, Gosling Emacs: https://en.wikipedia.org/wiki/Gosling_Emacs
Richard Stallman used Gosling Emacs as the starting code for GNU Emacs.
I am a developer and I do my personal budgeting on a spreadsheet. It was easier to setup and maintain, and follows my process better than the personal finance software I have used before. Could I have made a little program for this? Sure, but it would be time consuming and I have better projects to spend my time on.
For me, the crucial difference is that they're usable over ssh and tmux, not the type of cursor they have (if any).
To be honest I also think that if you are making heavy use of cloud based spread sheet software you have a 100% chance of being a drain on your organization and society as a whole, so I can't really count those missing features as a big downside.
How about for just personal (family/friends) usage?
I would tend to agree spreadsheets are a crutch in larger orgs, however they're best deployed as a prototype tool IMO.
There's plenty of room for > 1 and < 5 (write-access) person operations where cloud based xlsx sharing just makes far more sense than some expensive/excessive enterprise tools.
If it could, that'd be an excellent solution.