Sparse Matrices (2019)(matteding.github.io) |
Sparse Matrices (2019)(matteding.github.io) |
Can be a single tree for the matrix where keys are tuples of two integers. Or one tree per row, where keys are single integers, column index.
Unlike CSR, inserting elements is Log(n). RAM overhead is larger than CSR but still reasonable, much smaller than hash maps or red-black trees would be. Similar to CSR, reading rows is mostly sequential RAM reads i.e. pretty fast.
I used these trees in the code that builds large sparse matrices from something else. The domain was CAM/CAE, it was finite elements. As a nice side effect, with one tree per row I was able to parallelize the construction of that matrix, as different CPU cores can update different rows of the matrix in parallel.
No, this is not the mathematical definition. An n-by-n matrix is usually considered sparse if the number of nonzero elements is O(n). Which means that the ratio of nonzero elements goes to zero as the matrix grows.
[1] https://arrow.apache.org/docs/cpp/api/tensor.html?highlight=...
[2] https://github.com/apache/arrow/blob/master/python/pyarrow/t...
If you know how it's supposed to work in theory but struggle to bang it out, you could do eg leetcode.
It's a generalized CSR/CSC (to ndim > 2) format that uses a tree data structure where each path from root to leaf encodes a nonzero element.
Regarding the construction of the matrix, instead of insertions very often you can do it with tricks using kronecker products and the like.
Yep, that’s precisely why I used that thing.
> Most of my time is spent in algebraic operations with immutable matrices.
CSR is probably the way to go then.
Also, if you do that math on CPU and your matrix is not randomly sparse but made of small dense areas of non-zeroes, consider less compressed SIMD-friendly variant of CSR, where instead of individual elements you keep dense SIMD vectors.
For example, if you have AVX and need fp64 precision, keep 4 elements long slices of rows there. This way you can use 256-bit loads and FMA instructions to do that math.
Or if you spending most of time multiplying matrices as opposed to matrix*vector, might be reasonable to keep 2x2 dense blocks as opposed to 4x1.
But tbh, I think incremental updating of a matrix only makes sense if the other parts of your pipeline are also incrementalized.