What if SELECT, FROM, WHERE were functions?

What if SELECT, FROM, WHERE were functions?(remy.wang)

4 points by remywang 25 days ago | 4 comments

brof 22 days ago |

I've been looking into Prela the last few days and really like it :)

Conceptually, the two key operators `.select()` and `.and()` connect somewhat to the lineage of arrow notation [1], fork algebras [2], etc.

Left-to-right composition:

(>>>) :: (b -> c) -> (c -> d) -> (b -> d)

Fanout:

(&&&) :: (b -> c) -> (b -> d) -> (b -> (c, d))

The database-specific detail is enumerating the universe of entities to maintain shared correlations and using projections as ways to access information, so, in the above notation:

movie :: Movie -> Movie

title :: Movie -> String

movie >>> title :: Movie -> String

The specialization to then have Rust types be able to generate inlined query execution code at compile-time is quite neat. But, I wonder where the limitations of this approach are seen? In contrast to something like Soufflé which has a much more complicated optimizer but generates C++ code ultimately, it seems difficult to always rely on the Rust compiler. Likewise, some other projects like Crepe rely on Rust macros more heavily. And, the embeddability is a plus, but it does not look too easy currently to use from other languages as you would need to compile the query independently and provide something like an FFI wrapper.

Relatedly, some other posts about relational language design from the last couple days: https://news.ycombinator.com/item?id=49342530, https://news.ycombinator.com/item?id=49363617

And this tutorial looks interesting: https://northeastern-datalab.github.io/relational-language-t...

[1] https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/arro... [2] https://www.cosc.brocku.ca/Faculty/Winter/JoRMiCS/Vol1/PDF/v...

remywang 21 days ago | |

Thank you! Prela’s `.and` operator is exactly the fork in fork algebra. I also suspect you will get `.select` and `.and` if you lift >>> and &&& to the category of Rel.

> I wonder where the limitations of this approach are seen?

Compile time. Rustc takes forever to compile, much longer than the time it takes to run the query. There’s plan to build a JIT for Prela, which would also improve the interop.

Most of the language design is orthogonal to the embedded implementation though, and Prela could very well be implemented in a vectorized engine.

brof 20 days ago | | |

Got it! One more question if I may: in https://arxiv.org/pdf/2607.26356, all the JOB queries are around a second or less, if I read correctly.

My understanding is the query:

movie

    .with(keyword.eq("my-kw"))

    .select(title)

is inlined to something by the compiler approximately like

for movie in 0..movie_count {

    let lo = keyword_offsets[movie];

    let hi = keyword_offsets[movie + 1];

    for pos in lo..hi {

        let kw = keyword_ids[pos];

        if keyword_text[kw] == "my-kw" {

            emit(movie, movie_title[movie]);

            break;

        }

    }

}

So there's no index lookup on keyword, right? E.g., to use a hash index on keyword to find the resulting movie rows. If I wanted to do so, would I re-normalize the data in some way? This is what surprised me: that even without the secondary indexes, it is still the same (or more) performant than DuckDB.

Perhaps the index-lookup version would use something like

let movies_by_keyword: HashIdx<_, _> =

      keyword.text().inv().collect();

but it does not seem like it does (even though I assume DuckDB may).

let lo = keyword_offsets[movie]; let hi = keyword_offsets[movie + 1]; for pos in lo..hi { let kw = keyword_ids[pos]; if keyword_text[kw] == "my-kw" { emit(movie, movie_title[movie]); break; } }