Lovefield: Cross-browser, SQL-like, relational query engine on top of IndexedDB

Lovefield: Cross-browser, SQL-like, relational query engine on top of IndexedDB(github.com)

80 points by peter637 11 years ago | 11 comments

As someone who has extensively used/abused IndexedDB in the development of http://basketball-gm.com/ I can say that I'm really happy to see some serious libraries built on top of IDB. Years ago people said "yeah, it doesn't have all the features of Web SQL, but you could build something like that on top of it" but now that's finally starting to actually happen.

Other cool IDB projects are YDN-DB https://github.com/yathit/ydn-db which continues to develop and Dexie https://github.com/dfahlander/Dexie.js/wiki/Dexie.js which is also new.

I hope that some time in the near future I can stop writing ugly hacks in raw IDB to account for missing database features and browser incompatibilities. But I worry about losing performance if I delegate too much to one of these libraries (although I haven't really started seriously testing them yet).

xtacy 11 years ago |

This documentation page is an excellent overview for anyone who wishes to understand how a declarative language such as SQL is actually translated to an execution plan behind the scenes:

https://github.com/google/lovefield/blob/master/docs/dd/05_q...

streptomycin 11 years ago |

https://github.com/google/lovefield/blob/master/docs/dd/04_c...

Lovefield has an in-memory row cache, which is conceptually a big map of row ids to rows (and that is why Lovefield has unique row ids across the board). Currently the cache is a "dumb" cache: it contains exact duplica of what are persisted in the IndexedDB. The reason for doing that is to workaround IndexedDB inefficiency of handing bulk I/O, as described in backstore section. By caching all rows in memory, Lovefield avoids any additional round-trip required to load data from IndexedDB, with the price of memory usage.

How does that handle the case where the user has the app open in two tabs? Is there any synchronization?

ivanceras 11 years ago |

SQL-like? How about your use a real SQL used in SQLite, https://github.com/kripken/sql.js/

streptomycin 11 years ago | |

Not the same use case. Lovefield is built on IndexedDB which allows you to store persistent data client-side, not just in memory or on a server.

dangayle 11 years ago |

You need to support Python, Java AND Node.js for this? Sheesh.

vespakoen 11 years ago |

Here is a nice collection of node.js / browser stuff that wraps around IndexedDB / LevelDB / WebSQL and is mostly compatible with one another.

https://github.com/rvagg/node-levelup/wiki/Modules

VeejayRampay 11 years ago |

Now they only need to add a "commitToServer()" feature that uses some diff since the last commit and we got a stew going.

That tech is promising though, I like that we're opening new possibilities all the time.

batbomb 11 years ago | |

The problem there, however, is you don't have distributed transactions and locks/leases. You would need to build that into the server itself, which is obviously out of scope for this project.

If the server changes, as it could easily do if you had two separate browsers open (say, mobile safari and your desktop), you'd need quite a bit conflict resolution code.

eob 11 years ago | |

We did this with sqlite about six years back and got awesome performance results. It was a system called SyncKit. Still would have preferred relational storage in the browser, but glad to see at least we're moving toward something.

http://edwardbenson.com/papers/www2010-synckit.pdf

bjornlouser 11 years ago |

Why did you choose Lovefield as the project name over DealeyPlaza?