OCaml for Kafka stream processing, analytics and telemetry data

OCaml for Kafka stream processing, analytics and telemetry data(github.com)

4 points by tingfirst 297 days ago | 3 comments

tingfirst 297 days ago |

For OCaml users interested in data streaming processing (similar to Flink or Spark), but looking for a faster and more efficient option, check out this OCaml plugin Timeplus Proton. Concise, safe, highly performant and fun

-> Streaming Queries - Process large datasets with constant memory usage

-> Async Inserts - High-throughput data ingestion with automatic batching

-> Compression - LZ4 and ZSTD support for reduced network overhead

-> TLS Security - Secure connections with certificate validation

-> Connection Pooling - Efficient resource management for high-concurrency applications

-> Rich Data Types - Full support for #ClickHouse types including Arrays, Maps, Enums, DateTime64

-> Idiomatic OCaml - Functional API leveraging OCaml's strengths

rainmaker11 297 days ago |

Interesting. How does the performance look like?

gangtao 292 days ago | |

Micheal, the author of the project did some performance test and he might annouce it later, but one of his first test show Async Insert with Rate: 3422488 rows/second, but I beleive after optimization and tuning, it could be much faster.