Ask HN: Tools / approaches to logging and monitoring real-time transactions

3 points by bec123 7 years ago | 4 comments

Summary Looking for suggestions on tools and approaches to logging and displaying, with slice dice filtering etc, of transactions.

Requirement Each logged transaction is 1 line of data prefixed with a time stamp, the rest of the line containing things about the response time. The transaction data can be pumped out periodically to files which presumably would need to be ingested by a tool.

My thoughts so far .. Primarily from this requirement I am looking for a simple on-prem tool that will allow me to push these transactions into it for display filtering etc.

I have investigated tools online and found a lot of high priced confusing tools that are designed to do a lot more than I need and/or are unclear to me if they meet my simple needs. Several also seem to be cloud based which is no use to me.

tedmiston 7 years ago |

It’s kinda more for operations monitoring but check out Prometheus (https://prometheus.io/) for monitoring and Grafana (https://grafana.com) for visualization to see if they would work for your use case. Both are pretty flexible. Also, I think Datadog is a roughly equivalent SaaS offering of the two though I haven’t used it myself yet.

I like Apache Superset for business intelligence type dashboards.

rubyfan 7 years ago |

I have used Splunk in the past on-prem. They are expensive and to your point and a lot of capability that wasn’t used. We had investigated Gray Log, Fluentd, Logstash and others. If we started now, I’d say the ELK stack (Elasticseach, Logstash, Kibana) would be what we’d have gone with.

mtmail 7 years ago | |

On the visualization part you can also check https://prometheus.io/ and https://grafana.com/grafana

bec123 7 years ago | |

Thanks will take a look.