The experience of adding monitoring with a New Relic plugin should be MUCH better than with any other monitoring approach.
1. The Nginx plugin has a broken init.d script. Not the end of the world, but hardly a plug and play solution.
2. The Varnish plugin download is behind a self-signed SSL certificate (which is for a different domain no less). Furthermore the Gems it requires are loaded from a GitHub repository that is no longer available to the public. There simply doesn't seem to be a way to get this plugin installed.
The install process and software requirements for the plugins also appear to be wildly different. It would be incredible if New Relic provided the ability to install plugins over the command line from a central repository - something as basic as `newrelic install $plugin_name` would be ideal.
I really hope that these issues are worked out as I think this platform has real potential. It's just unfortunate that the quality of the plugins appears to be far from production ready.
As for the varnish plugin I had to edit the gemfile to have
gem "newrelic_plugin", :git => "git://github.com/newrelic-platform/newrelic_plugin.git", :tag => 'release/0.2.11'
In the future you are likely to see some improved installation options, however because you can truly monitor _anything_ with New Relic plugins (at least, for now, anything that emits or can be made to emit time-series numerical data) it is unrealistic to expect that there will be only one, consistent install experience for everything.
In other words, installing the plugin agent to monitor your MongoDB will probably always be different from installing the plugin agent to monitor your John Deere tractor.
I don't see how expanding their service offering is a bad thing? Surely that expensive contract would have yielded more value than before, given that you use any of the supported applications. If not, you would not be worse of than before?
It's very important to bring the relevant numbers together to get a cohesive view of the current state, as well as be able to correlate between sources to paint the bigger picture.
I also found this: http://newrelic.com/azure
Azure's Management Portal monitors your infrastructure. New Relic also does a little infrastructure monitoring.
Overall, they do different things - and, New Relic is both free to use and super-easy to install, so perhaps you can try it out and see what you think?
The big disadvantage I see is that I still need to manually install these plugins on each server, whereas with scount my understanding is it's 1 click installs?
Here's my (biased) quick take on the differences between New Relic and Scout plugins:
* It looks like New Relic supports 2 types of plugins (1) direct integrations w/other SaaS services and (2) plugins you place directly on your servers. With Scout, you typically don't need to touch your servers to add plugins, but we don't support direct integrations w/other services (the actual plugins are run on your servers). So, if you're monitoring something server-side (like MySQL), Scout will feel easier. If you're pulling in data from another service (like MongoHQ), New Relic will feel easier.
* New Relic supports Java and Ruby plugins, we just support Ruby.
As a whole, Scout is more of a leaner approach to monitoring than New Relic.
Plugins are a part of most monitoring products. It's really the experience that matters. I love well-maintained tools that just work, so our approach to a plugin system reflects that:
* Gardening - Our plugin repo (https://github.com/scoutapp/scout-plugins) has 1000+ commits from 45 authors. We work hard to make sure plugins work, even when they weren't originally created by us. All of our plugins are open-source, so anyone can submit a pull request.
* One language (Ruby) - Why do we only support Ruby? (1) It's an easy scripting language to grasp (2) we know Ruby very well, so when something breaks, we can fix it 95% of the time vs. having to wait for the plugin author to address it. If you've ever browsed through plugins in an open source system like Nagios or Munin, you'll see how disjointed things can get.
A real-world example:
We track plugin errors in Scout so we can look for trends and plugins w/issues. Today, a customer reported an issue with our Delayed Job plugin via our UI. Beside it, we captured the actual error. They told us their workaround fix, which we then incorporated:
https://github.com/scoutapp/scout-plugins/commit/02239c8785f...
...and added a help entry in case other users run into the same problem:
https://scoutapp.com/plugin_urls/281-delayed-job/help_entrie...
So the total time from the reported issue to the fix was just a couple of hours, which is neato.
Does SendGrid via New Relic mean they provide more granular tracking and longer term storage?
More granular - no: The SendGrid plugin presents roughly the same data you could access via the SendGrid API (at least AFAIK).
If you are consuming a published plugin, you get alerts on whatever summary metrics the plugin author selected. If you are authoring a plugin, you can pick up to five metrics to be your summary metrics, then you can create alert thresholds on those metrics.
Could you tell me what differs in your solution, and why it is better? And why you didn't choose Graphite/Carbon or statsd?
The issue is that we are now a lot of people that tries to solve the same kind of problems. Everyone seems to be writing collectors that reports information to backends.
On top of my head I can name:
http://opensource.brightcove.com/project/diamond/ http://collectl.sourceforge.net/ http://collectd.org/
They all implement some kind of RRD or Graphite service hooks. It would be nice if all I should do to upgrade was to point my existing collectd collectors to NewRelic or Scout and be good to go.
Instead, I now have to reconfigure my whole stack, install their agent, add their plugins for my software, make sure things are working and then switch.
I am not advocating that everyone should use Graphite, I am just advocating that we should have some consistency between these systems, so we can use the same collectors no matter what the backend is.
I had a feeling that Graphite+Statsd was where the community was heading given that both Etsy and Github are using that.