[colug-432] splunk?

[Rick Hornsby]

On October 14, 2016 at 11:33:08, Jeff Frontz (jeff.frontz at gmail.com) wrote:

Anybody heard-of/using splunk?  Is it a widely-deployed platform?


Splunk is an awesome, if expensive, tool.

At its most basic and at the risk of over-simplifying it, Splunk is log
aggregation. In most setups, hosts run a splunk agent (“forwarder”). This
forwarder sends the host's logs (system, application, etc) to Splunk in
real time. Now instead of trying to
grep-pipe-grep-awk-crap-regex-hell-start-over 12 log files across 6 hosts
in a cluster one file, one host at a time, you run a sql-like (emphasis on
the like) query in the Splunk UI, and Splunk does the work.

There are dashboards, and automated searches, and certain log formats[1]
that Splunk understands natively and is able to turn your logs into data.
For formats that Splunk doesn’t know, it can be taught. For a simple
example, apache access logs - statistics by user agent, or access times.
Splunk understands the access time as a field - as a query-able piece of
data within a log entry. This becomes super powerful when you start sending
in and teaching Splunk about your application’s logs.

Splunk really starts to shine at scale. If you have 2 servers with a small
number of log files, big deal they can be searched by hand. If you’re
trying to track down an issue across a cluster of a dozen or hundreds of
nodes - grep begins to look like the horrible tool for that job that it is.
It’s practically impossible to do any sort of log/event correlation using
shell commands, but Splunk makes that sort of thing possible.

While Splunk includes alerting based on searches and whatnot, Splunk is not
a real-time monitoring tool or a replacement for a near RT monitoring tools
like Zabbix, Nagios, etc. Treating it like one is usually a road to
horrible performance and bad experiences.

There are free alternatives like logstash, but I don’t have any personal
experience with logstash. I believe LogInsight is VMWare’s answer to
Splunk, but AFAIK it is very much a vmware-specific tool, and doesn’t work
as well when trying to use it as in a more generalized way like Splunk was
meant to be used.

If you have an opportunity to attend a Splunk demo, I’d definitely do it.
It’s not for everyone or for every shop, but it’s worth knowing about.


[1]
http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/Whysourcetypesmatter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.colug.net/pipermail/colug-432/attachments/20161014/960fec43/attachment.html