[colug-432] splunk?

[Rick Hornsby]

On October 14, 2016 at 12:03:45, Rick Troth (rmt at casita.net) wrote:

At my place of work, we ship Splunk embedded in appliances.
I gather that it is a closed source log handler which *can talk SYSLOG
protocol* and presses its affinity for CEF (Common Event Format). CEF
strikes me as a good thing because SYSLOG traffic can be way too free-form
for enterprise processing. (Just too little structure for huge volumes of
log traffic to be processed effectively without something like CEF.)

Badly formatted - or worse completely unformatted - logs are a huge problem
and it does require some thought behind how an application generates its
logs. Splunk can deal with and recognize lots of formats out of the box,
and can be taught others. The key is that the logs have to be in a format.
You can't simply dump whatever you want into your log files (like raw XML)
- or a variety of formats in a single log - and expect Splunk will figure
it out by magic.

Badly formatted (or missing) timestamps on log entries can also royally
screw things up while Splunk spends a ton of time trying to figure out what
the timestamp for the event should be.
Adhering to a CEF can make a massive difference in Splunk's usability and
performance by addressing both of the above.

I found Splunk's gigabyte licensing to be annoying. A customer can bump
into the wall and lose traffic. (I forget the details of the failure mode.)

Last I knew, you don't lose logs due to licensing violations. Basically,
it's a certain number of violation "strikes" for going over the log input
("indexing") limit per month. I believe it is 5 strikes, and then you can't
search your logs until you resolve the license issue by talking to Splunk -
or perhaps some amount of time passes and searches work again. You should
never lose logs.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.colug.net/pipermail/colug-432/attachments/20161014/3d492e88/attachment.html