I have been hearing this a lot over the past few days and thought its time I put my experience into words. What’s Splunk? Four months back, I had no idea. Of course I had heard of the company, but I had never used their platform. Today, four certifications and countless hours of splunking later, things are a lot different.
Quick question – What do you think you can do with Splunk?
a. Manage structured as well as unstructured data
b. Monitor real time and historical metrics
c. Handle machine generated logs
d. Build dashboards and visualizations
e. Generate reports
f. Develop applications
Well, it can do all of the above (and a lot more).
Splunk Enterprise is free to download and allows you to index up to 500 MB of data every day. The not so good part? It’s not an easy tool to learn, let alone master. There are tools which you can start learning if you start messing around, and Splunk isn’t one of them. It has its own Splunk Search Processing Language (SPL) which you need to be comfortable with.
I’ll give an example. This is one of the searches I used to analyze data over different time ranges. As you can see, these are terms and commands specific to the Splunk SPL.
sourcetype="Test" earliest=-30d@d
| stats dc(ID) as Last30days by date_hour
| eval Last30days= round(Last30days/30,2)
| appendcols[search earliest=-7d@d | stats dc(ID) as Last7days by date_hour]
| eval Last7days= round(Last7days/7,2)
| sort date_hour
| eval date_hour = case(date_hour<12and date_hour>0,tostring('date_hour') + "AM",date_hour=12,"Noon",date_hour>12,tostring('date_hour'-12) + "PM",date_hour=0,"Midnight")
| rename Last7days as"7 day Average", Last30days as"30 day Average"
The Splunk documentation is the best place to start exploring. Here you’ll find everything from installing a Splunk instance to building beautiful visualizations. Is it worth the effort? Oh yes. I was able to build this simple interactive dashboard for analyzing internal Splunk logs in under an hour. Imagine what you can come up with in an entire day.
I started my summer with the idea of developing a Splunk app, now I’m working on building a custom web application using the Splunk SDKs. From building dashboards with a drag and drop GUI to developing the underlying XML code, from writing complex search queries to using the pivot functionality, from connecting a SQL database to referencing a lookup table, from indexing archive files to creating a modular input.. there are so many interesting things I have done with Splunk, that the possibilities seem endless.
My intention behind writing this article is to provide an overview of Splunk to someone who might be thinking of looking into what this tool really is. For me, Splunk is like a vast ocean – there’s always so much to learn, so much to explore..
Happy Splunking!