Countless times I have edited a props.conf or inputs.conf file, restarted the Splunk services, and my changes didn’t have any effect.  I experience this while testing at home, and over and over at client sites.  It’s pretty easy to fat-finger some value or get the syntax wrong in a stanza, but when you are sure these things are all accurate, this can be pretty frustrating.

I have had pre-sales opportunities seriously consider counting Splunk out of contention, due to this scenario.  Making changes and getting no results makes it seem to the customer that Splunk is broken.

I am sure most of us are aware of the use of Precedence in Splunk, which is described in detail here:   http://docs.splunk.com/Documentation/Splunk/6.1.1/Admin/Wheretofindtheconfigurationfiles

It’s confusing at best.  I understand why it is built this way, but it still makes me want to see if how many times I could get my laptop to skip across the pool.

Recently, someone tipped me off to Btool.  It is embedded in Splunk v6 and will tell you what files and the values in the stanzas, are actually being used.   It is run in the command line and per the .conf file in question, you can see what is going on in the background of file precedence.

In addition to revealing what is happening with precedence, Btool can also check your configuration file syntax.   Below is a unix example of the syntax and some simple commands you can run to make Btool work for you:

On the splunk server–

splunk btool inputs  list –debug | egrep  -ie index  -e [\[] | more

On the forwarding host–
splunk btool outputs  list –debug
splunk btool inputs  list –debug

All of this will return what is being read from (config files) per index and sourcetype.  The precedence is listed in order, so it starts to make more sense.

You can also run–

splunk btool check

You can run this anywhere you have splunk/splunkforwarding installed and it will do a syntax check