Splunk Search

How to combine and chart data from multiple files in the same folder, but only if a certain type of file exists?

avinashreddy539
New Member

Hi,

I am new to splunk and need help with my use case below.
Whenever a request is made to my application, it will create a new unique folder and start writing the information to three files called 1.err, 2.xml, 3.xml. 1.err file will be created only whenever any error is identified., otherwise this file will not be created.

My 1.err file contains

FAIL_ERROR_MESSAGE = Error due to function: function name.

If the file 1.err exists in the folder then i need to get all the FAIL_ERROR_MESSAGE count and from 2.xml file i need to get
SENDID
attribute value
from that folder and from 3.xml file i need to get GFM value as shown in below.

          <name>GFM</name>
          <value>Test</value>
          <dataType>String</dataType>
        </attribute>

I need to prepare a chart where it can show error messages by SENDID and by GFM value. The data should be collected from all the folders where the folder contains 1.err file.

Thanks in advance.

Tags (3)
0 Karma

jeremiahc4
Builder

I'd say you'd want to setup your inputs.conf on the server where you are getting these .err files such that you index all the files using a regex "*" in place of the unique directory.

[monitor:///path/to/files/*/*.xml]
...
[monitor:///path/to/files/*/*.err]
...

Then you can extract that unique directory using rex expression. You can then put those together in a transaction and search for the failure string from the .err file and extract the values you need

base search |  rex "\/path\/to\/files\/(?P<transID>[\w\d]*\/.*" | transaction transID | xmlkv | table field1 field2 SENDID GFM
0 Karma

jeremiahc4
Builder

This is an interesting use case. I'm not aware of any way to tell splunk only to index the files if it sees a .err file. You can index all of them, then alert based on the presence of the error message.

You would need some kind of transaction ID that is shared between the 2 xml & 1 err file in order to get the specifics you need. Perhaps the unique directory name could be used as that transaction ID. You would then use the xmlkv command to extract the SENDID and GFM values extracted into your report also.

0 Karma

avinashreddy539
New Member

we can index all files. But while generating the report we are interested in only .err files. if we have .err file in folder then only we need to fetch the fields from other files in same folder.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...