Getting Data In

discard few fields and ingest required data using scripted input

ips_mandar
Builder

I want to discard few fields from monitoring input so not increase license usage
What will be best way to do it
It can be possible with SEDCMD but I am trying to know using scripted input
I am newbie in script writing ..can anyone guide me to write python script to take only required data in splunk.
What are the stepsto follow?
Thanks in advance. I am using Splunk 7.3 on Windows server.

0 Karma

woodcock
Esteemed Legend

If you are using a scripted input, then you can either edit the script to modify what it outputs, or, if you already have a SEDCMD that works, you can just add | sed "Your SEDCMD here" to the end of the command line.

ips_mandar
Builder

Thanks @woodcock
For example I have below props.conf

SEDCMD-aremoveheader = s/^\<\?xml[^\>]*\>\n*//g

Then What I need to write in script to run above in script(will it by .py?).
Note: the above props.conf is in Indexer and if I run only |sed ""using script it will not fetch the data from remote server. Since I want to fetch data from remote server.

0 Karma

woodcock
Esteemed Legend

You have something like this in your inputs.conf:

 [script:///path/to/your_script.sh]

Change it to this:

[script:///path/to/your_script.sh | sed "s/^\<\?xml[^\>]*\>\n*//g"]
0 Karma

woodcock
Esteemed Legend

You might need to specify the full path to the sed binary.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk will index whatever a scripted input writes to stdout. Your script can read any data at all, but the key is write only the fields you want in Splunk.

---
If this reply helps you, Karma would be appreciated.

ips_mandar
Builder

Thanks @richgalloway Can you please help me with sample script like python . for example I have csv file in which I want only field 2 ,field 3 ,field 5 to be extracted... Since I never written any script can you please help to provide sample script which will work like mentioned above.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Google can provide lots of examples. Here's one I crafted from the first result.

import csv

with open('my_csv.txt', mode='r') as csv_file:
    csv_reader = csv.DictReader(csv_file)
    for row in csv_reader:
        print(f'{row[2]},{row[3]},{row[5]}')
---
If this reply helps you, Karma would be appreciated.
0 Karma

ips_mandar
Builder

Thanks @richgalloway I will give this try and will keep posted.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...