Getting Data In

How to upload multiple XML files and have the sourcetype be defined as the name of the file automatically?

ektasiwani
Communicator

Hi,

I have a folder with 10 XML files and it can be more in the future.
My requirement is to upload this file, and the sourcetype name should be equal to name of file.
I want everything to be automatic.
For that, do I need to use a python script or can it be done by Splunk?
Is there any tutorial link for creating a sourcetype and uploading the file through a python script?

Thanks.

0 Karma
1 Solution

woodcock
Esteemed Legend

You can do it in Splunk by using props.conf and transforms.conf like this (be sure not to set a sourcetype= in your inputs.conf😞

#props.conf:
[source::<your stuff here>]
TRANSFORMS-sourcetypeSetFromFilename=sourcetypeSetFromFilename

Then

#transforms.conf:
[sourcetypeSetFromFilename]
SOURCE_KEY = MetaData:Source
REGEX = <your RegEx here with one capture group>
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

View solution in original post

0 Karma

woodcock
Esteemed Legend

You can do it in Splunk by using props.conf and transforms.conf like this (be sure not to set a sourcetype= in your inputs.conf😞

#props.conf:
[source::<your stuff here>]
TRANSFORMS-sourcetypeSetFromFilename=sourcetypeSetFromFilename

Then

#transforms.conf:
[sourcetypeSetFromFilename]
SOURCE_KEY = MetaData:Source
REGEX = <your RegEx here with one capture group>
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype
0 Karma

ektasiwani
Communicator

Hi,

I am getting it.
here is my pros file

My files are in splunk-input fiolder in local computer.

props.conf:

[source::/opt/splunk-input/]
TRANSFORMS-sourcetypeSetFromFilename=sourcetypeSetFromFilename

here is my transform file

#transforms.conf:
[sourcetypeSetFromFilename]
SOURCE_KEY = MetaData:Source
REGEX = .\/(.)
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

i copied both files in splunk/etc/system/local folder

And i am uploading files through setting->data->new
and for sourcetype i selected automatic.

0 Karma

woodcock
Esteemed Legend

I don't think it will work with [source::/opt/splunk-input/]; you need to use something with wildcards like [source::/opt/splunk-input/*], otherwise it will not match any of your sources. Other than that, it should work, once you bounce your forwarders.

0 Karma

ektasiwani
Communicator

Thanx for rply,

That regex in transforms.conf is to set the name of sourcetype? or for something else?

And if i will upload a folder with all the file through settings-->data input , sourcetype will be equal to folder name and one for all the files? or each file will have there sourcetype?

for example i have file 1 , file 2, file 3 in a folder name file. my requirement is to have three sourcetype with name file 1, file2, file 3.
but dont want to upload each file one by one.

0 Karma

woodcock
Esteemed Legend

Right, if you need to use the filename as the sourcetype, you use a RegEx like this:

 REGEX = .*\/(.*)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...