Getting Data In

Windows: How to upload multiple files with different sourcetypes?

edrivera3
Builder

OS: Windows

Hi,

I have a bunch of folders with five files, and I want to index just two of them. These two files have different custom sourcetypes. At this moment, I am uploading one file at a time so this is taking me a lot of time. I would appreciate your help with this matter. Thanks

0 Karma
1 Solution

edrivera3
Builder

Well, I found a solution to my problem.
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. Choose one-time index and select the directory you want to upload. Then add three points after the directory address
for recursive (...) i.e C://blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

After you finish uploading those files, delete the full path to your data in Settings>Data inputs>Files & Directories and repeat the process for the second files which have a different sourcetype.

View solution in original post

edrivera3
Builder

Well, I found a solution to my problem.
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. Choose one-time index and select the directory you want to upload. Then add three points after the directory address
for recursive (...) i.e C://blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

After you finish uploading those files, delete the full path to your data in Settings>Data inputs>Files & Directories and repeat the process for the second files which have a different sourcetype.

masonmorales
Influencer

Upload them all to a folder on your Splunk server. Then, do:

splunk add oneshot /tmp/yourfolder/file1 -index myindex -sourcetype sourcetypeA
splunk add oneshot /tmp/yourfolder/file2 -index myindex -sourcetype sourcetypeB
splunk add oneshot /tmp/yourfolder/file3 -index myindex -sourcetype sourcetypeC
etc.

masonmorales
Influencer

Combine with some bash scripting and voila.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Add some details. For example - you only want to index two of what - two of the files, not folders, right? Further more, are all 5 files the same file type? Are the two named in a particular fashion in each folder? Do the folders have a particular structure or naming ?

0 Karma

edrivera3
Builder

Ok. I have one directory which contains 70 subdirectories. Each subdirectory have five files with different extensions. I want to upload only two of those five files in each subdirectory. These two files have different sourcetypes and different filename. Also, you can find the same filenames in all 70 subdirectories.

0 Karma

masonmorales
Influencer

Assuming you are using Linux, you could produce a bash script using the find command and the splunk oneshot commands I listed below to accomplish what you have described.

0 Karma

edrivera3
Builder

No, I'm using Windows but I just found a half solution. Here:
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. In File or Directory, input the directory with recursive (...) i.e C:// blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

I just tried to do the same for the next type of file but splunk doesn't let me select the same directory because it was selected in the previous uploading. Is there a way to make splunk choose the same directory.

0 Karma

masonmorales
Influencer

Why not use a lower root of the directory?

0 Karma

edrivera3
Builder

If I use a lower directory I will be uploading data from other directories that I don't want.

0 Karma

tonykung
New Member

set up a forwarder then

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...