Getting Data In

How to override the sourcetype of events within the same source based on the event format?

norskedm
Explorer

I'm trying to override the sourcetype of events within the same source (for now, a file uploaded once and indexed - once I get it figured out, the source will be a scripted input from universal forwarders). I need to override the sourcetype of events in a source, based on the format of the event. If the event contains the word "share" (in a certain place) I'd like the source type to be "share"; likewise "dir" and "ext". The events' sourcetype defaults to "ext" (since this is the bulk of the events).

I am using Splunk documentation as a reference.

Here is a sample of the source data:

2014-08-11 22:14:54Z,foo900.example.com,share,seed,g:\seed,Disk,"General testing."
2014-08-11 17:14:54Z,foo900.example.com,dir,\\foo900.example.com\seed,182445977979,2014-07-17 17:00:28Z,2011-02-15 23:20:45Z
2014-08-11 17:14:54Z,foo900.example.com,ext,\\foo900.example.com\seed,.sgy,163108239992

Here is the props.conf:

[ext]
TRANSFORMS-change_sourcetype = transform_ext_sourcetype
SHOULD-LINEMERGE = false
REPORT-ext = transform_ext, transform_dir, transform_share

Here is the transforms.conf:

[transform_ext_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = [^,]+,[^,]+,(share|dir|ext),
FORMAT = sourcetype::$1

[transform_ext]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),ext,(?<share>[^,]+),(?<file_ext>[^,]*),(?<bytes_used>[^$]+)$

[transform_share]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),share,(?<share_name>[^,]+),(?<path>[^,]*),(?<share_type>[^,]+),(?<share_remarks>[^$]+)$

[transform_dir]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),dir,(?<share>[^,]+),(?<bytes_used>[^,]+),(?<last_access_datetime>[^,]+),(?<creation_datetime>[^$]+)$
0 Karma
1 Solution

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

View solution in original post

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

norskedm
Explorer

@lguinn The problem is it didn't work. A very-important part of the puzzle was left out of the documentation

0 Karma

lguinn2
Legend

What is the problem?

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...