Getting Data In

How to ingest logs from a single file and split them into 2 source types or indexes?

daniel_augustyn
Contributor

I've been trying to ingest logs from a single log file into 2 source types. For example, looking at the 'messages' file in Linux, I want to split the logs from this file into different source types or indexes.

Sourcetype 1:

Jul 10 09:20:01 systemd: Started Session 27535 of user root.

Sourcetype 2:

Jul 10 09:13:26 rsyslogd: [origin software="rsyslogd" swVersion="X.X.X" x-pid="18406" x-info="http://www.rsyslog.com"] rsyslogd

How should I approach this?

0 Karma
1 Solution

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

View solution in original post

ArchieCrozier
Path Finder

@sundareshr Appreciate the response.  Was looking for this one.

0 Karma

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...