Getting Data In

How to ingest logs from a single file and split them into 2 source types or indexes?

daniel_augustyn
Contributor

I've been trying to ingest logs from a single log file into 2 source types. For example, looking at the 'messages' file in Linux, I want to split the logs from this file into different source types or indexes.

Sourcetype 1:

Jul 10 09:20:01 systemd: Started Session 27535 of user root.

Sourcetype 2:

Jul 10 09:13:26 rsyslogd: [origin software="rsyslogd" swVersion="X.X.X" x-pid="18406" x-info="http://www.rsyslog.com"] rsyslogd

How should I approach this?

0 Karma
1 Solution

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

View solution in original post

ArchieCrozier
Path Finder

@sundareshr Appreciate the response.  Was looking for this one.

0 Karma

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...