Splunk Search

How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria
Engager

I want to change a source by removing a "hostname" from file path (string) using inputs.conf

Currently, the source is /var/log/syslog/<hostname>/filename.

What I am looking for is /var/log/syslog/filename

How do I remove "hostname" string from path? I need to add it to inputs.conf

thanks in advance.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You should be able to do it with transforms. See transforms.conf.example.

[hostoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMAT = host::$1$2
DEST_KEY = MetaData:Host
---
If this reply helps you, Karma would be appreciated.
0 Karma

meet_vadaria
Engager

I am trying to override source field of syslog. I think given block will help me to override host field. i tried for source by modifying little bit. still can't get it working.

input.conf
[monitor:///var/log/syslog/]
host_segment = 4
sourcetype = xxx

transforms.conf
[sourceoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMATE = source::$1$2
DEST_KEY = MetaData:Source

props.conf
[source::/var/log/syslog/]
TRANSFORMS-changesource = sourceoverride

I have config as above.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Sorry for misreading your question. Your modifications look good. Be careful, however, of the REGEX string. Make sure it matches your file paths. "filename", for example, is taken from your question, but is probably not part of the file path. You should modify it accordingly.

---
If this reply helps you, Karma would be appreciated.
0 Karma

meet_vadaria
Engager

I think this block is to change Host filed of events. I am trying to change Source field. I don't see any documents regarding that. I found splunk documents mentioning I can change source type with transforms.conf and props.conf but not source.

any idea if we can change Source using transforms.conf and props.conf?

transforms.conf
[set_sourcetype_my_log_for_some_hosts]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?).\w+\s+)[?(host1|host2|host3)[\w.-]]?\s
FORMAT = sourcetype::my_log
DEST_KEY = MetaData:Sourcetype

props.conf
[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_my_log_for_some_hosts

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Advancedsourcetypeoverrides

0 Karma

ddrillic
Ultra Champion

@meet_vadaria, which path would you like to monitor? If it's /var/log/syslog/filename can't you just specify it as such in input.conf?

0 Karma

meet_vadaria
Engager

filename is dynamic. I want to modify source of event to this. splunk forwarder by default shows file path which is /var/log/syslog// with hostname and filename dynamic. I want it to be modified to /var/log/syslog/. need to remove from file path.

0 Karma

ddrillic
Ultra Champion

At the level of input.conf, you can use the * and ... wildcards. If needed, you can change source later during index time.

0 Karma

meet_vadaria
Engager

I think you are not getting my point. here is example of my input.conf on splunk forwarder.

[monitor:///var/log/syslog//]
host_segment = 4
sourcetype = xxx

this input.conf monitors all log files sitting under /var/log/syslog// directory. let's say I have 3 log files there as foo.log, bar.log, xxx.log

now my default source in splunk for this logs(events) will be /var/log/syslog//foo.log or specific log file

I want to change it to /var/log/syslog/foo.log. I believe I can do it in input.conf by using regex. I guess I can have a regex to remove string from /var/log/syslog//foo.log and it will be right way to do in input.conf

I don't know how to do it in indexing time.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...