How to remove information from forwarded data?

[ddarmand]

Hello everyone,

This is my topology:

Splunk Forwarder (with local copy of data) -----> Main Splunk

The forwarder is adding sourcetype from regex etc... and it appears in the main Splunk, but I prefer to have raw data. How is it possible to have this without deleting rules on the Splunk forwarder?

Thanks,

Damien

[woodcock]

What makes you think that your sourcetype is being added to your raw event? I am unaware of any automatic (i.e. accidental) way that this could happen and to deliberately make it happen, although not difficult, is certainly something that takes some work. I suspect that your raw events are actually OK but you can see for yourself like this:

... | table sourcetype _raw

You will probably see that the field _raw always has an identical/unmodified copy of your raw events.

[woodcock]

If I understand you correctly, this is your situation:

We have two types of users: A and B both generating the same events that are coming through syslog.  For type-B users, we'd like the logs without modifications (added fields, e.g. sourcetype) by the heavy forwarder (just as they come by syslog).  For type-A users, we'd like to add/keep these modifications.

If this is correct, then you need to build a REGEX that can match type-A users but not type-B users. Once this is done, you need to modify the stanza in transforms.conf that is adding the fields so that it has a REGEX= line. That should do it.

[ddarmand]

It's added because i have some configurations to do this in props.conf transforms.conf ect... but i dont wan't to have these infos on the forwarded data

[woodcock]

OK, I am TOTALLY confused. You are deliberately adding stuff to your raw events but you would like to not do so? Back all the way up and FULLY explain your existing configurations (and maybe why they are that way) and the explain your desired end state. As it is, right now, I am utterly confounded.

[ddarmand]

Ok, sorry i will try to explain this :

http://zupimages.net/viewer.php?id=15/43/mnyy.png

As you can see, we have two types of users A and B.

I want to see on the main splunk for user B the logs without modifications by the heavy forwarder (as they come by syslog)

These modifications is adding sourcetype for example with props.conf and transforms.conf ect...

But user A need these modifications.

[asimagu]

so you are using a HW Forwarder?? why don't you try using a UF?

[ddarmand]

because i need to keep local copy of events on the forwarder !

[asimagu]

cool, the more info you provide , the better we will be able to assist 🙂