Eventgen: Where do I put the transforms.conf that I copied from production?

[DavidGuarneri]

In Eventgen, how do I apply a transforms.conf extraction that I copied from production using the replay mode? I've looked everywhere for the answer but not able to find it. Putting the props.conf and trasnforms.conf files in $SPLUNK_HOME/etc/apps/SA-Eventgen/local did not work, and it didn't work putting it in the Splunk local path. I see how to do a field replacement in eventgen.conf , but I have many extractions I need to copy from production, and these are in context (fieldname1 fieldname2 etc), so defining a single field by a single regex doesn't work for me. Instead, for example, all I get in the apache index is metadata fields like date, sourcetype, etc. I don't get access_request, status, IP, etc.

I need to be able to just drop in the transforms.conf somewhere and see the field/value pairs show up in Splunk for the index. The point is to replicate production, and that's not a thing if I'm hand coding stuff.

I'm surprised this is nowhere to be found in the tutorial or the documentation.

[DavidGuarneri · Accepted solution]

Came back to answer my own question. Replay mode not needed.

You should have a separate app where you are doing your configurations. For example, make a blank app in C:\Program Files\Splunk\etc\apps\mytesting

The "local" directory is where props.conf and transforms.conf should go. I've tested this, and it works.

Props.conf will have the name of the sourcetype as the stanza header/title. It will then have additional options and reference the configuration in transforms.conf.

In transforms.conf, Regexes can be on one line, or separately in multiple lines under the same stanza.

Below is an example:

props.conf:

[apache_access]
KV_MODE = none
REPORT-apache_extracts = ub_apache_extracts
FIELDALIAS-apache = user AS userid clientip AS src_ip

transforms.conf

[ub_apache_extracts]
REGEX = ^(?\d\d/..... blah blah blah

When you do a search, make sure you are in your "mytesting" app, or whatever you named it.

Copying stanzas from production may or may not work. Try the regexes in Splunk first before trying to make them work in transforms.conf. Do a little bit at a time.

[koshyk]

can you please mark answer as correct, if you are satisfied with the response. cheers.

[koshyk]

IMO, the best thing to do is
- to run as "sample" rather than replay mode
- Put output as a file
- Use your original TA containing the transforms.conf and just put inputs.conf
- This will ensure you index the data in exact same way as you do in PROD

I feel Eventgen should be isolated from the complexity of transforms and is better to do in your own app

[DavidGuarneri]

Output to file would probably work. The downside is that now I have to manage files from getting too big on the server. If the cleanup job fails, it could bring down the server. Secondly, the metadata that goes into eventgen, such as source and sourcetype, would be lost. For each index, I have one file output that has multiple sources and sourcetypes.