Hello,
I'm attempting to change the sourcetype and host on a single event. The tricky part is I want the second transform based on the change from the first transform
For Example,
My data comes in as
index=main
host=heavy_forwarder
sourcetype=aws:logbucket
I want the data to change to
index=main
host=amazonfsx.host
sourcetype=XmlWinEventLog
The catch is that I have other sourcetypes coming in as aws:logbucket and getting transformed to various other sourcetypes (cloudtrail, config, etc). On these events I do not want to run the regex to change the host value
If I have a props.conf file that states
TRANSFORMS-modify_data = aws_fsx_sourcetype, aws_fsx_host
And a transforms.conf of
[aws_fsx_sourcetype]
SOURCE_KEY = MetaData:Source
REGEX = ^source::s3:\/\/fsxbucket\/.*
FORMAT = sourcetype::XmlWinEventLog
DEST_KEY = MetaData:Sourcetype
[aws_fsx_host]
REGEX = <Computer>([^.<]+).*?<\/Computer>
FORMAT = host::$1
DEST_KEY = MetaData:Host
I'm worried this will have unexpected results on the other sourcetypes that aws:logucket has, like cloudtrail and config.
If I break it out with two separate transforms, like this
TRANSFORMS-modify_data = aws_fsx_sourcetype
TRANSFORMS-modify_data2 = aws_fsx_host
I'm worried the typing pipeline won't see the second transform.
What is the most effective way to accomplish this?
Thanks,
Nate
Altering host field is one of least desirable alterations but it has to be done from time to time. In your case, you probably have to use a calculated field.
I'm going to try that. I'm seeing now the Windows App has a default transformation called "WinEventXmlHostOverride" that will override the host with the "Computer" Xml value. Do you see any negative from doing this at search-time rather than index-time?
Calculated field and transforms are also search time. The general advice is to be very careful in making index time calculations. (Except filtering down.)