I have data incoming via TCP syslog. I have created the following transforms to process them:
etc/system/local/props.conf:
[source::tcp:1514]
TRANSFORMS-windows = set_sourcetype_snare, set_source_wineventlog
etc/system/local/transforms.conf:
[set_source_wineventlog]
REGEX = AgentDevice=WindowsLog.AgentLogFile=(.?)\s
FORMAT = source::WinEventLog:$1
DEST_KEY = MetaData:Source
[set_sourcetype_snare]
REGEX = AgentDevice=WindowsLog
FORMAT = sourcetype::windows_snare_syslog
DEST_KEY = MetaData:Sourcetype
These work as expected, and the source
and sourcetype
are set accordingly. However, I expected that setting these two fields would also trigger some other Splunk built-in transforms. For example:
[splunk@l1807s local]$ ~/bin/splunk btool props list windows_snare_syslog
[windows_snare_syslog]
...
TRANSFORMS = syslog-host
and
[splunk@l1807s local]$ cat ~/etc/apps/Splunk_TA_windows/default/props.conf
...
[source::(MonitorWare|NTSyslog|Snare|WinEventLog|WMI:WinEventLog)...]
...
FIELDALIAS-event_id_for_windows = RecordNumber as event_id
...
As far as I can tell, the default processing is not happening. I see the source
and sourcetype
fields as set by my transforms, however for example I don't find the field event_id
, and host
is incorrectly set. What am I doing wrong here, and how can I achieve the intended behaviour?
Although you can change the host
and sourcetype
fields, all configurations inside of props.conf
may reference the original/pre-overridden values if the overriding happens afterwards.
Are you sure about this? This documentation page states the contrary: http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Wheretofindtheconfigurationfiles
Yes, I am sure: I am battle-tested on this. Where in that link do you see anything to the contrary? Please post the text in a comment/followup.
Another thought: in my example, isn't event-id
a field extracted at search time? Shouldn't it pick up changes to source
and sourcetype
, which are made during indexing?
Under the heading "precedence in a global context" (which is the context where indexing happens), the system local directories have the highest priority. I just realized however that this applies only to the situation where the same attribute is defined in multiple files, and says nothing about the order of evaluation of different attributes, which is what you were referring to. Do you have a documentation link where the latter is documented?
Is the data coming in and the props/transforms on the same Splunk instance or is there some separation?
No, there isn't any separation. It's a single-machine test installation.
I recommend you take the transforms and other actions you desire from other sources/sourcetypes and copy them to your own props.conf and transforms.conf files. That will ensure they work and protect you from future changes made to the other apps.