When trying to use the field extraction in Hunk it is only allowing me to map the extraction to either the source or the host name. This is very unflexible as the host would cover a very large dataset and the source would be too specific and not cover the various log files in the directory. How are people getting around this?
Thanks,
The solution here would be to use props.conf to assign a sourcetype to the data (that's of the same or similar format) based on some source pattern, for example:
# assign all files under this dir sourcetype=foobar
[source::/path/to/data/...]
sourcetype = foobar
# assign all files named foobarbaz.log* sourcetype=foobarbaz
[source::.../foobarbaz.log*]
sourcetype = foobarbaz
Once this is done, you can then key field extractions off of the sourcetype