If I add INDEXED_EXTRACTIONS = w3c using a sourcetype other than iis, it does not work for defining the field names. Is there a .conf file that I can define a different sourcetype for this functionality? We have several groups with different IIS logs and I'd like to call the sourcetype iis_group1, iis_group2, etc.
Well, just in case anyone bumps into this, I guess it was quite a newbie problem. I managed to do it by deploying to the forwarder (instead of the indexer) a props.conf file with a copy of the [iis] default stanza but with the different name [iis_group1] in the example. This way the w3c fields in iis_group1 should be automatically extracted.
Give this a try
[iis_group1]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
INDEXED_EXTRACTIONS = w3c
detect_trailing_nulls = auto
category = Web
Update
Try this
[iis_group1]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
FIELD_DELIMITER = whitespace
FIELD_HEADER_REGEX = ^#Fields:\\s*(.*)
MISSING_VALUE_REGEX = -
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TIMESTAMP_FIELDS = date,time
Thanks for your response, somesoni2,
As suggested, I tried:
inputs.conf
# DOTNET IIS LOGS
[monitor://D:\Applications\*\*\W*\*.log]
sourcetype = iis_group1
index = dotnet
ignoreOlderThan = 5d
disabled = false
props.conf
[iis_group1]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
INDEXED_EXTRACTIONS = w3c
detect_trailing_nulls = auto
category = Web
That makes the data look good, but it is still not pulling in the standard IIS field names; c_ip, cs_bytes, cs_method, cs_uri_query, cs_uri_stem, cs_User_Agent, cs_username, etc
Results:
Interesting Fields
a d 15
# date_hour 1
# date_mday 1
# date_minute 1
a date_month 1
# date_second 12
a date_wday 1
hi, this is an old topic but I just found this exact same behavior. Has anyone ever found a justification for this? If I use sourcetype=iis, the w3c fields are automatically extracted. On the other hand, if I copy the [iis] stanza into my props.conf with another name and use this as the sourcetype, the fields are not extracted anymore.
Make sure you are sending a copy of your custom props.conf to the UF as well. I had the same issue but it started working when I put a copy of props.conf with my custom sourcetype on UF's.
I have created different stanzas using different sourcetypes, but the only time I can get the field extractions to work correctly is if I use [iss] as the sourcetype. If i use something like [iss_group1], the field names do not get extracted correctly. Running the same query (index=dotnet) see the examples below:
This one works using [iis] as the sourcetype:
Inputs.conf
# DOTNET IIS LOGS
[monitor://D:\Applications\*\*\W*\*.log]
sourcetype = iis
index = dotnet
ignoreOlderThan = 5d
disabled = false
props.conf
[iis]
INDEXED_EXTRACTIONS = w3c
Returns:
Interesting Fields
a c_ip 1
# cs_bytes 100+
a cs_method 2
a cs_uri_query 25
a cs_uri_stem 86
a cs_User_Agent 4
a cs_username 3
a date 1
If I change that sourcetype to [iis_group1] and use INDEXED_EXTRACTIONS = w3c, Splunk does not properly extract the field names:
inputs.conf:
# DOTNET IIS LOGS
[monitor://D:\Applications\*\*\W*\*.log]
sourcetype = iis_group1
index = dotnet
ignoreOlderThan = 5d
disabled = false
props.conf
[iis_group1]
INDEXED_EXTRACTIONS = w3c
Returns
Interesting Fields
# date_hour 24
# date_mday 5
# date_minute 60
a date_month 1
# date_second 60
a date_wday 5
# date_year 1
a date_zone 1
Create a different stanzas in the props.conf and use your stanzas (sourcetype) while indexing the log