I have a SH and 2 indexers in my setup. The two indexers when I log into those i can see the user field being extracted in the _audit index.
The SH does not auto extract that field. I thought I could just do the extraction myself by going through the extract more fields step but that also doesn't populate the field. Is there a log that would point to what the issue could be. I was on 6.6.0 this morning and have upgraded to 6.6.2 to see if that would fix the issue but it has not.
Audit:[timestamp=07-18-2017 08:54:20.038, user=admin, action=search, info=granted REST: /search/jobs/rt_md_23432565/results_preview][n/a]
I fixed it by commenting out the line
FIELDALIAS-user_for_splunk_endpoint_change = uid as user
in the file
/opt/splunk/etc/apps/Splunk_SA_CIM/default/props.conf
What is the search mode (little dropdown below the time range picker) you're using? Is it set to Fast Mode when you run your query on Search Heads (and different in Indexers)?
Verbose mode
My bad.. I didn't see the screenshot properly. Other fields like action and timestmap. Could there be any other field extraction (may be global) which is conflicting with Splunk's auto-extraction of field user? Do you have access to CLI on search head? If yes, then can you run following btool command and check if there is any specific field extraction setup for user?
/opt/splunk/bin/splunk btool props list audittrail --debug
Here is the output.
Does show the splunkappforwebanalytics using "EVAL-user = md5(clientip."_".http_user_agent)" but don't think that should interfere since when i did the extract new fields I labeled the field "audituser" at first because i had the same thought about something else using the field.
D:\Program Files\Splunk\etc\apps\search\local\props.conf [audittrail]
D:\Program Files\Splunk\etc\system\default\props.conf ANNOTATE_PUNCT = True
D:\Program Files\Splunk\etc\system\default\props.conf AUTO_KV_JSON = true
D:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE =
D:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE_DATE = True
D:\Program Files\Splunk\etc\system\default\props.conf CHARSET = AUTO
D:\Program Files\Splunk\etc\system\default\props.conf DATETIME_CONFIG = \etc\datetime.xml
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EVAL-file = if(match(file,"."),file,NULL)
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EVAL-http_channel = if(http_referer="-","Direct", if(like(http_referer_
omain,"%".site."%","Direct", if(isnull(http_channel), "Referal", http_channel)))
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EVAL-http_referer_domain = replace(http_referer_domain, "http(s|):\/\/"
"")
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EVAL-http_referer_hostname = replace(replace(replace(http_referer_domai
, "http(s|):\/\/", ""), "^(www|m|uk|r|l|tpc|lm).+", ""), "(.{1}[a-zA-Z]+)", "")
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EVAL-user = md5(clientip."".http_user_agent)
D:\Program Files\Splunk\etc\apps\splunk_monitoring_console\default\props.conf EXTRACT-apiEndTime = apiEndTime=\'(?[^\']?)\'
D:\Program Files\Splunk\etc\apps\splunk_monitoring_console\default\props.conf EXTRACT-apiStartTime = apiStartTime=\'(?[^\']?)\'
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf EXTRACT-http_locale = (?i)^(?:[^;\n]*;){3}\s+(?P[a-z]{2}(|
-][a-z]{2}));
D:\Program Files\Splunk\etc\apps\splunk_monitoring_console\default\props.conf EXTRACT-search_id = search_id=\'(?[^\']?)\'
D:\Program Files\Splunk\etc\apps\splunk_monitoring_console\default\props.conf EXTRACT-search_string = search=\'(?.?)\',\sautojoin
D:\Program Files\Splunk\etc\system\default\props.conf HEADER_MODE =
D:\Program Files\Splunk\etc\system\default\props.conf LEARN_MODEL = true
D:\Program Files\Splunk\etc\system\default\props.conf LEARN_SOURCETYPE = true
D:\Program Files\Splunk\etc\system\default\props.conf LINE_BREAKER_LOOKBEHIND = 100
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf LOOKUP-2_Channels = WA_channels Hostname AS http_referer_hostname OUTPU
Channel AS http_channel
D:\Program Files\Splunk\etc\apps\splunk_monitoring_console\default\props.conf LOOKUP-dmc_add_instance_info = dmc_assets host OUTPUTNEW machine search
group
D:\Program Files\Splunk\etc\apps\SplunkAppForWebAnalytics\default\props.conf LOOKUP-site = WA_settings source AS source host AS host OUTPUTNEW value
AS site
D:\Program Files\Splunk\etc\system\default\props.conf MATCH_LIMIT = 100000
D:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_AGO = 2000
D:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_HENCE = 2
D:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_AGO = 3600
D:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_HENCE = 604800
D:\Program Files\Splunk\etc\system\default\props.conf MAX_EVENTS = 256
D:\Program Files\Splunk\etc\system\default\props.conf MAX_TIMESTAMP_LOOKAHEAD = 128
D:\Program Files\Splunk\etc\system\default\props.conf MUST_BREAK_AFTER =
D:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_AFTER =
D:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_BEFORE =
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION = indexing
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-all = full
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-inner = inner
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-outer = outer
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-raw = none
D:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-standard = standard
D:\Program Files\Splunk\etc\system\default\props.conf SHOULD_LINEMERGE = True
D:\Program Files\Splunk\etc\system\default\props.conf TRANSFORMS =
D:\Program Files\Splunk\etc\system\default\props.conf TRUNCATE = 10000
D:\Program Files\Splunk\etc\system\default\props.conf detect_trailing_nulls = auto
D:\Program Files\Splunk\etc\system\default\props.conf maxDist = 100
D:\Program Files\Splunk\etc\system\default\props.conf priority =
D:\Program Files\Splunk\etc\system\default\props.conf sourcetype =
Well It was the "SplunkAppForWebAnalytic" app that was causing this issue. I stopped splunk, deleted the app and started it back up and now the user is being populated in the field list with a value.
Thanks for the command help.