All Apps and Add-ons

Update of Splunk WSA Plugin to support WSA 10.x

tmayer
Explorer

Hi,

The current plugin is supporting WSA with 9.x logs. Cisco has already released 10.0 for some time now and is coming out with 10.5 release in two weeks. Logs from 10.x Appliances are not correctly parsed because there have been some changes in the accesslogs on the WSA. Is there a timeline when the Splunk Plugin for WSA will support WSA with 10.x logs?

bigjohnmax
New Member

mine worked with this regex:
^(?<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<field2>[^\s]+)\s+(?P<x_elapsed_time>\w+\s+\d+\s+\d+:\d+:\d+)\s(?P<field4>\w+.\w+):\s+(?P<field5>\w+):\s(?P<field6>\d+.\d+)\s(?P<field7>\d+)\s(?<src_ip>[a-zA-Z0-9:.]*)\s+(?<txn_result_code>[A-Z_]*)\/(?<status>[0-9]*)\s+(?<bytes_in>[0-9]*)\s+(?<http_method>\w*)\s+(?<url>\S*)\s+["|']?(?<user>[^\s"']+)["|']?\s+(?<server_contact_mode>[^\/]+)\/(?<dest>\S*)\s+(?<http_content_type>\S*)\s+(?<acltag>\S*)\s+(?:<|&lt;)(?<x_webcat_code_abbr>[^,]+),(?<wbrs_score>[^,]+),["|']?(?<x_webroot_scanverdict>[0-9]{0,2}|\-|\w+)["|']?,["|']?(?<webroot_threat_name>[^,"']+)["|']?,(?<x_webroot_trr>[^,]+),(?<x_webroot_spyid>[^,]+),(?<x_webroot_trace_id>[^,]+),(?<x_mcafee_scanverdict>[^,]+),["|']?(?<x_mcafee_filename>[^,]+?)["|']?,(?<x_mcafee_scan_error>[^,]+),(?<x_mcafee_detecttype>[^,]+),(?<x_mcafee_av_virustype>[^,]+),["|']?(?<x_mcafee_virus_name>[^,]+?)["|']?,(?<x_sophos_scanverdict>[^,]+),(?<x_sophos_scancode>[^,]+),["|']?(?<x_sophos_file_name>[^,]+?)["|']?,["|']?(?<x_sophos_virus_name>[^,]+?)["|']?,(?<x_ids_verdict>[^,]+),(?<x_icap_verdict>[^,]+),(?<x_webcat_req_code_abbr>[^,]+),["|']?(?<x_webcat_resp_code_abbr>[^,]+?)["|']?,["|']?(?<x_resp_dvs_threat_name>[^,]+?)["|']?,["|']?(?<x_wbrs_threat_type>[^,"']+)["|']?,["|']?(?<x_avc_app>[^,"']+)["|']?,["|']?(?<x_avc_type>[^,"']+)["|']?,["|']?(?<x_avc_behavior>[^,"']+)["|']?,["|']?(?<x_request_rewrite>[^"',]+)["|']?,(?<x_avg_bw>[^,]+),(?<x_bw_throttled>[^,]+),(?<x_user_type>[^,]+),["|']?(?<x_resp_dvs_verdictname>[^,"']+)["|']?,["|']?(?<x_req_dvs_threat_name>[^,"']+)["|']?(,["|']?(?<x_amp_verdict>[^,"']+)["|']?,["|']?(?<x_amp_malware_name>[^"']+)["|']?,(?<x_amp_score>[^,]+),(?<x_amp_upload>[^,]+),["|']?(?<x_amp_filename>[^,]+?)["|']?,["|']?(?<x_amp_sha>[^"',]+)["|']?)?(,["|']?(?<x_file_verdict>[^"',]+)["|']?)?(,(?<x_archive_scan_verdict>[^,]+),["|']?(?<x_archive_scan_verdict_reason>[^"']+)["|']?),(?P<field42>[^,]+)(?:\>)\s(?:\")?(?P<field43>[^,]+)(?:\")?$

0 Karma

HomeGrownCoder
Engager

Here is a quick edit to hold you over until the application is patched. I added update place holders in for the new fields. I am working with cisco to try and get the correct field names for these values. It does not appear to be documented. The three capture groups before the last needs to added

(?[0-9.]+)\s+(?[0-9]+)\s+(?[a-zA-Z0-9:.]*)\s+(?[A-Z_]*)/(?[0-9]*)\s+(?[0-9]*)\s+(?\w*)\s+(?\S*)\s+"?(?[^\s"]*)"?\s+(?[^/]*)/(?\S*)\s+(?\S*)\s+(?\S*)\s+(?:<|&lt;)(?[^,]+),(?[^,]+),"*(?[0-9]{0,2}|\-|\w+)"*,"(?[^"]+)",(?[^,]+),(?[^,]+),(?[^,]+),(?[^,]+),"(?[^,]+)",(?[^,]+),(?[^,]+),(?[^,]+),"(?[^"]+)",(?[^,]+),(?[^,]+),"(?[^"]+)","(?[^"]+)",(?[^,]+),(?[^,]+),(?[^,]+),(?[^,]+),"(?[^"]+)","(?[^,]+)","(?[^,]+)","(?[^,]+)","(?[^,]+)","(?[^,]+)",(?[^,]+),(?[^,]+),(?[^,]+),"(?[^,]+)","(?[^,]+)"(,(?[^,]+),"(?[^"]+)",(?[^,]+),(?[^,]+),"(?[^"]+)","(?[^"]+)"),(?[^"]+),(?[^"]+),"(?[^"]+)"?(?:>|&gt;)\s+(?"[^"]*"|-)?\s*.*$
0 Karma

Alliant
New Member

Any update on this? I'm not sure how or where to apply what was posted above.

Thanks

0 Karma

Yorokobi
SplunkTrust
SplunkTrust

I removed the last capture group from the scan_verdict_info_extract in transforms.conf in order to get that extraction to work properly (and the related automatic lookups to work), leaving the new(?) fields unaccounted for.

It looks like the x-result-code for WSA 10.x has three additional fields and the capture group at the end of the giant regex (?:>|&gt;) breaks the entire extraction.

<IW_infr,3.9,0,"-",0,0,0,1,"-",-,-,-,"-",1,-,"-","-",-,-,IW_infr,-,"Unknown","-","Unknown","Unknown","-","-",288.91,0,Local,"Unknown","-",-,"-",-,-,"-","-",-,-,"-">

Our Ironports are currently running on 10.1.0-204 for Web

0 Karma

knalla
Path Finder

we are having the same issue, Can you please paste the stanza from transforms.conf.

0 Karma

mplaksin
Engager

Hi, we are having some problem.
Where wich transforms.conf you edited:

./etc/system/local/transforms.conf
./etc/system/default/transforms.conf
./etc/apps/Splunk_TA_cisco-wsa/default.old.20170421-151119/transforms.conf
./etc/apps/Splunk_TA_cisco-wsa/default/transforms.conf
./etc/apps/maps/default/transforms.conf
./etc/apps/search/default/transforms.conf
./etc/apps/SplunkforCiscoIronportWSA/default/transforms.conf
./etc/apps/splunk_monitoring_console/default/transforms.conf
./etc/apps/cisco_wsa_reporting/etc/system/default/transforms.conf
./etc/apps/cisco_wsa_reporting/etc/apps/cisco_cws_addon/default/transforms.conf
./etc/apps/cisco_wsa_reporting/etc/apps/cisco_wsa_reporting/default/transforms.conf
./etc/apps/cisco_wsa_reporting/etc/apps/splunk_management_console/default/transforms.conf
./etc/apps/Splunk_TA_cisco-asa/default/transforms.conf

Can you give one example before and after?

Thanks!

0 Karma

Yorokobi
SplunkTrust
SplunkTrust

$SPLUNK_HOME/etc/apps/Splunk_TA_cisco-wsa/local/transforms.conf

You may have to create the local/ directory then copy default/transforms.conf to local/ to edit. Afterwards, restart Splunk.

0 Karma

Yorokobi
SplunkTrust
SplunkTrust
[scan_verdict_info_extract]                                                                                                                                           
SOURCE_KEY=scan_verdict_info                                                                                                                                          
REGEX= (?:<|&lt;)(?<x_webcat_code_abbr>[^,]+),(?<wbrs_score>[^,]+),"*(?<x_webroot_scanverdict>[0-9]{0,2}|\-|\w+)"*,"(?<webroot_threat_name>[^"]+)",(?<x_webroot_trr>[^
,]+),(?<x_webroot_spyid>[^,]+),(?<x_webroot_trace_id>[^,]+),(?<x_mcafee_scanverdict>[^,]+),"(?<x_mcafee_filename>[^,]+)",(?<x_mcafee_scan_error>[^,]+),(?<x_mcafee_det
ecttype>[^,]+),(?<x_mcafee_av_virustype>[^,]+),"(?<x_mcafee_virus_name>[^"]+)",(?<x_sophos_scanverdict>[^,]+),(?<x_sophos_scancode>[^,]+),"(?<x_sophos_file_name>[^"]+
)","(?<x_sophos_virus_name>[^"]+)",(?<x_ids_verdict>[^,]+),(?<x_icap_verdict>[^,]+),(?<x_webcat_req_code_abbr>[^,]+),(?<x_webcat_resp_code_abbr>[^,]+),"(?<x_resp_dvs_
threat_name>[^"]+)","(?<x_wbrs_threat_type>[^,]+)","(?<x_avc_app>[^,]+)","(?<x_avc_type>[^,]+)","(?<x_avc_behavior>[^,]+)","(?<x_request_rewrite>[^,]+)",(?<x_avg_bw>[
^,]+),(?<x_bw_throttled>[^,]+),(?<x_user_type>[^,]+),"(?<x_resp_dvs_verdictname>[^,]+)","(?<x_req_dvs_threat_name>[^,]+)"(,(?<x_amp_verdict>[^,]+),"(?<x_amp_malware_n
ame>[^"]+)",(?<x_amp_score>[^,]+),(?<x_amp_upload>[^,]+),"(?<x_amp_filename>[^"]+)","(?<x_amp_sha>[^"]+)")?(?:>|&gt;)
0 Karma

tmayer
Explorer

bumping up the question... Anyone has any insights?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...