Has anyone had issues with the Splunk Add-on for F5 BIG-IP setting/separating the sourcetypes?
According to the documentation, if the sourcetype is set to f5:bigip:syslog, the data should be separated into its specific sourcetypes (f5:bigip:apm:syslog, f5:bigip:asm:syslog, f5:bigip:icontrol, etc), but all of our sourcetypes are still coming in as f5:bigip:syslog.
Is there anything specific that I need to change in the configuration files so that these sourcetypes are automatically updated?
Thanks.
@jonesnadiam wrote:Has anyone had issues with the Splunk Add-on for F5 BIG-IP setting/separating the sourcetypes?
According to the documentation, if the sourcetype is set to f5:bigip:syslog, the data should be separated into its specific sourcetypes (f5:bigip:apm:syslog, f5:bigip:asm:syslog, f5:bigip:icontrol, etc), but all of our sourcetypes are still coming in as f5:bigip:syslog.Is there anything specific that I need to change in the configuration files so that these sourcetypes are automatically updated?
Thanks.
yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format
and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs
which will not be presented on Datamodels Or dashboards
so i make a new files at local folder
props.conf
and tags.conf
### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled
[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled
AND eventtypes.conf
[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)
[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)
and now everything is working fine and data are tagged
this should be added to addons in the next release
the same issue with F5-addons too
when i change logging profile for F5 v15.1 as below options
1. F5 Logging Profile (Syslog ) ==> addon F5-big IP not working as log come in below format not like
F5-addons format at props/transform files.
130>Sep 30 10:39:44 F5-01.*.com ASM:unit_hostname="F5-01.*.com"
and F5-add on match only below format
<131>Sep 12 23:53:50 F5-01.*.com ASM:f5_asm=Splunk-F5-AS
when i change logging profile from F5 v15.1 to pre-define template format call "Splunk"
2. F5 Logging Profile (Splunk) ==> logs come in duplicated event parameters
when i change logging profile for F5 v15.1 to custom template at Splunk-Docs
Configure F5 Logging Profiles for ASM
https://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup
3. 2. F5 Logging Profile (Custom template) ==> logs come in duplicated event parameters too
We're having the same issue with the f5 add-on. The problem is the add-on because if you don't use the add-on the event braking works with 8.2 just fine. The sub sorucetype [f5:bigip:apm:syslog] works fine. The root sourcetype [f5:bigip:syslog] isn't parsing correctly for TCP stream or Syslog format.
BSD_format.txt
Event 1:
May 17 23:33:58 %masked_host% apmd[13742]: 01490115:5: /Common/fs.training.np2.navy.mil_modern:Common:cbd11dc1: Following rule 'CAC' from item 'Logon Authentication Type Switch' to terminalout 'CAC'
Event 2:
May 17 23:34:01 %masked_host% /Common/fs.training.np2.navy.mil_modern: Common:cbd11dc1: iRule access_policy_default CLIENTSSL_CLIENTCERT | DEBUG | Got 2 certs ||
We had our F5 SME provide two different format that configurable with the F5 appliance.
Here is the line brake by the TA.
[f5:bigip:syslog]LINE_BREAKER = ([\r\n]+)(.*)(f5_irule|[^"]f5_asm|:\s(?:\d{4})[0-9A-Fa-f]{4}:\d+:\s+(?:[^:\s]{1,100}:[^:\s]{1,100}:\s*)?[0-9a-f]{8}:|( debug | info | warning | err | notice | alert | crit | emerg ))
Fails to parse the above two events.
Splunk_format.txt
Fails to line brake with for all the events.
[f5:bigip:syslog]LINE_BREAKER = ([\r\n]+)(.*)(f5_irule|[^"]f5_asm|:\s(?:\d{4})[0-9A-Fa-f]{4}:\d+:\s+(?:[^:\s]{1,100}:[^:\s]{1,100}:\s*)?[0-9a-f]{8}:|( debug | info | warning | err | notice | alert | crit | emerg ))
Were you able to make progress?
yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format
and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs
which will not be presented on Datamodels Or dashboards
so i make a new files at local folder
props.conf
and tags.conf
### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled
[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled
AND eventtypes.conf
[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)
[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)
and now every thing is working fine and data are tagged
this should be added to addons in next release
@esmat777 Could you explain in more which sourcetype worked after fix the f5_bigip_asm_syslog? I don't see any difference between your prop.conf, eventtpe.conf and tags. Are you saying you had the Splunk system admin to change the regex on f5:bigip:asm:syslog? If yes, how are you are ingestion the F5 logs? TCP or syslog? Are you able to see the following sourcetypes: TRANSFORMS-sourcetype=f5_bigip-irule-default, f5_bigip-irule-http, f5_bigip-irule-dns-request, f5_bigip-irule-dns-response, f5_bigip-irule-lb-failed, f5_bigip-syslog-asm, f5-bigip-apm-syslog, f5_bigip-irule-exclude-audit, f5_bigip-secure, f5_bigip-ltm-ssl-error, f5_bigip-ltm-tcl-error, f5_bigip-ltm-traffic, f5_bigip-ltm-log-error.
Can you create idea of your fix so, that it could be included in the next release of the add-on?
You need to make sure the TA is installed on the indexer or heavy forwarder if you are using them. Due to the line breaks etc a Universal forwarder will not expand the events in the transforms to give you the expanded source types.
Yes. This isn't the issue. It's different beast entirely.
this is not an issue where to install the addon
it is related to a raw data format which is not compatible with addon
yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format
and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs
which will not be presented on Datamodels Or dashboards
so i make a new files at local folder
props.conf
and tags.conf
### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled
[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled
AND eventtypes.conf
[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)
[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)
and now everything is working fine and data are tagged
this should be added to addons in the next release @jonesnadiam
This could happened if F5 is not configured well (sending logs format).
Should be like this:
http://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup
Regards,
Wojtek
Hmm i got same problem, splunk not separating sourcetypes.
I try follow http://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup but the splunk only receive syslog audit.
Hi,
Got same problem here. Add-on cannot extract fields for ASM audit events.
Sourcetype: f5:bigip:syslog
Port: udp:9514
Have you guys get through this issue?
Hello,
Did you get any answer about this question ? I have the same problem for all my F5:big IP logs
yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format
and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs
which will not be presented on Datamodels Or dashboards
so i make a new files at local folder
props.conf
and tags.conf
### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled
[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled
AND eventtypes.conf
[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)
[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)
and now everything is working fine and data are tagged
this should be added to addons in the next release @pvuong @jonesnadiam @ianbow_concur @sirajnp