Unable to get the Cisco Networks App working

[damode]

I have a Kiwi Syslog server where all Cisco logs are getting stored in folders with the host IP as the folder name.
I have configured an app in the universal forwarder that reads those logs. Below is the inputs.conf,

[monitor://E:\SyslogData\10.10.10*\log*]
host_segment = 2
index = test_index
sourcetype = syslog
disabled = 0

When the Kiwi log file format is set to Kiwi format ISO yyyy-mm-dd (Tab delimited) while data encoding is UTF-8. The host value extracted is Local0.Info.

And with the above settings, the logs in kiwi look like below :

2018-08-01 16:06:30 Local0.Info 10.10.10.X.X    1 153232424240.525452732241 cfC_G_AP07 flows allow src=13.13.140.249 dst=12.12.122.21 mac=43:43:5C:83:25:CA protocol=t3p sport=7240 dport=6224

The only exception when the extraction with the above inputs.conf works well is if the log format is changed in Kiwi to either "message text only (no priority) or raw logging", however, even in that case the data does not get populated in the Cisco Networks app. It still shows no data.

Last thing I tried was I uninstalled my custom app I had deployed in UF that had the above inputs.conf and deployed Cisco Networks add-on in the UF with the above inputs.conf, in the hope that maybe transforms.conf from the add-on would properly parse data and send to Indexer. But after doin that, I have stopped receiving data altogether in Splunk.

Arch Info : 1 S.H, 1 Indexer, 1 HF

I have tried numerous way to try to make this work but nothing has helped. Please help

[FrankVl]

Two things:

Try and configure Kiwi to write the full raw log as it received it from the network. That should ensure it is in the format as the TA expects it.

When using sourcetype=syslog, you automatically get the syslog hostname extraction transform for free, that Splunk has built in in some default config file. Either use the cisco TA's specific sourcetype, or override the syslog hostname transform to block that host field extraction (since you are already populating the host field using the host_segment setting, so you don't want that to get overwritten). You can do that by adding the following to a local props.conf file.

[source::E:\SyslogData\10.10.10*\log*]
TRANSFORMS = 

Which prevents the TRANSFORMS = syslog-host from system/default/props.conf from getting applied.

[damode]

Hi @FrankVI,
Configured Kiwi log file format as raw logs

1. In 1st attempt, removed host_segment=2 to check whether Cisco's add-on extracts the IP but it didnt work as expected. Resulting host value = syslog server IP.
2. 2nd attempt, brought back host_segment=2, host IP extracted properly. However, the Cisco networks app is still not showing any data. it looks like transforms is still not getting applied. Its already using the sourcetype = syslog as shown above in monitoring stanza of in the inputs.conf

[FrankVl]

And you have deployed the add-on to your indexers and search heads as well?

What does the data in the log file now look like?

[damode]

Yes, the add-on has been deployed literally everywhere, the S.H, indexer, heavy forwarder.

The data is in this format now,

<134>1 14245934.01624555 tfC_G_Af14 flows allow src=14.15.150.50 dst=152.56.5.51 mac=50:5E:55:53:82:30 protocol=tcp sport=3560 dport=56

Thats the Cisco PIX PFSS format (raw logging) set in Kiwi

[FrankVl]

When I look at the props and transforms files of the add-on, that doesn't include anything that seems to match these type of logs.
I'm wondering if this particular cisco product is supported by the add-on, or whether there is still something weird going on with the log format somehow.

[damode]

You are right. There was a miscommunication with the client. These logs are actually from Meraki!
Thanks for your assistance on this.