TA-meraki: How do I remove single quotes from syslog not working with props/transforms?

[ColinJacksonPS]

I've got data coming in from Meraki APs. It's mostly good, but I can't get rid of the single quotes at index time.

Data:

Nov 15 17:08:27 192.168.255.1 1 1542301707.870571288 main_office events content_filtering_block url='https://baddymcbadface.com/...' category0='Spyware and Adware' server='123.123.123.123:443' client_ip='192.168.204.80'

Here's what I have set up in my props and transforms.conf

Transforms:

[meraki-singlequotes]
REGEX = \s*([^=]+)='([^']*)'
FORMAT = $1::$

Props:

[source:meraki]
REPORT-singlequotes = meraki-singlequotes

I've also tried changing this with [sourcetype:meraki-hq], but that doesn't work as well.

I'm bringing this in via syslog. inputs.conf for this is in the SPLUNK_HOME/etc/apps/launcher/local/ I've added them there, but no dice.
I've also put them in the TA-meraki as well, but it's not taking. Does anyone have any suggestions?

[ColinJacksonPS · Accepted solution]

Here are my updated conf files. Looks like it's still not taking. Thanks @laurie_gellatly and @FrankVl

Props.conf
[source::meraki]
SEDCMD-singlequotes=s/\s*([^=]+)='([^']*)'/g

Transforms.conf (though, I think if I'm doing SEDCMD command, I don't need transforms anymore)
[meraki-singlequotes]
REGEX = \s*([^=]+)='([^']*)'
FORMAT = $1::$2

Anonymized output:

Nov 19 18:42:58 192.168.200.15 1 1542652978.961504488 Board_Room events type=disassociation radio='1' vap='0' client_mac='E4:2B:DD:DD:DD:DD' channel='36' duration='141.595751334' auth_neg_dur='0.839737584' last_auth_ago='140.746016874' is_8021x='1' full_conn='1.309590753' ip_resp='1.309590753' ip_src='192.168.207.2' arp_resp='0.839737584' arp_src='192.168.207.2' dns_server='222.222.222.222'

[FrankVl]

That’s not how SEDCMD works. It should be s/<regex matching what needs to be replaced>/<what it needs to be replaced with>/g (the g is to keep applying it, rather than replacing just the first match).

In your case, that could be as simple as s/'//g.

[ColinJacksonPS]

That did it. Thanks.

[laurie_gellatly]

And remember it won't fix what you've already ingested into the index. Only new ingestions will be altered!

[FrankVl]

I now have a moment to take a closer look and provide a more extensive answer (apart from my earlier comment about REPORT not being performed at indextime), there are several problems with your attempts:

• A source based stanza in props.conf needs to be defined by [source::<source>], note the double :.
• If you want to use a sourcetype based stanza instead, it is just [<sourcetype>], no sourcetype: in front of the actual sourcetype.
• The regex isn't perfect, first fieldname will be the entire start of the event up until the first =; see also: https://regex101.com/r/1LJ1Wl/1
• as @laurie_gellatly mentions: you're missing a 2 at the end of your FORMAT string.

[laurie_gellatly]

Also your transform needs to $1::$2
And as already stated it happens at search not index time.
If it MUST happen at index then you could look at using sedcmd.

...Laurie:{)

[FrankVl]

A REPORT transform works at searchtime, not at indextime.

Can you perhaps explain a bit further what is and what isn't working, what output do you get now and what is still wrong with that which your are trying to fix?