All Apps and Add-ons

Tenable Network Security PVS App for Splunk: How to troubleshoot why field extractions are not working?

ccsfdave
Builder

Greetings,

I have a few PVS's coming through syslog via TCP. I have set index=pvs, sourcetype=pvs:internal (for these, there will be "externals" coming down the pipe in a few weeks) and the host=.

I have attempted to comment out the syslog stanza of the props.conf and collapsed the extract into the local/props.conf stanza automagically created when I set the sourcetype on the heavy forwarder to [pvs:internal]. So I now have this in my /opt/splunk/etc/apps/pvs/local/props.conf:

[pvs:internal]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
TRANSFORMS-changesourcetype = set_sourcetype_pvs
EXTRACT-PVS,src,src_port,dest,dest_port,protocol,PVS_pluginid,PVS_eventname,PVS_data,PVS_data2,PVS_risk = (?P<PVS>pvs): (?P<src>[^:]+):(?P<src_port>\d{1,5})\|(?P<dest>[^:]+):(?P<dest_port>\d{1,5})\|(?P<protocol>\d{1,3})\|(?P<PVS_pluginid>\d{1,5})\|(?P<PVS_eventname>[^\|]+)\|(?P<PVS_data>[^\|]+)\|(?P<PVS_data2>[^\|]+)?\|(?P<PVS_risk>[^\|]+)

When I went through the regex of the extract into https://regex101.com/ it seems to grab every other event (which may be a different issue), but I wanted to verify the regex.

Anyway, I am not getting any extractions which is my real issue. Can anyone offer suggestions?

Thanks,

Dave

0 Karma
1 Solution

ccsfdave
Builder

Well, I can't say I have learned anything from this but what I did is just do a field extraction through the GUI and pasted:

(?P<PVS>pvs): (?P<src>[^:]+):(?P<src_port>\d{1,5})\|(?P<dest>[^:]+):(?P<dest_port>\d{1,5})\|(?P<protocol>\d{1,3})\|(?P<PVS_pluginid>\d{1,5})\|(?P<PVS_eventname>[^\|]+)\|(?P<PVS_data>[^\|]+)\|(?P<PVS_data2>[^\|]+)?\|(?P<PVS_risk>[^\|]+)

Into the regex window. It is now working. Does anyone know whether the field extractor changes the props.conf on the deployment-apps or apps directory?

View solution in original post

0 Karma

mokuso
Explorer

I agree with @rich7177 and I would have expected the linemerge setting to fix your issue. Can you try adding both of the following to your props.conf:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)

regex101 is going to only return the first match unless you add the 'g' global modifier.
https://regex101.com/r/dY6wN0/1

0 Karma

ccsfdave
Builder

Well, I can't say I have learned anything from this but what I did is just do a field extraction through the GUI and pasted:

(?P<PVS>pvs): (?P<src>[^:]+):(?P<src_port>\d{1,5})\|(?P<dest>[^:]+):(?P<dest_port>\d{1,5})\|(?P<protocol>\d{1,3})\|(?P<PVS_pluginid>\d{1,5})\|(?P<PVS_eventname>[^\|]+)\|(?P<PVS_data>[^\|]+)\|(?P<PVS_data2>[^\|]+)?\|(?P<PVS_risk>[^\|]+)

Into the regex window. It is now working. Does anyone know whether the field extractor changes the props.conf on the deployment-apps or apps directory?

0 Karma

Richfez
SplunkTrust
SplunkTrust

I believe your issue is one of two things. Your regex is grabbing the beginning of the next event as part of its last extraction, so it may be possible to just fix it by adding a $ at the very end of your regex.

However, I think the better fix is to adjust your linebreaking. If you add to your props.conf the one line

SHOULD_LINEMERGE = false

It may clear this up better. Can you try that?

0 Karma

ccsfdave
Builder

@rich7177 thanks for the response

Unfortunately the linemerge didn't have any affect.

I put a $ in the regex tester: https://regex101.com/#pcre and it did nothing. What does sorta work is put an new line (\n) just before the last parenthesis however it reads everything except the last entry. Which may be ok as logs roll in but it is not tested. I suppose I will.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Could you paste a couple of the events themselves? If it's necessary to sanitize them, please be careful to not change too much.

0 Karma

ccsfdave
Builder
May 13 09:15:44 10.x.x.x pvs: 10.x.x.x:53|10.y.y.y:53|17|7024|DNS Client Queries|PVS has observed this host perform a DNS lookup. The most recent DNS query performed was for: |hostname.some.ORG to the server at 10.y.y.y|NONE
May 13 09:15:42 10.x.x.x pvs: 10.y.y.y32:61565|10.z.z.z:80|6|7041|HTTP request detection|The following GET/POST request was observed:|DIP: 10.z.z.z:80;URI: /department/Admin/ProjectManage/Lists/Develop%20New%20PM%20tracking/Research;Referer: None;Host: hostname2.some.org;Query: YES;PROTO: 1.0|NONE
May 13 09:15:42 10.x.x.x pvs: 10.y.y.y32:61566|10.z.z.z:80|6|7041|HTTP request detection|The following GET/POST request was observed:|DIP: 10.z.z.z:80;URI: /committee_name/committees/budget_perf/DeptITPlansFY201516ThroughFY201920/CON;Referer: None;Host: hostname2.some.org;Query: NO;PROTO: 1.0|NONE
May 13 09:15:42 10.x.x.x pvs: 10.y.y.y32:61565|10.z.z.z:80|6|7041|HTTP request detection|The following GET/POST request was observed:|DIP: 10.z.z.z:80;URI: /department/Admin/ProjectManage/Lists/Develop%20New%20PM%20tracking/Research;Referer: None;Host: hostname2.some.org;Query: NO;PROTO: 1.0|NONE
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...