Splunk Search

Field Extraction issue

rajgowd1
Communicator

HI Experts,
i am able to exact 4th and 5th fields from below log but i am able to exact get the value if the 4th or 5th filed is HOSTNAME but if it is IPaddress then i am not able retrieve.here is the sample log

tcp 0 0 10.40.88.178:7171 10.40.88.175:50326 ESTABLISHED
tcp 0 0 12b8-splfwd02.ndm.nsro:7171 10.40.88.170:50326 TIME_WAIT
tcp 0 0 10.40.88.178:7171 poc-card-luna2.nad.ns:50326 TIME_WAIT
tcp 0 0 12b8-splfwd02.ndm.nsro:7171 10.30.88.145:50326 ESTABLISHED
tcp 0 0 10.40.88.178:7171 poc-card-luna4.nad.ns:50326 SYNC_SENT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 poc-card-luna1.nad.ns:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 10.30.88.117:46756 SYNC_SENT
tcp 0 0 10.40.88.178:7171 poc-card-luna2.nad.ns:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 10.40.83.157:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 poc-card-luna4.nad.ns:46756 ESTABLISHED
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna1.nad.ns:46756 TIME_WAIT
tcp 0 0 10.40.88.178:7171 poc-card-luna3.nad.ns:46756 SYNC_SENT
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna2.nad.ns:46756 SYNC_SENT
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna4.nad.ns:46756 ESTABLISHED
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna2.nad.ns:46756 ESTABLISHED

Tags (1)
0 Karma
1 Solution

govindsinghrawa
Path Finder

Below Regex can be used in field extractor in write your own regular expression syntax:


^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)

During search time this regex can also be used with rex to extract fields if fields are not already extracted:


yourBaseSearch
|rex field=_raw "^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)"
| table hostOrIp1, hostOrIp2

NOTE: Please remove the space in each of the tags, example "< protocol>", "< field1>" etc.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you're using the Splunk UI you're probably using the automatic field extractor which is most likely producing a regex string that is too specific. Try this regex, instead. It skips the first three space-delimited fields and extracts the next two space-delimited fields as 'src' and 'dest', respectively.

(?:[^\s]+\s){3}(?P<src>[^\s]+)\s(?<dest>[^\s]+)

Perhaps you want the 4th and 5th fields to be the host name/IP address and port. That regex string is

(?:[^ ]+ ){3}(?P<src>[^:]+):(?<dest>\d+)
---
If this reply helps you, Karma would be appreciated.
0 Karma

govindsinghrawa
Path Finder

Below Regex can be used in field extractor in write your own regular expression syntax:


^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)

During search time this regex can also be used with rex to extract fields if fields are not already extracted:


yourBaseSearch
|rex field=_raw "^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)"
| table hostOrIp1, hostOrIp2

NOTE: Please remove the space in each of the tags, example "< protocol>", "< field1>" etc.

0 Karma

rajgowd1
Communicator

Thank you govindsinghrawat.i am trying to create visualization by using all your experts answers.
is there way to create visualization by using similar search

both are giving same results

index=netstat|rex field=_raw "^(?[\S]+)\s*?(?[\S]+)\s*?(?[\S]+)\s*?(?[^:]+):(?[\S]+)\s*?(?[^:]+):(?[\S]+)\s.*?(?[\S]+)"
| eventstats count by state hostOrIp1 hostOrIp2 |eval CountConnStatus=count+" - "+state|chart values(CountConnStatus) by hostOrIp1 hostOrIp2

index=netstat |rex "(?[a-zA-Z0-9][^\s]+)\s(?[^\s]+)\s(?.*)" |eventstats count by CONN_STATUS CBOL_SERVER HSM |eval CountConnStatus=count+" - "+CONN_STATUS|chart values(CountConnStatus) by CBOL_SERVER HSM

0 Karma

govindsinghrawa
Path Finder

Hi @rajgowd

The above regex which I gave is to extract fields. Once the fields are extracted then you can use them i your commands.

Now coming to visulaizations. Visualizations can only be created depending on how you end your search, for example :

if you end your search with timechart command then you will immediately have visualization options for line, bar etch. Example:
yourBaseSearch | timechart count by yourField

If you have statistics being generated as aggregations like min, max, sum etc you can plot them using timechart, chart etc.

0 Karma

rajgowd1
Communicator

Hi Rich,thanks for your response.
From logs,I am trying to extract using splunk UI.
I think I am able extract multiple fields.

0 Karma

rajgowd1
Communicator

Hi Rich,thanks for your response.
From logs,I am trying to extract using splunk UI.
I think I am able extract multiple fields.

0 Karma

sundareshr
Legend

You have few options.

1) Edit props/transforms per instructions in the blog post. Your data appears to be space delimited, so you will specific " " in your transforms instead of ","

http://blogs.splunk.com/2013/03/11/quick-n-dirty-delimited-data-sourcetypes-and-you/

2) The other option is to use Interactive Field Extractor (IFX) to extract the fields as space delim. IFX will generate the similar props/transforms edits for the extractions.

3) Use regex in your SPL like this

base search | rex "(?<f1>[^\s]+)\s(?<f2>[^\s]+)\s(?<f3>[^\s]+)\s(?<f4>[^\s]+)\s(?<f5>[^\s]+)\s(?<f6>[^\s]+)" | table f*
0 Karma

richgalloway
SplunkTrust
SplunkTrust

How are you extracting the fields? Is this a search-time or index-time extraction?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...