How to write the regex and break and extract a field to get the required data?

[kamal_jagga]

I need to create a report of failed services.
From the logs, I get the data in the below mentioned format:

*******$$$$$$$$$$$$$##########ServiceInvoker,1.4,S,400,0,ServiceName########***
ServiceInvoker,1.4,S,400,0,ServiceName1**********########

!!!!!!!!!!!!!!ServiceInvoker,1.4,S,400,0,ServiceName2

*******$$$$$$$$$$$$$##########ServiceInvoker,1.4,S,400,0,ServiceName3########***
ServiceInvoker,1.3,F,400,0,ServiceName2**********########

!!!!!!!!!!!!!!ServiceInvoker,1.4,S,400,0,ServiceName1

Note: *,#,! is any random log data
Service failure is determined by the status here (eg. S for Success, F for Failed as above).

Now I am able to write the search and search the time instance in which the Service fails.

But if i try and extract the required set of data as a field.

(ServiceInvoker,1.4,S,400,0,ServiceName2)
It gives the whole set of data and I am not sure how to break or use it.

Req is
1. parse all the service invoker values.
2. If the status is F then provide the servicename.

Kindly advise.

[richgalloway]

Once you have the extracted data, use rex to parse it. Regex strings do not provide for conditional parsing, so you'll end up with a servicename regardless of status.

... | rex field=foo "[^,]+,[^,]+,(?P<Status>[^,]+),[^,]+,[^,]+,(?P<Servicename>.*)" | ...

[kamal_jagga]

Hey ,

I am new to this. So if i ask you some silly question. Kindly forgive.

Now when i try to use the above

......|rex field=ServiceInvoker "[^,]+,[^,]+,(?P[^,]+),[^,]+,[^,]+,(?P.*)" | table ServiceInvoker STATUS Servicename

I am getting the table with only values in STATUS and other columns are blank.

1. not sure of the value thats coming in field. Will it be Status or Servicename.
2. Also, my requirement is to display all the services with status F or SE.

[kamal_jagga]

index=cricket_index_idx sourcetype=tv (STATUS="F" OR STATUS="SE") HOST=cricket (STEP=) STEP="europe/cricket/ronaldo/" |rex field=ServiceInvoker "[^,]+,[^,]+,(?P[^,]+),[^,]+,[^,]+,(?P.)" | table ServiceInvoker STATUS Servicename

this is my whole query.
It gives me output of the server. Now the data contains various sets of ServiceInvoker,1.4,S,400,0,ServiceName2 with different services and different status.

I need services with above status only.

[richgalloway]

First, code is easier to read if it's put on a line that starts with 4 spaces.

Since the STATUS field is created by the rex command, it cannot be used before the rex command. Unless you already have a STATUS field in your event, which means rex should be creating a different field name.

Field names are case-sensitive in Splunk.

What does this do?

index=cricket_index_idx sourcetype=tv (ServiceInvoker="*,F,*" OR ServiceInvoker="*,SE,*") HOST=cricket STEP="europe/cricket/ronaldo/" |rex field=ServiceInvoker "[^,]+,[^,]+,(?P<STATUS>[^,]+),[^,]+,[^,]+,(?P<Servicename>.*)" | table ServiceInvoker STATUS Servicename

[kamal_jagga]

index=cricket_index_idx sourcetype=tv (ServiceInvoker=",F," OR ServiceInvoker=",SE,") HOST=cricket STEP="europe/cricket/ronaldo/" |rex field=ServiceInvoker "[^,]+,[^,]+,(?P[^,]+),[^,]+,[^,]+,(?P.*)" | table ServiceInvoker STATUS Servicename

This gives an error and no output is displayed.

And, Yes STATUS is a already extracted field.

And if I update the name in rex. It just generates blank table and does not provide any output.
index=cricket_index_idx sourcetype=tv (STATUS=",F," OR STATUS=",SE,") HOST=cricket STEP="europe/cricket/ronaldo/" |rex field=ServiceInvoker "[^,]+,[^,]+,(?P[^,]+),[^,]+,[^,]+,(?P.*)" | table ServiceInvoker service_status Servicename

[richgalloway]

I think we need to take a step back. If STATUS is already a separate field then there's no need to extract it from ServiceInvoker. It also calls into question the contents of ServiceInvoker. If you're willing, please share the result of the following (anonymize as needed) so we can see your fields and sample data (1 or 2 rows):

index=cricket_index_idx sourcetype=tv (ServiceInvoker="*,F,*" OR ServiceInvoker="*,SE,*") HOST=cricket STEP="europe/cricket/ronaldo/" | fields - _* | table *

[kamal_jagga]

index=cricket_index_idx sourcetype=tv (ServiceInvoker=",F," OR ServiceInvoker=",SE,") HOST=cricket STEP="europe/cricket/ronaldo/" | fields - _* | table *

This gave error message.
When i tried the following

** index=cricket_index_idx sourcetype=tv (service_status=",F," OR service_statusr=",SE,") HOST=cricket STEP="europe/cricket/ronaldo/" | fields - _* | table ***
It gave the whole log data again.

Sample data is appserver logs. In this the pattern from server appears many times. Sometimes the status is S i.e Success and at other times it is F or SE i.e not successful. We need to capture all the instances when it is F or SE. And in this pattern only, after the status, service name for which the status is F or SE is mentioned.

So, we need to check the status. if the status is F or SE then capture Service name

$$$$$$$$$$$$$##########ServiceInvoker,1.4,S,400,0,ServiceName########*
ServiceInvoker,1.4,S,400,0,ServiceName1**########

!!!!!!!!!!!!!!ServiceInvoker,1.4,S,400,0,ServiceName2