Getting Data In

Displaying specific text from logs

ssingh313
Path Finder

I have a table on my dashboard that displays particular information from logs but I am trying to add an event name to it.
This is how the data appears in my log.
6/7/16
9:31:13.213 AM 09:31:13.2130 [0] 2.1.000.00 distribute_event: message EventAttachedDataChange.

5/30/16
9:30:13.613 AM 09:30:13.6130 [0] 2.1.000.00 distribute_user_event: message EventDiverted

So what I want is to get the distribute_event message as the Event Name. That means on my table there should be an Event Name column and the event name should display "EventAttachedDataChange" or "EventDiverted"

Also currently my logs show 2 timestamps as can be seen above one created by splunk which is the left most timestamp (with the date) and the other timestamp is created when the log is generated (which is right next to the splunk timestamp). Currently my table displays the splunk time stamp and date but is there a way I can keep the date generated by splunk but the timestamp that was generated when the log wad created since that is more accurate.

0 Karma
1 Solution

sundareshr
Legend

For the timestamp, splunk would have extracted the timestamp from the filed in your log file, at the time of indexing. From the looks of it, the time appears to be the same. What, do you think is not right? the date? To fix that you will need to update your props.conf. Please share your current props.conf to fix the timestamp issue. Fortunately, getting the event name is easier 🙂 To get the distribute event message, you can use the following regex.

.... | rex "distribute_event:\smessage\s(?<eventname>[^\t\n]+)"

View solution in original post

sundareshr
Legend

Try this

... | rex "(Trc|distribute_event|distribute_user_event):?\s\w+\s(?<EventName>\w+)"
0 Karma

gcusello
SplunkTrust
SplunkTrust

you have to extract fields using the guided procedure or regex.
For the first try something like this:
| regex "distribute_event: message\s+(?.*)"

about the second timestamp:
if you need only the time you can extract a field like:
| regex "\d+/\d+/\d+\s+\d+:\d+:\d+.\d+\s+\w+\s+(?\d+:\d+:\d+.\d+)"
Instead, if you want date and timestamp you have to extract two fields and connect them with eval
| eval newfield=field1+" "+field2

Verify regexes in the site https://regex101.com/

Bye.
Giuseppe

0 Karma

sundareshr
Legend

For the timestamp, splunk would have extracted the timestamp from the filed in your log file, at the time of indexing. From the looks of it, the time appears to be the same. What, do you think is not right? the date? To fix that you will need to update your props.conf. Please share your current props.conf to fix the timestamp issue. Fortunately, getting the event name is easier 🙂 To get the distribute event message, you can use the following regex.

.... | rex "distribute_event:\smessage\s(?<eventname>[^\t\n]+)"

ssingh313
Path Finder

Sorry didn't realize this earlier. Event names can also be displayed this way.

7/09/16
7:39:54.235AM 07:39:54.235 Trc 01234 RequestedDistributeEvent received from [xxx]

So it doesn't have a specific distribute_event but I need the RequestedDistributEvent to show up as the EventName. Is there a possible way to do this?

0 Karma

sundareshr
Legend

Try this instead

.... | rex "(?<EventName>RequestedDistributeEvent|EventAttachedDataChange|EventDiverted)"
0 Karma

ssingh313
Path Finder

that particular query won't work because there are many different event names besides those three mentioned. Is there a way to keep the previous query but add to it so it can display the name for a log that display the event name is this particular way?

7/09/16
7:39:54.235AM 07:39:54.235 Trc 01234 RequestedDistributeEvent received from [xxx]

0 Karma

sundareshr
Legend

Will Trc 01234 always prefix the EventName? Can you share a couple of examples of this type of event

0 Karma

ssingh313
Path Finder

yes the prefix will always be the same. Here are a few examples:

08:27:48.526 Trc 01234 RequestDistributeEvent received from [xxx]

06:31:25.488 Trc 01234 RequestDistributeEvent received from [xxx]
09:06:51.846 Trc 01234 RequestDistributeEvent received from [xxx]
03:32:37.964 Trc 01234 RequestDistributeEvent received from [xxx]

0 Karma

ssingh313
Path Finder

Also with the time stamp there might be a 10th of a second difference in the logs but Splunk would round it and combine multiple logs. I'm trying to keep the logs separate because rounding the time combines the 2 events within the same time stamp. Example:

7/1/16
7:23:46.846 AM 07:23:46.8460 [0] 4.0.000.00 distribute_user_event: message EventDiverted

7/1/16
7:23:46.846 AM 07:23:46.846 Trc 01234 RequestDistributeEvent received from [000]

So is there a way to keep the date from the splunk time stamp but get the exact time from the log

0 Karma

sundareshr
Legend

The best way to do this would be to change the props.conf to nanoseconds instead of microseconds. By the time it comes to the search, your data is already assigned a timestamp

0 Karma

ssingh313
Path Finder

Thanks this works! As you mentioned I changed the (? to (?&ltEventName&gt...
but like you can see above in the data one is distribute_event and the other is distribute_user_event so would I need to add a or/and statement in between to make sue that the name is displayed given either case

0 Karma

sundareshr
Legend

I missed the second part. Try this

.... | rex "[distribute_event|distribute_user_event]:\smessage\s(?<EventName>[^\t\n\r]+)" 

If both can appear in the same log, do this

.... | rex max_match=0 "[distribute_event|distribute_user_event]:\smessage\s(?<EventName>[^\t\n\r]+)" | mvexpand EventName  

ssingh313
Path Finder

This works thanks!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...