Getting Data In

Why is Splunk not parsing the correct datetime?

dpanych
Communicator

I have logs that contain the following datetime format:

29-06-2016_00-08-17

The props contain:

[odb]
TIME_PREFIX = ".+",".+","
TIME_FORMAT = %d-%m-%Y_%H-%M-%S
MAX_TIMESTAMP_LOOKAHEAD = 50
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
pulldown_type = true
SEDCMD-01_change_delims_in_oracle_logs = s/\|;\|/,/g
SEDCMD-02_remove_the_end = s/"\|\|\?--END---\?\|\|//g
REPORT-set_delimiters_oracle_logs = REPORT-delims_odb_logs

What could be the problem?

0 Karma
1 Solution

woodcock
Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

View solution in original post

woodcock
Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

woodcock
Esteemed Legend

Also, this needs to be deployed to your indexers, the splunk instances must be restarted, and only newly-indexed (post-restart) data will be effected (bad events will stay bad).

0 Karma

jplumsdaine22
Influencer

Does it make a difference if you use a capture group instead of a non capture group there? As in:

(?:"[^"]*",){3}"
0 Karma

gcusello
SplunkTrust
SplunkTrust

you have to use \ before " so the prefix is
\"\w+\",\"\w+\",\"\w+\",\"

Bye.
Giuseppe

0 Karma

woodcock
Esteemed Legend

In this case, they are the same.

0 Karma

dpanych
Communicator

I'm no regex guru, but how do I make the regex above work with the |;| delims? ---- "aaa"|;|"OIM"|;|"DELETE"|;|"29-06-2016_01-53-16"

0 Karma

jplumsdaine22
Influencer

Just modify the regex to pick the delimiters instead of a comma

 ("[^"]*"\|;\|){3}"

dpanych
Communicator

Thank you guys, this worked!

0 Karma

jplumsdaine22
Influencer

https://regex101.com/ is your friend

0 Karma

woodcock
Esteemed Legend

We cannot say without you posting a sample event.

0 Karma

dpanych
Communicator

Here's what the logs looks like after the sed:

"host1","MON","LOGOFF","30-06-2016_11-15-01","","0"
"host2","ODS","UPDATE","30-06-2016_12-51-05","UPDATE DS_ATTRSTORE SET ATTRVAL = :B1 WHERE ENTRY = :B2 AND ATTRNAME = 'modname'","0"
"host3","ODS","UPDATE","30-06-2016_08-28-43","UPDATE DS_ATTRSTORE SET ATTRVER = :B4 || CHR(94) || :B5 || CHR(94) || :B6 , ATTRVAL = :B3 , ATTRKIND = :B2 , ATTRSTYPE = :B1 WHERE ENTRY = :B8 AND ATTRNAME = :B7","0"
0 Karma

jplumsdaine22
Influencer

Can you post a sample event as well?

0 Karma

maciep
Champion

Not sure if I'm reading that correctly, but are you replacing the delimiters in the raw text from "|;|" to ","? If so, is that the same comma you're expecting to be there for the time prefix? If it is, I'm wondering if the timestamp recognition happens before sedcmd is applied?

Also, be sure 50 characters is enough to look ahead (assuming you did, but thought I'd mention it).

0 Karma

gcusello
SplunkTrust
SplunkTrust

I found a similar problem: time recognition is done before the character replacing.
Bye.
Giuseppe

0 Karma

dpanych
Communicator

So I updated the time prefix to include the delims before the sed, TIME_PREFIX = ".+"|;|".+"|;|".+"|;|"
and still no luck.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...