Getting Data In

timestamp extra

crazyeva
Contributor

I put some oracle-exported data into splunk, with props.conf:

NO_BINARY_CHECK = true
CHARSET = GB2312
SHOULD_LINEMERGE = false
TIME_PREFIX = (?(?:\d{2}(?:\/\d{2}){2}\s\d{2}(?::\d{2}){2})|(?:\d{4}(?:-\d{2}){2}\s\d{2}(?::\d{2}){2}))(?=(?:^[^^]*){27}$)

same result,easy to read: TIMEPREFIX = (?[^\^]*)(?=(?:\^[^\^]*){27}$)

Most timestamps are extracted correctly, but two of them are unexpected:

event 1, 7/15/12 9:35:17.000 PM should be "05/03/12 15:56:32" and event 2, 7/15/12 9:27:02.000 PM should be "04/11/12 19:15:18" stong characters in raw

1 » 7/15/12 9:35:17.000 PM

2012-05-03 15:57:45^INSERT^ "ipb-a-cjx-cx600-101SHELL/5/CMDRECORD(l): Record command information. (Task vt0 Ip 58.246.74.188 User gongchuang Command efu np-2 slot 2 ingress display status )"^129920652^"ipb-a-cjx-cx600-1"^"124.75.5.14"^"Syslog Probe on nmman5-pd"^"adsl-CX600"^""^"%%01SHELL"^2^"01SHELL/5/CMDRECORD(l): Record command information. (Task vt0 Ip 58.246.74.188 User gongchuang Command efu np-2 slot 2 ingress display status )"^05/03/12 15:57:22^05/03/12 15:56:32^05/03/12 15:56:32^05/03/12 15:57:22^0^1^1^200^0^""^65534^0^0^0^"TROU5"^60637884^""^""^0^0^""^"124.75.5.14"^""^""^""^""^""^""^""
FirstOccurrence=05/03/12 15:56:32 Options| InternalLast=05/03/12 15:57:22 Options| StateChange=05/03/12 15:57:22 Options

2 » 7/15/12 9:27:02.000 PM

2012-04-11 19:16:49^UPDATE^ "ipb-a-yh-9312-101SHELL/6/DISPLAY_CMDRECORD(l): Record command information. (Task vt0 Ip 124.74.213.3 User root Command display igmp-snooping port-info vlan 51 )DISPLAY_CMDRECORD(l):%%01SHELL/6/DISPLAY_CMDRECORD(l):Recordcommand"^126438075^"ipb-a-yh-9312-1"^"124.75.192.222"^"Syslog Probe on nmman5-pd"^"9312"^"DISPLAY_CMDRECORD(l):"^"%%01SHELL/6/DISPLAY_CMDRECORD(l):Recordcommand"^1^"01SHELL/6/DISPLAY_CMDRECORD(l): Record command information. (Task vt0 Ip 124.74.213.3 User root Command display igmp-snooping port-info vlan 51 )"^04/11/12 19:15:58^04/10/12 19:15:16^04/11/12 19:15:18^04/11/12 19:15:18^0^1^97^200^0^""^65534^0^0^0^"TROU5"^58515545^""^""^0^0^""^"124.75.192.222"^""^""^""^""^""^""^""
FirstOccurrence=04/10/12 19:15:16 Options| InternalLast=04/11/12 19:15:18 Options| StateChange=04/11/12 19:15:58

Tags (1)
0 Karma
1 Solution

mloven_splunk
Splunk Employee
Splunk Employee

crazyeva,

That first TIME_PREFIX makes my head hurt. Let's go with something closer to your second example.

I would do something along these lines:

TIME_PREFIX = ([\r\n]+)(?([^^]+^){14})

View solution in original post

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

crazyeva,

That first TIME_PREFIX makes my head hurt. Let's go with something closer to your second example.

I would do something along these lines:

TIME_PREFIX = ([\r\n]+)(?([^^]+^){14})

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

TIME_PREFIX is used to tell Splunk what comes before the timestamp.

Also, in that TIME_PREFIX that I provided, the regex in the first set of parenthesis will match any number of returns or newlines.

0 Karma

crazyeva
Contributor

Sorry to reply so late.
The number of "^" is not sure from line starts, so i tried to find timestamp from their tails.
By the way, I am confused that TIME_PREFIX attempts to match what is before timestamp or just to match timestamp?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...