Getting Data In

Why is Splunk not picking up datetime in the following logs?

vikas_gopal
Builder

![alt text][1]HI Experts,

I have the following 2 logs. Why 2? Because I know BREAK_ONLY_BEFORE = Path=

I want the timestamp: "2006-09-21, 02:57:11.58"

{{"2006-09-21, 02:57:11.58", 122, 11, "Path=/LoginUser Query=CrmId=ClientABC&ContentItemId=TotalAccess&SessionId=3A1785URH117BEA&Ticket=646A1DA4STF896EE&SessionTime=25368&ReturnUrl=http://www.clientabc.com, Method=GET, IP=209.51.249.195, Content=", ""}} 
{{"2006-09-21, 02:57:11.60", 122, 15, "UserData:<User CrmId="clientabc" UserId="p12345678"><EntitlementList></EntitlementList></User>", ""}} 
{{"2006-09-21, 02:57:11.60", 122, 15, "New Cookie: SessionId=3A1785URH117BEA&Ticket=646A1DA4STF896EE&CrmId=clientabc&UserId=p12345678&AccountId=&AgentHost=man&AgentId=man, MANUser: Version=1&Name=&Debit=&Credit=&AccessTime=&BillDay=&Status=&Language=&Country=&Email=&EmailNotify=&Pin=&PinPayment=&PinAmount=&PinPG=&PinPGRate=&PinMenu=&", ""}}
{{"2006-09-21, 02:57:11.58", 122, 11, "Path=/LoginUser Query=CrmId=ClientABC&ContentItemId=TotalAccess&SessionId=3A1785URH117BEA&Ticket=646A1DA4STF896EE&SessionTime=25368&ReturnUrl=http://www.clientabc.com, Method=GET, IP=209.51.249.195, Content=", ""}} 
{{"2006-09-21, 02:57:11.60", 122, 15, "UserData:<User CrmId="clientabc" UserId="p12345678"><EntitlementList></EntitlementList></User>", ""}} 
{{"2006-09-21, 02:57:11.60", 122, 15, "New Cookie: SessionId=3A1785URH117BEA&Ticket=646A1DA4STF896EE&CrmId=clientabc&UserId=p12345678&AccountId=&AgentHost=man&AgentId=man, MANUser: Version=1&Name=&Debit=&Credit=&AccessTime=&BillDay=&Status=&Language=&Country=&Email=&EmailNotify=&Pin=&PinPayment=&PinAmount=&PinPG=&PinPGRate=&PinMenu=&", ""}}

I have the below settings at source type selection in preview mode

[ test123 ]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=Path=
CHARSET=AUTO
MAX_TIMESTAMP_LOOKAHEAD=30][1]
TIME_FORMAT= %Y-%m-%d, %H:%M:%S
TIME_PREFIX={{"
category=Custom
disabled=false
pulldown_type=true

With the above settings, I am getting the below error as shown in the snippet.

alt text

0 Karma
1 Solution

whrg
Motivator

The error message is pretty much self-explanatory. Splunk does not want to index data which is too old. You events are from 2006.

According to props.conf regarding MAX_DAYS_AGO:

Defaults to 2000 (days), maximum 10951.

So set MAX_DAYS_AGO=10000 and you should be fine if you really want to index this old data.

View solution in original post

0 Karma

whrg
Motivator

The error message is pretty much self-explanatory. Splunk does not want to index data which is too old. You events are from 2006.

According to props.conf regarding MAX_DAYS_AGO:

Defaults to 2000 (days), maximum 10951.

So set MAX_DAYS_AGO=10000 and you should be fine if you really want to index this old data.

0 Karma

vikas_gopal
Builder

Ahaaa!!! worked like a charm Thank you so much

0 Karma

soumyasaha25
Contributor

is there any particular reason why you want to merge 3 lines of raw data into 1 rather than breaking them into 3 separate events, as you get a significant boost to processing speed when you use LINE_BREAKER to delimit multi-line events as opposed to using SHOULD_LINEMERGE to reassemble individual lines into multi-line events.

0 Karma

vikas_gopal
Builder

No Specific Reason I am just doing testing and playing with the data was not aware of MAX_DAYS_AGo Setting

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...