Deployment Architecture

Unable to remove header from CSV file pulling from S3

rajasha
Explorer

Can someone help here please. I'm trying to remove the header which is currently adding as header as a events in the parsing which needs to remove. 

Also time stamp is not correct. Below is config from props.conf

KV_MODE = auto
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = NONE
CHARSET=UTF-8
INDEXED_EXTRACTIONS=CSV
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=Date,Time
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIMESTAMP_FIELDS = Date,Time
FIELD_DELIMITER = ,
FIELD_QUOTE = "
CHECK_FOR_HEADER = true

@splunk @BSplunk 

Labels (3)
0 Karma

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share a sample header and events.  Also, what is incorrect about the timestamp?

---
If this reply helps you, Karma would be appreciated.

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma
Get Updates on the Splunk Community!

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...