Getting Data In

Why is this linebreak is not working with JSON data?

fisuser1
Contributor

Any ideas why this linebreak is not working with JSON data? I've even set the sourcetype to _json, but still no luck.

[prd_thousand_eyes_rest]
INDEXED_EXTRACTIONS = json
**BREAK_ONLY_BEFORE={|**
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=300
SHOULD_LINEMERGE=false
category=Structured
disabled=false
pulldown_type=true

_raw
[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]
0 Karma

jconger
Splunk Employee
Splunk Employee

There are 2 main problems here:

1) The _raw data is not JSON. JSON should look like

{"key1":"value1", "key2":"value2"}

Instead, you have

{|field1=value1||field2=value2|}

2) The _raw data looks like it has arrays of collections. Are you collecting this data programmatically, or reading it from a file? If reading from a file, you could use props.conf to strip out the square brackets using SEDCMD, set up your line breaker, and regex the keys and values. If you are getting this data programmatically, you could format it before streaming it over to Splunk.

richgalloway
SplunkTrust
SplunkTrust

BREAK_ONLY_BEFORE only applies when SHOULD_LINEMERGE=true.

---
If this reply helps you, Karma would be appreciated.
0 Karma

ssadanala1
Contributor

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)[\{|

0 Karma

fisuser1
Contributor

thanks, unfortunetly still seeing the same behavior. not breaking before "{|type" with in the json bracket.

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...