This is a long question.
We have a Heavy Forwarder and an Indexer cluster (managed through indexer cluster master.) I have a scripted input that pulls some data which is in "array of json" format. To remove the complication of array of jason, I am using SEDCMD, which works perfect. But my LINE_BREAKER does not work.
The custom add-on which has the input is hosted on the Heavy Forwarder and the props.conf is present on both HF as well as Indexers. The props.conf works perfect if I upload the data to a Single Instance Splunk Enterprise but does not work in HF--> Indexer scenario.
I have tried implementing combinations of the props.conf on both HF and Indexers, but LINE_BREAKER does not work.
Below my props.conf, I have used several combinations of the LINE_BREAKER as well as MUST_BREAK_AFTER (LINE_MERGE = TRUE)
[testdata_api]
SHOULD_LINEMERGE = 0
category = Splunk App Add-on Builder
pulldown_type = 1
#LINE_BREAKER = ((?<!"),|[\r\n]+)
#LINE_BREAKER = (\}\s+)
LINE_BREAKER = ((\}\,)|(\}\s+))
#SEDCMD-remove_prefix = s/{\n\"Report_Entry":\s\[//g
#SEDCMD-remove_trailing_commas = s/\},/}/g
#SEDCMD-remove_footer = s/\]\s\}//g
#SHOULD_LINEMERGE = true
#MUST_BREAK_AFTER = (\}\s)
Below is the sample of how my data looks like
{
"Report_Entry":
[{
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4",
"field5": "value5",
"field6": "value6",
"field7": "value7",
"field8": "value8",
"field9": "value9",
"field10": "value10"
},
{
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4",
"field5": "value5",
"field6": "value6",
"field7": "value7",
"field8": "value8",
"field9": "value9",
"field10": "value10"
},
{
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4",
"field5": "value5",
"field6": "value6",
"field7": "value7",
"field8": "value8",
"field9": "value9",
"field10": "value10"
},
{
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4",
"field5": "value5",
"field6": "value6",
"field7": "value7",
"field8": "value8",
"field9": "value9",
"field10": "value10"
}]
}
Below is the out put of btool
/opt/splunk/etc/apps/TA-testdata_api/local/props.conf [testdata_api]
/opt/splunk/etc/system/default/props.conf ADD_EXTRA_TIME_FIELDS = True
/opt/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf AUTO_KV_JSON = true
/opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE =
/opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf CHARSET = UTF-8
/opt/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf DEPTH_LIMIT = 1000
/opt/splunk/etc/system/default/props.conf HEADER_MODE =
/opt/splunk/etc/system/default/props.conf LEARN_MODEL = true
/opt/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true
/opt/splunk/etc/apps/TA-testdata_api/local/props.conf LINE_BREAKER = (\}\,)
/opt/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100
/opt/splunk/etc/system/default/props.conf MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000
/opt/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2
/opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600
/opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800
/opt/splunk/etc/system/default/props.conf MAX_EVENTS = 256
/opt/splunk/etc/system/default/props.conf MAX_TIMESTAMP_LOOKAHEAD = 128
/opt/splunk/etc/system/default/props.conf MUST_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE =
/opt/splunk/etc/system/default/props.conf SEGMENTATION = indexing
/opt/splunk/etc/system/default/props.conf SEGMENTATION-all = full
/opt/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner
/opt/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer
/opt/splunk/etc/system/default/props.conf SEGMENTATION-raw = none
/opt/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard
/opt/splunk/etc/apps/TA-testdata_api/local/props.conf SHOULD_LINEMERGE = false
/opt/splunk/etc/system/default/props.conf TRANSFORMS =
/opt/splunk/etc/system/default/props.conf TRUNCATE = 10000
/opt/splunk/etc/apps/TA-testdata_api/local/props.conf category = Splunk App Add-on Builder
/opt/splunk/etc/system/default/props.conf detect_trailing_nulls = false
/opt/splunk/etc/system/default/props.conf maxDist = 100
/opt/splunk/etc/system/default/props.conf priority =
/opt/splunk/etc/apps/TA-testdata_api/local/props.conf pulldown_type = 1
/opt/splunk/etc/system/default/props.conf sourcetype =
I need to know, if there is any precedence of LINE_BREAKER over SEDCMD? or what causes the LINE_BREAKER to fail in my case?
... View more