How to edit my props.conf to line break my events properly?

[AmitKapila]

I am trying to have separate BrkrName events.

I have a script ./iibqueuemonitor.sh that outputs:

EventType=Broker,BrkrName=MBIB001P01,Status=RUNNING
EventType=Broker,BrkrName=MBIB001P02,Status=RUNNING

But in Splunk Web, when I use this search:

index="test" source="iibqueuemonitor.sh" sourcetype="metro:iibcorpqmon" host="myhostcompany" EventType=Broker

It does not treat the 2 lines as independent events.

My inputs.conf looks like this:

[script://./bin/iibqueuemonitor.sh]
index = test
source = iibqueuemonitor.sh
sourcetype = metro:iibcorpqmon
interval = 60
disabled = 0

My props.conf:

3piib01 bin]# cat /opt/splunkforwarder/etc/apps/Metro_TA_iibcorp/local/props.conf
[metro:iibcorpqmon]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = EventType=
MUST_BREAK_AFTER

/opt/splunkforwarder/bin/splunk cmd btool props list metro:iibcorpqmon
[metro:iibcorpqmon]
ANNOTATE_PUNCT = True
BREAK_ONLY_BEFORE = EventType=
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
detect_trailing_nulls = false
maxDist = 100
priority =
sourcetype =

I tried it after changing /opt/splunkforwarder/etc/apps/Metro_TA_iibcorp/local/props.conf but it fails again to split the EventType.

[metro:iibcorpqmon]
SHOULD_LINEMERGE = false
DATETIME_CONFIG = current
LINE_BREAKER=([\r\n]+)

Is the props.conf in the correct place?

[AmitKapila]

Hi Andrei,

At the moment it is working as intended so I am not sure what advantage your kind suggestion may have. There are more EventTypes coming from the script that uses the universal forwarder other than Broker. Now that line breaking is working SPLUNK indexer is treating these as separate events which is what I wanted. Your solution above seems to work for only 1 type of EventType. I have about a dozen different types of EventType.

Cheers,

Amit.

[andrei1bc]

Could a regex transformation work in your case ?

props.conf :

[sourcetype_name_goes_here]
category = Custom
pulldown_type = 1
disabled = false
TRANSFORMS-event = transform_name_goes_here

transforms.conf :

[transform_name_goes_here]
REGEX = ^.\w+.(?<EventType>\w+).\w+.(?<BrokerName>\w+).\w+.(?<Status>\w+).+
FORMAT = EventType::$1 BrokerName::$2 Status::$3
WRITE_META = true

Add the extra options

[woodcock]

Your problem is that you have SHOULD_LINEMERGE=True instead of what you should have which is SHOULD_LINEMERGE=false. Fix this in props.conf, deploy to your indexers, restart the splunk instances there and check events that have been forwarded/indexed AFTER the restart ONLY.

[AmitKapila]

I restarted the service as you suggested. It does not work. This might be because I have put the props.conf in the wrong place. Is there a way to debug this so that I can see if the props.conf is being read?

[AmitKapila]

Ok it is now working. Instead of putting the:

[:]
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false

into $SPLUNK_HOME/etc/deployment-apps//default/props.conf I put it into /opt/splunk/etc/system/local/props.conf then restarted the service. It is now treating the string as separate events. Thanks for your help.

[AmitKapila]

Still not working post changes made:

1) On somesplunkindexerserver I created /opt/splunk/etc/deployment-apps//default/props.conf.
2) I used these settings:

[metro:iibcorpqmon]
LINE_BREAKER = =([\r\n]+)
TIME_FORMAT = %y%m%dT%H:%M:%S
SHOULD_LINEMERGE = false

3) I refreshed using http://splunk.service...:/en-GB/debug/refresh
4) Now the script that gets data from the server that the forwarder runs on has disappeared.

[woodcock]

A debug/refresh will definitely NOT work; there are constantly new things added to the API and it might be that a props.conf refresh will do it but I know that a restart will.

[AmitKapila]

[serverthatcollectsdata]# ./splunk list forward-server
Active forwards:
somesplunkindexerserver:
Configured but inactive forwards:
None
[serverthatcollectsdata]#

On the I can see /opt/splunk. Where should the props.conf be placed (/opt/splunk/etc/system/local/props.conf ?) and should the "service splunk restart" command be reissued?

Many thanks,

Amit Kapila

[woodcock]

Yes, that will restart the splunk service, if it is configured as a service.

[woodcock]

The file should reside inside the default folder of an app created for this purpose that is named similarly to the app the you created for the forwarder (e.g. MyThing_inputs on forwarder and MyThing_props on your indexers).

[somesoni2]

No. Your current instance is a universal forwarder and it doesn't do event parsing. You should be putting those props.conf changes (2nd one with SHOULD_LINEMERGE=false) to the full Splunk Enterprise instance this universal forwarder is sending data to, which could be an Indexer or heavy/intermediate forwarder.