Getting Data In

Events not breaking correctly - using mv-add

guimilare
Communicator

Hello Splunkers.

I'm helping a client to find out why some of his events are not being broken correctly.
They are currently running a Search Head Cluster with 3 SHs, 2 Indexers, 1 Master Cluster and 1 License/Deployer.
Here is a example of log:

--
tstamp="20160105 23:59:39.893"
IdCmd=01
Port=01
tstampResp="20160105 23:59:40.390"
Cmd="XXXXXX"
tipoAcao=ABC
Pri=I
Rsgmt=H
mainkey=12345678
acao="A";vAcao="000100000000A";resp="O"
acao="A";vAcao="000200000000A";resp="O"
acao="A";vAcao="000300000000A";resp="O"
acao="A";vAcao="000400000000A";resp="O"
acao="A";vAcao="000500000000A";resp="O"
acao="A";vAcao="000600000000A";resp="O"
acao="A";vAcao="000700000000A";resp="O"
acao="A";vAcao="000800000000A";resp="O"
acao="A";vAcao="000900000000A";resp="O"
acao="A";vAcao="001000000000A";resp="O"
acao="A";vAcao="001100000000A";resp="O"
acao="A";vAcao="001200000000A";resp="O"
acao="A";vAcao="001300000000A";resp="O"
acao="A";vAcao="001400000000A";resp="O"
acao="A";vAcao="001500000000A";resp="O"
acao="A";vAcao="001600000000A";resp="O"
acao="A";vAcao="006700000000A";resp="O"
acao="A";vAcao="006A00000000A";resp="O"
acao="A";vAcao="006B00000000A";resp="O"
acao="A";vAcao="006C00000000A";resp="O"
acao="A";vAcao="006E00000000A";resp="O"
acao="A";vAcao="006F00000000A";resp="O"
acao="A";vAcao="007000000000A";resp="O"
acao="A";vAcao="007200000000A";resp="O"
acao="A";vAcao="007400000000A";resp="O"
acao="A";vAcao="007500000000A";resp="O"
acao="A";vAcao="007600000000A";resp="O"
acao="A";vAcao="007700000000A";resp="O"
acao="A";vAcao="007800000000A";resp="O"
acao="A";vAcao="007900000000A";resp="O"
acao="A";vAcao="007B00000000A";resp="O"
acao="A";vAcao="007E00000000A";resp="O"
acao="A";vAcao="008000000000A";resp="O"
acao="A";vAcao="008200000000A";resp="O"
acao="A";vAcao="008900000000A";resp="O"
acao="A";vAcao="008A00000000A";resp="O"
acao="A";vAcao="008E00000000A";resp="O"
acao="A";vAcao="008F00000000A";resp="O"
acao="A";vAcao="009800000000A";resp="O"
acao="A";vAcao="009B00000000A";resp="O"
acao="A";vAcao="009D00000000A";resp="O"
acao="A";vAcao="009F00000000A";resp="O"
acao="A";vAcao="00A000000000A";resp="O"
acao="A";vAcao="00AA00000000A";resp="O"
acao="A";vAcao="00AB00000000A";resp="O"
acao="A";vAcao="00AC00000000A";resp="O"
acao="A";vAcao="00B500000000A";resp="O"
acao="A";vAcao="00C000000000A";resp="O"
acao="A";vAcao="00C100000000A";resp="O"
acao="A";vAcao="00C200000000A";resp="O"
acao="A";vAcao="01AA00000000A";resp="O"
acao="A";vAcao="021100000000A";resp="O"
acao="A";vAcao="021200000000A";resp="O"
acao="A";vAcao="039100000000A";resp="O"
acao="A";vAcao="039C00000000A";resp="O"
acao="A";vAcao="01C100000000A";resp="O"
acao="A";vAcao="000500000000A";resp="O"
acao="A";vAcao="001400000000A";resp="O"
acao="A";vAcao="005300000000A";resp="O"
acao="A";vAcao="005C00000000A";resp="O"
acao="A";vAcao="008400000000A";resp="O"
acao="A";vAcao="001600000000A";resp="O"
acao="A";vAcao="00F300000000A";resp="O"
acao="A";vAcao="00F000000000A";resp="O"
acao="A";vAcao="01C200000000A";resp="O"
acao="A";vAcao="00EF00000000A";resp="O"
acao="A";vAcao="01C300000000A";resp="O"
acao="A";vAcao="01C400000000A";resp="O"
acao="A";vAcao="00EE00000000A";resp="O"
acao="A";vAcao="01C600000000A";resp="O"
acao="A";vAcao="01C500000000A";resp="O"
acao="A";vAcao="01D700000000A";resp="O"
acao="A";vAcao="00EC00000000A";resp="O"
acao="A";vAcao="01C700000000A";resp="O"
acao="A";vAcao="00ED00000000A";resp="O"
acao="A";vAcao="01DE00000000A";resp="O"
acao="A";vAcao="01DD00000000A";resp="O"
acao="A";vAcao="01E200000000A";resp="O"
acao="A";vAcao="01C800000000A";resp="O"
acao="A";vAcao="01E100000000A";resp="O"
acao="A";vAcao="01E000000000A";resp="O"
acao="A";vAcao="01C900000000A";resp="O"
acao="A";vAcao="01CA00000000A";resp="O"
acao="A";vAcao="00F400000000A";resp="O"

The log above always starts with -- and can have hundreads of acao="A";vAcao="XXXX00000000A";resp="O" lines.
Here is the props.conf that I'm using:

SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=--
disabled=false
TIME_PREFIX=\d+\=\"
TIME_FORMAT=%Y%m%d %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=30

And here is the transforms.conf that I'm using:

REGEX = acao=\"(.*?)\";vAcao=\"(.*?)\";resp=\"(.*?)\" 
FORMAT = acao::$1 vAcao::$2 resp::$3 
MV_ADD = true

Sometimes, the event is not broken correctly. It breaks on acao="A";vAcao="XXXX00000000A";resp="O" lines.
I thought that I should include the TRUNCATE option in my props.conf.
However, when I try to Distribute Bundle Configuration using the UI at the Master, I receive an error message saying that I can't use TRUNCATE at my stanza.
So now I'm a bit lost.
Do you guys have any ideias that can help me?

Od course I can go directly to the Indexers and put the TRUNCATE option "by hand", but since they will soon add more indexers, this will not be feasible without the Master's Distribute Bundle Configuration.

Thanks in advance!
Regards,
GMA

0 Karma
1 Solution

lguinn2
Legend

Your props.conf does not specify MAX_EVENTS - which defaults to 256. So, when you have an event with many, many lines, Splunk breaks even without a --

Add this to the props.conf

MAX_EVENTS = 1024

Or some other number that is larger than the maximum possible number of events. Setting TRUNCATE would not have solved this problem anyway.

View solution in original post

0 Karma

lguinn2
Legend

Your props.conf does not specify MAX_EVENTS - which defaults to 256. So, when you have an event with many, many lines, Splunk breaks even without a --

Add this to the props.conf

MAX_EVENTS = 1024

Or some other number that is larger than the maximum possible number of events. Setting TRUNCATE would not have solved this problem anyway.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

From where you're ingesting this log? Is it a data monitoring setup on Search Head OR on a Forwarder? Which location you're putting the props.conf for the sourcetype (with TRUNCATE)?

0 Karma

guimilare
Communicator

Hi somesoni2. I'm getting this data on a Forwarder. /opt/splunk/etc/master-apps/app/local/props.conf is the path on the Master. When I use Truncate, I can't use the Distribute Bundle Configuration on Master's UI.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Are you able to push the bundle from the command line/shell on the Indexer Master?? using http://docs.splunk.com/Documentation/Splunk/6.2.0/Indexer/Updatepeerconfigurations#3._Apply_the_bund...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...