Getting Data In
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question
Solved! Jump to solution

Long logs split into multiple events in Splunk

erwinpastor
Explorer
‎05-01-2015 05:47 AM

Good day everyone!

I have the following config in props so that it creates a new event only if it encounters a new line with a date but the logs are still being break down into several events.

[sourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD = 25
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE = True
MAX_EVENTS = 10000

Am I missing something, ie adding TRUNCATE = 100000 (or to higher value) ? Or is there any specific parameter to add in props for xml kind of log?

See sample logs below.
"===================" denotes new events created in Splunk

2015-05-01 11:03:00,818 INFO [HTTP Handler 10.241.43.96] appName=CSAccountTransactionEntSvc.findTransactions|service=CSAccountTransactionEntSvc.provider.ws.v1.CSAccountTransactionEntSvc_.services.findTransactions|event=End Audit|AuditTrackingID=XXXXXXX|AccountID=XXXXXXX|user=Tester
<?xml version="1.0"?>
<applicationName>CSAccountTransactionEntSvc.findTransactions</applicationName>
<eventName>End Audit</eventName>
<level>INFO</level>
<content>
  <document>
    <AuditHeader>
      <Message>
        <AuditTrackingID>XXXXXXX</AuditTrackingID>
        <OperationName>findTransactions</OperationName>
        <CreationDateTime>2014-06-29T23:20:56Z</CreationDateTime>
        <ServiceVersion>1.1</ServiceVersion>
        <Organisation>MBL</Organisation>
      </Message>
      <OperationInitiator>
        <System>Online</System>
        <Component>Comp</Component>
        <User>Tester</User>
      </OperationInitiator>
      <MessageInitiator>
        <System>Online</System>
      </MessageInitiator>
    </AuditHeader>
    <StatusLog>
      <MaxItemStatus>Error</MaxItemStatus>
      <Item>
        <Status>Error</Status>
        <Type>Technical Error</Type>
        <Code>ESB_E1000</Code>
        <Description>Exception Occured - [ISC.0049.9010] Service 'CSAccountTransactionEntSvc.operation.findTransactions.pub:findTransactions' invoking unknown service 'ESBCommonUtilsEnablerSvc.pub.statusLog:mapStatusLogFromValidationErrors' at 'Transformer ESBCommonUtilsEnablerSvc.pub.statusLog:mapStatusLogFromValidationErrors'. The service may have been renamed, moved or disabled. ; Error Source - CSAccountTransactionEntSvc.provider.ws.v1.CSAccountTransactionEntSvc_.services:findTransactions</Description>
        <System>ESB</System>
      </Item>
    </StatusLog>
  </document>
</content>
<identifiers>
  <id1>
    <idType>AuditTrackingID</idType>
    <idValue>XXXXXXX</idValue>
  </id1>
  <id2>
    <idType>AccountID</idType>
    <idValue>XXXXXXX</idValue>
  </id2>
</identifiers>
<extendedAttributes>
  <applicationServerName>Online</applicationServerName>
  <applicationTimestamp>2014-06-29T23:20:56Z</applicationTimestamp>
  <applicationTimestampPattern>yyyy-MM-dd'T'hh:mm:ss.S'Z'</applicationTimestampPattern>
  <serviceName>CSAccountTransactionEntSvc.provider.ws.v1.CSAccountTransactionEntSvc_.services:findTransactions</serviceName>

==================================================

 </id2>
</identifiers>
<extendedAttributes>
  <applicationServerName>Online</applicationServerName>
  <applicationTimestamp>2014-06-29T23:20:56Z</applicationTimestamp>
  <applicationTimestampPattern>yyyy-MM-dd'T'hh:mm:ss.S'Z'</applicationTimestampPattern>
  <serviceName>CSAccountTransactionEntSvc.provider.ws.v1.CSAccountTransactionEntSvc_.services:findTransactions</serviceName>
  <user>Tester</user>
</extendedAttributes>

==================================================

 </RangeList>
        <SearchItemList>
          <SearchItem>
            <ItemType>
              <Code>TxnStatus</Code>
              <Name></Name>
            </ItemType>
            <ItemValue>Posted</ItemValue>
          </SearchItem>
        </SearchItemList>
      </FindTransactionsInSearchCriteria>
    </FindTransactionsInput>
  </document>
  <extendedDocument>
    <AuditHeader>
      <Message>
        <AuditTrackingID>XXXXXXX</AuditTrackingID>
        <OperationName>findTransactions</OperationName>
        <CreationDateTime>2014-06-29T23:20:56Z</CreationDateTime>
        <ServiceVersion>1.1</ServiceVersion>
        <Organisation>MBL</Organisation>
      </Message>
      <OperationInitiator>
        <System>Online</System>
        <Component>Comp</Component>
        <User>Tester</User>
      </OperationInitiator>
      <MessageInitiator>
        <System>Online</System>
      </MessageInitiator>
    </AuditHeader>
    <StatusLog>
      <MaxItemStatus>Error</MaxItemStatus>
      <Item>
        <Status>Error</Status>
        <Type>Technical Error</Type>
        <Code>ESB_E1000</Code>
        <Description>Exception Occured - [ISC.0049.9010] Service 'CSAccountTransactionEntSvc.operation.findTransactions.pub:findTransactions' invoking unknown service 'ESBCommonUtilsEnablerSvc.pub.statusLog:mapStatusLogFromValidationErrors' at 'Transformer ESBCommonUtilsEnablerSvc.pub.statusLog:mapStatusLogFromValidationErrors'. The service may have been renamed, moved or disabled. ; Error Source - CSAccountTransactionEntSvc.provider.ws.v1.CSAccountTransactionEntSvc_.services:findTransactions</Description>
        <System>ESB</System>
      </Item>
    </StatusLog>
  </extendedDocument>
</content>
<identifiers>
  <id1>
    <idType>AuditTrackingID</idType>
    <idValue>XXXXXXX</idValue>
  </id1>
  <id2>
    <idType>AccountID</idType>
    <idValue>XXXXXXX</idValue>

==================================================

<ServiceVersion>1.1</ServiceVersion>
        <Organisation>MBL</Organisation>
      </Message>
      <OperationInitiator>
        <System>Online</System>
        <Component>Comp</Component>
        <User>Tester</User>
      </OperationInitiator>
      <MessageInitiator>
        <System>Online</System>
      </MessageInitiator>
    </AuditHeader>
    <FindTransactionsInput>
      <FindTransactionsInAccount>
        <AccountBase>
          <AccountID>XXXXXXXXXXXXX</AccountID>
          <CountryCode>AU</CountryCode>
          <AccountNo>XXXXXXXXX</AccountNo>
          <BSBNo>XXXXXX</BSBNo>
          <CurrencyCode>AUD</CurrencyCode>
          <AccountName>XXXXXXX</AccountName>
          <AccountShortName>XXXXX</AccountShortName>
          <SourceSystem>SAP</SourceSystem>
        </AccountBase>
      </FindTransactionsInAccount>
      <FindTransactionsInSearchCriteria>
        <IndicatorList>
          <Indicator>
            <IndicatorType>
              <Code>RunningBalance</Code>
              <Name></Name>
            </IndicatorType>
            <IndicatorValue>Y</IndicatorValue>
          </Indicator>
          <Indicator>
            <IndicatorType>
              <Code>AccountBalances</Code>
              <Name></Name>
            </IndicatorType>
            <IndicatorValue>Y</IndicatorValue>
          </Indicator>
        </IndicatorList>
        <MaxResults>10</MaxResults>
        <RangeList>
          <Range>
            <RangeType>
              <Code>Amount</Code>
            </RangeType>
            <LowerValue>100.00</LowerValue>
            <UpperValue>10.00</UpperValue>
          </Range>

==================================================        

<System>Online</System>
          <Component>Comp</Component>
          <User>Tester</User>
        </OperationInitiator>
        <MessageInitiator>
          <System>Online</System>
        </MessageInitiator>
      </AuditHeader>
      <FindTransactionsInput>
        <FindTransactionsInAccount>
          <AccountBase>
            <AccountID>XXXXXXXXXXXXX</AccountID>
            <CountryCode>AU</CountryCode>
            <AccountNo>XXXXXXXXX</AccountNo>
            <BSBNo>XXXXXX</BSBNo>
            <CurrencyCode>AUD</CurrencyCode>
            <AccountName>XXXXXXX</AccountName>
            <AccountShortName>XXXXX</AccountShortName>
            <SourceSystem>SAP</SourceSystem>
          </AccountBase>
        </FindTransactionsInAccount>
        <FindTransactionsInSearchCriteria>
          <IndicatorList>
            <Indicator>
              <IndicatorType>
                <Code>RunningBalance</Code>
                <Name></Name>
              </IndicatorType>
              <IndicatorValue>Y</IndicatorValue>
            </Indicator>
            <Indicator>
              <IndicatorType>
                <Code>AccountBalances</Code>
                <Name></Name>
              </IndicatorType>
              <IndicatorValue>Y</IndicatorValue>
            </Indicator>
          </IndicatorList>
          <MaxResults>10</MaxResults>
          <RangeList>
            <Range>
              <RangeType>
                <Code>Amount</Code>
              </RangeType>
              <LowerValue>100.00</LowerValue>
              <UpperValue>10.00</UpperValue>
            </Range>
          </RangeList>
          <SearchItemList>
            <SearchItem>
              <ItemType>
0 Karma
Reply
1 Solution
Solved! Jump to solution
Solution

cpetterborg
SplunkTrust
SplunkTrust
‎05-02-2015 09:21 AM

Try using LINE_BREAKER if you have a regular expression capable way to define the end of the event. Like, if you have the exact same ending XML tag the doesn't appear anywhere else in the event, then that will work. I didn't see that was the case in the data you included.

You might also be able to use:
BREAK_ONLY_BEFORE_DATE = true

This will break only when it sees a new line with a timestamp.

If you use TRUNCATE, if you set it to 0, it will do an unlimited event, but be careful in case you have the possibility of some really bad days coming in.

View solution in original post

0 Karma
Reply
Solved! Jump to solution
Solution

cpetterborg
SplunkTrust
SplunkTrust
‎05-02-2015 09:21 AM

Try using LINE_BREAKER if you have a regular expression capable way to define the end of the event. Like, if you have the exact same ending XML tag the doesn't appear anywhere else in the event, then that will work. I didn't see that was the case in the data you included.

You might also be able to use:
BREAK_ONLY_BEFORE_DATE = true

This will break only when it sees a new line with a timestamp.

If you use TRUNCATE, if you set it to 0, it will do an unlimited event, but be careful in case you have the possibility of some really bad days coming in.

0 Karma
Reply
Solved! Jump to solution

erwinpastor
Explorer
‎05-03-2015 12:48 AM

Thanks! I have already added BREAK_ONLY_BEFORE_DATE = true in my original setup but it still splitting the log into few events.

[sourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD = 25
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE = True
MAX_EVENTS = 10000

LINE_BREAKER will not work, as what you have noticed, I don't have exact same ending tag when the line breaks. I'll probably try adding TRUNCATE but instead of setting it to 0, I'll put a value and play around which value works with the logs.

Any additional idea or recommendation is highly appreciated.

0 Karma
Reply
Solved! Jump to solution

erwinpastor
Explorer
‎05-03-2015 07:34 PM

Tried adding the following but the log is still breaking down to few events

MAX_EVENTS = 10000
TRUNCATE = 100000

I'm quite hesistant to set TRUNCATE to 0 as it may cause problems with Splunk performance. Any suggestions?

0 Karma
Reply
Solved! Jump to solution

acharlieh
Influencer
‎05-03-2015 09:08 PM

I don't think TRUNCATE will help you here, that's because TRUNCATE is used for line breaking, the most number of characters on a "line". As you're not redefining what ends a line by changing the definition of LINE_BREAKER, I don't think that's it. However event breaking is only done once, are you reindexing in between changing your props.conf defintion? Also are you changing these definitions on the Splunk Node that's doing parsing (A Heavy Forwarder or Indexer... NOT the Universal Forwarder)

Reply
Solved! Jump to solution

erwinpastor
Explorer
‎05-03-2015 09:16 PM

Thanks for the feedback. Yes, I'm doing it in the heavy forwarder. I've added a LINE_BREAKER and set the expression to a dummy string. Seems to be working OK now.

0 Karma
Reply
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...