Why is my search returning a log response value that is truncated after the boolean string "true" in my sample response?

[psable]

Hi,

I am very new to Splunk and am trying to get all the fields in response. One of the fields is actually a JSON string with a structure similar to this:

2016-06-17 09:16:17,899 [-] [HttpClientWorker-12]  INFO SERVICE_LOGGER To: http://www.w3.org/2005/08/addressing/anonymous|WSAction: |SOAPAction: |log_uid = 1234|log_Response = {"resultCode":true,"score":{"-- rest of the response here...

I am trying to fetch the response field for log_Response above with a search to show me all the fields in tabular format:

index=pi* host="ServerName" "log_Response" "log_uid" | table log_Response

The above search only returns a string for log_Response until 'true' string in response and rest of the string content is missing. i.e.

Response in table:

{"resultCode":true   // rest of the content missing

Does anyone know what the issue could be? There are other req/resp for other services which work fine, but this is the only one that has this truncation issue, and I think it's because of the boolean value in the response log.

Thank you!

[somesoni2 · Accepted solution]

The default field extraction (of kv pair) will terminate at first space/comma which is what it's doing it here. You probably would need to setup some custom search time field extract to get the full value in log_Response field. We can help you will regex if you can provide a full events (mask sensitive data). Meanwhile give this a try

index=pi host="ServerName" "log_Response" "log_uid" | rex "log_Response=(?<log_Response>.+)$" | table log_Response

[psable]

Hi,
Thanks for the reply, I tried above regex but it didn't work. I noticed that one of the fields after doing "table *" instead of "table log_response" is "meta" with value "truncated" . I wonder if this is the culprit? How can I tell splunk not to truncate the field string?

edit: I also see the total number of characters is exactly 10000, how can i increase the size? I am accessing splunk logs through web browser with some splunk queries in it which i can control but i dont have any control on configuration files

Thank you!

[somesoni2]

The meta=truncated means that size of event is beyond the limit set in limits.conf (see this
http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configureeventlinebreaking#Multiline_event_li... ). Means, you won't see the full event in _raw itself but it should not affect field extraction (it does but not directly).

There was a missing space in the regex. Try this

index=pi host="ServerName" "log_Response" "log_uid" | rex "log_Response\s*=\s*(?<log_Response>.+)$" | table log_Response

[psable]

Man, you are the wizard !!! Thank you so much, it worked !!!!

[psable]

Can you tell me how to mark this as an answer 🙂

[somesoni2]

You should be now.

[psable]

Done, have a nice day !