Hi,
I am very new to Splunk and am trying to get all the fields in response. One of the fields is actually a JSON string with a structure similar to this:
2016-06-17 09:16:17,899 [-] [HttpClientWorker-12] INFO SERVICE_LOGGER To: http://www.w3.org/2005/08/addressing/anonymous|WSAction: |SOAPAction: |log_uid = 1234|log_Response = {"resultCode":true,"score":{"-- rest of the response here...
I am trying to fetch the response field for log_Response above with a search to show me all the fields in tabular format:
index=pi* host="ServerName" "log_Response" "log_uid" | table log_Response
The above search only returns a string for log_Response until 'true' string in response and rest of the string content is missing. i.e.
Response in table:
{"resultCode":true // rest of the content missing
Does anyone know what the issue could be? There are other req/resp for other services which work fine, but this is the only one that has this truncation issue, and I think it's because of the boolean value in the response log.
Thank you!
The default field extraction (of kv pair) will terminate at first space/comma which is what it's doing it here. You probably would need to setup some custom search time field extract to get the full value in log_Response field. We can help you will regex if you can provide a full events (mask sensitive data). Meanwhile give this a try
index=pi host="ServerName" "log_Response" "log_uid" | rex "log_Response=(?<log_Response>.+)$" | table log_Response
The default field extraction (of kv pair) will terminate at first space/comma which is what it's doing it here. You probably would need to setup some custom search time field extract to get the full value in log_Response field. We can help you will regex if you can provide a full events (mask sensitive data). Meanwhile give this a try
index=pi host="ServerName" "log_Response" "log_uid" | rex "log_Response=(?<log_Response>.+)$" | table log_Response
Hi,
Thanks for the reply, I tried above regex but it didn't work. I noticed that one of the fields after doing "table *" instead of "table log_response" is "meta" with value "truncated" . I wonder if this is the culprit? How can I tell splunk not to truncate the field string?
edit: I also see the total number of characters is exactly 10000, how can i increase the size? I am accessing splunk logs through web browser with some splunk queries in it which i can control but i dont have any control on configuration files
Thank you!
The meta=truncated means that size of event is beyond the limit set in limits.conf (see this
http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configureeventlinebreaking#Multiline_event_li... ). Means, you won't see the full event in _raw itself but it should not affect field extraction (it does but not directly).
There was a missing space in the regex. Try this
index=pi host="ServerName" "log_Response" "log_uid" | rex "log_Response\s*=\s*(?<log_Response>.+)$" | table log_Response
Man, you are the wizard !!! Thank you so much, it worked !!!!
Can you tell me how to mark this as an answer 🙂
You should be now.
Done, have a nice day !