Getting Data In

Unable to retrieve the value where key contains ": " Symbol

karthi25
Path Finder

I have splunk log as follows:

   2018-02-21T18:29:31.958125+00:00 EQM-SCMS.Test-SCMS-qlab02.tfbhardGoodsSCMS-test fa4cbb7b-26fa-425e-968d-05dabde7c79a[[APP/PROC/WEB/0]]: cf_foundation=px-npe01 cf_app_name=tfb_hardGoods_SCMS-test cf_app_id=fa4cbb7b-26fa-425e-968d-05dabde7c79a cf_org_name=EQM-SCMS cf_org_id=56f5bed9-cbdc-4ae3-a1e8-73072442a1fe cf_space_name=Test-SCMS-qlab02 cf_space_id=ba8816e1-36d0-4857-9396-87dbf162aead .source.s_cf_apps {"@timestamp":"2018-02-21T10:29:31.957-08:00","@version":1,"message":"Acknowledging Message for consumerTag : cb6ba2fe-dc20-42e4-88ae-b94d3f17f611, deliveryTag : 32","logger_name":"com.tmobile.deep.AMQPEventConsumer","thread_name":"pool-2-thread-4","level":"INFO","level_value":20000,"APP_NAME":"tfb_hardGoods_SCMS","eventID: ":"d100ecb2-5821-4089-b8aa-902c0a8629a0","eventType: ":"TBCCommissionUpgradeOrderFeed"}

2018-02-21T18:19:58.059358+00:00 EQM-SCMS.Test-SCMS-qlab02.tfbhardGoodsSCMS-test fa4cbb7b-26fa-425e-968d-05dabde7c79a[[APP/PROC/WEB/0]]: cf_foundation=px-npe01 cf_app_name=tfb_hardGoods_SCMS-test cf_app_id=fa4cbb7b-26fa-425e-968d-05dabde7c79a cf_org_name=EQM-SCMS cf_org_id=56f5bed9-cbdc-4ae3-a1e8-73072442a1fe cf_space_name=Test-SCMS-qlab02 cf_space_id=ba8816e1-36d0-4857-9396-87dbf162aead .source.s_cf_apps {"@timestamp":"2018-02-21T10:19:58.059-08:00","@version":1,"message":"Before publishing the Event to Kafka:::LoggingHeader [eventType=TBCCommissionUpgradeOrderFeed, clientEventId=null, deepEventId=d2afaee0-1733-11e8-a03a-a96c3fa64a03, eventTime=03-21-2018 10:19:57, eventProducerId=TBC, filteredSystemsList=null, filteredExceptionMessage=null]","logger_name":"com.tmobile.deep.AMQPEventConsumer","thread_name":"pool-2-thread-3","level":"INFO","level_value":20000,"APP_NAME":"tfb_hardGoods_SCMS","eventID: ":"ca2f9346-ac31-465f-b306-11beb2543f52","eventType: ":"TBCCommissionUpgradeOrderFeed"}

2018-02-20T20:48:39.782740+00:00 EQM-SCMS.Test-SCMS-qlab02.tfbhardGoodsSCMS-test fa4cbb7b-26fa-425e-968d-05dabde7c79a[[APP/PROC/WEB/0]]: cf_foundation=px-npe01 cf_app_name=tfb_hardGoods_SCMS-test cf_app_id=fa4cbb7b-26fa-425e-968d-05dabde7c79a cf_org_name=EQM-SCMS cf_org_id=56f5bed9-cbdc-4ae3-a1e8-73072442a1fe cf_space_name=Test-SCMS-qlab02 cf_space_id=ba8816e1-36d0-4857-9396-87dbf162aead .source.s_cf_apps {"@timestamp":"2018-02-20T12:48:39.782-08:00","@version":1,"message":"Event with id : Snehal45678 , will be redelivered after 731757 ms from now ","logger_name":"com.tmobile.deep.AMQPWaitExchangePublisher","thread_name":"pool-2-thread-3","level":"INFO","level_value":20000,"APP_NAME":"tfb_hardGoods_SCMS","eventID: ":"Snehal45678","eventType: ":"TBCCommissionUpgradeOrderFeed"}

Now I want to get the unique events ID , I have tried the query:

index=cloudfoundry sourcetype=cloudfoundry_apps "cf_foundation=***" "cf_org_name=testOrg" "cf_space_name=Test" "cf_app_name=app-test" | stats count by eventID: 

But it returns nothing. Can anyone please help me to get it.

0 Karma
1 Solution

mayurr98
Super Champion

You can try something like this

index=cloudfoundry sourcetype=cloudfoundry_apps "cf_foundation=***" "cf_org_name=testOrg" "cf_space_name=Test" "cf_app_name=app-test" | rex field=_raw "eventID\:\s\"\:\"(?<eventID>[^\"]+)"| stats count by eventID

what transforms you have written for extracting the fields?
If you want to extract the fields in index time properly.
Then

in transfroms.conf

[test]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = \"([^\"]+)\":\"([^\"]+)

and in props.conf

[<your_sourcetype>]
REPORT-test = test

and then try your query

 index=cloudfoundry sourcetype=cloudfoundry_apps "cf_foundation=***" "cf_org_name=testOrg" "cf_space_name=Test" "cf_app_name=app-test"| stats count by eventID:

let me know if this helps!

View solution in original post

mayurr98
Super Champion

You can try something like this

index=cloudfoundry sourcetype=cloudfoundry_apps "cf_foundation=***" "cf_org_name=testOrg" "cf_space_name=Test" "cf_app_name=app-test" | rex field=_raw "eventID\:\s\"\:\"(?<eventID>[^\"]+)"| stats count by eventID

what transforms you have written for extracting the fields?
If you want to extract the fields in index time properly.
Then

in transfroms.conf

[test]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = \"([^\"]+)\":\"([^\"]+)

and in props.conf

[<your_sourcetype>]
REPORT-test = test

and then try your query

 index=cloudfoundry sourcetype=cloudfoundry_apps "cf_foundation=***" "cf_org_name=testOrg" "cf_space_name=Test" "cf_app_name=app-test"| stats count by eventID:

let me know if this helps!

karthi25
Path Finder

@mayurr98 thanks. It's works as expected

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...