Splunk Search

Regex Help; exceeded configured match_limit

JoshuaJohn
Contributor

I have this query that works in all regex assist sites but is too greedy for my Splunk Environment. I am unable to add it to props, and it must be in the query itself.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+).+?(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+).+?(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+).+?(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+).+?(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+).+?(?:\"isProduction\\\" : )(?<Prod>[^,]+).+?(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+).+?(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+).+?(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+).+?(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+).+?(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+).+?(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"

Here is what a log looks like

{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01  StartEvent {\n  \"appDetails\" : {\n    \"version\" : \"1\",\n    \"number\" : \"1\",\n    \"ver\" : \"11.1.0\",\n    \"release\" : \"11.1.1\",\n    \"storeId\" : \"0120\",\n    \"terminalId\" : \"41\"\n  },\n  \"capTime\" : \"2019-01-01 10:59:48\",\n  \"MSR\" : {\n    \"MSRReach\" : \"Scanner not connected\",\n    \"version\" : \"Not available\",\n    \"isConnected\" : true,\n    \"SB\" : \"\"\n  },\n  \"isProduction\" : true,\n  \"pinpad\" : {\n    \"serialNumber\" : \"111-112-111-111\",\n    \"OSVersion\" : \"AWS12343\",\n    \"model\" : \"910X5\"\n  },\n  \"MDM\" : {\n    \"lastSeen\" : \"Not Available\"\n  },\n  \"device\" : {\n    \"Serial\" : \"BV12AXAJ013J\",\n    \"battery\" : \"100.0\",\n    \"network\" : \"ReachableViaWiFi\",\n    \"osVersion\" : \"11.0.1\",\n    \"memory\" : \"100\"\n  }\n} EndEvent\n"

Any ideas to improve efficiency?

Tags (2)
0 Karma
1 Solution

JoshuaJohn
Contributor

Well this is ugly, but it works.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)" 
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)" 
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)" 
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)" 
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)" 
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)" 
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)" 
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)" 
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)" 
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)" 
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)" 
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"

View solution in original post

woodcock
Esteemed Legend

Have you tried piping your events through | kv or | spath? You will be surprised at how much gets done well enough!

JoshuaJohn
Contributor

I haven't, but I will look into it. Looks promising especially for my data.

0 Karma

JoshuaJohn
Contributor

Well this is ugly, but it works.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)" 
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)" 
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)" 
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)" 
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)" 
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)" 
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)" 
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)" 
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)" 
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)" 
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)" 
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...