I have this query that works in all regex assist sites but is too greedy for my Splunk Environment. I am unable to add it to props, and it must be in the query itself.
| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+).+?(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+).+?(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+).+?(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+).+?(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+).+?(?:\"isProduction\\\" : )(?<Prod>[^,]+).+?(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+).+?(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+).+?(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+).+?(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+).+?(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+).+?(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"
Here is what a log looks like
{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01 StartEvent {\n \"appDetails\" : {\n \"version\" : \"1\",\n \"number\" : \"1\",\n \"ver\" : \"11.1.0\",\n \"release\" : \"11.1.1\",\n \"storeId\" : \"0120\",\n \"terminalId\" : \"41\"\n },\n \"capTime\" : \"2019-01-01 10:59:48\",\n \"MSR\" : {\n \"MSRReach\" : \"Scanner not connected\",\n \"version\" : \"Not available\",\n \"isConnected\" : true,\n \"SB\" : \"\"\n },\n \"isProduction\" : true,\n \"pinpad\" : {\n \"serialNumber\" : \"111-112-111-111\",\n \"OSVersion\" : \"AWS12343\",\n \"model\" : \"910X5\"\n },\n \"MDM\" : {\n \"lastSeen\" : \"Not Available\"\n },\n \"device\" : {\n \"Serial\" : \"BV12AXAJ013J\",\n \"battery\" : \"100.0\",\n \"network\" : \"ReachableViaWiFi\",\n \"osVersion\" : \"11.0.1\",\n \"memory\" : \"100\"\n }\n} EndEvent\n"
Any ideas to improve efficiency?
Well this is ugly, but it works.
| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)"
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)"
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)"
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)"
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)"
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)"
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)"
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)"
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)"
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)"
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)"
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"
Have you tried piping your events through | kv
or | spath
? You will be surprised at how much gets done well enough!
I haven't, but I will look into it. Looks promising especially for my data.
Well this is ugly, but it works.
| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)"
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)"
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)"
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)"
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)"
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)"
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)"
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)"
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)"
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)"
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)"
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"