Data:
message:
================>
Request Details:
[requestId:123122313-3453-1122-1112222]
[requestMethod = GET]
[requestUrl = http://api.dev.com/testing.json?FromDate=2019-03-06T11:00:00&ToDate=2019-03-06T12:00:00]
[requestHeaders = {x-amzn-trace-id=Root=66-2342-2342243h2hhhgh424jh3, x-forwarded-proto=https, host=api.dev.com, x-forwarded-port=443, **x-forwarded-for**=xx.xxx.xx.xxx, accept-encoding=gzip, deflate, **user-agent**=python-requests/5.26.9, accept=*/*}]
Query:
index=test_prod sourcetype="test-search-api" | rex field=message "x-forwarded-for=(?P<client_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}[^,]+)"| search client_ip="*"*
Expected output:
I'm getting all the IPs, whereas I want only those IPS in which user-agent=SFDC.
Rest of all is good with the query.
The following might be quicker to execute (untested)
index=test_prod sourcetype="test-search-api" SFDC user-agent
| rex field=message "x-forwarded-for=(?P<client_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}[^,]+)"
By putting the search restriction "SFDC user-agent" before the first pipe should mean less work for the rex command as fewer events to process and therefore quicker.
Depending on how field extraction is done user_agent=SFDC instead of just "SFDC user-agent" might be even quicker again. If no field extraction is done try "user_agent=SFDC"
index=test_prod sourcetype="test-search-api" user-agent=SFDC | rex field=message "x-forwarded-for=(?P< client_ip>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}[^,]+)"| table client_ip
Please remove the same from above query in between P< and client_ip.
Hi @changj
Try by adding the condition
|rex field=message "x-forwarded-for=(?P<client_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}[^,]+).*user-agent=(?P<user_agent>.*)," | where user_agent="SFDC"