I have a csv file with data in the following format...
logsource,Critical,Buffer Overflow,15:05:27 13 Mar 2017,,sourceserver_172.92.110.10 (172.92.110.10),,somedesthost_172.92.110.18 (172.92.110.18),",N/A/1080",Application Servers Protection Violation,buffer overflow,,,2,01548914,
logsource,Medium,Application Activity,12:11:23 31 Mar 2017,,172.92.110.83,,somedestination_172.92.110.88 (172.92.110.88),file-wbt-server [tcp/19380],,,,,22,01547076,
logsource,Informational,Application Activity,23:56:20 31 Mar 2017,India,202.91.201.54,United States,ABCDEF_172.74.18.13 (172.74.168.13),tcp/443,,,,,22,01553030,
logsource,Informational,Application Activity,23:49:22 31 Mar 2017,,somesource1_172.92.18.7 (172.92.18.7),United States,14.35.2.46,tcp/1443,,,,,3,01552984,
I need to extract the source IP address from the 6th fields in each row and save in a field "src_ip_address"
eg. from line 1, src_ip_address = 172.92.110.10
from line 2, src_ip_addres = 172.92.110.83
Similarly I need to extract the destination IP address from the 8th field and store the values in a "dst_ip_address" field
eg.
from line 1, dst_ip_address = 172.92.110.88
from line 2, dst_ip_address = 172.92.110.18
Is this possible? if yes, it will be of immense help if someone could show a clue.
Thanks !
Try using rex in query this way -
| rex "([^\,]*\,){5}[^\,]*(?<src_ip_address>(\d+\.){3}\d+)[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip_address>(\d+\.){3}\d+)"
You used the regex tag on your message, but it's not clear if you want to the extraction at index time or search time.
For search-time extraction, try this rex
command.
... | rex "(?:[^,]*?,){5}(?<src_ip_address>[^,]*?),[^,]*?,(?<dst_ip_address>[^,]*?)," | ...
For index-time extraction, try putting this stanza in your props.conf file.
[mysourcetype]
INDEXED_EXTRACTIONS = CSV
FIELD_NAMES = logsource, severity, condition, timestamp, field5, src_ip_address, field7, dst_ip_address, field9, field10, field11, field12, field13, field14, field15
TIMESTAMP_FIELDS = timestamp
Hi, I could do the search - time extraction using the rex command -
... | rex field=_raw "([^\,]*\,){5}[^\,]*(?<src_ip>\b((\d+\.){3}\d+))[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip>\b(\d+\.){3}\d+)"
Now is there a way to do the extraction at the time of indexing ?
I tried your suggestion to add a stanza to the props.conf file, it did not work.
Add this under sourcetype -
EXTRACT_FIELDS = ([^\,]*\,){5}[^\,]*(?<src_ip>\b((\d+\.){3}\d+))[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip>\b(\d+\.){3}\d+)
Thanks that worked !
Try using rex in query this way -
| rex "([^\,]*\,){5}[^\,]*(?<src_ip_address>(\d+\.){3}\d+)[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip_address>(\d+\.){3}\d+)"
Thank you for the regular expression, it's working except that it is not showing the first digit in the first octet of the src and dst ip addresses.
... | rex "([^\,]*\,){5}[^\,]*(?<src_ip_address>(\d+\.){3}\d+)[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip_address>(\d+\.){3}\d+)"
| top 5 src_ip_address, dst_ip_address
src_ip_address dst_ip_address
0.92.46.23 0.92.138.71
0.92.46.25 0.92.138.71
trying to fix it... any quick suggestions?
thanks again.
This one did the magic!... could you please let me know if this is the right fix?
... | rex field=_raw "([^\,]\,){5}[^\,](?\b((\d+.){3}\d+))[^\,]\,[^\,]\,[^\,]*(?\b(\d+.){3}\d+)"