How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
@TheGU wrote:How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
bluecoat server is awsome man
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
I found that transaction may be up to 2kB. Very hard to write a regex to cover all of them. However your answer is Ok. Thanks.