All Apps and Add-ons

Why am I unable to extract fields in Splunk for Squid app?

bortnikl
New Member

I am using Splunk version 6.1.3 and Squid App 0.2v.
My props.conf contain:

[squid]
TIME_FORMAT = %s.%3N
MAX_TIMESTAMP_LOOKAHEAD = 15
KV_MODE = none
SHOULD_LINEMERGE = false
REPORT-squid = squid

My transforms.conf contain:

[squid]
REGEX = ^\d+\.\d+\s+(\d+)\s+([0-9\.]*)\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:]*)://)?([^/:]+):?(\d+)?(/?[^ ]*))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.*)$
FORMAT = duration::$1 clientip::$2 action::$3 http_status::$4 bytes::$5 method::$6 uri::$7 proto::$8 uri_host::$9 uri_port::$10 uri_path::$11 username::$12 hierarchy::$13 server_ip::$14 content_type::$15

I am receiving squid logs at UDP 514. Sourcetype is manually set to "squid" (I just wrote squid in the field)
When I try search sourcetype="squid", things goes fine and I have results, but if I put command sourcetype="squid" action="*" , I do not have any results.
When I run squid app, there are no results too. Job inspector shows me this information:

This search has completed, but did not match any events. The terms specified in the highlighted portion of the search and this part is highlighted:

search sourcetype="squid" action="*" | eval reqcount=1

and

The following messages were returned by the search subsystem:

DEBUG: base lispy: [ AND sourcetype::squid ]
DEBUG: search context: user="admin", app="SplunkforSquid", bs-pathname="/opt/splunk/etc"

As a test, I tried to change sourcetype to "syslog" and run search

"*search sourcetype="syslog" action="**" 

.. and It gave me results, but when I tried to put search like:

*search sourcetype="syslog" clientip="*" uri_host="*" uri_path="*"*

.. I didn`t get any results. There are no clientip, uri_host or uri_path fields extracted, or even indexed.

Can you help me with this problem?

0 Karma
1 Solution

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

View solution in original post

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

bortnikl
New Member

Here is a sample: Aug 25 12:39:35 x.x.x.x Aug 25 12:39:37 somename (squid): 1408963177.419 138515 X.X.X.X TCP_MISS/000 10925 CONNECT safebrowsing-cache.google.com:443 - DEFAULT_PARENT/servername -

0 Karma

Ayn
Legend

Well not custom really, it's what Squid comes with by default out of the box. But yeah, assumptions regarding the format are definitely made.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...