We are currently sending all of our Palo Alto syslogs to a syslog server that collects multiple machines syslogs and forwards them via a universal forwarder to our splunk instance.
We filtered out all logs tagged with the palo alto device name and set the sourcetype to pan_log
heres the piece of our inputs.conf broken out for the palo alto logs from our syslog server
The index=syslog is the generic index name we use for all syslogs rather than 'main' or 'default' etc.
we also made an update to the macros.conf on the application side via our search head and included the index name under :
definition = index=syslog sourcetype="pan_threat" NOT "THREAT,url"
definition = index=syslog sourcetype="pan_traffic"
definition = index=syslog sourcetype="pan_system"
definition = index=syslog sourcetype="pan_config"
definition = index=syslog sourcetype="pan_threat" "THREAT,url"
Oddly enough under this dir
the inputs.conf listed there is empty..? is this correct?
Now as it stands I am able to see under splunk deployment monitor a pan_log sourcetype that is receiving traffic but I am unable to view any data under the palo alto app or by doing an independent search such as sourcetype="pan_log" or 'pan_threat' etc.
Any help would be greatly appreciated.
adding to the summary indexing discussion: please take a look at this post: http://splunk-base.splunk.com/answers/5837/summary-indexing-on-a-search-head . also, if you plan on using multiple indexers, i would discourage the use of summary indexes for now. admittedly, the summary indexing use is not the best in this app. the summaries are of a high dimensionality, which results in a low summarized to raw data ratio. ultimately, the summaries will become very large. i am working on a better strategy for this.
'pan_threat' host="pa*" : i was unable to recreate this issue. this search works ok on a newly installed splunk instance with a fresh install of the app.
the reason for : This - index="pan_logs" pan_threat | bin _time span=5m | fillnull vsys app category src_ip dst_ip severity RISK threat_id CATEGORY | stats count by vsys app threat_id severity category src_ip dst_ip log_subtype CATEGORY RISK _time works
because the pan_logs index is not in the default search path of the user running the search.
i appreciate your feedback. i have added several things to my to do list for the next version of the app. happy to talk to you in person about some of this.
The app's main dashboard page has inline searches. Those searches use index=pan_logs. Other views have searches built on the macros. You have already modified those macros. But adding the index=syslog was not neccessary for those views.
Lastly, it is a good practice to keep different log types separated by indexes. I would not recommend sending all syslog type logs into one index.
You shouldn't be editing anything in the default folder. Anything you want to modify should be in the local folder. I believe stanza/section's in local supersede anything in default. Here is what my inputs.conf in /opt/splunk/etc/apps/SplunkforPaloAltoNetworks/local
[udp://514] connection_host = ip sourcetype = pan_log index = pan_logs no_appending_timestamp = true
Ok hate to use the answer box because this is only kinda an answer based off everything I've gathered from people on this thread, so if another noob is searching and finds this hopefully this overview will get them part way there pretty quickly without sifting and piecing together all this from the whole thread, aside from my added issue at the bottom that is
Overview: Distributed search
and tag them as pan_log (initially was merging into an existing index -
now using a newly created index of 'pan__logs')
- I was able to view data by searching pan_log -
- **N**o transforms occurring -
- I had not created an app props dir on the indexers for the props and
transforms conf's -
- **C**reated an app directory and populated the props and transforms
under its default dir ie /opt/splunk/etc/apps/pan_props/default
- **R**estarted indexers in sequence.
- **R**eceived errors regarding index for pan_logs (index was created on
the search head but had not been created on the indexers)
- **C**reated the index on indexers and updated index.conf to match
- **R**estarted in sequence again -
- **P**A data now being transformed and residing in it's own index.
- **H**owever macros are not searchable so no 'pan_ threat' just
index=pan __logs sourcetype="pan_ _threat" so none of the saved searches work.
- **S**o just to see what would happen, re specified the index on the base
macros on macros.conf on the search head (I know Monzy you said it was un-neccesary to modify the macros but for some reason it seemed to work so I went with it) ie:index=pan_logs
- **R**estarted the splunk service on the search head and macros started
- **A**ble to search based off macros for 'pan_system' etc
- **A**ble to use saved searches. Progress!
I am still having a few other issues but I'll post those as a comment on this.
I have an existing syslog server that was already receiving PaloAlto logs. I've installed the universal forwarder on the syslog server and it is successfully sending the data to splunk. The PA app seems to be working ok, with the exception that searches by username always return no data. If I use the default splunk search page, I would like the 'host' field to be populated with the hostname of the firewall that generated the log message. Right now, it is always populated with the name of the logfile "firewall.log". I have tried several iterations of transforms.conf and props.conf on the syslog/forwarder host. Here are the current contents:
transforms.conf [PAN_Firewall] DEST_KEY = MetaData:Host REGEX = \b\w*-\w*-\d\b FORMAT = host::$1 props.conf [source::/var/log/syslog/firewall.log] TRANSFORMS-PAN=PAN_Firewall