Quick question, any reason you're writing regex for the other objects in that data? It looks like a json object. With the correct sourcetype the url field should exist already, then you could use a transform like the below (Hoping for feedback on performance from @cpetterborg on the below being the master of regex 🙂 😞 [extract_page_from_url] SOURCE_KEY = url REGEX = \S+\/([^\/\?]+) FORMAT = page::$1 Also, check out the URL toolbox app (https://splunkbase.splunk.com/app/2734/), very handy when working with URL's ... View more

The regex doesn't have 3 capture groups, hence the index out of bounds... 3 doesn't exist. The use of ?: says this is a non-capturing group. For now to make the errors stop I would create a new transforms.conf in Splunk_TA_microsoft-cloudservices/local and modify the FORMAT parameter so it doesn't reference the non-existant capture group. The below should do the trick: [o365_invalid_account] FORMAT = account_name::$1 url::$2 account_status::"invalid" ... View more

I've previously asked for message tracking on the add-on too. Would be really handy In the mean time it's possible to use the 'Get-MessageTrace' powershell command in a simple script. Don't expect the below to work by copying it verbatim as it's not complete but it should give you an idea what I've done to deal with this for the interim. It's not perfect but it does work. If anyone wants to offer suggests to wrap some more smarts around it go your hardest, feedback is more than welcome 🙂 ... <session stuff> ... For ($i=1; $i -le 10; $i++) { Get-MessageTrace -PageSize 5000 -Page $i -StartDate ([DateTime]::Now.AddMinutes(-30)) -EndDate ([DateTime]::Now.AddMinutes(-15)) | Select @{Name="time";Expression={$_."Received"}}, @{Name="message_id";Expression={$_."MessageID"}}, @{Name="recipient";Expression={$_."RecipientAddress"}}, @{Name="sender";Expression={$_."SenderAddress"}}, @{Name="src_ip";Expression={$_."FromIP"}}, @{Name="dest_ip";Expression={$_."ToIP"}}, @{Name="subject";Expression={$_."Subject"}}, @{Name="status";Expression={$_."Status"}}, @{Name="size";Expression={$_."Size"}} | Export-Csv "c:\temp\message_trace\$((get-date ([DateTime]::Now) -Format yyyyMMddTHHmm))_page_$i.csv" -NoTypeInformation } ... View more

Had the issue with Windows app too running on SLES and Splunk Enterprise 6.4.2 Warning: I don't know the flow on effects of changes to mime-types so I don't know the broader impact of this. But along the lines of @malmoore suggestion I commented out the line: text/x-js js in the file /etc/mime-types and restarted Splunk and the app worked RHEL doesn't have 'text/x-js js' in /etc/mime-types so I guess that explains why it only seems to present as an issue on SLES ... View more

So the macro "cim_Network_Traffic_indexes" contains (index="network_summary" OR index="network_summary2" OR index="network_summary3")"? That seems strange, the network_summary* indexes are from ES, not the original data, that could be the issue. I am not sure why this would be the case, maybe something with your deployment? The macro "cim_Network_Traffic_indexes" should define the indexes to use in the data model. These cim_* macros are really to improve performance Assuming there is a reason for the network_summary indexes listed in the macro, you could add the real data index to that macro and give it a go, i.e. add " OR index=" in the brackets ... View more

Go to settings>advanced search>search macros. Select 'Splunk Common Information Model' (Splunk_SA_CIM) from the 'app context' menu. It should be listed there. do you get anything by searching just the tags: tag=network tag=communicate ... View more

Assuming the current CIM data models here, do you get search results from the root object of the data model: ( cim_Network_Traffic_indexes ) tag=network tag=communicate If no, have you defined indexes in the cim_Network_Traffic_indexes macro? Or do you have the index in your default searched indexes? ... View more

Thanks for that Andy! I've edited my answer. ... View more

What Splunk architecture do you have? As sourcetype=pan_logs works, I suspect you might have simply incorrectly named the sourcetype incorrectly in inputs.conf. It should be 'sourcetype=pan_log' not 'sourcetype=pan_logs'. The app will rewrite sourcetype=pan_log to other sourcetypes based on data being sent in with a sourcetype of 'pan_log'. If sourcetype=pan_logs happens to be a typo and the sourcetype is actually 'pan_log', then I suspect you have a distributed architecture. In a distributed architecture there are some processes (in particular the sourcetype renaming) which are performed by the indexers as data goes in. The main config you need is in the props.conf and transforms.conf files (should be here: $SPLUNK_HOME/etc/apps/SplunkforPaloAltoNetworks/default/). The configuration in these files is what rewrites the pan_log sourcetype to each respective sourcetype based on text in each log message. Hope that helps ... View more

Hi @theouhuios how did you solve this? ... View more

I think acharlieh is right. Assuming the log is being written by syslog you should be able have it write the correct permissions. However if this isn't possible, you might be able to use 'setfacl' Unix command on the directory the file is in to add explicit perms for your Splunk user. Something like this: setfacl -m u:splunkuser:r /var/log ... View more

You should be able to do it. What operating system are you using on the Splunk server you are runnnng db connect on? The key information is the last line of that error. if db connect is on a Linux host it won't be part of a trusted domain Have you followed ssievert_splunk advice? ... View more

I suspect DB connect running from a Linux host or a windows host on a different domain For the sake of testing can you try using sql authentication instead of windows authentication? ... View more

A few ways I could think of doing it. If you have the commonname of the group: | ldapsearch domain="<domain>" search=(&(objectClass=group)(cn=<groupCommonName>)) attrs="member" | mvexpand member | ldapfilter domain="<domain>" search=(&(objectClass=user)(distinguishedName="$member$")) attrs="sAMAccountName" The easiest but least efficient way if you have the distinguished name for the group (Warning, this may cause excessive load on AD, Thanks Andy!): | ldapsearch domain="<domain>" search=(&(objectClass=user)(memberOf="<GroupDN>")) attrs=sAMAccountName ... View more

What else is configured from the sourcetype you're assigning it? Did you restart Splunkd after modifying the sourcetype? line breaking is one of those things that needs a restart Something else to check, do you have a distributed deployment? Where did you apply the props.conf changes? Line breaking should be performed on the system that you are running DB connect on (expecting DB Connect is on a search head) assuming you are sending 'cooked' data to the indexer, which is the default for tcpout (in outputs.conf). If it is not sending cooked data, then linebreaking will need to be done at the indexer. ... View more

I don't use the profile Block List text box, so can't really answer for that specific use. Usually setting the action to block should send a syslog message If it doesn't log using the block list text box in the URL profile, you could try creating a custom category and add that to the URL profile. ... View more

You don't need to do anything special with the Palo Splunk App Assuming you have the appropriate security policy configured for url filtering, and that same policy is configured to forward syslog messages correctly. There are a couple of things to look at. Most of the url logs are informational events. Check your syslog profile is set to send informational events. Also in the URL filtering configuration (Objects>security profiles>URL filtering). Set the desired categories to an action of 'alert' and it will syslog them out. Then in splunk they will appear as a sourcetype of "pan_threat" There is also an option to 'log container page only' which will not log all content. Uncheck that and you should get everything. ... View more

I installed it 2 weeks ago, and also had issues getting it started. It should be fine on udp 2055. Not sure what happened but for some reason the following directories were not being created: ./Splunk_TA_flowfix/nfdump-ascii ./Splunk_TA_flowfix/nfdump-binary After creating them manually it worked fine. Also worth looking at, the flowfix.sh script has hard coded paths. The configure.sh script assumes/expects it is being run from the directory the script is in. Also be aware of the paths in flowfix.sh if you ever move the TA to another host. ... View more

Agree with above, there are many variables. Very simply put from my experience if search performance is at all important go for as many indexers as you can wrangle the budget for (yeah, helpful statement... I know). If I had my time again, I would have spread our hardware budget across more, lower spec'd servers (fewer cpu cores, less memory and less disk capacity (Still highest iops as possible). From my understanding search is not multithreaded which is what I see as the issue with a single massive server. ... View more

Hi, It sounds like a drilldown would be more what you are looking for. There are examples on how to achieve this in the example apps Examples App Cheers Mat ... View more

You could try the multisearch command ... View more