Yeah, the doc team was really responsive when I reached out once before. I just wanted to make sure it's actually an error and not just a case of Idiot User. ... View more

Yeah, reverting to 3.1.4 fixed it for me. ... View more

I see that in v7.0.0 of the Splunk Add-on for Windows, the addon includes the config necessary to convert perfmon values to metrics (source). Looks like the default collection method is multikv, but if you change it to single then there are configs in the props.conf and transforms.conf to convert the resulting value to metric form. ... View more

Just FYI for any future readers. In my case I was using the Splunk App for Windows Infrastructure v2.0.0, which does not have a windows_apps.csv lookup file. But my search heads (I have a search cluster with a deployer server) still had the lookup file in the app's directory. I guess the deployment doesn't delete files. My fix was to remove the splunk_app_windows_infrastructure directory from my search heads before re-deploying from the deployer. That way I didn't have leftover files from older versions of the app. ... View more

I'm not terribly familiar with regex, but it looks like you might have a reliable pattern of two closing curlies - }} - followed eventually by a double quote. So perhaps: ,"sql_queries":".*?}}.*?" https://regex101.com/r/IYPCl3/1 So, grab everything until you get to the first set of double closing curlies, and then grab everything after that until the first double quote. ... View more

I notice that both your queries above say "span=1h". Is the second one - the one with the lower result - supposed to be "span=1d"? If so, here's a possibility: For span=1h, it counts the distinct Session IDs in each hour and sums them up. For span=1d, it's counting all the distinct IDs in the day. Let's say you have Session ID 12345 that appears at 2 pm and at 7 pm. Perhaps with span=1h it gets counted twice - once for the 2pm span and once for the 7 pm span. But with span=1d it just gets counted once. ... View more

Perfect, thanks! That gives me a good option to filter it out at collection type. The filter is a bit of an interesting challenge. I only caught this because one of the common mount points - /mnt - showed up in the data, but we've got NFS filesystems on various mount points other than /mnt. It looks like the filesystems we'd want to include always start with /dev, so a piped command like | egrep '^\/dev' would catch them all. I took a look at a Linux server (we don't have that many), and it looks like the ones which don't start with /dev are already being filtered out by your egrep -v : $ df -k -P Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/root xxx xxx xxx 32% / devtmpfs xxx xxx xxx 0% /dev tmpfs xxx xxx xxx 0% /dev/shm tmpfs xxx xxx xxx 3% /run tmpfs xxx 0 xxx 0% /sys/fs/cgroup /dev/mapper/lvol0 xxx xxx xxx 19% /fs1 /dev/sda1 xxx xxx xxx 26% /boot tmpfs xxx 0 xxx 0% /run/user/1417441623 tmpfs xxx 0 xxx 0% /run/user/995 ` Alternatively, I see a -T local flag in the AIX df command that takes care of it nicely; I'm not familiar enough with the Linux version though (maybe -x). ... View more

This Reddit post has this from user ItsJohnLocke: j.ho answered this over in the Slack channel with the following: "Looks like it was introduced to fix SPL-72576: Results differ between stats/chart/timechart and tstats when grouping by a multivalued field with duplicate values per event" I got the same answer from a Splunker that I talked to: apparently this was a bug with datamodels in a previous version of Splunk, and people generally keep this in their searches even though they're not sure if it's still needed anymore. I usually see dedup_splitvals=t in tstats searches that have prestats=true - something like: "| tstats values(field) as field from datamodel dm prestats=true | stats dedup_splitvals=t etc . If you remove "prestats=true" then the data shows up in stats form, so ... do you really need the subsequent stats command? ... View more

Oh, that's just brilliant! I know there's documentation and dashboard examples that probably cover this sort of thing, but it's near impossible to find time for that. Coming across these posts is super helpful. ... View more

Minor typo fix for Option 1 - for the addition to the search queries so that the search only triggers when the token is set. Instead of this | eval dummy="$tokShowPanelA$" | fields - "$tokShowPanelA$" It should be this (i.e. the field to hide is dummy). Otherwise the dummy field does show up when the search is run: | eval dummy="$tokShowPanelA$" | fields - "dummy" ... View more

Both the Universal Forwarder and HEC use SSL to encrypt the in-flight data (you should replace the default certs). It sounds like you want an authentication method - like the HEC tokens - for the UF. I've never used it myself, but it looks like the UF also has authentication tokens that you can use. Here's the link to that section of the documentation. It looks very similar to HEC (but there's apparently no GUI method to configure it). ... View more

I stumbled across this post when having the same challenge. This stats-based command was really helpful - but just an FYI to future readers that neither of these searches will work if you're at 0% free space. It's not something you can fix by changing the search - it's just the inherent problem of trying to use freeMB and freePerc to figure out used or total. It's possible that hitting 0.000...% free space is super rare - but this is still annoying to me. The problem is simply that perfmon doesn't collect used or total metrics. In my case I think I'll end up using WMI for this. ... View more

Incidentally (and I'm not sure if this is a symptom of the same issue), I've actually had the search results show the same events multiple times (as though they were different events). As before, explicit sorting fixes this. ... View more

Thanks for this - multikv was another item I was interested in since it apparently reduces the amount of data indexed (which would be worth the memory cost on the Windows servers). ... View more

Thanks - I wasn't sure if it was something I had screwed up. I'll poke them with a support case and see what happens. ... View more

The SharePoint message is coming from the event logs as well - from the Application log as opposed to the Security log. So it's all event log messages. It seems like the app assumes that an event ID number is unique to one message across the event viewer. If that's the thinking, then it's definitely wrong - as this example shows. ... View more

I had the same problem. In my case I started with DB Connect v3. I added drivers and pushed it to my search cluster, but I didn't do the config until a while later. When I tried I got the same error, and also nothing in the logs. In my case, I downloaded and installed the Java SDK and then specified the path in the DBX config. But it didn't work - until I restarted Splunk. Perhaps the environment settings for the splunk process hadn't picked up ... some necessary parameter for the Java SDK. Restarting splunk started a new process with a fresh environment, so .... I dunno. But it works now. ... View more

Yes - that's perfect! It's a little disconcerting actually - to get exactly what I need as soon as I need it. I'll be looking forward to the new release (I may just pull from github if I have time). This was one of the few pieces for which I was considering installing the Splunk app for Unix and Linux. ... View more