Dashboards & Visualizations

Why does the Pie chart in dashboard in scheduled PDF gives wrong results but works okay in dashboard or when test email is sent?

davidwaugh
Path Finder

Hi I have a strange problem.

I have a dashboard with a several pie charts. When I view the dashboards in Splunk they are correct. When I send a test email from the scheduled dashboard it works to.

However when I schedule the pdf to be sent, then one of the pie charts shows incorrectly, even though a similar search based on a corresponding search does produce the correct results.

The search that generates the Pie Chart is:

| inputlookup SwiftCompliancePolicy 
| table Host,Policy |join Host type=left [ |inputlookup SwiftDeviceIPHostname |table NewServerName,InsideIP,MgmtIP |rename NewServerName as Host |eval Host=lower(Host)] |search Policy=linux* |eval device_ip=coalesce(MgmtIP,InsideIP) |join device_ip type=left [search index=linux* |rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" |stats latest(_time) by device_ip] |join Host type=left [|inputlookup SwiftCompliance|table hostname,service |rename hostname AS lowershost|eval Host=lower(lowershost)] |search service !=Remove* |search service != "*A1c" |search service !=*2HOP |search Host!= ace* |rename latest(_time) as Last_Time|eval DateDiff=(now()-Last_Time)/(24*3600) |eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") |chart count by Compliant|append [| stats count | eval Compliant=split("Compliant,Non Compliant",",") | mvexpand Compliant] | stats sum(count) as count by Compliant |eval Compliant=Compliant." : ".count

The Search that generates the corresponding table is: (Shows "non compliant")

 | inputlookup SwiftCompliancePolicy 
 | table Host,Policy |join Host type=left [ |inputlookup SwiftDeviceIPHostname |table NewServerName,InsideIP,MgmtIP |rename NewServerName as Host |eval Host=lower(Host)] |search Policy=linux* |eval device_ip=coalesce(MgmtIP,InsideIP) |join device_ip type=left [|search index=linux* |rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" |stats latest(_time) by device_ip] |join Host type=left [|inputlookup SwiftCompliance|table hostname,service |rename hostname AS lowershost|eval Host=lower(lowershost)] |search service !=Remove* |search service != "*A1c" |search service !=*2HOP |search Host!= ace* |rename latest(_time) as Last_Time|eval DateDiff=(now()-Last_Time)/(24*3600) |eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") |search Compliant="Non Compliant" |table Host,Compliant

Here is the Pie chart on the dashboard:

alt text

But here is the Pie Chart when sent as a pdf.

alt text

Running We are running Splunk on Windows.
Splunk Enterprise
Version:
7.2.1
Build:
be11b2c46e23

Any ideas how to troubleshoot this?

0 Karma
1 Solution

davidwaugh
Path Finder

Hello, I managed to solve this by changed my dashboard panels to scheduled reports. These then ran in the background and the results were then immediately ready for the PDF to be generated. It didn't have to wait for the searches to finish and then generate the report.

Thanks for all the help.

View solution in original post

davidwaugh
Path Finder

Hello, I managed to solve this by changed my dashboard panels to scheduled reports. These then ran in the background and the results were then immediately ready for the PDF to be generated. It didn't have to wait for the searches to finish and then generate the report.

Thanks for all the help.

davidwaugh
Path Finder

When I scheduled the report to run hourly, I got an error in the python.log

https://answers.splunk.com/answers/511942/error-in-pdf-rendering.html

0 Karma

cmerriman
Super Champion

try something like this for the pie chart:

index=linux* | rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" | stats latest(_time) as Last_Time by device_ip
|join device_ip type=left 
    [| inputlookup SwiftDeviceIPHostname |lookup SwiftCompliance hostname AS NewServerName OUTPUT service
    | table NewServerName,InsideIP,MgmtIP,service | eval Host=lower(NewServerName) | eval device_ip=coalesce(MgmtIP,InsideIP)|lookup SwiftCompliancePolicy Host OUTPUT Policy| search Policy=linux*  service !=Remove* service != "*A1c" service !=*2HOP Host!= ace* ] 
| eval DateDiff=(now()-Last_Time)/(24*3600) 
| eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") 
| chart count by Compliant 
| eval Compliant=Compliant." : ".count

and see what you get. I think one issue you might be facing is the search time. I know I used to have issues with searches that used too much memory or too much time that the PDFs wouldn't render properly. If this works, use similar logic to re-write the table SPL. Use the job inspector to check the search optimization. Let me know if there are any questions or if it isn't working, it'll be helpful to have some sample data to work with.

0 Karma

davidwaugh
Path Finder

Thanks. One problem with the reworked query is that if there are no events, then there are no matches in the join. I still want to see a row in the table even if there are no events.

0 Karma

Richfez
SplunkTrust
SplunkTrust

I think the inestimable @cmerriman has a good theory on why it may not be working for the scheduled version, though it works for "regular use." Could you test that her search at least does indeed schedule OK?

If it does, then we can continue tweaking some things. Like this for speed...

index=linux* 
| rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" 
| stats latest(_time) as Last_Time by device_ip
| lookup SwiftDeviceIPHostname device_ip OUTPUT NewServerName MgmtIP InsideIP
| lookup SwiftCompliance hostname AS NewServerName OUTPUT service
| eval Host=lower(NewServerName) 
| eval device_ip=coalesce(MgmtIP,InsideIP) 
| lookup SwiftCompliancePolicy Host OUTPUT Policy
| search Policy="linux*" service!="Remove*" service!="*A1c" service!="*2HOP" Host!="ace*" ] 
| eval DateDiff=(now()-Last_Time)/(24*3600) 
| eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") 
| chart count by Compliant 
| eval Compliant=Compliant." : ".count

And then we can flip it back around for including all rows. But, which rows? Like, what's your one source of "all the things?"

But we need to know this is resolving at least the problem of not being able to schedule it?

0 Karma

cmerriman
Super Champion

agree with the ever-brilliant @rich7177 that we should at least test that the PDF is generating properly or not.

if you are looking to add a line when no data is present, you can add something like this to the end (tweak as needed)

|appendpipe [stats count|where count=0|eval Compliant="No Data"]
0 Karma

niketn
Legend

@davidwaugh try the Smart PDF Exporter app from Splunkbase

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

davidwaugh
Path Finder

Here is my code:

| inputlookup SwiftCompliancePolicy 
| table Host,Policy |join Host type=left [ |inputlookup SwiftDeviceIPHostname |table NewServerName,InsideIP,MgmtIP |rename NewServerName as Host |eval Host=lower(Host)] |search Policy=linux* |eval device_ip=coalesce(MgmtIP,InsideIP) |join device_ip type=left [search index=linux* |rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" |stats latest(_time) by device_ip] |join Host type=left [|inputlookup SwiftCompliance|table hostname,service |rename hostname AS lowershost|eval Host=lower(lowershost)] |search service !=Remove* |search service != "*A1c" |search service !=*2HOP |search Host!= ace* |rename latest(_time) as Last_Time|eval DateDiff=(now()-Last_Time)/(24*3600) |eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") |chart count by Compliant|append [| stats count | eval Compliant=split("Compliant,Non Compliant",",") | mvexpand Compliant] | stats sum(count) as count by Compliant |eval Compliant=Compliant." : ".count

And to generate corresponding table of non compliant systems

| inputlookup SwiftCompliancePolicy 
| table Host,Policy |join Host type=left [ |inputlookup SwiftDeviceIPHostname |table NewServerName,InsideIP,MgmtIP |rename NewServerName as Host |eval Host=lower(Host)] |search Policy=linux* |eval device_ip=coalesce(MgmtIP,InsideIP) |join device_ip type=left [|search index=linux* |rex field=source ".*\/(?<device_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/.*" |stats latest(_time) by device_ip] |join Host type=left [|inputlookup SwiftCompliance|table hostname,service |rename hostname AS lowershost|eval Host=lower(lowershost)] |search service !=Remove* |search service != "*A1c" |search service !=*2HOP |search Host!= ace* |rename latest(_time) as Last_Time|eval DateDiff=(now()-Last_Time)/(24*3600) |eval Compliant = if(DateDiff <=1,"Compliant","Non Compliant") |search Compliant="Non Compliant" |table Host,Compliant
0 Karma

Richfez
SplunkTrust
SplunkTrust

EDIT: I reformatted your SPL to use the 'code' button (the little 1010 button in the toolbar) so the special characters come through properly. That worked for some of them, but others may have already been eaten by the editor - could you edit your post and re-paste the searches?

EDIT 2: Thanks, those look better (the extractions were messed up in the regex). You added them as an Answer - be careful doing that because once answered, a lot of people won't even look at the question. But no worries! I fixed that, converted it to to a comment, then copied and pasted your corrected code back into the original question! All good now.

0 Karma

davidwaugh
Path Finder

Thanks very much. Sorry I am new to Splunk so still very much learning!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...