Why is scheduled report pdf email not received if it takes more than one hour?

[safe_splunk]

Hi,

I prepared a dashboard and a scheduled pdf delivery, but when it takes more than an hour, I receive no email at all. When it takes less than 1 hour, I receive pdf report properly.

What should I do to receive report if it takes more than an hour?

Thanks

[ronogle]

It looks like there is a bug in the Splunk sendemail and rest simplerequest code. I had the situation where most of my reports would generate the PDFs correctly for emails. However, I had a few alerts and 1 report that kept giving me a time out issue.

In the /etc/apps/search/bin/sendemail.py script as mentioned by mzorzi, there is a default PDF timeout value that is defined. Later in the sendemail.py code there is a rest remplerequest call that sends the timeout to that set of scripts.

Those scripts are located in $SPLUNK_HOME/lib/python2.7/site-packages/splunk/rest. The initialization script, init.py, has a default timeout value called SPLUNKD_CONNECTION_TIMEOUT. You can see where the simplerequest gets called with the timeout coming in from the sendemail program.

I think what happens is the SplunkD timeout is occurring before the rest timeout happens. Therefore, you have to update the splunkd connection timeout value because it will time out before the other time out happens.

In my opinion, the timeout sent in from the calling program should override the default values. The other problem is that to fix this, I had to change the default value in the python directory. Next time we upgrade, this will get over-written.

[mzorzi · Accepted solution]

This is the timeout that needs to increase: from the file

./etc/apps/search/bin/sendemail.py

PDF_REPORT_SERVER_TIMEOUT = 600

PDFGEN_SIMPLE_REQUEST_TIMEOUT = 3600

As you can see, the second parameter is used further down in the simpleRequest call, so it needs to be increased:

response, content = simpleRequest("pdfgen/render", sessionKey = sessionKey, getargs = parameters, timeout = PDFGEN_SIMPLE_REQUEST_TIMEOUT)

[adminsecdsi]

hi i have the same probleme on my Splunk Version ............................................6.3.3
i want generate a globaly report than make more than 5 hours to complete

i check
PDF_REPORT_SERVER_TIMEOUT = 600
PDFGEN_SIMPLE_REQUEST_TIMEOUT = 3600 is put on sendemail.py

but always i have time out on sendmail and receice

what can i do for my sendemail wait at max 24 hours before crash when sending a dashboard

Cordialy
romuald

[safe_splunk]

Exactly as you mentioned solved all problems.
So Anyone that has report searches that take more than an hour with pdf scheduled delivery must raise these intervals to receive email report. Otherwise it won't work.

Thanks Matteo.

[ankireddy007]

Hi,

Please change default settings in "pdf_server.conf" in $SPLUNK_HOME/etc/system/default.
Note: Please do any changes locally by copying into $SPLUNK_HOME/etc/system/local

tools.sessions.timeout = <integer>
   * Specifies the number of minutes of inactivity before a user session expires.
   * Defaults to 60

response.timeout = <integer>
   * Specifies the number of seconds to wait for the server to complete a response.
   * Some requests such as uploading large files can take a long time.
   * Defaults to 7200

[safe_splunk]

Also when i manually execute I got this error.

Script execution failed for external search command 'sendemail'

But i have other reports takes less then 1 hour to finish they all work and i receive pdf emails.
But i have problem on only on this report

[safe_splunk]

No it did not work. I did exactly as you mentioned.

This is inspect OUTPUT below. some ip/mail/address i change.

None | sendemail "server=x" "use_ssl=0" "use_tls=0" "to=test@gmail.com," "cc=" "bcc=" "from=splunk@warn.x.com" "subject=Splunk Report: _ScheduledView__search_engine_bots" "format=html" "sssummary=Saved Search [_ScheduledView__search_engine_bots]: always(0)" "sslink=search/@go?sid=" "ssquery=| noop" "ssname=_ScheduledView__search_engine_bots" "inline=0" "sendresults=1" "sendpdf=1" "pdfview=search_engine_bots" "searchid=" "width_sort_columns=1" "graceful=True" maxinputs="10000" maxtime="60m" "papersize=letter" "paperorientation=landscape"

When i look jobs searches takes 1 hour and 15 minutes and they go till finish but scheduled delivery pdf stops exact as 1:00 hour.

over the time range:

1/1/70 2:00:01.000 AM – 1/1/70 2:00:02.000 AM
did not return any data. Possible solutions are to:

relax the primary search criteria
widen the time range of the search
check that the default search indexes for your account include the desired indexes
This search is an instance of the saved search: ScheduledView_search_engine_bots.

The following messages were returned by the search subsystem:

DEBUG: The 'sendemail' command is implemented as an external script and may cause the search to be significantly slower.
DEBUG: search context: user="admin", app="search", bs-pathname="/opt/splunk/etc"
ERROR: Script execution failed for external search command 'sendemail'
Learn more about troubleshooting empty search results at Splunk Documentation.

(SID: scheduler_adminsearch_RMD5f3b1710c906edb28_at_1405687200_30)