All Apps and Add-ons

Why does Website Monitoring not work per the schedule when we have connection timeout URLS?

allurirohan
Explorer

We are facing the issue when Website Monitoring app has connection timeout URLS in configuration and looks like its blocking the other monitors to run as per schedule.

Scenario: If i have 100 URL's running every 5 mins which are giving some status code (200,401,404 etc), everything working perfectly as per the schedule
But, when I append a couple of URL's for the same inputs file which are giving timeout error status, then its blocking all the other monitors to run as per the schedule and 5 mins schedule is messing up and running randomly every 10mins, 18mins, etc....

Is there a workaround for this scenario ?

Tags (1)
0 Karma
1 Solution

LukeMurphey
Champion

This is due to the fact that the Website Monitoring input currently runs all of the inputs under a single instance. I have a couple of options in mind that would prevent this sort of issue. I am investigating further in http://lukemurphey.com/issues/1524.

Update:
I have a version of Website Monitoring with inputs re-written to use multi-threading. It isn't released yet as I am still doing testing but I think I will likely have a new version available within a few days. I was able to reproduce the issue you observed but the new version has no problem executing all of the inputs while still keeping up.

Update [2]:
Version 2.0 is up on Splunk-base. I don't have it set as the default yet so you will have to manually select it from the version drop-down. I have it deployed in production and will make it the default version once I run it for a few more days.

View solution in original post

LukeMurphey
Champion

This is due to the fact that the Website Monitoring input currently runs all of the inputs under a single instance. I have a couple of options in mind that would prevent this sort of issue. I am investigating further in http://lukemurphey.com/issues/1524.

Update:
I have a version of Website Monitoring with inputs re-written to use multi-threading. It isn't released yet as I am still doing testing but I think I will likely have a new version available within a few days. I was able to reproduce the issue you observed but the new version has no problem executing all of the inputs while still keeping up.

Update [2]:
Version 2.0 is up on Splunk-base. I don't have it set as the default yet so you will have to manually select it from the version drop-down. I have it deployed in production and will make it the default version once I run it for a few more days.

allurirohan
Explorer

Ok.Thanks Luke, when can we expect v 2.0 ? 🙂

0 Karma

LukeMurphey
Champion

Haven't figured it out yet. I just made that version based on this Answers post. This issue is the most significant change slated for 2.0 by far.

I'll let you know once I figure out how I'm going to handle this.

0 Karma

allurirohan
Explorer

HI Luke, we just saw the issue http://lukemurphey.com/issues/1524 is resolved.But we dont see software v2.0 in downloads..can you please check and upload the v2.0 file

0 Karma

LukeMurphey
Champion

Its on Splunk-base: https://splunkbase.splunk.com/app/1493/

Look for the dropdown at the right-side of the release notes and select 2.0 and then click the download button.

0 Karma

allurirohan
Explorer

Thanks Luke, looks like its an issue with IE,firefox. Tried it in chrome and able to download and test.It is working as per the schedule now , even with high volume>

one observation found was , for the connection failed URL's its showing blank in response_code. In previous version it used to show connection failed but its blank now. Any idea ?

0 Karma

LukeMurphey
Champion

Where is it showing up blank? I have seen that issue on the Status Overview page but I thought I had fixed it there.

0 Karma

allurirohan
Explorer

It is showing on the status overview page. Where I am getting 400,404,200 for others but the failed ones are showing blank in respose_code column.

0 Karma

LukeMurphey
Champion

ok, I'll test it tonight. I know I saw this issue on my Windows test box but I thought sure I fixed it. Something must have slipped through.

0 Karma

LukeMurphey
Champion

It turns out I had fixed this but in a later build than the one I had uploaded to Splunk-base. I uploaded the build on Splunk-base; that issue should be fixed now.

0 Karma

allurirohan
Explorer

ok..Thanks Luke

0 Karma

LukeMurphey
Champion

If I understand correctly, your seeing inputs getting hung up on URLs that have timeout (since the other inputs are waiting on the input to timeout before they can execute).

Do I have that right?

0 Karma

allurirohan
Explorer

Correct.I dont have any issue when i have all URL's which are result in some status code. Is there a work around?

0 Karma

LukeMurphey
Champion

I don't have a workaround yet. I'm looking into a change in the backend that would prevent the inputs from hanging one another up.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...