Hi
Whenever i am adding inputs in the Website Monitoring app it fails (http 401 response). I figured out that it was, by default, taking Authentication type as Basic.
Is there any way we switch to Windows authentication?
Regards
Shrirang
I built the app with the ability auto-detect the type of authentication required and it has support for NTLM. However, the auto-detection doesn't work with all websites since not all provide hints as to the type of authentication required. I think the solution is to add support for manually telling the input what type of authentication to use.
I opened http://lukemurphey.net/issues/1545 to look into this.
Hi Luke
I am also getting same error pasted by Shrirang_Ambekar
"File "E:\APPS\Splunk\Python-2.7\Lib\hashlib.py", line 97, in _get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md4"
Status is shown as "Connection Failed" So it not working yet.
Any advice ?
Thanks
KKN
To provide some back ground, what i have in my lab is Windows 2008 R2 server
I finally got this one fixed. Its slated for release in version 2.0.1.
Hi @LukeMurphey,
Do you know when version 2.0.1 will be ready?
Are there any plans for https authentication?
Thanks for a great app
I just pushed the update: 2.0.1 is now available.
BTW: if by HTTPS authentication you mean, client-side SSL authentication, then this is already supported. The inputs.conf allows you to specify the path to the certificate and key per the client_certificate and client_certificate_key fields. See the inputs.conf.spec for more details.
I think you are on to something. I just tried it on Windows with Splunk 6.5 and MD4 didn't work. It did work on *nix though.
Do you happen to know if this website requires kerberos authentication or NTLM authentication? Website Monitoring doesn't support Kerberos yet.
I built the app with the ability auto-detect the type of authentication required and it has support for NTLM. However, the auto-detection doesn't work with all websites since not all provide hints as to the type of authentication required. I think the solution is to add support for manually telling the input what type of authentication to use.
I opened http://lukemurphey.net/issues/1545 to look into this.
I downloaded the latest version and I am still getting a 401 error trying to hit a page with windows authentication. Is there an example of how the data input in the inputs.conf should look?
Hi Luke
Thanks for your effort. I would surely test it and let you know
Regards
Shrirang
Hi Luke
Thanks it seems it worked . pasting some portion of the log
2016-10-27 15:38:36,877 DEBUG Discovered auth_type=ntlm
2016-10-27 15:38:37,105 ERROR A general exception was thrown when executing a web request
Traceback (most recent call last):
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\web_ping.py", line 355, in ping
http = requests.get(url.geturl(), proxies=proxies, timeout=timeout, cert=cert, verify=False, auth=auth, headers=headers)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests\api.py", line 65, in get
return request('get', url, **kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests\api.py", line 49, in request
response = session.request(method=method, url=url, **kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests\sessions.py", line 461, in request
resp = self.send(prep, **send_kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests\sessions.py", line 579, in send
r = dispatch_hook('response', hooks, r, **kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests\hooks.py", line 41, in dispatch_hook
_hook_data = hook(hook_data, **kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests_ntlm\requests_ntlm.py", line 119, in response_hook
'Authorization', r, kwargs)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\website_monitoring_app\requests_ntlm\requests_ntlm.py", line 102, in retry_using_http_NTLM_auth
NegotiateFlags
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\ntlm3\ntlm.py", line 249, in create_NTLM_AUTHENTICATE_MESSAGE
NtChallengeResponse = calc_resp(create_NT_hashed_password_v1(password), nonce)
File "E:\APPS\Splunk\etc\apps\website_monitoring\bin\ntlm3\ntlm.py", line 403, in create_NT_hashed_password_v1
digest = hashlib.new('md4', passwd.encode('utf-16le')).digest()
File "E:\APPS\Splunk\Python-2.7\Lib\hashlib.py", line 116, in __py_new
return __get_builtin_constructor(name)(string)
File "E:\APPS\Splunk\Python-2.7\Lib\hashlib.py", line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
Few questions though
i am getting the availability of the site but Maximum response time is shown 0.
Maximum response time only shows up when you get an error from the website - is this normal .
Also the HTTP response code for the successful site appears as blank in the event
Regards
Shrirang
Sorry, I missed your response (didn't get the email for some reason).
The script is complaining that the MD4 hash algorithm is not available. Could you let me know what version of Splunk you are running (and what platform)?
Second, could you try running Splunk's python this and telling me what happens?
bin/splunk cmd python -c "import hashlib;print hashlib.new('md4', 'test').hexdigest()"
That should return "db346d691d7acc4dc2625db19f9e3f52" if MD4 is supported.
For reference, I'm investigating this in http://lukemurphey.net/issues/1558 too.
It turns out that the server authentication auto-detection wasn't working properly. I put a fix in the 2.0 version of Website Monitoring that is about to release.
Could you test this build and see if it works? This is a little tricky for me to test because I don't have a web-server around that uses NTLM authentication. I tried to make a setup for testing (which now passes after I fixed the auto detection) but I can't be 100% sure if this fixes it.
Hello Luke,
I downloaded version 2.0 couple of days back. One URL worked fine for me, but other did not work. I believe something is going on around auth type. Please look at the below log info. ( Not sure if there is way to enforce auth type & if so where)
Appreciate your help & and thanks for this app.
2016-12-14 11:04:13,283 INFO Added thread to the queue for stanza=web_ping://e3-idn-base, thread_count=1
2016-12-14 11:04:19,566 INFO Performing ping, url="MY https URL"
2016-12-14 11:04:49,608 ERROR Unable to determine authentication type
Traceback (most recent call last):
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/web_ping.py", line 169, in determine_auth_type
http = requests.get(url.geturl(), proxies=proxies, timeout=timeout, cert=cert, verify=False)
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/website_monitoring_app/requests/api.py", line 65, in get
return request('get', url, kwargs)
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/website_monitoring_app/requests/api.py", line 49, in request
response = session.request(method=method, url=url, **kwargs)
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/website_monitoring_app/requests/sessions.py", line 461, in request
resp = self.send(prep, **send_kwargs)
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/website_monitoring_app/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/data/splunk_imdc_w/etc/apps/website_monitoring/bin/website_monitoring_app/requests/adapters.py", line 419, in send
raise ConnectTimeout(e, request=request)
ConnectTimeout: HTTPSConnectionPool(host='MY URL*', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(, 'Connection to **MY URL* timed out. (connect timeout=30)'))
2016-12-14 11:04:49,611 INFO Authentication type could not be automatically discovered; auth_type=basic
2016-12-14 11:05:23,569 INFO Removed inactive threads, thread_count=0, removed_thread_count=1
Luke - Any update on this please? I am getting time_out = true for some of my URLs. Appreciate your help.
I have a build with a fix but I don't have a good NTLM website to test it on. I have a test server setup but I don't know if this adequately reproduces a real NTLM server.
Would you be willing to test it out? You can download it here: http://lukemurphey.net/attachments/download/400/website_monitoring.tar.gz
Sure. Let me try it out. Will get back.
@sdaniels yes true but haven't seen any resolution hence posted the question again . I referred to your question only the first time I encountered 401.
Similar thread here that might help. https://answers.splunk.com/answers/334288/website-monitoring-when-monitoring-a-url-that-has.html