The basis issue is that there is enough skew between your Splunk instance time and the time of the system clock at Okta.
In the SAML assertion, there is a set of time parameters:
If the response occurs outside that time slice, the Splunk will shut things down. Some times with a more descriptive message, but definitely with the 'The conditions saml response failed validation Verify the time in the response from IDP is in UTC time format.' message as part of the resulting error.
You can try a couple of things:
- Set up an NTP date service in your Splunk Instance (search heads, indexers, cluster managers, the whole ball of wax! - all of them!).
- Capture the SAML conversation with a SAML tracer plugin within your browser (chrome, firefox, etc. all have a SAML tracer plugin - this makes it easier to see all of the XML passed between Okta and Splunk to determine what the time values are in the NotBefore and NotOnOrAfter conditions as well as the time stamps in the responses
I've not found a way to add additional time buffer for Okta. In other IdPs (such as ADFS for instance), you can 'tune' the amount of time between the before/after conditions to make it a bit more tolerant of delays between Okta (IdP) and Splunk.
