Sorry for the long post but I need some help 😄
After watching the Splunking Jenkins presentation, we started setting the Splunk App for Jenkins with half success.
We have a Splunk cloud instance, so Splunk Support installed the Splunk App for Jenkins for us. After configuring everything the best we could, in the app, some sections have no results, some sections do.
What works:
In all pages, I can select the "host" in the "Jenkins Master" box.
Overview > Master/Slave Topology works!
Jenkins Health works!
Jenkins Slave works!
What doesn't work:
Overview > Jenkins Build Status History and Latest Builds shows "No results found"
Build Analysis > I can only select the "host" in the "Jenkins Master" box. Every other box, I can only select "*", again, it says "No results found"
Audit Trails shows "No results found" in both sections
Somewhat works:
Test Results > I can see results, but they are not correct. (For some builds that are unstable (test failed), it only shows the passing tests, but not the failing ones. I am using python's nosetests which outputs a xunit format report which Jenkins publishes)
Here is our configuration...
1. Collector and token setup by support
Support gave us the event collector URL and a token. We tested the token against the indexes:
curl -k [collectorURL] -H 'Authorization: Splunk [token]' -d '{"event":"test message", "index":"jenkins"}'
curl -k [collectorURL] -H 'Authorization: Splunk [token]' -d '{"event":"test message", "index":"jenkins_console"}'
curl -k [collectorURL] -H 'Authorization: Splunk [token]' -d '{"event":"test message", "index":"jenkins_artifact"}'
curl -k [collectorURL] -H 'Authorization: Splunk [token]' -d '{"event":"test message", "index":"jenkins_statistics"}'
In splunk search, we run: index=jenkins* "test message" |table _raw, index
All good at this point, we got all the 4 "test message".
(Note: when contacting support, they created a HEC and asked for the index and sourcetype for it. We said index=debug, sourcetype=xml. I don't know if this is relevant, but the information was not clear in the Jenkins plugin documentation about what to say to support, now it seems as if we don't specify the index in the metadata, the default goes to "debug" and we get a funny error if metadata is simply host=mymasterhost)
2. Jenkins configuration
We entered our splunk host, port=443, the token given by support, and our Jenkins MetaData is:
sourcetype=_json
source=jenkins
host=masterhostname
#custom index, uncomment the settings after index created in Splunk
index=jenkins_artifact
build_report.index=jenkins
file.index=jenkins_artifact
console_log.index=jenkins_console
queue_info.index=jenkins_statistics
slave_info.index=jenkins_statistics
jenkins_config.enabled=true
jenkins_config.monitoring=true
3. Jobs
We did not change our jobs.
Am I missing anything? I am also confused why Test Results doesn't show me the right results (fail tests doesn't show up). Funny enough, sometimes it does show me failed tests, so it's not consistent, I haven't figured out a pattern yet... if someone can help me with this as well...
... View more