I've installed and configured both the Splunk Plugin and Splunk Plugin Extension on my Jenkins instance, and our devops guy installed and configured the Jenkins plugin for our Splunk instance. I can see my Jenkins jobs appear on the "Splunk App for Jenkins" dashboard, where it indicates success/fail of a job. However, when I select a specific job and then on the left hand side select "Logs and Artifacts" I would expect to be able to view the console logs, however I just see a line of text saying "Click Here Choose Console Output or Build Artifacts" where "Click Here" looks like a link but doesn't do anything.
That particular behavior looks buggy, but it wouldn't surprise me in the least if our configuration is simply not correct, or if I'm looking in the wrong place. As an extra step, I searched for a hash that appeared in the console logs and that returned 0 results. In the Jenkins logs I see nothing in between consecutive runs of my job. I've tried with and without the sendSplunkConsoleLog call in my Jenkinsfile, and observe no difference.
Can someone point me in the right direction to get our console logs into Splunk, or begin a debugging process to move in that direction?
If your job type is Pipeline, you need to explicitly declare what you want to send as console output inside your groovy. The default console output won't be sent to Splunk unless you use non-pipeline jobs.
https://wiki.jenkins.io/display/JENKINS/Splunk+Plugin+for+Pipeline+Job+Support
Here are the two examples included on the wiki page. I have tested both scripted and non-scripted in my environment and they both work.
#scripted pipeline
sendSplunkConsoleLog {
node{
sh "echo testjob";
}
}
#declarative pipeline
pipeline {
agent any
options {
timeout(time: 1, unit: 'HOURS')
sendSplunkConsoleLog()
}
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
}
Hi,
can you please explain more about it. Like do we need to copy as it as and paste or we need to make some changes in that code. If yes then what and where?
Thanks
I am also facing this issue where the console logs are not visible in the splunk. I have a free-style type of projects in my Jenkins server. Neither do I know how to custom the metadata information. Can anyone suggest
could you get any data from index=jenkins_console sourcetype=text:jenkins?
I would try to test the token and app setup using
hec_host=<splunk-host>
hec_port=8088
hec_token=<splunk-token>
#send data manually
curl -k "https://${hec_host}:${hec_port}/services/collector/raw?host=test-host&index=jenkins_console&sourcetype=text:jenkins&source=/job/dummy-job/1" -H "Authorization: Splunk ${hec_token}" -d "hello there"
If data is searchable, I would try to check jenkins.log for plugin error message
Thanks for the reply. I ran the curl and was able to find that entry by searching in Splunk, though nothing appeared in the "Splunk App for Jenkins." It's an improvement though over what I was seeing; it's more important to have the console logs than the special dashboard.
There was nothing in jenkins.log, just one line from a month ago saying that the log had been turned over.
I'm not sure what next step to take for investigating this further.
I don't know that particular app, but the first thing I'd do is check the search behind the console, and execute it directly in the search bar. It's possible that there is a permissions issue that the console is handling for you, or that some knowledge objects needed for the search are currently limited to use within the app itself, and might need to be exported to be able to find the actual data. Alternatively, the app is pulling data from Jenkins on the fly... which is possible, since Splunk can do that from various relational databases, but it's not certain, one way or the other.
Can u explain it more, as I am facing the same issue and new to splunk. Awaiting response
I'm fairly new to Splunk, so I'm not sure how to "check the search behind the console". It may be that when I said "As an extra step, I searched for a hash that appeared in the console logs" that was what you meant by "execute it directly in the search bar" - that wasn't through the "Splunk App for Jenkins".
I appreciate the reply but I'm not sure what action(s) to take based on it.