Add-on for JIRA 2.2.1 and 2.2.0 not parsing JIRA fields correctly

[worshamn]

I have version 2.1.0 working fine but it is missing many fields. When I try to install 2.2.0 or 2.2.1 the output is much different. Many new JIRA fields appear, most with the literal value of "null". Some fields that were previously working, like "Project," are now filled with what looks like JSON that includes information that does not seem helpful (like "avatarUrls"), with the "Project" data buried in it. The instructions for what needs to be the jira.conf file are a bit confusing and I have tried making changes to it with no success.

I see the app is now published by Splunk but not supported by Splunk which means this question may really go unanswered. The original GitHub repo had this exact problem listed as an issue https://github.com/firebus/splunk-jira/issues/33

If this is a config setting, it is not obvious what the solution is, so I ask here if anyone knows how to fix this or if anyone else is experiencing this?

A further complication is that the jira command has an entirely different output than the indexing option this app provides. The the indexing output is almost preferable because it is in straight JSON, it does not convert the custom field names like the command does.

[worshamn]

I suppose this will be one of the times I get to answer my own question at least until someone either updates the app or incorporates the following changes into it (I would have just done a pull request but there is no current repo on GitHub and again not sure who is supporting this). Either way I can now move on with my obsessive compulsive life.

Note: I chose to deal with SLA fields by creating 4 subfields instead (relates to https://answers.splunk.com/answers/302052/how-to-get-sla-values-of-jira-in-splunk-events-aft.html) and if you want to see the previous way it was outputting, just add debug=True to the parse_value function call parse_value(v, field, row, debug=True) on the new line 261.

I made the following changes to the Add-on for JIRA 2.2.1, specifically in $SPLUNK_HOME/etc/apps/jira/bin/jira_rest.py (I don't know if there are issues with any of the other commands it offers I was only concerned with the | jira jqlsearch "<query>" command. Maybe there is a way to get JIRA's API to respond more consistently but I haven't found a way, so I essentially had to make rules for all the various ways the fields come back and grab what was deemed important and remove what was not.

Add library import

import ast

Original lines 256-259:

            elif isinstance(v, basestring) == True:
                row[field] = v
            else:
                row[field] = json.dumps(v)

Change to (will now start on line 257):

            elif isinstance(v, basestring) == True:
                if '{}' not in v:
                    row[field] = v
            elif isinstance(v, (int, long, float, complex)) == True:
                row[field] = v
            else:
                parse_value(v, field, row)

Add the following 2 functions into the code:

def parse_value(v, field, row, debug=False):
    outputSelector = [
{'value':'completedCycles'},
{'value':'outwardIssue', 'output': lambda vDict: vDict['outwardIssue']['key']},
{'value':'key', 'output': lambda vDict: vDict['key']},
{'value':'name', 'output': lambda vDict: vDict['name']},
{'value':'value', 'output': lambda vDict: vDict['value']},
{'value':'watchCount', 'output': lambda vDict: vDict['watchCount']},
{'value':'progress', 'output': lambda vDict: vDict['progress']},
{'value':'requestType', 'output': lambda vDict: vDict['requestType']['name']}
                     ]
    if debug == True:
        row[field] = json.dumps(v)
    elif v != None:
        if isinstance(v, dict) == True:
            vDict = ast.literal_eval(str(v))
            if vDict:
                if outputSelector:
                    for i in outputSelector:
                        if i['value'] in vDict:
                            if i['value'] == 'completedCycles':
                                #for SLA incomplete fields
                                if 'ongoingCycle' in vDict:
                                    slaFieldCreate(vDict['ongoingCycle'], row, field)
                                    break
                                #for SLA complete fields
                                else:
                                    if isinstance(vDict['completedCycles'], collections.Iterable) == True:
                                        fList = ast.literal_eval(str(vDict['completedCycles']))
                                        if fList:
                                            for i in fList:
                                                if isinstance(i, dict) == True:
                                                    if i:
                                                        slaFieldCreate(i, row, field)
                                            break
                                    break
                            else:
                                row[field] = i['output'](vDict)
                                break
                elif field.lower() in vDict:
                    row[field] = vDict[field.lower()]
                else:
                    row[field] = vDict
        elif isinstance(v, collections.Iterable) == True:
            vList = ast.literal_eval(str(v))
            if vList:
                    outputlist = []
                    for i in vList:
                        if isinstance(i, dict) == True:
                            iDict = ast.literal_eval(str(i))
                            if iDict:
                                for item in outputSelector:
                                    if item['value'] in iDict:
                                        outputlist.append(item['output'](iDict))
                                        break
                        #for Sprint names
                        elif 'name=' in i:
                            match = re.search('name=([^,]+)', i)
                            outputlist.append(match.group(1))
                        else:
                            outputlist.append(i)
                    row[field] = outputlist

def slaFieldCreate(value, row, field):
    slaValues = ['elapsedTime','remainingTime','goalDuration','breached']
    for slaType in slaValues:
        if slaType in value:
           if slaType == 'breached':
               row[field + " " + slaType] = value[slaType]
           else:
               row[field + " " + slaType] = value[slaType]['millis']

[worshamn]

updated it to add the breached SLA value