All Apps and Add-ons

How to obtain data through the FreshDesk API in REST API Modular Input?

gonzaloAgui
Explorer

Hi,
I need to obtain data from FreshDesk through the FreshDesk API using a url param named "updated_since" (this is a datetime string with format "%Y-%m-%dT%H:%M:%SZ") like a checkpoint from the last ticket indexed. And adding 1 second to the checkpoint because otherwise the last ticket is doubled.
The problem is (I THINK) that the URL Arguments passed from the Response Handler to the rest.py change the format to this:

URL Arguments:
updated_since=1970-10-10 10:10:10

And this generate a Bad request to the API:
03-26-2018 16:27:32.652 -0300 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 400 Client Error: Bad Request

My response handler script:

class FreshDeskResponseHandler:
    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):

        if response_type == "json":
            output = json.loads(raw_response_output)
            datetime_format = '%Y-%m-%d %H:%M:%S'
            last_update_at = "1970-10-10 10:10:10"
            for freshdesk_event in output:
                print_xml_stream(json.dumps(freshdesk_event))
                if "updated_at" in freshdesk_event:
                    update_at_date = (freshdesk_event["updated_at"].replace("T", " "))[:-1]
                    if update_at_date > last_update_at:
                        update_one_sec = str(datetime.datetime.strptime(update_at_date, datetime_format) + datetime.timedelta(seconds=1))
                        update_one_sec = (update_one_sec.replace(" ", "T")) + str('Z')
                        last_update_at = update_one_sec
            if not "params" in req_args:
                req_args["params"] = {}

            req_args["params"]["updated_since"] = str(last_update_at.replace(" ", "T") + str('Z'))

        else:
            print_xml_stream(raw_response_output)

My inputs .conf:

[rest://TEST - input freshdeck]
auth_password = PASSWORD
auth_type = basic
auth_user = USERNAME
endpoint = https://DOMAIN.freshdesk.com/api/v2/tickets
http_method = GET
index_error_response_codes = 0
response_type = json
sequential_mode = 0
sourcetype = json_freshdesk
streaming_request = 0
cookies = _x_w=24_5
disabled = 0
host = fresh_desk_api
index = testfresh
polling_interval = 60
response_handler_args =
response_handler = FreshDeskResponseHandler
url_args = updated_since=1970-10-10 10:10:10

Has someone encountered a similar problem or knows a workaround to this?
Thanks in advance!

0 Karma

Damien_Dallimor
Ultra Champion

Are you able to get a trace of the actual HTTP GET request sent out , maybe using wireshark or something similar to capture it.

0 Karma

gonzaloAgui
Explorer

Hi Damien, thank you for your quick response. I am trying to trace the http request but failing in the attempt. Could you guide me please? the instance of splunk is in a virtual machine Centos without graphical interface and I do not have access to the server that hosts it.
I have tried with tcpdump, netstat, lsof, iptraf without success.

0 Karma

Damien_Dallimor
Ultra Champion

Try using Splunk Stream 🙂 https://splunkbase.splunk.com/app/1809/

0 Karma

gonzaloAgui
Explorer

I'm already configure splunk stream, but I still can't see the URL with the parameters or the body of the package.
I see data from this sources:

stream:Splunk_DNSIntegrity
stream:Splunk_DNSRequestResponse

stream:Splunk_SSLActivity

stream:Splunk_HTTPURI

stream:http

Is there something I'm missing? let me know if you need a sample of the events.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...