Hello,
I have a problem with Splunk Entreprise 6.5.2 et Splunk DB Connect 3.1.3 :
Splunk DB Connect don't index data from database.
In logs, I see :
2018-05-28 14:53:51.863 +0200 [QuartzScheduler_Worker-27] INFO org.easybatch.core.job.BatchJob - Job 'testdbinput' finished with status: FAILED
2018-05-28 14:53:51.863 +0200 [QuartzScheduler_Worker-27] ERROR org.easybatch.core.job.BatchJob - Unable to write records
java.io.IOException: HTTP Error 400: Bad Request
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36
....
2018-05-28 14:53:51.863 +0200 [QuartzScheduler_Worker-27] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch
java.io.IOException: HTTP Error 400: Bad Request
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
...
2018-05-28 14:53:51.850 +0200 [QuartzScheduler_Worker-27] INFO c.s.d.s.dbinput.recordwriter.HttpEventCollector - action=writing_events_via_http_event_collector record_count=5
When I configure my input, the request is OK :
I have disabled SSL, and I put a tcpdump in the server to see request :
{"time":"1527509442,533","event":"2018-05-28 14:10:42.533, action=\"SUPPRESSION_CONTRAT\"","host":"xxxxx","source":"testdbinput","sourcetype":"defautkv_xxxxx","index":"test"}
When I test to send this data with a curl :
curl -k https://127.0.0.1:8088/services/collector/event -H "Authorization: Splunk 761bdb35-0b8c-4780-xxxx-xxxxxx" -d '{"time":"1527509442,533","event":"2018-05-28 14:10:42.533, action=\"SUPPRESSION_CONTRAT\"","host":"xxxxx","source":"testdbinput","sourcetype":"xxxxx","index":"test"}'
{"text":"Error in handling indexed fields","code":15}
For me the field time isn't correct : 1527509442,533 ==> 1527509442.533
curl -k https://127.0.0.1:8088/services/collector/event -H "Authorization: Splunk 761bdb35-0b8c-4780-xxxx-xxxxxx" -d '{"time":"1527509442.533","event":"2018-05-28 14:10:42.533, action=\"SUPPRESSION_CONTRAT\"","host":"xxxxx","source":"testdbinput","sourcetype":"xxxxx","index":"test"}'
{"text":"Success","code":0}
Is it a bug in Splunk DB Connect ?
Thank you in advance,
Cordially
Hi
Can anyone show an example of how to change the locale environment variables:
LANG=C
LC_ALL=C
Thanks for all reply
In Linux, type locale at the prompt. I'm not sure how to do it in Windows.
Thank you, I wil try to change this in windows for the user running splunk.
You have to change your locale environment variables:
LANG=C
LC_ALL=C
I had the same issue, and your suggestion worked for me. My splunk user was using "fr_FR.UTF-8",
I changed with LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8
Thank you for your help
Your suggestion worked for me too.
3.1.1 version works properly as well. But I had to completely remove the app in console first. After upgrade I see each time that task server cannot be run on port 9998 or any other free port.
How did you get version 3.1.1? I can only download version 2.4.1 or 3.1.3 on splunkbase.
Thanks.
Hi, is it possible to get a copy of the older version please?
Let me know your email I'll send you a link to the file stored in my Google drive.
At this time I have downloaded the version 2.4.1 and it's working properly but I would like to update to the latest version...
I also have the issue with the metadata field "time" is not being formated correctly. It is using a comma instead of a dot. In the documentation, under metadata, it says it should be a dot with the default settings: https://docs.splunk.com/Documentation/Splunk/7.1.1/Data/FormateventsforHTTPEventCollector
Again, if you find a workaround it would be much appriciated if you let me know. Thanks.
Relevant event from my log where you see the event being created incorrectly with a badly formated time field:
2018-05-30 15:09:48.365 +0200 [QuartzScheduler_Worker-22] DEBUG c.s.d.s.dbinput.task.processors.EventMarshaller - action=finish_format_hec_events record=Record: {header=[number=2, source="blueprism", creationDate="2018-05-30 15:09:48.365"], payload=[{"time":"1527685788,365","event":"2018-05-30 15:09:48.365, resourceid=\"9EAD88A2-725A-4806-897F-8F1C8B1022AD\", name=\"NOLB2373_debug\", status=\"Ready\", processesrunning=\"0\", actionsrunning=\"0\", unitsallocated=\"0\", lastupdated=\"2018-05-09 14:12:21.64\", AttributeID=\"4\", diagnostics=\"0\", logtoeventlog=\"1\", FQDN=\"NOLB2373.mistral.mistralnett.com\", ssl=\"0\", userID=\"6D34DB81-1665-4324-89B4-21A0B878100B\"","host":"NOLB2373\\SQLEXPRESS","source":"blueprism","sourcetype":"blue_prism","index":"resources"}]}
I have the same problem. It would be much appriciated if you could update your post if you find a solution. Thanks
Sorry, I can't edit and I want to add this information :
=> I have tested with the version 2.4.1. It is OK, Data is indexed correctly...
So it is a bug in the 3.1.1 version ?