ResultsReaderJson to output JSON string

[1234testtest]

Hi,
(Pardon my ignorance) I would like to know how to get the JSON string from ResultsReaderJson or any other API. We need to send the output in JSON format to a URL.

Here is what I plan to achieve. To get the output from Splunk in the form of JSON and feed this input to another RESTful service, say ABC.

For ABC, if we directly give through curl in the following format, it works.
curl -d '{ "index": "myindex", "data": 2340 ' http://localhost/test/mytest

I have tried to read INputstream - but it doesnt give the values.

            Args outputArgs = new Args();
            outputArgs.put("output_mode","json");

InputStream results=  jobSavedSearch.getResults(outputArgs);
            ResultsReaderJson resultsReader = new ResultsReaderJson(results);
            InputStreamReader resultsStreamReader = new InputStreamReader(results, "UTF8");
            OutputStreamWriter writer = new OutputStreamWriter(System.out);
             int size = 1024;
             int count = 0;
                char[] buffer = new char[size];
                try
                {

                while (true) {
                    count = resultsStreamReader.read(buffer);
                    System.out.println("countis "+ count);
                    if (count == -1) break;
                    writer.write(buffer, 0, count);

                }

                writer.write("\n");
                writer.close();
                resultsStreamReader.close();
                }

Nothing gets printed and nothing in buffer - so not able to get the data in the form of JSON from splunk.

I have tried using Java the data shown in the the curl example above and it works ( below is code if need be)

String baseURL = "http://localhost/test/mytest";
String urlString = baseURL;
System.out.println("Request: " + urlString);
String dataString =
"{ \"index\": \"myindex\", \"data\": 2340 }";
System.out.println("Data: " + dataString);
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
conn.setRequestProperty("Accept", "application/json");
conn.setReadTimeout(10000); // time out in 10 seconds
conn.setDoInput(true);
conn.setDoOutput(true);
DataOutputStream output =
new DataOutputStream(conn.getOutputStream ());
output.write(dataString.getBytes());
output.flush ();
output.close ();

//

[Neeraj_Luthra · Accepted solution]

I see now what it is that you wanted. Should be straightforward ... build your JSON string as you loop through the raw input stream. And simply pass it to the org.json.JSONObject and ask for "results".

Args outputArgs = new Args();
outputArgs.put("output_mode","json");
Job job = service.getJobs().create("search index=_internal | head 1000 | top status");
while (!job.isDone()) {
    Thread.sleep(1000);
}
InputStream results = job.getResults(outputArgs);
BufferedReader br = new BufferedReader(new InputStreamReader(results));
StringBuilder sb = new StringBuilder();
String line;
while ((line=br.readLine()) != null) {
    sb.append(line);
}
System.out.println(sb.toString());
JSONObject jo = new JSONObject(sb.toString());
System.out.println(jo.get("results"));

You can also download the pre-built java-json.jar

[gracex]

Above code produced json, however it is not valid json:

Error: Parse error on line 1:
..., "highlighted":{}}[{"_bkt":"main~2206~
----------------------^
Expecting 'EOF', '}', ',', ']', got '['

Below is json produced but failed with validation:

{"preview":false,"init_offset":0,"messages":[],"fields":[{"name":"_bkt"},{"name":"_cd"},{"name":"_indextime"},{"name":"_raw"},{"name":"_serial"},{"name":"_si"},{"name":"_sourcetype"},{"name":"_subsecond"},{"name":"_time"},{"name":"host"},{"name":"index"},{"name":"linecount"},{"name":"source"},{"name":"sourcetype"},{"name":"splunk_server"}],"results":[{"_bkt":"main~2206~9496990C-834A-4C64-1111-1C64F0EF946D","_cd":"2206:205793331","_indextime":"1572153782","_raw":"2019-10-27 05:23:01.566 DEBUG (Nof1DownloadConsumer.java:39) - === Nof1 downloader start ===","_serial":"0","_si":["ip-172-66-99-54","main"],"_sourcetype":"kubernetes","_subsecond":".566","_time":"2019-10-27T05:23:01.566+00:00","host":"enzyme-curation-sqa-55545b6f5c-2wkfm","index":"main","linecount":"1","source":"/var/log/containers/55545b6f5c-2wkfm_enzyme--sqa_enzyme-curation-sqa-03357d02d3aeb771b8f2e238d431b427a30b23c032ad2edbca6aeb3717f63691.log","sourcetype":"kubernetes","splunk_server":"ip-172-31-27-54"}], "highlighted":{}}
[{"_bkt":"main~2206~9496990C-834A-4C64-8E76-1C64F0DC946D","_cd":"2206:205793331","_serial":"0","_raw":"2019-10-27 05:23:01.566 DEBUG (Nof1DownloadConsumer.java:39) - === Nof1 downloader start ===","splunk_server":"ip-172-31-27-54","index":"main","source":"/var/log/containers/enzyme-curation-sqa-55545b6f5c-2wkfm_enzyme-curation-sqa_enzyme-curation-sqa-03357d02d3aeb771b8f2e238d431b427a30b23c032ad2edbca6aeb3717f63691.log","_indextime":"1572153782","_subsecond":".566","linecount":"1","_si":["ip-172-31-27-54","main"],"host":"enzyme-curation-sqa-55545b6f5c-2wkfm","_sourcetype":"kubernetes","sourcetype":"kubernetes","_time":"2019-10-27T05:23:01.566+00:00"}]

[Neeraj_Luthra]

It doesn't look like you are actually using the resultsReader object in your code at all. So no output doesn't surprise me. Here is a simplified version of the code that you can try:

InputStream results = job.getResults(outputArgs);
ResultsReaderJson resultsReader = new ResultsReaderJson(results);

Map<String, String> map = null;
while ((map=resultsReader.getNextEvent()) != null) {
    for (String key: map.keySet()) {
        System.out.println("   " + key + ":  " + map.get(key));
    }
}   

Also, I can't tell which version of Java SDK (0.8 or 1.0) and Splunk (5.0 or less) you are using. Please provide these details if you continue to run into issues.

-Neeraj.

[1234testtest]

Thank you - this is giving me the string results and along with that lot of other unwanted data

{"preview":false,"init_offset":0,"messages":[{"type":"DEBUG","text":"base lispy: [ OR sourcetype::test1 sourcetype::test2 ]"},{"type":"DEBUG","text":"search context: user=\"admin\", app=\"myapp\", bs-pathname=\"c:\Program Files\Splunk\etc\""}],"results":[{"count":"61"}]}

I was assuming that I could get a JSON object of just
{"count":"61"}.
Thanks a lot once again

[Neeraj_Luthra]

ResultsReaderJson is basically a helper class that parses through the JSON data for you. If you want to get the JSON object itself, you dont have to use any of the ResultsReader classes. Simply get the data directly from the input stream. How about this:

InputStream results = job.getResults(outputArgs);

BufferedReader br = new BufferedReader(new InputStreamReader(results));

String line = null;

while ((line=br.readLine()) != null) {

    System.out.println(line);

}

[1234testtest]

Thank you. Jre 1.7, Splunk 5.0 are the versions. I get this piece of code outputs the data - but not in the form of JSON- how do I get the JSON object which can be fed into another system? Or atleast as String object which has JSON format is also fine.