Monitoring Splunk

Java sdk Splunk queries performance challenge

1234testtest
Path Finder

Hi, I am using JAVA SDK of Splunk and using struts2 as the framework. I have a performance challenge.
Each time the page loads, the queries take too long time to execute (i have 6 of them in each page). So, I implemented Ajax.
But even then, the first time the page loads - takes too long (about 30-40 seconds) (for splunk server connection and then querying the saved searches) before the user sees a response. I am using non-blocking mode of executing which was included in the website (http://dev.splunk.com/view/splunk-java-sdk-how-to/SP-CAAAEKY).

Could you pl suggest a better way of improving the performance.

   Job jobSavedSearch = null;

        // Run the saved search
        try {
            jobSavedSearch = savedSearch.dispatch();
            amJob.job = jobSavedSearch;
        } catch (InterruptedException e1) {
            e1.printStackTrace();
        }

        System.out.println("Waiting for the job to finish...\n");

        // Wait for the job to finish
        while (!jobSavedSearch.isDone()) {
            try {
                Thread.sleep(500);
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
Tags (3)
0 Karma
1 Solution

Neeraj_Luthra
Splunk Employee
Splunk Employee

The connection to Splunk is most likely not adding any significant delay. My initial guess is that your saved searches are expensive and are taking quite some time to return.

Unless these queries absolutely need to return real-time data, I would recommend scheduling these saved searches on Splunk to run at regular intervals. You can do this from the Splunk UI and pick time intervals like once a minute, hour, day, month etc., whatever suits your business need.

Finally, try the following in your Java code:

SavedSearch mySavedSearch = service.getSavedSearches().get("mySavedSearchName");
Job[] jobs = mySavedSearch.history();
Job myJob = null;
try {
    myJob = (jobs.length > 0) ? jobs[0] : mySavedSearch.dispatch();
} catch (InterruptedException e) {
    e.printStackTrace();
}
System.out.println("Waiting for the job to finish for saved search - " + mySavedSearch.getName() + " ...\n");

while (!myJob.isDone()) {
    try {
        Thread.sleep(500);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}
System.out.println("Job for saved search - " + mySavedSearch.getName() + " finished.\n");

View solution in original post

Neeraj_Luthra
Splunk Employee
Splunk Employee

The connection to Splunk is most likely not adding any significant delay. My initial guess is that your saved searches are expensive and are taking quite some time to return.

Unless these queries absolutely need to return real-time data, I would recommend scheduling these saved searches on Splunk to run at regular intervals. You can do this from the Splunk UI and pick time intervals like once a minute, hour, day, month etc., whatever suits your business need.

Finally, try the following in your Java code:

SavedSearch mySavedSearch = service.getSavedSearches().get("mySavedSearchName");
Job[] jobs = mySavedSearch.history();
Job myJob = null;
try {
    myJob = (jobs.length > 0) ? jobs[0] : mySavedSearch.dispatch();
} catch (InterruptedException e) {
    e.printStackTrace();
}
System.out.println("Waiting for the job to finish for saved search - " + mySavedSearch.getName() + " ...\n");

while (!myJob.isDone()) {
    try {
        Thread.sleep(500);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}
System.out.println("Job for saved search - " + mySavedSearch.getName() + " finished.\n");

1234testtest
Path Finder

Thank you..

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

Summary Indexing can definitely help but I suggest that you read through Report Acceleration and Summary Indexing along with their use cases at http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Aboutsummaryindexing.

0 Karma

1234testtest
Path Finder

Hi Neeraj,
Thank you for the guidance. Could you kindly suggest does summary indexing help.

0 Karma

1234testtest
Path Finder

Thank you. I see that in 5.x version of SPlunk,there is summary indexing, which seemingly is much faster. http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Usesummaryindexing

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

That won't help. An index in Splunk is not like one in a typical RDBMS where indexes are created for performance improvement. If you have scheduled your saved searches and are still looking for further improvement, I recommend reading through the Splunk search manual - http://docs.splunk.com/Documentation/Splunk/latest/SearchReference.

0 Karma

1234testtest
Path Finder

Does usage of indexes help in improving the performance? Am just evaluating other options as well. Thank you.

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee
  1. The code is set up to dispatch the saved search in the event that the history is empty. That is probably why you are still getting results.
  2. After you login into Splunk web, select the appropriate app from the "App" drop down in the top right corner, then click on "Jobs". The URL is something like this - http://$host:$port/en-US/app/$app/job_management. Here you should see jobs for your saved searches, assuming they are scheduled.
0 Karma

1234testtest
Path Finder

I have now tried with multiple queries. There are few observations.
1. I dont have scheduled search- but still am getting the results from the savedsearch with the above code. What could be the reason.
2. Could you kindly clarify if there is any splunk web equivalent for the above code - how do I check from where it is getting savedsearch.history.
Kindly help

0 Karma

1234testtest
Path Finder

Thank you. It worked!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...