Getting Data In

Limit search results to a sourcetype

sandy1978
New Member

What can I do to limit search results for one or more sourcetypes.

I am able to get the results through the Splunkweb UI but I need to get the same set of results from a command line as well. My sourcetype name is BIC_CS.

curl -u username:password -k https://hostname:8089/services/search/jobs -d "search=search sourcetype=BIC_CS"

When I create this search job, it results in 0 events whereas from UI, I get all events in this sourcetype.

Any help will be greatly appreciated!

Thanks

Tags (1)
0 Karma

Lowell
Super Champion

Have you tried:

splunk search "sourcetype=BIC_CS" --auth user:password

That's probably the easiest way. For more info run:

splunk help search


You can also run a search in "oneshot" mode and return the results directly. The results will be returned in an XML format (which isn't all that easy to parse with standard command line tools, but I think that's your only option in "oneshot" mode.) This example searches from -2h@h to -15m@m and will not run longer than 60 seconds before timing out

curl -u user:password -k https://splunk.example.com:8089/services/search/jobs -d "search=search sourcetype%3dBIC_CS&exec_mode=oneshot&earliest_time=-2h@h&latest_time=-15m@m&max_time=60"

Notice that you have to encode any "="s with %3ds in your search because of standard HTTP URL quoting conventions. There are a bunch of other quoting rules you have to deal with as well. You can quickly URL quote a search string with a command like this: (Simply insert your own search between the double quotes)

python -c 'import urllib; print urllib.quote("search tag::host=production size>100")'

The output should look like:

search%20tag%3A%3Ahost%3Dproduction%20size%3E100


If you don't have command line access to the server, then I would suggest using one of the existing splunk-search client packages that already exist. You can certainly write your own (which I've done for a high-level integration platform that had the necessary HTTP and XML handling capabilities built in), or do some shell scripting with curl (as shown above), but you may find that tedious and it may not scale well.

If you don't have much programming experience, than I would recommend getting started with the Python search SDK. First because it pretty easy language to learn and teaches good programming practices. And secondly because splunk uses it internally (and it is therefore well tested an up to date) and you may find it coming in handy down the road with other splunk integration tasks.

Documentation resources:

Unfortunately, in your situation there are a few extra gotchas because you would need to use the python SDK remotely. (Of course, you could simply install a local copy of splunk, and just don't run it, but that seems a little silly.) I feel like there should be a good answer to this question, so I've asked How to install the Python SDK on a remote machine? Hopefully someone will provide some easier instructions on setting this up.

sandy1978
New Member

Lowell, Thanks a lot for helping me figure out the issue with URLencoding. This tells me how long it's been since I did any serious coding!

0 Karma

sandy1978
New Member

Here is a copy paste from Developer Manual from Splunk. It gives an example of searching for *.

Create a search job by POSTing to the search/jobs/ endpoint. Set your search as the POST payload.
For example:
curl -u admin:changeme -k https://localhost:8089/services/search/jobs -d"search=search *"

Now when I need to limit the search results to a specific sourcetype, how should I write the same query? The name of the sourcetype is BIC_CS.

I don't have access to Splunk CIL and I need to run this command line query remotely.

Thanks
Sandy

0 Karma

Genti
Splunk Employee
Splunk Employee

yeap, that should do it!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...