Reporting

Export large search date range in raw format

kerrj712
Explorer

OK,

So I have been at this all day and I cannot see a solution. Part of my frustration is that the documentation says one thing, the wiki says another. e.g. use "quotes" no don't use quotes.

My user wants all messages for the last 60 days for a single host, sent to him in a syslog format so he can forward to the vendor.

From the GUI I can get results but it is greater than 10,000 lines so exporting it is heck! ( sorry folks but those links posted here on how to increment a export suck.)

From the command line I don't get any errors but splunk will not under any circumstances report over 60 days. Doesn't matter if I used starttime="m/d/y:h:m:s or if I use daysago=60 etc. The search will not go back far enough.

Can anyone tell me how to get ./splunk search host="foo" daysago=60 > myfoofile.txt to work?

Tags (1)

kerrj712
Explorer

I agree. I won't give CLI access to the sysadmins, (separation of duties). But I would like for them to be able to do this on the GUI instead of having to ask me to pull it for them.

0 Karma

zscgeek
Path Finder

Use case is sending logs for a problem report to a vendor. I don't want my users touching CLI on search head boxes AND I don't see being able to give the vendors direct splunk access. Oftin the logs for the problem that they request are far larger then 10k lines

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

It would be helpful to know some of the use cases for a 10,000+ event manual export from the GUI, and whether all or part of whatever you need it for can be performed within Splunk itself.

0 Karma

zscgeek
Path Finder

I would really love an option to do this in the GUI as well. requiring CLI access is highly suboptimal for many use cases.

Stephen_Sorkin
Splunk Employee
Splunk Employee

By default, the Splunk CLI will output 100 results. In Splunk 4.1, for simple searches, you can export an unlimited number of results from the CLI using -maxout 0. For example, you should search:

./splunk search 'host=foo earliest=-60d' -maxout 0 > myfoofile.txt

ranjyotiprakash
Communicator

Thanks a lot Stephen 🙂 +2

0 Karma

gsawyer1
Engager

I am trying the same thing, attempting to export windows event logs sent to the Indexer via WMI. I narrowed the CLI command down to what I expected would be a simple export of only 73 events, which I verified within the web GUI's search interface. I can see that the file gets created at the file location I specified, but the file's size remains 0KB, and I have yet to get the prompt back; I'm assuming this means that Splunk is still processing the command, but exactly how long should it take to export 73 windows events to a text file, and how will I know when the job is done? Also, what is the syntax for specifying a period of time such as from day x to day y within a range, using a CLI search command?

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

I really would like this to be in the UI as well. The main problem is how the Python appserver relays data from splunkd, but this is a technical issue that we need to work through.

0 Karma

kerrj712
Explorer

Stephen, Thanks much. This did the trick. For what it is worth, I think a better option is to allow this from the GUI as a submitted batch job that is throttled back so it doesnt break the app.

I don't have a problem with allowing a user to pull large data, as long as they don't want it in 2 minutes. Let it cook low and slow so they get their file for the vendor.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...