Replacing Results Rather Than Appending

[parkerio]

A script has been set up to produce a txt file every 10 mins

The txt file has been set to import as follows:

'Continuously index data from a file or directory this Splunk instance can access'

Rather than the results appending each time the txt file is created / imported, I would like the results to be completely overwritten each time.

I have tried the source type as:
csv
syslog
automatic

No Joy, any help would be much appreciated.

Cheers

[parkerio · Accepted solution]

Thanks for the help.

What I needed was a dashboard that shows just the current data from a script that is set to run every hour.

When the script ran the first time the data was correct, the second time it ran the data was doubled, the third time trebled etc-

I have actually worked out a way to do this by running a script every hour and getting the saved search to start one hour earlier to now.

By doing this even though the data is accumulating the dashboard just shows data from one hour ago which is the current data.

I suspect this is not the best way to achieve my needs but it works.

Cheers

[dwaddle]

How big is the text file? What are you doing with it, precisely? If it is both relatively small (tens of megabytes) and only used as ancillary data to "enhance" existing events, a lookup may be appropriate.

Without doing some strange things, Splunk isn't really set up to understand the idea of "replace those events with these".

I would recommend you edit your question with some additional detail behind your use case. (Don't append a new answer to the original question - that just gets confusing) This will help folks be more able to provide an answer that meets your needs.

[MHibbin]

Do you mean you would like the results removed from Splunk each time? - If so, the following URI would be useful I think...

http://docs.splunk.com/Documentation/Splunk/latest/admin/RemovedatafromSplunk

When I want to remove events from Splunk I use $SPLUNK_HOME/bin/splunk clean eventdata -index <indexname> -f

BE WARNED, THIS WILL REMOVE ALL EVENTS THE INDEX...! Maybe if this is what you mean you can create a seperate index just for this data, and then build this command into your script.

There is also a section on the above webpage, which may also help... If you set the files to be dynamically named (e.g. including date in file name), you may be able to monitor the directory... and then use the "delete" search time option (also in the link)...

So if, for example, the file was called "splunk_import_20111125.txt", you could remove all events with the source relating to this (providing you leave the source as default)

Or do you mean, you would like Splunk to start reading the file from the beginning each time? If so, you must ensure that the "Follow Tail" option is de-selected (under Advanced Options, when adding a File or Directory input using SplunkWeb), or using ...

followTail = 0

Under the appropriate stanza in your inputs.conf file.