Dashboards & Visualizations

best way to index automatically and update dashboard with unique file name

Javo222
Path Finder

Hi,

I have a script that uploads a file named file.log to a folder on my computer (let's say C:\Users\Public\Documents\splunk_data). I have managed to extract some fields like CPU_usage or Memory_usage and draw it in a dashboard, using the following string:

source="file.log" Total_CPU_Usage=* | timechart span=1h max(Total_CPU_Usage)

That works fine, however now my script will upload a new file.log to this folder and I would like splunk to Index it automatically and also update the dashboard with this new data ONLY. I don't want to see the old data.

How can I:
- Make sure Splunk indexes automatically this file? It seems it's not indexed using "Continuously index data from a file or directory this Splunk instance can access" to "C:\Users\Public\Documents\splunk_data".
- Make sure the old data is removed from the server, or at least not shown in the dashboard.

At the moment, I'm renaming any new file into fileX.log and uploading it manually. Then I go to the dashboard and change the source to fileX.log . I'm sure there is a better way to do that 🙂

thanks!

Tags (2)
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Assuming you have access to physical server where Splunk resides and both the file and splunk are on same server, do the following:-

  1. Go the machine where Splunk is installed and go to the local folder of the splunk app where you're creating your dashboard. (it would be something like c:\Program Files\Splunk\etc\apps\YourAppName\local.
  2. Create/update a file name inputs.conf and put following

    [monitor://C:\Users\Public\Documents\splunk_data\file.log]
    disabled = false
    followTail = 0
    sourcetype = YourSourceType
    index = YourIndex

  3. Restart your splunk instance. This will enable automatic monitoring on the file C:\Users\Public\Documents\splunk_data\file.log and will update all changes to splunk automatically. The same can be done from Splunk Web as well See this (http://docs.splunk.com/Documentation/Splunk/6.1.4/Data/UseSplunkWeb)

  4. Change your dashboard query to this to get the latest indexed file.

    source="file.log" Total_CPU_Usage=* _indextime=[search source="file.log" | head 1 | eval Date=_indextime | return $Date] | timechart span=1h max(Total_CPU_Usage)

Updated

source="file.log" Total_CPU_Usage=*  [search  source="file.log" | head 1 | eval Date=_indextime | eval search="_indextime=".Date | table search] | timechart span=1h max(Total_CPU_Usage)
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try with updated query.

0 Karma

Javo222
Path Finder

Yes I have access to the server and the file. I've tried your instructions but the search seems wrong, I can't see what is missing. I get the following error:

Error in 'search' command: Unable to parse the search: Comparator '=' is missing a term on the right hand side.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...