Getting Data In

size and timestamp of lookup files

mmol
Explorer

The lookup table files view only shows some basic info about the file.
I often like to know the size and the age of the files.
I've seen some pyhton additions to show that, but the lookup table file view seems to be the logical place to add such info.

Can this be added in a future splunk release?

0 Karma

darrenfuller
Contributor

If you were looking to add this now, (and assuming you are on Linux) you could make a quick and dirty script that collects that info into index ...

ie. if you made an app on your search head

mkdir /opt/splunk/etc/apps/lookup_evaluator
mkdir /opt/splunk/etc/apps/lookup_evaluator/bin
mkdir /opt/splunk/etc/apps/lookup_evaluator/local
mkdir /opt/splunk/etc/apps/lookup_evaluator/metadata

And then made a script that grabs what you are looking for:

/opt/splunk/bin/lookup_evaluator/bin# cat ./lookup_lookups.sh 
#!/bin/bash
ls -lah $(/opt/splunk/bin/splunk btool lookups list --debug | egrep ".csv|.kmz" | sed -r "s/(.+)\s+\[\w+\.\w+\]/\1/g")

And then made an inputs.conf to run that script

/opt/splunk/etc/apps/lookup_evaluator/local# cat ./inputs.conf 
[script://./bin/lookup_lookups.sh]
interval = 60
index = main
sourcetype = lookuplookups

And then made a props.conf to evaluate the output from the script

/opt/splunk/etc/apps/lookup_evaluator/local# cat ./props.conf 
[lookuplookups]
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
DATETIME_CONFIG = CURRENT
EXTRACT-01-lookuplookupsfields = ^(?<permissions>\S+)\s+(?<linkcount>\d+)\s+(?<owner>\S+)\s+(?<group>\S+)\s+(?<size>\S+)\s+(?<last_modified_date>\w{3}\s+\d+\s\d+\:\d+)\s+(?<path>.+)
EVAL-last_modified_timestamp = strptime(last_modified_date, "%b %d %H:%M")
EVAL-sec_since_last_modified = now() - last_modified_timestamp

and then finally adding /local/app.conf and metadata/local.meta files to make this a proper app:

/opt/splunk/etc/apps/lookup_evaluator/local# cat ./app.conf 
[install]
is_configured = true

[ui]
is_visible = 0
label = lookup_evaluator

[launcher]
author = Me
description = Collect last modified time and size of lookups in Splunk
version = 0.0.1

[package]
id = lookup_evaluator

&

/opt/splunk/etc/apps/lookup_evaluator/metadata# cat ./local.meta 
[]
access = read : [*], write : [admin]
export = system

You would end up with events that gave you the data you were looking for and it would be timestamped over time so you could track growth and change over time of your lookup files.

Hope this helps.
./D

mmol
Explorer

Thanks for your comments Darren,
Unfortunately I have no access at the command line level so implementing this requires me to go to the application managment team.
It seems such an easy feature to add to the gui....

0 Karma

darrenfuller
Contributor

If you have sufficient access on Splunk, if you create all the files, then compress them into a tar.gz file, then use: Apps -> Manage Apps -> Install app from file, to load the app onto your splunk installation, you may not need command line access. Then restart the Splunk instance and it should work fine.

Splunk will run on your laptop/desktop so you can create and test the configs before you install them on the production instance..

0 Karma

richgalloway
SplunkTrust
SplunkTrust

https://ideas.splunk.com

---
If this reply helps you, Karma would be appreciated.
0 Karma

mmol
Explorer

Thanks for the pointer! Just created it as a new idea...
https://ideas.splunk.com/ideas/EID-I-168

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...