Getting Data In

Monitor:// for a file that has a FOOTER

narwhal
Splunk Employee
Splunk Employee

I'm trying to monitor a html table in an html file that is updated regularly. My gotcha is that there are about 15 lines at the bottom of the HTML that finish up the page. I already know how to use props/transforms to only keep the entries that I want, but I'm worried that Splunk will see the file get bigger but really only eat the footer again and again. Is there some way that (without a preprocessor--which isn't out of the question) that I can get Splunk to not only see that there is more data in the file, but to eat the new entries not just the same footer over and over as it gets pushed deeper into the file?

Tags (1)
1 Solution

narwhal
Splunk Employee
Splunk Employee

I have taken the preprocessor route on this issue. Again, I have a programmatically created HTML file that has 202 lines html gunk at the top and 17 lines at the bottom. I want to strip that off and have just the "table" html with the "rows" indexed into Splunk. So, I now have an every minute fired off Linux cron task that does a combination of head & tail to clean it up and create a new file, and I monitor that file.

My script loops and does this for each file, but the important part is how to use head/tail to accomplish my goal.

head -n -17 filename.html | tail -n +202 > filename.html.table

Hope that helps someone...

View solution in original post

0 Karma

narwhal
Splunk Employee
Splunk Employee

I have taken the preprocessor route on this issue. Again, I have a programmatically created HTML file that has 202 lines html gunk at the top and 17 lines at the bottom. I want to strip that off and have just the "table" html with the "rows" indexed into Splunk. So, I now have an every minute fired off Linux cron task that does a combination of head & tail to clean it up and create a new file, and I monitor that file.

My script loops and does this for each file, but the important part is how to use head/tail to accomplish my goal.

head -n -17 filename.html | tail -n +202 > filename.html.table

Hope that helps someone...

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...