Monitoring large amounts of SCADA type data and producing reports

[streddy]

Hi,

I'd like to use Splunk to be able to monitor a large amount of SCADA type data

e.g. a sensor which updates each second.

I'd like to be able to create a dashboard to be able to better understand the data and notify when something is out of range e.g. a temperature guage goes above a limit.

I'd also like to be able to generate reports periodically and on demand.

e.g. a report that showed fuel use over a month

I'm a beginner exploring Splunk to see if my company should put the reasouces into it to implement it.

Any tips or ideas is appreciated.

[mkumawat]

Hi,

I am also doing this project for my college's final year. Any advice and suggestion is appreciable!! I found out that for SCADA I need to install Kepware plug in in Splunk. Please help me with more knowledge.
Thanks in advance.

[remoraninja]

Hi all (hi Nick!)

I've just contributed to a discussion on Linkedin re Splunk and SCADA and thought I'd poke my head in here too.

We (Remora - remora.com.au) are actively participating in SCADA data monitoring with Splunk. Our clients are utility companies (currently only utilities in Australia). We have a number of tried-and-true methods for interfacing DIRECTLY with SCADA devices (to PLCs and also straight to RTUs) using a number of old-school protocols, then passing that data as name-value pairs to Splunk. We then do some beautiful things with the data, far beyond what traditional "process historians" etc do with the data. We can achieve better results in predictive failure analysis (PFA) than any other system we've seen (and better than any of our clients have seen) in a MUCH shorter amount of time (and data). I call it "the other side of the mirror" (Alice In Wonderland style) for Splunk - there may be a lot we can do with computer-generated data, but we enjoy analysis of machine-generated data - REAL machines. We have a case study of our biggest client coming out via Splunk in the next few weeks, watch out for it!

[sideview]

Absolutely. I've helped a number of people from the SCADA world get accustomed to Splunk. I believe Splunk is and has been proven to be a fantastic solution to take over a lot of monitoring and reporting in the SCADA world.

A while ago we built an app called the Sideview Process Historian, which you can see here - http://sideviewapps.com/apps/sideview-process-historian/

In hindsight a better name for it would be "Splunk for OSISoft PI System", but hindsight is always 20/20.

Step 1 of course is to get the data coming into Splunk as plain ascii text, ideally each point is on its own line. Generally step 2 is to turn it from Point, Value data into more of a simple key-value pair. I find that this is easiest to do in Splunk's search language and NOT at index time, but you might have to get used to Splunk before you believe that. And last but not least, is to construct the dashboards and reporting tools that you need.

Feel free to email me and I'm happy to jump on the phone too or to give a webex of the Process Historian. Cheers. nick [at] sideviewapps.com

[bhavna_jain]

Hi,
Above discussion is really informative. I would like to have a webex of Process historian app, since I am exploring how to analyze SCADA data through Splunk. please let me know when can we schedule the same.