Getting Data In

Filtering event data

markb81
New Member

Hi,

I'm probably asking something that has been asked a thousand times. I searched the forums but I'm not really sure what to look for.
I'm really new to splunk. I got it to work that my firewall has been logging it's syslogs to splunk and that splunk is indexing

That results in large lines of text that are not really readable. Is there a good way to make a default view that opens every time I want to search my firewall, with just vdom, source ip, destination ip, protocol etc.... just the regular data.

Kind regards,

Mark

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you're seeing large lines of text, then it's likely your firewall data is not being indexed properly. While Splunk will do a lot right out of the box, it's often better to tell Splunk how to process your data.

If you would share a few sample events as well as your props.conf settings for the sourcetype you are using, we can help you get the data indexed better.

Once you've improved the indexing, searching the data should be easier.

---
If this reply helps you, Karma would be appreciated.
0 Karma

markb81
New Member

Hi Richard,

Thanks. This has been really helpful. I played around with it and got a very nice dashboard. This is what I now have:

source=Fortinet sourcetype=fgt_traffic | rename vd as vdom | table vdom srcintf srcip dstintf dstip dstport proto action policyid service appcat

I have a couple of more questions.

  1. This particular code looks very interesting, however if I include it I get zero results in my search results.

    (srcip=$field1$ OR dstip=$field1$)
    
  2. When saving as a dashboard I get a nice HTML like table which looks great, only cannot be searched. Is there a way to search that table?

  3. For some reason the action colum remains blank, altough there is a action in each and every line, and the action field is called action. Do you have a clue why the action column remains empty?

To give an example. When I filter for this: source=Fortinet sourcetype=fgt_traffic

These are the results. Action in this case is "timeout".

Apr 11 07:13:51 10.60.0.254 date=2018-04-11 time=16:14:21 devname="forti" devid="FGT60E4Q1xxxxxx" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="OUTSIDE" eventtime=1523456061 srcip=192.168.xx.xx srcport=28706 srcintf="VL_CL_OUT1" srcintfrole="undefined" dstip=172.17.xx.xx dstport=3483 dstintf="wan1" dstintfrole="wan" poluuid="baa8a562-1bc6-51e8-f019-abf69408feb1" sessionid=39112475 proto=6 action="timeout" policyid=1 policytype="policy" service="tcp/3483" dstcountry="Reserved" srccountry="Reserved" trandisp="snat" transip=195.xx.2xx.xxx transport=28706 duration=19 sentbyte=132 rcvdbyte=0 sentpkt=3 rcvdpkt=0 appcat="unscanned" crscore=5 craction=262144 crlevel="low"

  1. Is there a way to get the date and time in 2 columns in this format: dd-mm-yyyy hh:mm:ss
0 Karma

richgalloway
SplunkTrust
SplunkTrust
  1. The search will return no results until the field1 token has a value. That's done by the text box you created with "Add input" in my previous comment.

  2. Tables can be sorted, but not searched. Well, it could have been done using Advanced XML, but that's deprecated.

  3. Usually, an unexpectedly empty field is the result of a spelling error, but that's not the case here, it seems. I ran a test with your sample event and it displayed as it should (not empty). What are the props.conf settings for the fgt_traffic sourcetype?

To get the date and time columns into that format, try eval datetime=strftime(strptime(date." ".time, "%Y-%m-%d %H:%M:%S"), "%d-%m-%Y %H:%M:%S"). The 'date." ".time` construct concatenates the two columns with a space between them. The resulting string is parsed and then reformatted.

---
If this reply helps you, Karma would be appreciated.
0 Karma

markb81
New Member

Hi Richard,

Well that is something I can absolutely work with. If I enter it in my search filter I get a nice table with all the necessary data.
How to work with this. Is this a view I can save and use as favorite view?

What I have seen in an organization is that all the events were more or less readable like this table, and they could just enter an ip address in the search field to search all their firewall logs for that specific IP address. Is that something I can acchieve?

Kind regards,

Mark

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Mark,

You can save the search as a view by clicking "Save As" and selecting "Dashboard Panel".

To add the ability to enter a specific IP address to search for, edit the dashboard. Click "Add input" and choose Text. Click the pencil icon near the text box to customize it. Give the field a label and click Apply. To use the text field in your search, click the magnifying glass and select Edit Search. Change the search to this one

index=foo vd=* (srcip=$field1$ OR dstip=$field1$) | rename vd as vdom | table vdom srcip dstip proto sentbyte rcvdbyte

click Apply then save the dashboard and you should be able to search for a random IP address.

---
If this reply helps you, Karma would be appreciated.
0 Karma

markb81
New Member

Hi,

Thank you. Well I think the data is indexed properly. It uses the sourcetype of the fortigate splunk add-on.

1 entry looks like this:

Mar 24 06:44:04 10.60.0.254 date=2018-03-24 time=14:45:31 devname="forti" devid="FGT60E4Q17027315" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="OUTSIDE" eventtime=1521899131 srcip=192.168.20.50 srcport=61664 srcintf="VL_IN_OUT1" srcintfrole="undefined" dstip=72.26.1zz.zz dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="79991b00-1bc7-51e8-14b6-99478dcc1a0c" sessionid=25527977 proto=6 action="close" policyid=2 policytype="policy" service="HTTPS" dstcountry="United States" srccountry="Reserved" trandisp="snat" transip=195.35.xx.xx transport=61664 duration=1 sentbyte=333 rcvdbyte=387 sentpkt=5 rcvdpkt=5 appcat="unscanned"

There is a lot of data in there. I'm only interested in a little bit of it, and stripping the rest of it in some view would make it a lot more readable.

Thanks for thinking with me.

Kind regards,

Mark

0 Karma

richgalloway
SplunkTrust
SplunkTrust

So that's not as bad as I originally envisioned. Luckily for you it's in key=value format, which Splunk eats for breakfast.

See if this search gets you started.

index=foo vd=* srcip=* dstip=* | rename vd as vdom | table vdom srcip dstip proto sentbyte rcvdbyte
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...