hello,
I am currently testing Splunk, with a single instance on a VM.
I have some trouble getting information out of logs correctly.
The log I am analysing has the following fields:
Time Stamp, Action, Source, Destination, Translated, Source, Translated Dest, Duration, Bytes Sent, Bytes Received, Application, and Reason.
some sample data:
========================================================================================================================
Entire Traffic Log list
Current system time is Thu, 25 Apr 2019 09:38:19
========================================================================================================================
Time Stamp Action Source Destination Translated Source Translated Dest Duration Bytes Sent Bytes Received Application Reason
2019-04-25 09:38:19 Permit
10.11.100.139:49573 192.168.3.2:9090 10.11.100.139:49573 192.168.3.2:9090 0 sec 0 0 TCP PORT 9090 Creation 2019-04-25 09:38:19 Permit
10.11.100.104:52934 <public IP>:443 <public IP>:30233 <public IP>:443 0 sec 0 0 HTTPS Creation 2019-04-25 09:38:19 Deny
10.10.1.50:60239 <public IP>:443 0.0.0.0:0 0.0.0.0:0 0 sec 0 28 HTTPS Traffic Denied 2019-04-25 09:38:19 Permit
10.11.100.139:49572 192.168.3.2:9090 10.11.100.139:49572 192.168.3.2:9090 0 sec 0 0 TCP PORT 9090 Creation 2019-04-25 09:38:19 Permit
10.11.100.133:50622 <public IP>:443 <public IP>:32209 <public IP>:443 0 sec 0 0 HTTPS Creation 2019-04-25 09:38:19 Permit
10.11.100.139:49571 192.168.3.2:9090 10.11.100.139:49571 192.168.3.2:9090 0 sec 0 0 TCP PORT 9090 Creation 2019-04-25 09:38:19 Permit
10.11.100.39:51561 <public IP>:443 <public IP>:57732 <public IP>:443 0 sec 0 0 HTTPS
That's the first few lines of the log.
I have replaced public IPs with <public IP>
for obvious reasons.
When I try to transform all these so I can select on them more easily, I run into errors.
What is the best way to get the data out?
I guess I have to change a props.conf file. How do I find the one that contains the sourcetype I created?
your question/problem seems to be very generic. would be good to put the actual event message etc.
So at which point you stuck?
1. Are you able to index data into Splunk? Check if inputs.conf is correct
2. Did you specify the indextime settings correctly? (ie. timestamp, source, host, sourcetype, line break etc.) all within props.conf
3. Once (1) and (2) is complete, ensure you extract all basic things like sourcetype, time etc.
Hello Koshyk,
1) The data goes into Splunk fine.
2) Yes. the line break is fine.
Even Sourcetype, time, source are extracted correctly. But an event is just that, an event. I cannot select on sourceIP, or protocol.
I can select a sourceIP from any event, but not all SourceIPs, because Splunk does not see them as key/value pairs. It only sees the default key/value pairs.
My question is: where do I define them? in the inputs.conf file?
If I look for "inputs.conf" I get 26 hits (the VM is both Indexer, UF ánd SH...)
In SPLUNK\etc\system\default** the sourcetype I configured does not appear in either **inputs.conf or props.conf.
I updated the question with an example...
Can anyone help me?
you need to provide sample events and we can write the props.conf for you
these settings are normally in props.conf with sometimes, (for complex extractions) transforms.conf
I changed the question to include some sample events.