Splunk Search

How can I replace only the field value if found using Automatic lookups?

horst_poehlmann
Explorer

I have a problem with my checkpoint logs and automatic lookup tables (although the problem is not specific to checkpoint logs but rather automatic lookups).

The original log lines look something like this:

loc=1959996|time= 6Nov2014 11:48:49|action=accept|orig=fw|i/f_dir=inbound|i/f_name=Lan1|has_accounting=1|product=VPN-1 & FireWall-1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={999999999-63E0-11E4-8888-999999999999};mgmt=cpmgr;date=1415077659;policy_name=POL1]|service_id=domain-udp|src=SERVER1|s_port=64444|dst=DESTSERV1|service=domain-udp|proto=udp|rule=77

loc=1960000|time= 6Nov2014 11:48:49|action=accept|orig=fw|i/f_dir=inbound|i/f_name=Lan2|has_accounting=1|product=VPN-1 & FireWall-1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={999999999-63E0-11E4-8888-999999999999};mgmt=cpmgr;date=1415077659;policy_name=POL1]|service_id=https|src=10.0.0.1|s_port=55555|dst=10.1.0.1|service=https|proto=tcp|rule=222

When splunk creates the fields, I end up with src=SERVER1 and src_ip=SERVER1 (same for dest and dest_ip)

What I want to do is replace src_ip with the actual IP of SERVER1, so I created a csv from the checkpoint config (using CPrules.pl) and that all works fine.

However, there are also lines that are not defined and therefore already contain an IP address (see logline 2 above), which means it can't find it in the lookup table and therefore returns a blank field. I use the "Overwrite field values" feature in automatic lookups.

My Question is, can I somehow keep the original value if it wasn't found in the lookup table?

Also, when defining lookups in the Web GUI, where is the information stored?

PS: I use splunk 6.1

Thanks

0 Karma
1 Solution

musskopf
Builder

Hello horst.poehlmann,

All configuration is stored inside $SPLUNK_HOME/etc//local on the files props.conf and transforms.conf when permissions are "App" or "Global".

Related to your query, I believe the best thing is to output the IP from the automatic lookup to a new field, something like: src_ip_resolved and use an eval command to define it the src_ip should be replaced with the content of src_ip_resolved or not:

Example

<your search> | eval src_ip=IF(ISNULL(src_ip_resolved),src_ip,src_ip_resolved)

On the example above, you initial search should use the automatic lookup definition returning the new field src_ip_resolved, later you check: IF the lookup result was null, you use the src_ip, ELSE use src_ip_resolved instead. You'll need to repeat it for the dest_ip field as well.

View solution in original post

musskopf
Builder

Hello horst.poehlmann,

All configuration is stored inside $SPLUNK_HOME/etc//local on the files props.conf and transforms.conf when permissions are "App" or "Global".

Related to your query, I believe the best thing is to output the IP from the automatic lookup to a new field, something like: src_ip_resolved and use an eval command to define it the src_ip should be replaced with the content of src_ip_resolved or not:

Example

<your search> | eval src_ip=IF(ISNULL(src_ip_resolved),src_ip,src_ip_resolved)

On the example above, you initial search should use the automatic lookup definition returning the new field src_ip_resolved, later you check: IF the lookup result was null, you use the src_ip, ELSE use src_ip_resolved instead. You'll need to repeat it for the dest_ip field as well.

musskopf
Builder

That's good! Eval functions like those ones, with a simple IF are very light... you can use the "JOB -> Inspect" panel to have a look how much time they take to run.

0 Karma

horst_poehlmann
Explorer

I could not find any of the variables I used in my lookup definitions using the GUI in the files under $SPLUNK_HOME. I grep'd for a particular string. Not that important at this stage though.

Thanks, the eval function did the trick. Not sure how efficient that is as I had to use 4 separate eval functions (src, dest port and protocol), but it does work.

Appreciate your answer.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...