How do I install, configure and utilize the Splunk for amMap apps flashmaps in my Splunk instance?
YOU WILL NEED TO INSTALL THE MAXIND APP ON SPLUNKBASE FOR GEO LOCATION INFORMATION
The app can be fond here: http://splunk-base.splunk.com/apps/22282/geo-location-lookup-script-powered-by-maxmind
By now you have installed the amMap application and should have installed supporting MAXMIND app as well. You'll notice an empty map on the default landing page of this app. That map is set to be populated by your data on an hourly basis, mapping the top 100 public IP's in your instance that are recorded in the last hour. If you are running Splunk Free you will need to manually populate this map. You'll notice the drill down searches aren't working but I'll explain that later on in this doc.
In order to do a first run and see if things are working correctly you can run the following search:
| rex "(?<ip>d+.d+.d+.d+)"| search ip!=192.168 ip!=0.0. ip!=10.| stats count by ip | head 100 | eval count_label="Event" | eval iterator="ip" | eval iterator_label="IP" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as ip | mapit
If this search completes but does not display any results then you should see a populated map on the landing page.
If you do not see anything try the search but remove the | mapit and make sure you are getting back a results table with populated geo info. If a table is returning but the geo fields are empty you have most likely do not have any public IP's in your data for the geoip translation to operate on.
It is possible to create maps on other assets such as internal IPs, subnets, hostnames etc. That is explained further down in this doc under Populating the map with csv lookups.
For the purpose of this doc we are going to use the following example search:
eventtype=firewall src_ip=* | stats count by src_ip | eval count_label="Event" | eval iterator="src_ip" | eval iterator_label="src_ip" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as src_ip | mapit
In the first segment of the search you will see that we are searching across our firewall data looking for events with a src_ip field. (Instructions on how to create a field in splunk can be found here)
This part of the search represents the results you are interested in. You may want to add additional values to have results that represent a particular threat or web traffic or something else you would like to see represented geographically on the flash map.
Next we need to create a stats table for that IP field, which is done here in the example search:
| stats count by src_ip
The next step is to create the required fields necessary for the map_results.py script to run. These are the varibles which drive your flashmap. That is happening in the above search here:
| eval count_label="Event" | eval iterator="src_ip" | eval iterator_label="IP" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap"
These eval statements create the REQUIRED fields for the map to work:
count_label - What we want to display on the mouse over (i.e. Security Events, BotNet events, etc.) the default in the example above is simply Events.
iterator - What should the script iterate on, in this case we are counting up the src_ip counts for each location, this is represented in the flashmap as the unique field count.
iterator_label - For better formatting we give a name to the iterator, in this case its an IP. We will use this field like this in the mouse over for a location: Unique <iterator_label>(s)
movie_color - This is the color of the balloon on the map. This can be dynamic using a rangemap or eval for multiple colors.
app - This is the app where the map data will be written to.
output_file - this is the xml file where the map data will be written to. The output file will go into the appserver/static/xml_out directory of the specified app. The map homepage of this app is already set to read from home_threat_data.xml
OPTIONAL zoom - if you are familiar with amMap and would like to set the default zoom for your map you can do so by eval'ing the zoom values into a string i.e "zoom="334%" zoom_x="-128.58%" zoom_y="-113.11%""
Next we need to add the geo information for mapping. This can be done with the MAXMIND geoip lookup app OR by using a lookup on internal or non-ip values.
the example above is done with the MAXMIND app using the following syntax:
lookup geoip clientip as src_ip
This command returns the following fields to your results which are required by the amMap app. client_city client_region client_country client_lat client_lon
Finally we call the mapit script. If all of the above fields are present the script should run and output your flashmap.
If you would like to populate the map using something other than public IPs you can do so by creating a CSV lookup documented here: http://www.splunk.com/base/Documentation/latest/Knowledge/Addfieldsfromexternaldatasources
Your lookup must return the following fields in order for mapit to run properly: client_city client_region client_country client_lat client_lon
If your searches are failing when you drill down it is most likley becuase you are not interating on a searchable field in Splunk. Documentation on how to create fields in your results can be found here: http://www.splunk.com/base/Documentation/4.1.3/User/ExtractNewFields
The sample map lives inside of the ammap_view view. You will see the following stanza:
<module name="ServerSideInclude" layoutPanel="panel_row1_col1">
This refers to ammap.html in the appserver/static directory. That HTML file specifies the xml being used, we generate the xml with the output_file field in our search.
<module name="ServerSideInclude" layoutPanel="panel_row1_col1">
The map_results log file is indexed into the Splunk internal index. You can view that log with the following search: index="_internal" source="*ammap_map_results.log" Additional debugging statements can be added by un-commenting anywhere you see logger() being called.
I've been playing with lots of different outputs for amMap maps. Please send me your ideas and I may have a parser you can use now or may need to do a little tweaking but in any case please send feed back as I'd love to hear how we can make this feature useful for you! firstname.lastname@example.org 6/4/2010
I am looking for a way to make the dots on the map a differnet color based on the sourcetype of the search. Like firewall denies are yellow and idps events are red. Then have a method to drop or shade one color or the other from the map. Simular to some of the samples on the Ammap website.
I took away the "... | mapit" to see if any data is returned, and I do get data back. However, I get "no results returned" when I add the "mapit" command.
What am I doing wrong?
I am using the command used above (plus an earliest command to reduce the data):
Thanks for your help.
Has anyone used ammap without a lookup? The necessary fields exist in my results and I'd like to bypass creating a lookup. See the link below.