Hey guys,
We are monitoring 2 specific CSV Log files on one indexer. I setup the appropriate custom field extractions for the CSV files in the props.conf and transform.conf files for both the indexer and the search head.
If I search directly on the indexer it works fine. However, if when I try to search the same files through the search head I am not able to see the custom field extractions I have created.
Any thoughts?
Here is what I have for the props.conf file for both the indexer and the search head:
[palo_alto_traffic]
REPORT-paextract = paloalto_traffic_extractions
KV_MODE = none
CHECK_FOR_HEADER = true
TRANSFORMS-NoHeader = NoHeader_paloalto
[palo_alto_threat]
REPORT-paextract = paloalto_threat_extractions
KV_MODE = none
CHECK_FOR_HEADER = true
TRANSFORMS-NoHeader = NoHeader_paloalto
and here is the contents of the transforms.conf file for both the search head and the indexer:
[paloalto_traffic_extractions]
DELIMS = ","
FIELDS = "Domain" , "Receive_Time" , "Serial_Number" , "Threat_Content_Type" , "Config_Version" , "Generate_Time" , "Source_address" , "Destination_address" , "NAT_Source_IP" , "NAT_Destination_IP" , "Rule" , "Source_User" , "Destination_User" , "Application" , "Virtual_System" , "Source_Zone", "Destination_Zone" , "Inbound_Interface", "Outbound_Interface" , "Log_Setting" , "Time_Logged" , "Session_ID" , "Repeat_Count" , "Source_Port" , "Destination_Port" , "NAT_Source_Port" , "NAT_Destination_Port" , "Flags" , "IP_Protocol" , "Action" , "Bytes" , "Bytes_Sent" , "Bytes_Received" , "Packets" , "Start_Time" , "Elapsed_Time_Sec" , "Category" , "Padding"
[paloalto_threat_extractions]
DELIMS = ","
FIELDS = "Domain" , "Receive_Time" , "Serial_Number" , "Type" , "Threat_Content_Type" , "Config_Version" , "Generate_Time" , "Source_address" , "Destination_address" , "NAT_Source_IP" , "NAT_Destination_IP" , "Rule" , "Source_User" , "Destination_User" , "Application" , "Virtual_System" , "Source_Zone" , "Destination_Zone" , "Inbound_Interface" , "Outbound_Interface" , "Log_Setting" , "Time_Logged" , "Session_ID" , "Repeat_Count" , "Source_Port" , "Destination_Port" , "NAT_Source_Port" , "NAT_Destination_Port" , "Flags" , "IP_Protocol" , "Action" , "URL" , "Threat_Content_Name" , "Category" , "Severity" , "Direction"
[NoHeader_paloalto]
REGEX = Domain,Receive Time,Serial #,Type,Threat/Content Type, ...
DEST_KEY = queue
FORMAT = nullQueue
Let me know.
Thanks.
Brian
Actually looked at the logs on this and looks like splunk did not like my custom field extractions for some reason... I will look into this further. Just weird that it works on the indexer and not the Search Head... maybe there is a conflict in the configs. I'll narrow it down and let you guys know.
Thanks for the help as always.
Brian
Actually looked at the logs on this and looks like splunk did not like my custom field extractions for some reason... I will look into this further. Just weird that it works on the indexer and not the Search Head... maybe there is a conflict in the configs. I'll narrow it down and let you guys know.
Thanks for the help as always.
Brian
$SPLUNK_HOME/etc/system/local.... I also took the liberty of setting the CHECK_FOR_HEADER = false...
And finally, can you let us know exactly where on each machine these files are relative to $SPLUNK_HOME?
not answering the question here (and it doesn't affect your problem), but CHECK_FOR_HEADER
should be false
if you're specifying your fields.
actually you shouldn't need a restart to change search-time extractions.
and yes I did restart the splunk instance for both the search head and the indexer.