I am trying to extract a field from a Windows event which can contain multiple values. At the search line I can do this easily, but I am having trouble setting this up to happen automatically. An example of the event I want to extract from is below. I basically want to extract the SAN name fields. SAN:DNS=amazonaws.com&DNS=*.amazonaws.com&DNS=*.us-east-1.amazonaws.com&DNS=*.us-west-2.amazonaws.com&DNS=*.us-west-1.amazonaws.com&DNS=*.eu-west-1.amazonaws.com&DNS=*.ap-southeast-1.amazonaws.com&DNS=*.ap-southeast-2.amazonaws.com&DNS=*.ap-northeast-1.amazonaws.com&DNS=*.sa-east-1.amazonaws.com CertificateTemplate:xx UserAgent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.59 Safari/537.36 ccm:xx.xx.xx.xx Disposition: 3 SKI: xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx Subject: CN=amazonaws.com, OU=xxx, O=xxx, L=xxx, S=xxx, C=xx Using the following search from the Splunk search box everything works fine. index="CA_Logs" host=ca1 msad_action="approved a certificate request and issued a certificate." | rex max_match=50 "(?ms)(DNS|IPAddress)=(?<san>[\w\.\-\*]+)" It correctly extracts a field called "san" and finds all of the following SAN's: amazonaws.com *.amazonaws.com *.us-east-1.amazonaws.com *.us-west-2.amazonaws.com *.us-west-1.amazonaws.com *.eu-west-1.amazonaws.com *.ap-southeast-1.amazonaws.com *.ap-southeast-2.amazonaws.com *.ap-northeast-1.amazonaws.com *.sa-east-1.amazonaws.com I've tried setting this up via the Splunkweb: Settings > Fields Field Extractions Name: WinEventLog:Security : EXTRACT-CA-SAN-fields Type: Inline Extraction/Transform: (?ms)(DNS|IPAddress)=(?[\w.-*]+) I've tried adding max_match=50 to this but it breaks the match as I assume max_match=50 becomes part of the regex. Settings > Fields Field transformations Name: CA_SAN_MV Regular Expression: san="(?ms)(DNS|IPAddress)=(?[\w.-*]+)" or Regular Expression: (?ms)(DNS|IPAddress)=(?[\w.-*]+) or Regular Expression: "(?ms)(DNS|IPAddress)=(?[\w.-*]+)" Source Key: _raw or Source Key: Message Create multivalued fields: Ticked Automatically clean field names: Ticked Via Field Extractions it works but only catches the first instance of SAN=xxxx. ... View more

I would like to find the first and last event per day over a given time range. So far I have figured out how to find just the first and last event for a given time range but if the time range is 5 days I'll get the earliest event for the first day and the last event on the last day. I'm just using the _time field to sort the date. I've tried the following. index="security" user="123456" EventCode=4624 OR EventCode=4634 earliest=-7d@d | stats earliest(_time) AS Earliest, latest(_time) AS Latest | eval FirstEvent=strftime(Earliest,"%+") | eval LastEvent=strftime(Latest,"%+") This produces kind of what I want except over the 7 day time range. FirstEvent LastEvent Fri Aug 1 11:10:25 EST 2014 Fri Aug 8 05:57:06 EST 2014 If someone could help me figure out how to do it day by day like below that would be excellent. FirstEvent LastEvent Fri Aug 1 11:10:25 EST 2014 Fri Aug 1 16:57:06 EST 2014 Fri Aug 2 11:10:25 EST 2014 Fri Aug 2 16:57:06 EST 2014 Fri Aug 3 11:10:25 EST 2014 Fri Aug 3 16:57:06 EST 2014 Fri Aug 4 11:10:25 EST 2014 Fri Aug 4 16:57:06 EST 2014 And so on. ... View more

I am having a problem with field extraction of some Windows event logs. I have an example log below. 08/05/2014 09:59:00 AM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4887 EventType=0 Type=Information ComputerName=issuingca.my.domain TaskCategory=Certification Services OpCode=Info RecordNumber=1981173 Keywords=Audit Success Message=Certificate Services approved a certificate request and issued a certificate. Request ID: 5001 Requester: my\user1 Attributes: CertificateTemplate:DomainInternalIPSECv1 ccm:issuingca.my.domain Disposition: 3 SKI: aa aa d2 64 ef a1 46 0f e2 2e 92 aa ce ec 1d 81 99 4b aa aa Subject: CN=TestSubject So if I search for this and use rex to extract the fields I don't get any problem. Example below. index="ca_logs" host="issuingca*" EventCode=4887 | rex "(?ms)EventCode=4887\s*\n.*Message=Certificate Services (?<action>.*?)[:\.]\s*\n.*?Requester:\s*(?<requester_domain>[^\\\\]+)\\\\(?<requester_name>.*?)\s*\n.*CertificateTemplate:(?<certificate_template>.*?)\s*\n.*" Running the above, I extract the fields I'm expecting. I get: action requester_domain requester_name certificate_template So I then create a new field extraction on Splunk web. Settings > Fields > field extractions > New. Destination app = activedirectory Name = 4887-fields Apply to = sourcetype = WinEventLog:Security Type = inline Extraction/Transform = (?ms)EventCode=4887\s*\n.*Message=Certificate Services (?<msad_action>.*?)[:\.]\s*\n.*?Requester:\s*(?<requester_domain>[^\\\\]+)\\\\(?<requester_name>.*?)\s*\n.*CertificateTemplate:(?<certificate_template>.*?)\s*\n.* It creates a new rule called "WinEventLog:Security : EXTRACT-4887-fields". But when I search "index="ca_logs" host="issuingca*" EventCode=4887", I don't get the field extraction occurring. I've set the permissions so that its available to the activedirectory application (this app only instead of private). Can anyone shed light on what I'm doing wrong? ... View more

Hi All, I have a scenario where I am indexing event logs from Windows servers across 5 different time zones: Australia/Queensland Australia/NSW NZ Australia/South Australia/Victoria Australia/West All of the servers are sending their security log to forwarders and from there they are sent to the indexers. So I've got an issue where when I search logs in real time nothing is showing up, I've got to look forwarder 2 hours or backward 2 hours to see logs coming in. I'm trying to figure out how on a forwarder I can set the TZ for specific servers. Looking at http://docs.splunk.com/Documentation/Splunk/4.1.8/admin/ApplyTimezoneOffsetstotimestamps, it looks like it should be simple but I can't seem to figure it out. Is there something I can put on the UniversalForwarder config or on the Indexer that will allow me to apply a Time Zone to specific servers? Reading the article above it seems to suggest the Time Zone should be that of the server sending the data. But they all hit the props stanza [WinEventLog://Security] so I'm a bit stuck. My forwarder configuration is as follows. inputs.conf [splunktcp://9997] disabled = 0 props.conf [WinEventLog:Security] TRANSFORMS-security = security_null, security_truncate, security_index transforms.conf [security_null] REGEX = . DEST_KEY = queue FORMAT = nullQueue [security_truncate] REGEX = (?msi)(.*)(This event is generated|Certificate information is only provided) DEST_KEY = _raw FORMAT = $1 [security_index] REGEX = (?msi)^EventCode=(list of event Id's removed for brevity)\D DEST_KEY = queue FORMAT = indexQueue Everything is working excellent, except how can I specify a different timezone for different servers in the props.conf if the stanza is looking for [WinEventLog:Security]? The configuration for the servers is all identical. inputs.conf [WinEventLog://Security] checkpointInterval = 5 current_only = 1 disabled = 0 index = security start_from = oldest outputs.conf [tcpout] defaultGroup = forwarder_9997 [tcpout:forwarder_9997] server = forwarder:9997 [tcpout-server://forwarder:9997] ... View more

I am having some issues getting my heavy forwarder to forward events. The configuration I'm trying to achieve is as follows. Server running universal forwarder, forward all security logs to Heavy Forwarder | Heavy Forwarder receives the events, filters them on event id and forwards them to the indexer | Logs get indexed on the indexer (in a index called test_index) I can get the logs to go from the UF to the HF, and if I set the HF to store a local copy of forwarded events I can see the events logging on the forwarders local index. But my filtered logs from the heavy forwarder are not getting sent to the indexer. Basic example of the configuration I'm trying to get working is for a server running the UF to send event logs to the HF and filter the logon and logoff event ids 4624 and 4634 and forward these to the indexer. What am I doing wrong? 🙂 Universal Forwarder Configuration inputs.conf [WinEventLog:Security] index = test_index disabled = 0 start_from = oldest current_only = 1 checkpointInterval = 5 outputs.conf [tcpout] defaultGroup = heavyforwarder_9997 [tcpout:heavyforwarder_9997] server = heavyforwarder:9997 [tcpout-server://heavyforwarder:9997] Heavy Forwarder Configuration inputs.conf [WinEventLog:Security] disabled = 0 start_from = oldest current_only = 1 checkpointInterval = 5 #index = test_index outputs.conf [tcpout] defaultGroup = splunkindexer_9997 indexAndForward = 1 [tcpout:splunkindexer_9997] autoLB = true server = splunkindexer:9997 props.conf [WinEventLog:Security] TRANSFORMS-security= events-null, events-filter transforms.conf [events-filter] REGEX=(?msi)^EventCode=(4624|4634)\D DEST_KEY = queue FORMAT = indexQueue [events-null] REGEX = . DEST_KEY = queue FORMAT = nullQueue ... View more