One file with two sourcetype Need Regex

[hartfoml]

I have my DNS and DHCP logs in one file and I would like to set "TZ = UTC" on the sourcetype. My problem is what would the sourcetype be since the file has both DNS and DHCP in the file.

Here is an example of the logs:

Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: DHCP_RenewLease: Host= IP=x.x.x.x MAC=001b786eb865 Domain=ndc.nasa.gov ClientID=01001b786eb865
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Sent DHCPACK to Client MAC= 001b786eb865 ciaddr= x.x.x.x yiaddr= x.x.x.x client ID= 01001b786eb865
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.161 notify: zone 154.146.in-addr.arpa/IN: sending notifies (serial 98815)
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR started
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR ended

So I thought maybe i can create two sourcetypes with two monitors useing REGEX to monitor the one file for each of the diferent sourctypes.

Can I do this? What would the REGEX in the [monitor:///myfile] stanza in the inputs.conf look like to run the regex?

Example:

[monitor:////myfile/location]
TZ = UTC
regex="/opt/qip/usr/bin/dhcpd"
sourcetype=dhcp
host=nameserver.mydomain
index=network

Example:
[monitor:////myfile/location]
TZ = UTC
regex="named["
sourcetype=dns
host=nameserver.mydomain
index=network

[kristian_kolb]

Overlapping [monitor] stanzas in this way won't work. What you can do is to perform an index-time transform of the sourcetype. Assuming that you currently have one [monitor] that specifies one sourcetype, like this

[monitor:///path/to/file]
sourcetype=dns
other params here

Then on the indexer (or where your parsing takes place) you can do like so;

props.conf

[dns]
TRANSFORMS-blah = split_dhcp
other params here

transforms.conf

[split_dhcp]
REGEX = dhcpd\[
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::dhcp

For more info see;

http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

Hope this helps,

K

[kristian_kolb]

If the data passes through a Heavy Forwarder, that is where you must do this configuration. This operation is done during the parsing phase, which takes place only once.

See http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings

[hartfoml]

Thanks so much for the help. I will try it out right now.

One question if you don't mind

Can I do this on the heavy forwarder before it is sent to the indexer? I have 3 indexers do I have to setup the props and transforms on all three or can i just do this on the Heavy Forwarder