Getting Data In

Can you index evtx on linux heavy forwarder? 4.3

r999
Path Finder

Does not make it clear here:

http://splunk-base.splunk.com/answers/141/can-splunk-index-windows-event-logevtevtx-files

Does this absolutely have to be a windows forwarder?

Tags (2)
1 Solution

Kate_Lawrence-G
Contributor

Unfortunately yes - event files are actually binary and Splunk needs to utilize native Windows APIs to extract information from these files, you need to run Splunk on windows.

@Kate

View solution in original post

Kate_Lawrence-G
Contributor

Unfortunately yes - event files are actually binary and Splunk needs to utilize native Windows APIs to extract information from these files, you need to run Splunk on windows.

@Kate

rhysbee
New Member

Has there been any changes to this since newer releases of Splunk Enterprise 6.5 or 7.0.2?

We use Splunk Enterprise 6.5.0 on Linux for Index cluster and capturing live AD Domain Controller security logs using universal forwarder. For Audit purposes, we want to re-import some periods from archived .evtx files where live capture had failed.

I can successfully Index .evtx files on a standalone Windows Splunk Enterprise 7.0.2 server by setting up a folder monitor and Automatic source type.
When I tried on Windows Splunk Enterprise 6.5.1 server the folder monitor fails to index .evtx files at all.

Is there any way to forward these on to Linux Indexer?
Can this be achieved with the "indexAndForward" attribute in "outputs.conf"?

One option could be to copy bucket folders from Windows Splunk indexer to Linux Splunk indexer. Copy to linux - chown - then run repair, but we would like to automate/simplify the process as much as possible.

Alternately we have also looked at scripts using either logparser.exe / python-evtx / get-winevent and output to .csv/.xml but the formatting is different than existing events. Also logparser is unable to pull the EventRecordID field we are using to validate there are no missing records.

0 Karma

Kate_Lawrence-G
Contributor

Hmm...OK maybe we could tackle this in a different way.

The next things I'd try doing is going back to the universal forwarder on the windows boxens, and then a windows(there seem to be differing comments on whether it will work on linux) indexer with the http://splunk-base.splunk.com/apps/22315/splunk-app-for-windows app loaded?
This might be able to read those EVT files correctly?

It definitely sounds like an API related issue that the forwarder can't parse the files so the indexer is likely rejecting it as binary input. Maybe with the app on the indexer will do the trick.

0 Karma

r999
Path Finder

No this didnt work either. there must be some config to tell it to cook the data? the queue = winparsing didint work.

0 Karma

Chubbybunny
Splunk Employee
Splunk Employee

@bob999: try configuring your Windows forwarder to use an input like so.

[monitor://c:\import_exported_EVT\] 
host_segment = 3 
recursive = true 
queue = winparsing 
crcSalt = <source>
0 Karma

r999
Path Finder

thanks. i have tried with full splunk. still the same. do i need to configure anything special in inputs, props or transforms.conf to make sure it cooks the data first... i only have inputs.conf.

do i need to do something different in inputs.conf?

[monitor://$SPLUNK_HOME\evtmon
recursive = false
sourcetype = sevtx
index = indevtx

i tried [WinEventLog://$SPLUNK_HOME\evtmon but nothing gets forwarded

0 Karma

Kate_Lawrence-G
Contributor

well the universal forwarder will just forwarder data and NOT parse it (i.e. run it through the props/transforms actions) so it will not have the api available to parse and the linux indexer will not have the api either...so.

I'd recommend a heavy forwarder (or basically a full splunk instance with the web turned off) on the windows host in this case so that you can parse the data at read time and then forward it over already cooked to the linux indexer.

0 Karma

r999
Path Finder

i just tried on a windows universal forwader, forwading to a linux indexer.

the forwarder has read the file and sent to indexer but it has indexed in its binary format:

4:53:58.000 AM

\x00\x1\x00\x4\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x2!\x00\x4\x000\x00t'0@\x00\x00\x00\x00......... etc!

Does the indexer have to be windows too!!!?

0 Karma

Chubbybunny
Splunk Employee
Splunk Employee

additional details:
07-10-2012 11:36:11.180 -0700 INFO TailingProcessor - Ignoring file '/home/Chubbybunny/tmp/Sec_EVT.evt' due to: binary

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...