Need help to auto extract fields in non standard files

[anhtrantech]

Hello,
I am working on this for a while but i can't make it work correctly. I hope someone can help me to do this
I have multiple log files like bellow. It is not a standard log file.

Thank you very much for your help.

[esix_splunk · Accepted solution]

You should also set the sourcetype here at your inputs. This will be beneficial down the line for parsing. The sourcetype set here should match the props on the indexer/search head.

[anhtran]

Hello esix_splunk,
After I extracted the index=sqltem with the sourcetype=temp-log , i have starttime, endtime, user_id, dbname, instruments_processed, inst_skipped, error_nums

Then I have another index=jobinf with the sourcetype=jobinfo with the field
jobid
user_id
database
status
jobstarttime
jobfinishtime

As you see only user_id is a common field.

I would like to have a table with combining the fields on both index. How can I do that?

Thank you very much.

[esix_splunk]

First, make sure you are capturing this as a multiline event and the event boundaries are correct. For the fields, use the below in props.conf, change sourcetypename to match yours...

[sourcetypename]
SHOULD_LINEMERGE = true
EXTRACT-starttime = started at (?P<start_time>\d{2}\:\d{2}:\d{2})\,
EXTRACT-endtime = ended at (?P<end_time>\d{2}\:\d{2}:\d{2})\,
EXTRACT-user_id = User Id\: (?P<user_id>[^\s]+)
EXTRACT-dbname = DB Name: (?P<dbname>[^\s]+)
EXTRACT-instruments_processed = instruments processed\: (?P<instruments_processed>\d+)
EXTRACT-inst_skipped = instruments skipped\: (?P<instruments_skipped>\d+)
EXTRACT-error_nums = Number of errors\: (?P<error_num>\d+)
EXTRACT-warn_nums = Number of warnings\: (?P<warn_num>\d+)