I'm trying to extract fields from a log and failing miserably.
In my first attempt I used a props.conf to specify the delimiter and field names:
[ipoz]
FIELD_NAMES = "Priority","Date","Thread","Category","Message"
FIELD_DELIMITER="\t"
That didn't work for some reason, so I tried using props and transforms first by specifying the delimiter/fields again, and later switching to regular expression like so:
<props.conf>
[ipoz]
REPORT-IPOZ=IPOZ-DELIM
<transforms.conf>
[IPOZ-DELIM]
#DELIMS="\t"
#FIELDS="Priority","Date","Thread","Category","Message"
REGEX=(?<Priority>.+)\t(?<Date>.+)\t(?<Thread>.+)\t(?<Category>.+)\t(?<Message>.+)
I have some control over the formatting of the log, so I can change the delimiter, but since the 5th field can contain commas I feel like using tab for a delimiter is the right choice.
Can anyone help with this??? I have confirmed that the log is indeed tab delimited by checking the logger configuration (log4j), pasting the log into Notepad++ and showing characters, and using a regex tester to validate the regex.
Here is the relevant section of inputs.conf
[monitor://<path to the not so very good software company>\ipoz.log]
index=casuite
sourcetype=ipoz
disabled = 0
queue = parsingQueue
Below is a sample of the log I am trying to parse.
ERROR 2019-08-21 10:53:32,386 [0x00001cb4] [eiam.server.ipoz.sponsorinterfacev1] [src/Poz.cpp:2047] bool __cdecl eiam::server::poz::Poz::detach(const class eiam::core::String &)
INFO 2019-08-21 11:19:32,821 [0x0000222c] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
ERROR 2019-08-21 11:23:33,224 [0x00002358] [eiam.server.ipoz.sponsorinterfacev1] SponsorInterfaceV1::clientDetach: detach failed [sessionId: 43d70a8e665b51f5363dc44ad1f5537d-5d5c41dd-cb06650-1e2, clienthost: 10.33.52.44:60103]
ERROR 2019-08-21 11:23:33,224 [0x00002358] [eiam.server.ipoz.sponsorinterfacev1] Exception[-704]: session expired
ERROR 2019-08-21 11:23:33,224 [0x00002358] [eiam.server.ipoz.sponsorinterfacev1] [src/SessionManager.cpp:286] class eiam::server::dirobj::Session *__cdecl eiam::server::poz::SessionManager::retrieveSession(const class eiam::core::String &)
ERROR 2019-08-21 11:23:33,224 [0x00002358] [eiam.server.ipoz.sponsorinterfacev1] [src/Poz.cpp:3433] class eiam::server::dirobj::Session *__cdecl eiam::server::poz::Poz::retrieveSession(const class eiam::core::String &)
ERROR 2019-08-21 11:23:33,224 [0x00002358] [eiam.server.ipoz.sponsorinterfacev1] [src/Poz.cpp:2047] bool __cdecl eiam::server::poz::Poz::detach(const class eiam::core::String &)
INFO 2019-08-21 11:42:41,448 [0x00000bec] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:09:29,304 [0x000012b0] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:10:15,716 [0x00001150] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:14:52,428 [0x000003b8] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:19:49,863 [0x00001e48] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:24:26,612 [0x00001858] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
INFO 2019-08-21 13:27:33,143 [0x00001698] [eiam.server.ipoz.sponsor] Sponsor::Sponsor: Poz initialized
... View more