Hi,
I am looking to extract fields from multi-line events. Some of the events are more than 20 lines. When I am trying to extract events, it trims out the event with more than 20 lines. I guess it's its limit (See attachment).
Is it possible to extract a field from an event with more than 20 lines ? I am looking to extract a field from the last 2 lines.
Event with more than 25 lines:
2019-01-21T14:54:51.774+0100: 344911.303: [GC pause (G1 Evacuation Pause) (young), 0.0082884 secs]
[Parallel Time: 3.0 ms, GC Workers: 11]
[GC Worker Start (ms): Min: 344911303.7, Avg: 344911303.8, Max: 344911304.5, Diff: 0.8]
[Ext Root Scanning (ms): Min: 0.0, Avg: 0.6, Max: 0.7, Diff: 0.7, Sum: 6.5]
[Update RS (ms): Min: 0.0, Avg: 1.0, Max: 2.1, Diff: 2.1, Sum: 11.4]
[Processed Buffers: Min: 0, Avg: 13.1, Max: 24, Diff: 24, Sum: 144]
[Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.2]
[Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[Object Copy (ms): Min: 0.0, Avg: 1.0, Max: 1.9, Diff: 1.8, Sum: 10.7]
[Termination (ms): Min: 0.0, Avg: 0.1, Max: 0.2, Diff: 0.2, Sum: 1.6]
[Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 11]
[GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.2]
[GC Worker Total (ms): Min: 2.1, Avg: 2.8, Max: 2.9, Diff: 0.8, Sum: 30.6]
[GC Worker End (ms): Min: 344911306.6, Avg: 344911306.6, Max: 344911306.6, Diff: 0.0]
[Code Root Fixup: 0.0 ms]
[Code Root Purge: 0.0 ms]
[Clear CT: 0.3 ms]
[Other: 5.0 ms]
[Choose CSet: 0.0 ms]
[Ref Proc: 3.7 ms]
[Ref Enq: 0.3 ms]
[Redirty Cards: 0.3 ms]
[Humongous Register: 0.0 ms]
[Humongous Reclaim: 0.0 ms]
[Free CSet: 0.1 ms]
[Eden: 129.0M(129.0M)->0.0B(127.0M) Survivors: 6144.0K->7168.0K Heap: 223.2M(256.0M)->95.0M(256.0M)]
[Times: user=0.03 sys=0.00, real=0.01 secs]
Hello,
You can use the rex
command to field during search time. https://docs.splunk.com/Documentation/Splunk/7.2.3/SearchReference/Rex
Conversely, you can extract fields by clicking on an event event actions >> extract fields
. You can get more information from the following documentation
https://docs.splunk.com/Documentation/Splunk/7.2.3/Knowledge/ExtractfieldsinteractivelywithIFX
You can also use regex101.com to test your regex code
@agoyal
Did you try to set TRUNCATE value in propos.conf?
I am not using TRUNCATE and I guess using TRUNCATE won't work in this case. I can see full event is ingested. it just when I am trying to do field extract I can't see full event.
I have tried TRUNCATE = 0 but no impact.
props.conf
[G1_BETA]
MAX_TIMESTAMP_LOOKAHEAD = 30
BREAK_ONLY_BEFORE = ^\d\d\d\d
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1
disabled = false
Its possible, the splunk extractor is not showing all 20lines. you can use EXTRACT-yourfield to extract what you need. Pls check regex101.com and add the regex directly in the props and test it out.