Splunk Search

Setting fields from logs with different row values

juancarlos_pola
Explorer

Hello everybody,

I have a question that might have been responded before but I have a log file from a server that looks like this

10:01pm load_disk:  200
10:01pm sessions: 2
10:01pm maxpage: 201
10:01pm datadisk: 30
10:01pm memory: 10
10:02pm load_disk:201
10:02pm sessions: 3
10:02pm maxpage: 202
10:02pm datadisk: 31
10:02pm memory: 11
....

As you can see, it is a log file that shows many different values on each row but the values repeat (in this case) every 5 rows. I would like to extract a field for each line that defines a log:
field1 -> load_disk
field2 -> sessions
...

When I try to extract using the Extract Field option in the search, it does not show me all the lines of the log ( this log has a huge number of lines that are different and repeat ) maybe because of window size but I cannot see all the lines of the log (before repeating).

Best regards,

Juan

Tags (2)
0 Karma

lguinn2
Legend

You could do this:

in props.conf

[yoursourcetypehere]
REPORT-eaf=extract-all-fields

in transforms.conf

[extract-all-fields]
FORMAT = $1::$2
REGEX  = (\S+?)\s*:\s*(\S+)

This should give you all the field extractions.

0 Karma

tom_frotscher
Builder

I do not completely understand your question. Do you want to put all fields, in this case load_disk, sessions, maxpage, datadisk and memory on one row?

So you want to gor from your example to something like this:

10:01pm load_disk: 200, sessions: 2, maxpage: 201, datadisk: 30, memory: 10
10:02pm load_disk: 201, sessions: 3, maxpage: 202, datadisk: 31, memory: 11
0 Karma

juancarlos_pola
Explorer

Hello Tom,

Thank you for your reply and I am sorry for the long silence.

What I actually wanted from the previous log was to set a field for each log variable as follows:

field1 = load_disk
field2 = sessions
field3 = maxpage
field4 = datadisk
field5 = memory

so I can draw a line chart with all these fields inside one chart:

source=/admin/server/* | timechart first(field1) first(field2) avg(field3) first(field4) avg(field5)

My issue now is that the REAL log has more than 80 different variables and they repeat every 10 minutes, so I would like to set a field for each log variable. I saw some usage of the regex editor but I am quite new in Splunk so I would like some advice or if you have had previous experience with such kind of log files

Thank you in advance.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...