what is the splunk command that when search all and see all different kind of log as a whole?

[hoyeunglee]

what is the splunk command that when search all and see all different kind of log as a whole
and that can parse any delimiter and any format to get column name and value and result a big table like Excel
such as if there is no such column if come from different kind of log, it fill in empty.

xxxxxxxx, xxxxx product : hello
log xxxxx ,   xxxxx serial number=3000
xxxxxxxx, xxxxx product : hello2


product serial number
hello
                    3000
hello2

[woodcock]

I get it now, you probably are looking for kv (also known as extract😞
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Extract

Be sure to check out the commands in the See also section.

[hoyeunglee]

can you demonstrate how to become a table?

hostname | extract pairdelim="=", kvdelim="=" | multikv fields Hostname

is there any industrial experience knowledge that can extract any delimiter?

[woodcock]

using the same character for both delims is bound to be wrong; try this:

...| extract kvdelim="=:" | rename number AS serial_number

[hoyeunglee]

i find spacy that can be trained to identify ip address etc , originally i think splunk can identify header for values automatically.

[hoyeunglee]

sometimes log do not have column name , or it has column name but the delimiter is a space, it seems splunk is not using meta learning to automate get header and value pair in log. it need manually get header and value.

[woodcock]

If your event has a header row and then other rows (in the same event), like output from df command, then use multikv.

[jslee]

If you want to search all log, you must have "index=* | " in your SPL

[woodcock]

You probably mean | table * but if not, perhaps you might like fieldsummary or maybe even cluster.

[DalJeanis]

@hoyeunglee - Lots of good here. They probably told you what you need to know if we understood your question right.

I'm betting that @woodcock's fieldsummary is the one you are looking for.

The one suggestion that I'd add is that you should put a |head 10 or | dedup index at the beginning before | fieldsummary, so that you don't end up with a completely unmanageable result.

[hoyeunglee]

i find meta learning can do data preprocessing such as auto pick column and value, is splunk using this?

[hoyeunglee]

i updated question, actually i mean a big excel table after parse with regex for any format and any delimiter

[HiroshiSatoh]

search all

->index=* OR index=_*

However, if all indexes have authority.

all different kind of log

->stats count,dc(source),values(source) by sourcetype

An example.

such as if there is no such column if come from different kind of log, it fill in empty.
->Except for the default field, it is defined for each source and source type. It will not be displayed if it does not meet the conditions.
https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Data/Aboutdefaultfields

[hoyeunglee]

i updated question, actually i mean a big excel table after parse with regex for any format and any delimiter