Why am I not able to get any dynamic content using the collect command with the marker option?

[bjoernjensen]

Hey,

I am trying to use collect together with the marker-Option. Unfortunately I am not able to get any dynamic content for the marker string:

index=_internal file=* | head 10 | table _time file | collect  index=test_temp marker=file

yields into _raw entries like this:

03/26/2015 23:59:27 +0100, info_search_time=1427410768.113, file=shelper, file

What I would like of course is the content of the field file and not the string file. I have already tried:
... marker='file'
... marker=\'file\'
... marker=\\'file\\'
... marker=\\\'file\\\'

But the marker is always set to the string.

I thought of using map-command. But this is very ugly since map starts a search for each event going into map (maxsearches could be adjusted, but .... naah)

I also tried to create a macro mycollect(2):

collect index=$index$ marker=$marker$

But the same result for either

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,file)`

or

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,'file')`

So, has anyone an idea?
Thanks in advance!

[bjoernjensen · Accepted solution]

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

[masonmorales]

Right, defining it as a field using an eval is the way to go for your use case. As you saw in the docs, marker has to be a string, which is useful for separating out different sets of data in the summary index.

For example:

| collect index="mysummaryindex" marker="report=top_sales"

Allows me to search on that data using:

index=mysummaryindex report=top_sales

[packman]

I'm having the same issue, I get no error in the search but the data is never indexed if I use a field value instead of just text.