Knowledge Management

Why am I not able to get any dynamic content using the collect command with the marker option?

bjoernjensen
Contributor

Hey,

I am trying to use collect together with the marker-Option. Unfortunately I am not able to get any dynamic content for the marker string:

index=_internal file=* | head 10 | table _time file | collect  index=test_temp marker=file

yields into _raw entries like this:

03/26/2015 23:59:27 +0100, info_search_time=1427410768.113, file=shelper, file

What I would like of course is the content of the field file and not the string file. I have already tried:
... marker='file'
... marker=\'file\'
... marker=\\'file\\'
... marker=\\\'file\\\'

But the marker is always set to the string.

I thought of using map-command. But this is very ugly since map starts a search for each event going into map (maxsearches could be adjusted, but .... naah)

I also tried to create a macro mycollect(2):

collect index=$index$ marker=$marker$

But the same result for either

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,file)`

or

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,'file')`

So, has anyone an idea?
Thanks in advance!

1 Solution

bjoernjensen
Contributor

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

View solution in original post

bjoernjensen
Contributor

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

masonmorales
Influencer

Right, defining it as a field using an eval is the way to go for your use case. As you saw in the docs, marker has to be a string, which is useful for separating out different sets of data in the summary index.

For example:

| collect index="mysummaryindex" marker="report=top_sales"

Allows me to search on that data using:

index=mysummaryindex report=top_sales

packman
Engager

I'm having the same issue, I get no error in the search but the data is never indexed if I use a field value instead of just text.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...