how to index only some filed

[davidepala]

hi all! i'm collecting some events from windows security log. As i understand the index volume is proportional to the number of field in every single event. my question is: there is a way to index only some field? i'll reduce the index volume with this selection?

[divyanshukakwan · Accepted solution]

You can remove the fields that you don't want to be indexed. To do this, create a new stanza in props.conf

[<spec>]
SEDCMD-filter = s/<pattern to search field>//g

This will substitute the fields with empty string.

In your case, you can do something like this in SEDCMD (assuming the fields are tab separated:

SEDCMD-filter = s/(.+)\t(.+)\t(.+)\t(.+)/\1\2/g 

depending on which fields you want to select, modify the replacement part of the sed string

Hope it helps.

[bangalorep]

As you've already indexed large amounts of data, from your description i think you're looking at discarding specific data and keeping the rest then look at this doc:
http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad#Discard_specif...

[davidepala]

tnx for the info but i want extract only some field BEFORE the index ...for example:

2018/24/01-10:53 EventID 4028 Source Security Message This is the text of the event Key 23

i want index only

2018/24/01-10:53 EventID 4028 Source Security

[493669]

Hi @davidepala,
You can Keep specific events and discard the rest using setnull and setparsing
Keeping only some events and discarding the rest requires two transforms. In this scenario, which is opposite of the previous, the setnull transform routes all events to nullQueue while the setparsing transform selects the sshd events and sends them on to indexQueue.

As with other index-time field extractions, processing of transforms happens in the order that you specify them, from left to right. The key difference is the order in which you specify the stanzas. In this example, the setnull stanza must appear first in the list. This is because if you set it last, it matches all events and sends them to the nullQueue, and as it is the last transform, it effectively throws all of the events away, even those that previously matched the setparsing stanza.

When you set the setnull transform first, it matches all events and tags them to be sent to the nullQueue. The setparsing transform then follows, and tags events that match [sshd] to go to the indexQueue. The result is that the events that contain [sshd] get passed on, while all other events get dropped.

1. Edit props.conf and add the following:

   [source::/var/log/messages]
   TRANSFORMS-set= setnull,setparsing

2. Edit transforms.conf and add the following:

   [setnull]
       REGEX = .
       DEST_KEY = queue
       FORMAT = nullQueue
   

   [setparsing]
   REGEX = [sshd]
   DEST_KEY = queue
   FORMAT = indexQueue

3. Restart Splunk Enterprise.
   Refer http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad