How to have certain fields displayed at the search level and another set of fields displayed at the report/dashboard panel level?

[andakun_222]

Sample Log:

[02.22.2017 03:48:33.985]  INFO - [CargoHub.com.aa.cargo.SPL.AirWaybillSCPSModule] TID[WMQJCAResourceAdapter : 7288] SID[sabre:AWBReplication] RID[601528076] [<== com.ibm.bpe.generated.Abstract_PT_ ==> MQ AWB Message Processing took :18666.0 milliseconds for AWB # 89536053]

As per the business requirement,

I want to extract two different kind of fields set, from the same log. In other words i want certain fields (common across all the log entries like logtime, loglevel etc) to be displayed at the search level and certain fields (event specific fields) to be displayed at the report/dashboard level. So I created two field extractions,

I created below field extraction, which displays the basic fields (logtime,loglevel) at search level.

\[(?.*)\]\s+(?.*) - \[(?.*)\] TID\[(?.*)\]\s+SID\[(?.*)\]\s+RID\[(?.*)\] [<== com.ibm.bpe.generated.Abstract_PT_ ==> MQ AWB Message Processing took :18666.0 milliseconds for AWB # 89536053]

I created below field extraction, which displays event specific fields and I want to show this values from Report/dashboard panel
and I don't want this fields to be available at search level under (selecting fields and interesting fields).

.*MQ AWB Message Processing took :(?.*) milliseconds for AWB # (?.*)]

Since i can only able to define field extraction at source, host or sourcetype level. So by default both basic fields and event specific fields are populating at search level itself. How to resolve it?

Is there any way to achieve my requirement through event type?

[DalJeanis]

If you don't want it to show up in "interesting fields", then you generally cannot define the extraction to the system. That means you have to build that particular extraction into the panels somehow instead.

As an alternative, you might want to consider field-level encryption by user role. https://docs.splunk.com/Documentation/Splunk/6.5.2/Security/UseaccesscontroltosecureSplunkdata

[gpullis · Accepted solution]

I'm thinking, do your "I only want it sometimes" extractions with the rex command and don't define them in your conf files.

http://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Rex

[andakun_222]

Thanks gpullis, i added rex command at query level and created report/dashboard out of it. Its working as expected.

[gpullis]

Are you doing this for security reasons or for performance reasons?

[somesoni2]

I don't think access to a saved field extraction can be managed at that granular level. A dashboard also runs a search and have access to same set of extracted fields. The best you can do would to have generic fields (that you want accessible from both search level and dashboard level) as saved field extraction, and for remaining, only dashboard specific fields, keep it inline in the dashboard search (or create macro for frequently used fields).

[andakun_222]

@somesoni2,
Thanks for your response.
The reason why we are trying to avoid getting the same fields to be displayed at both search and report/dashboard level is to avoid redundancy. Also we don't want the system to extract the event specific fields unnecessarily until an action to view the report or dashboard is made. Is there any way to define a regular expression which will be executed only when call made to report or dashboard instead of executing at generic level.