How do I edit my search to include an app in a list that is using source=metrics.log?

[sunnyparmar]

Hi,

All my apps are indexing their data _internal index and using source=license_usage.log, but one of my apps is using source=metrics.logs. Now because of this, in the final result, all apps are showing except the one using the metrics.log source.

My question is: How can I include the metrics.log source file in my below given search so that one app will also be become the part of final list of results? Any comments would be appreciated.

My search is:

index=_internal source=*license_usage.log OR source=/opt/splunk/var/log/splunk/*metrics.log type="Usage" | eval indexname = if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx)  | eval sourcetypename = st  | bin _time span=1d  | stats sum(b) as b by _time, pool, indexname, sourcetypename | eval GB=round(b/1024/1024/1024, 3)| fields _time, indexname, sourcetypename, GB |  join sourcetypename [       | rest /services/saved/sourcetypes       | fields title, "eai:acl.app"       | rename title AS sourcetypename, "eai:acl.app" AS app_name  ] | chart sum(GB) by app_name

Thanks in Advance

[javiergn]

Hi, if you just want to find out how much license each app is consuming then take a look at my answer here:

https://answers.splunk.com/answers/351898/how-do-i-search-the-license-usage-by-sourcetype-fo.html

The query is just as follows:

index=_internal source=*license_usage.log type="Usage" 
  | eval indexname = if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx)
  | eval sourcetypename = st
  | bin _time span=1d 
  | stats sum(b) as b by _time, pool, indexname, sourcetypename
  | eval GB=round(b/1024/1024/1024, 3)
  | fields _time, indexname, sourcetypename, GB
  | join sourcetypename [
      | rest /services/saved/sourcetypes
      | fields title, "eai:acl.app"
      | rename title AS sourcetypename, "eai:acl.app" AS app_name
 ]

[sunnyparmar]

thanks for replying but as i mentioned earlier in my question that one of my app is using source = metrics.log and rest of the apps using license_usage.log source so in this case by your above given query which i am using currently not enlisting that app which is using source = metrics.log. Even in your query you have defined only source=*license_usage.log so do you have any idea here that how i would add one more source (metrics.log) in the same query.

Thanks

[javiergn]

What about this?

(index=_internal source=*license_usage.log type="Usage") OR 
((index=* OR index=_*) source=/opt/splunk/var/log/splunk/*metrics.log)
   | eval indexname = if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx)
   | eval sourcetypename = st
   | bin _time span=1d 
   | stats sum(b) as b by _time, pool, indexname, sourcetypename
   | eval GB=round(b/1024/1024/1024, 3)
   | fields _time, indexname, sourcetypename, GB
   | join sourcetypename [
       | rest /services/saved/sourcetypes
       | fields title, "eai:acl.app"
       | rename title AS sourcetypename, "eai:acl.app" AS app_name
  ]

[sunnyparmar]

sorry but still not showing the result for missing app. thanks

[javiergn]

Hi,

Sorry it's not easy to simulate your exact configuration.
Would you be able to provide an event sample for the missing data?
Ideally I would like to see: index name, sourcetype, source, app name, and any other field you find relevant.

Also, if you run the following:

| rest /services/saved/sourcetypes
| fields title, "eai:acl.app"
| rename title AS sourcetypename, "eai:acl.app" AS app_name

Is your app there by any chance?

Thanks,
Javier

[sunnyparmar]

Hi Javier,

Sorry for replying late. Get into the deep and dig some useful information.

1. My missing apps are not enlisted in the final result of your query that you have provided me above.
2. One of my missing app dashboard showing all system logs related data like basic hosts metrics, cpu load and memory etc. Data is indexing for this app in _internal but only related to splunk forwarder logs i.e. metrics and splunkd logs.
3. In my second app, it is not using indexes, all data are fetching via sourcetype and eventtype through queries in panels and besides this it is not indexing its data neither in _internal nor in main.

Now could you please let me know on the basis of my above question that what type of data inputs you want from my side?

Thanks
Sunny

[javiergn]

Hi,

I still don't get what you are trying to achieve. I also want to clarify the following comment you made earlier:

3-In my second app, it is not using indexes, all data are fetching via sourcetype and eventtype through queries in panels and besides this it is not indexing its data neither in _internal nor in main.

All your data is part of an index. You don't always have to explicitly use it when running a search, but it'll be there, trust me. Unless it's coming from lookups or external sources, there's always an index associated to your events.

Also, please keep in mind internal indexes (those that start with underscore) are not counting towards your daily license usage. They don't consume your license and that's the reason you can't create or inject data into them. Otherwise it would be very easy to bypass the Splunk license. If your missing app is only consuming data from internal indexes you won't be able to see it with the license usage queries

Let me know if that helps.

If not, please simply provide a few bullet points summarising what you are trying to achieve. It is not clear though. Are you trying to estimate the license usage per app and sourcetype? Anything else?

If you app wasn't listed in the query above try the following one then:

| rest /services/apps/local 
| rename label as AppName
| table AppName, version

Thanks,
J

[sunnyparmar]

Hi J,

I have started getting data for qv-monitoring app in _internal index. I have changed my searched type so now i am getting data in _internal by this way and i am going to paste some sample data below so might be it can help you

02-11-2016 08:30:29.824 +0000 INFO LicenseUsage - type=Usage s="WinEventLog:System" st="WinEventLog:System" h=BWQV o="" idx="qv-winevents" i="9F5915D4-4E58-41BA-A379-4912FE7CD558" pool="auto_generated_pool_enterprise" b=378 poolsz=33285996544

02-11-2016 08:42:37.798 +0000 INFO LicenseUsage - type=Usage s="Perfmon:CPUTime" st="Perfmon:CPUTime" h=BWQV o="" idx="qv-perfmon" i="9F5915D4-4E58-41BA-A379-4912FE7CD558" pool="auto_generated_pool_enterprise" b=1629 poolsz=33285996544

so could you now adjust it with the main query so that i can get license usage data for this app as well.

Thanks

[javiergn]

Hi, there's not too much to adjust, it'll be basically be the first query I posted above. Please remember this is based on data sources using non internal indexes and therefore consuming license:

index=_internal source=*license_usage.log type="Usage" 
| eval indexname = if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx)
| eval sourcetypename = st
| bin _time span=1d 
| stats sum(b) as b by _time, pool, indexname, sourcetypename
| eval GB=round(b/1024/1024/1024, 3)
| fields _time, indexname, sourcetypename, GB
| join sourcetypename [
       | rest /services/saved/sourcetypes
       | fields title, "eai:acl.app"
       | rename title AS sourcetypename, "eai:acl.app" AS app_name
]

[sunnyparmar]

ok thanks J..

Regards

[sunnyparmar]

Hi J,

Thanks for replying. I executed your query and in this my both apps are showing
1. Alusta tenant validation with version 16.1
2. Q.V monitoring with version 0.1

I am trying to find license usage by app and when i am executing the main query of yours my all apps are showing except the above two. If I will split it into only this much part

(index=_internal source=*license_usage.log type="Usage")

which is the part of our main query, then it is showing the indexes of my qv-monitoring app but when we added rest of the query with it then it is not showing license usage for my qv-monitoring app.

Now let's talk about second app i.e. Alusta Tenant validation, all the dashboard which comes under this app don't have indexes, all queries are running by sourcetype and eventtypes so in this case how could we calculate license usage for this app?

and last but not least as you said above that we can't count license usage from internal indexes so what does this parameter means? Please explain me why we use index=_internal and source=*license_usage.log for calculating license as i have not clear perspective about this?

I hope my answers can help you to troubleshoot the case. Thanks for your extensive help.

Thanks