How Can I have Sources or SourceTypes w/o Events Appear in Search Results?

[andrewkenth]

I have some files (sources w/ configured sourcetypes) that often times do not contain any events. Is there a way that I can have the search show that the file was attempted to be read but no events were listed? Currently I'm looking at all files by date (actually removing the date from the filename...) overtime in order to asertain if I am missing any files. However, when the file has no events it does not appear in my search results. I'd like files that had no events but were read to appear in my search results.

Example Search String:
index=charlesriver | bucket span=1d _time | stats count first(_time) as Date by _time, source | eval Date=strftime(Date,"%m/%d/%Y") | eval source=replace(source,"/apps/wcm-splunk/work/crd/prod/","") | eval source=substr(source,1,len(source)-13) | table Date, source, count | chart count over source by Date | sort -Date

[lguinn2]

Yes you can. However, the source must have had data at some time in the past. This won't help you identify data that has never been indexed.

| metadata type=sources index=charlesriver 
| join source type=outer [ search index=charlesriver 
    | bucket span=1d _time 
    | stats count  by _time, source 
    | eval Date=strftime(_time,"%m/%d/%Y")  
    | stats count over source by Date ]
| eval source=replace(source,"/apps/wcm-splunk/work/crd/prod/","")  
| eval source=substr(source,1,len(source)-13) 
| sort -Date
| xyseries source Date count

This is a bit complex, but it should perform okay. There are two searches: the outer search simply identifies all the sources that have ever been indexed in charlesriver. The inner search does the same as your original search - it counts all the sources by date. The results of the two searches are joined together, and then the source string is formatted. You should probably run this once without the final two commands, just to see the output before xyseries formats it.

[somesoni2]

If the files don't have the events during first time itself (means the data input was configured with/for empty file), searching _internal index is the only way to get those files. If the file was configured and had data earlier (data got indexed once, may be log back,which doesn't fall into search's timeframe) and now there are no data now (rolled over file or new file dumped everytime), then it is possible to get list of such files using metadata command.

[gfuente]

Hello

I think you are not going to get this with a "regular" search, as you have not any events to look for. Instead of that, you should search in the _internal index, and search for the indexers saying about which files are being monitored. Something like:

index="_internal" "stanza" | rex "monitor\:\/\/(?<file>[^\r]+)" | table host, file

This will give you a list of host and files beign monitored, regardless of being indexing events from those files

regards