I have two Splunk 4.1.3 instances that index the same data. Some searches work on one instance but not the other. The searches that do not work on one instance will work if I add a wildcard.
Example event:
Oct 20 12:00:00 10.10.10.10 webserver01 apache_requests=123 instance=worker
Example searches:
1) search: apache
returns the event on both instance 1 and instance 2
2) search: apache_requests
returns the event on instance 1 but not instance 2
3) search: apache_
returns the event on instance 1 but not instance 2
4) search: apache_*
returns the event on both instance 1 and instance 2
5) search: apache_requests*
returns the event on both instance 1 and instance 2
Any idea why I have to include a wildcard in searches on instance 2?
Do the events show the same sourcetype on both instances?
Is segmentation (link) configured the same way for that sourcetype / source / host?
Thanks. Unfortunately, the person who configured the segmenter file on instance 2 no longer works at our company. I'll try to figure out the intent of the changes.
segmentation can be set indexer-wide, per-index, or per-sourcetype/source/host. What is right depends on your particular needs. However, I don't know why you would have different segmenter configuration on different indexes on the same version, unless someone just made a mistake. In that case, you should try to find out what the intended change was.
The [search], [standard], and [inner] stanzas in segmenters.conf differ between the two instances. Should I update the stanzas on instance 2 to match instance 1?