I have the same sourcetype log files separated in different directories by day, and different files for every hour. Some of the log files are only a few lines, while some are thousands of lines. I cannot control how the log files are stored.
The issue is that when I monitor the directories, I get two separate indexes: **sourcetype* and sourcetype-too_small.*
What I want is one index, **sourcetype, which contains every event from all the files.
I do not care if the file is too small, since it should be merged with all the other files anyway.
How do I achieve this?
Thanks.
From your question "The issue is that when I monitor the directories, I get two separate **indexes: sourcetype and sourcetype-too_small." , did you mean two sourcetypes not indexes?
Splunk will automatically assign a source type if you don't provide it explicitly in your inputs.conf.
If you want all log files from the directories under same sourcetype , include the sourcetype parameter also in your inputs.conf
For eg:
[monitor:///apache/.../*.log]
index=<your index name>
sourcetype=<your sourceype>
Does it help?