Is there a specific reason you want the data to route through heavy forwarders?  If it isn't because of fears of some sort of network straining, and you aren't doing parsing, it may make sense to just forward directly to Splunk Cloud.  That will make the load on your heavy forwarders a bit more predictable and may allow you to have fewer.  I know at one point you could only have one set of certs per instance of splunkd. ... View more

I often find these modular input views struggle to load when the system is under-powered or under heavy load.  Obviously that's not the only cause, but it may be worth looking into if you aren't sure there aren't issues there. ... View more

Are you able to telnet to port 8000 on the Splunk server?  Are you able to connect to any other ports?  I am assuming yes.  Additionally, if it is a fresh install of Splunk Enterprise then HTTPS is likely disabled.  If your browser is automatically rerouting all requests to HTTPS, you may not be able to connect.  Turn off anything like HTTPS Everywhere or any applicable browser settings and try to reach the HTTP page explicitly.  Lastly, from my experience with Splunk on Ubuntu is that it comes with UFW enabled and not iptables.  Have you double checked there isn't another host based firewall? Apologies if these seems simple, but you've already done a lot of troubleshooting steps that I would perform. ... View more

@frapei It looks like the issue is that some events start with a 3 digit code and some don't.  If that isn't due to the parser, and the regex from @asridhara does not work, you need to address those cases. ... View more

[vmw-syslog] Tranforms-routing=vmwarelogs,discarlogs [vmwarelogs] REGEX=(logged out|Rejected password for user|Cannot login|logged in as|Accepted user for user|was updated on host|Password was changed for account|Destroy VM called) DEST_KEY=_TCP_ROUTING FORMAT=target1 [discarlogs] REGEX = . DEST_KEY = queue FORMAT = nullQueue   I have not been able to test on my own, but I believe your problem lies with trying to apply both transforms with one line of props.  It is possible for your events to match on both, so the the events are sent to the null queue, overriding the assignment to TCP ROUTING.  I would try reversing the order as @saravanan90 suggested, but also reducing the complexity of your [vmwarelogs] regex to one kind of event, until you have the routing down, then expand it to cover the other events. The example given in Splunk documentation supports this order. Props [source::/var/log/messages] TRANSFORMS-set= setnull,setparsing Transforms [setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setparsing] REGEX = \[sshd\] DEST_KEY = queue FORMAT = indexQueue https://docs.splunk.com/Documentation/Splunk/8.1.2/Forwarding/Routeandfilterdatad#Filter_event_data_and_send_to_queues Lastly, check to make sure that the data is set to go through the parsing queue whereever you are attempting to apply props. ... View more

I agree with @richgalloway that you should not assign time in default. Please review the precedence of configuration files: https://medium.com/splunkuserdeveloperadministrator/splunk-configuration-files-precedence-explained-1b5c20b7b41c You likely have something overriding the timezone in your app local or system local directory.  I would check the props where the sourcetype is defined. Additionally, if this data is forwarded, the indexer may not parse and change the time unless you specifically tell it to go to the parsing queue.  Forwarded data will not always go through each queue that data would go through in a single-instance deployment. https://wiki.splunk.com/Community:HowIndexingWorks Often times when I have a item not being applied from props, I find it is because I have the props stanza "applied" after the data has gone through the parsing queue, so it is not really applied. ... View more

Can you explain this further? ... View more

I had this issue as well with the newest version. Thanks. It fixed my issue. ... View more

Is there any chance you could post the full configurations for this? ... View more

Can you check that you don't have any apps or add-ons that are possibly changing that sourcetype? I had an issue yesterday that was very similar. The field I wanted was not extracted and after I manually extracted it would not show up in interesting fields. Turns out there was a conflict between *nix add-ons. ... View more

Unfortunately this did not solve it. Thank you though. ... View more