The 2nd piece of hardware has a 128GB SSD.
If this is moderately recent it will easily best the 1200 IOP reference recommendation.
For an "all time" search on 1.2 GB of data it should suck it up in < 5 seconds.
So if you're getting similar performance on the VM as the i5 box, something is seriously bottlenecked in the software (or you're doing it wrong).
You have 4x the cores, 2x the ram, infinity x disk io - thats just not right.
In my experience transaction and spath are hideously slow - i bet you use both.
So, rework your searches to not use transaction - its possible 95% of the time.
If you can change the log format to key value pairs instead of JSON, do that too.
Host the scrubbed sample data and your queries and lay down the challenge.
Now, at the risk of being downvoted to oblivion...
Splunk is a swiss army knife. It stores your data, searches your data, creates pretty charts, alerts, creates pdfs etc etc etc.
However the biggest advantage is the mentality of how to store and retrieve data.
With splunk you just throw throw logs at it and worry about parsing it into fields later.
A classic RDBMS is the opposite. You define your data, design a schema, then throw your data at it.
So... if
a) your data is highly structured
b) you know exactly what you need to present from your data already
c) you have the inhouse experience to design a performant RDBMS
d) you have the inhouse experience to present it nicely.
e) you already have support contracts in place with the alternative vendors.
Rolling out apache with mod_<insert_language_of_choice>, open ldap and mysql / postgresql will probably give you better performance on a single node. And you can spend the license cost on better kit.
</end_flame_bait>
... View more