It's flat files with an index, and is proprietary. The Splunk Documentation has the answer to pretty much every question you have, at least insofar as you actually need to know it. It'll tell you how it stores the data, how data moves through the pipelines to be stored, how searches work, how they work in a distributed environment and so on.
To get you started, here's Splunk does with your data when you get it in and how Splunk stores that data. I recommend reading those and following links, and perhaps asking more questions as you come across them. But really, it is fairly well documented so I'd start there.
"How much data"? Well, "instant" is relative. There are people reasonably happy with their search speeds who have hundreds of TB or even petabytes. They use indexer clusters, and sometimes search head clusters (both in the docs!), then may spend some time making sure they're writing smart searches. For what its worth, this is no different than a larger SQL instance or any other large, data-sourced program - you just need enough hardware, then DBAs (or in this case Splunk folks) to optimize your queries (searches).
Happy Splunking!
