Hello,
I got a partitionned Hive table by field dt (in the YYYYMMDD format)
Example :
/mywarehouse/my.db/foo/dt=20160207/part-m-00000
I got a Hunk Index on top of that :
[foo]
vix.provider = my_hive_provider
vix.input.1.path = /mywarehouse/my.db/foo/...
vix.input.1.splitter.hive.dbname = my
vix.input.1.splitter.hive.tablename = foo
vix.input.1.splitter.hive.fileformat = orc
The problem is, when I select a date from the timepicker, I would like Hunk to go directly to the dt partition (because now it makes a full scan of the DB)
I try to modify the following:
vix.input.1.path = /mywarehouse/my.db/foo/${dt}/...
and to add in props.conf :
[foo]
TIME_PREFIX="dt":
TIME_FORMAT = %Y%m%d
[source::.../mywarehouse/my.db/foo/*/*]
sourcetype = foo
But none of it matter...
Could someone lend me a hand on that?
Thanks
I finally did like with HDFS Indexes, i.e. add in indexes.conf
vix.input.1.et.format = yyyyMMdd
vix.input.1.et.regex = /mywarehouse/my.db/foo/dt=(\d+)
vix.input.1.lt.format = yyyyMMdd
vix.input.1.lt.offset = 86400
vix.input.1.lt.regex = /mywarehouse/my.db/foo/dt=(\d+)
I finally did like with HDFS Indexes, i.e. add in indexes.conf
vix.input.1.et.format = yyyyMMdd
vix.input.1.et.regex = /mywarehouse/my.db/foo/dt=(\d+)
vix.input.1.lt.format = yyyyMMdd
vix.input.1.lt.offset = 86400
vix.input.1.lt.regex = /mywarehouse/my.db/foo/dt=(\d+)
Instead of vix.input.1.path = /mywarehouse/my.db/foo/${dt}/...
try
In the VIX UI, select the option to customize timestamp format
See this document: http://docs.splunk.com/Documentation/Hunk/latest/Hunk/Addavirtualindex