csv file indexing not keep the data order

[ralzate]

Hello

Context :
I have a sqlplus session on server, it generates 6 csv files in append mode every minutes. Each file have an epoch column as _time
I installed a forwarder on this server. Forwarder works in monitor mode.
I created a dashboard with timechart refreshed every minutes.

Issue :
Sometimes I see a gap in the end of the chart and the datatable show no values (or zero) for 1-3 min and we see data after this gap (data are sorted correctly in the csv file and data are correct). After around 10 min, this gap was fill with missing values.

I think i have not setup correctly the forwarder. Is it normal ? how to correct that ?

Thanks
Régis

[gcusello]

Hi ralzate,
there could be three issues:

• database sometimes if overloaded so sqlplus session has a delay;
• Splunk Indexer is overloaded so there's a queue;
• it could be a network congestion.

You can test the second situation using Monitoring Console: how many logs are you ingesting?
If there's an overloading of your Indexers without a large amount of data, check Indexer's resources especially CPU and disk performances, always using Monitoring Console: in this way I found a problem in two situation of my projects!

Bye.
Giuseppe

[ralzate]

Hello,

Thanks for your answer.
I'm ingesting only few files (around 40) from this server. Total size is around 400MB per day.

Database overloaded : No, i confirm that all data are written in file instantaneously
Network congestion : I have no overall view of the network but for me it works fine.
Splunk indexer : I'm not admin user and i have no access to the monitor. I will ask to admin if they see something.

But I find strange that the indexer ingest last value before old value.

Regards
Régis

[gcusello]

Hi ralzate,
the most of 400 MB are concentrated in few time or distributed in the day?
Could you share your inputs.conf?
About the monitor console, if you have access to the web interface you have Monitor Console [Settings -- Monitor Console]
Bye.
Giuseppe

[ralzate]

Hi Giuseppe

Data are distributed in the day. Script generates only the difference with the previous exec.

Inputs.conf
[monitor:///data/*/applcsf/splunk/ifop/dbOracleTopActivity*.csv]
sourcetype = st_csvm_dbOracleTopActivity
index = idxm_appl
disable = 0

[monitor:///data/*/applcsf/splunk/ifop/dbOracleCpuCount*.csv]
sourcetype = st_csvm_dbOracleCpuCount
index = idxm_appl
disable = 0

[monitor:///data/*/applcsf/splunk/ifop/dbOracleTempSize*.csv] 
sourcetype = st_csv_dbOracleTempSize 
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/dbOracleTempUsed*.csv] 
sourcetype = st_csv_dbOracleTempUsed 
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/fopPrgQueue*.csv] 
sourcetype = st_csv_fopPrgQueue 
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/fopDbRequestActivity*.csv] 
sourcetype = st_csv_fopDbRequestActivity 
index = idx_appl 
disable = 0

[monitor:///data/*/applcsf/splunk/ifop/fopRequests*.csv] 
sourcetype = st_csv_fopRequests
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/fopConflict*.csv] 
sourcetype = st_csv_fopConflict
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/fopQueue*.csv] 
sourcetype = st_csv_fopQueue
index = idx_appl 
disable = 0 

[monitor:///data/*/applcsf/splunk/ifop/fopLock*.csv] 
sourcetype = st_csv_fopLock
index = idx_appl 
disable = 0 

And i have no web interface access to monitor console. I will try to ask the admin to give me access on it.

Regards
Régis