Splunk Enterprise

pipeline paralllelization ingest_pipe=0 not processing data

aab5272
Engager

Hi ,

I changed the pipelineparalleization=2

and i was able to see the data being processed through both pipelines, but after increasing the typingqueue , my one of ingest_pipe is not processing the data?

Can some one please let me know why would this happen. It was working before.

Thanks

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

You can use this search to look at your queues size per pipeline, and see where it gets full. ( we are using perc90 of the queue filled size)

  index=_internal host=* source=*metrics.log sourcetype=splunkd group=queue (name=parsingqueue OR name=aggqueue OR name=typingqueue OR name=indexqueue) | eval name=case(name=="aggqueue","2 - Aggregation Queue",name=="indexqueue","4 - Indexing Queue",name=="parsingqueue","1 - Parsing Queue",name=="typingqueue","3 - Typing Queue") | eval ingest_pipe = if(isnotnull(ingest_pipe), ingest_pipe, "none") | search ingest_pipe=* | eval max=if(isnotnull(max_size_kb),max_size_kb,max_size)  | eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size)  | eval fill_perc=round((curr/max)*100,2) | eval name=name."-".ingest_pipe | timechart  Perc90(fill_perc) by name useother=false limit=15

We will order the processor in the order they run, so that way you can see if one is full and is backing up the previous ones.
1- parsing -> 2 -aggregation -> 3 -typing -> 4- indexing

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...