Why is the search not showing all the expected res...

sarit_s · ‎04-22-2019

Hello,
I have log file that contains the following rows (im showing only those who relevant to my question)

<0> 11/04/19 08:05:58 : Head Optimization Wizard: Head 6, Next Voltage = 26.477612
<0> 11/04/19 08:07:47 : Head Optimization Wizard: Save voltage 26.100883 for head Head 6
<5> 11/04/19 08:16:16 : E2prom - physical Head Index = 0, Part Number = 390006-010, Serial Number = 11708L0420, Head Type = Gen4 type B

i'm trying to create a table with Serial Number Next Voltage Save Voltage Head Type
but my search returns only Serial Number and Head Type and not Next Voltage and Save Voltage

this is my query:

index="pj"  | search serial OR "Head Type" OR voltage NOT Disabling NOT Enabling OR "Head Optimization Wizard" OR "was selected"   | rex "Serial Number = (?<LogSerialNumber>\S+) " | rename Type AS HeadType  |append[search "Next Voltage" OR "save Voltage"] | rex "Next Voltage = (?<NextVoltage>\S+)" | rex "save Voltage = (?<SaveVoltage>\S+) "|table _time  LogSerialNumber HeadType NextVoltage SaveVoltage

First it's very very very slow and second as I said Next Voltage and Save Voltage are empty

What am I missing?

sarit_s · ‎04-26-2019

hi, thanks for your reply
it is not showing all the results, only this four:

head ModelVoltage SaveVoltage missing_nozzles
why it is not showing _time ?

sarit_s · ‎04-28-2019

hello
i have some updates:

the data in my query should be sort by head
so each head in the table will have the relevnt HeadSerialNumber and other results..
someone gave me a hint that the way im running my query is not right but didn't tell me what is the right way 🙂
maybe someone can help me here ?
this is the last query:

(index=*_pj OR index=other)
Head Optimization Wizard (was selected) OR (num of missing) OR (Save voltage) OR (Support voltage) OR (Model voltage) |append [search index=*_pj OR index=other AND ("Serial Number") OR ("Head Type")]
|`SerialNumber`
|`Region`
| rex "Serial Number = (?<HeadSerialNumber>\S+)"
| rex field=HeadSerialNumber mode=sed "s/,//"
| rex "Head Type = (?<HeadType>[\w\ ]+)"
| rex "Head Optimization Wizard: (?<WizardMode>\S+) was selected"
| rex "Nozzles for head Head (?<head>\d+) is (?<missing_nozzles>\d+)"
| rex "Save voltage (?<Voltage_after_last_wizard_run>[\d\.]+) for head Head (?<head>\d+)"
| rex "Support voltage (?<Voltage_before_last_wizard_run>[\d\.]+) for head Head (?<head>\d+)"
| rex "Model voltage (?<Voltage_before_last_wizard_run>[\d\.]+) for head Head (?<head>\d+)"
| stats max(_time) as _time values(Region) as Region values(HeadSerialNumber) as HeadSerialNumber values(HeadType) as HeadType values(WizardMode) as WizardMode values(head) as head values(missing_nozzles) as missing_nozzles values(Voltage_after_last_wizard_run) as Voltage_after_last_wizard_run values(Voltage_before_last_wizard_run) as Voltage_before_last_wizard_run by SerialNumber
| table _time Region SerialNumber  head HeadType HeadSerialNumber  WizardMode missing_nozzles Voltage_after_last_wizard_run Voltage_before_last_wizard_run

the table looks like:

> _time Region  SerialNumber    head    HeadType    HeadSerialNumber    WizardMode  missing_nozzles Voltage_after_last_wizard_run   Voltage_before_last_wizard_run
> 2019-04-23 14:46:02   AMER    8500012  0 1
> 2 3 4 5 6 7 Gen4 type E6   11809L0606
> 11809L0631 11809L0717 11809L0739
> HS/HM  0 1 2 3 5
> 24.174477
> 24.721434
> 26.681322
> 27.387808
> 27.897087
> 28.085215
> 28.767229
> 29.118530
> 21.7998
> 22.2444
> 26.6693
> 26.9673
> 27.2549
> 27.9447
> 28.3945
> 28.7302 2019-04-25 06:03:32   AMER    8500065  0 1 2 3 4 5 6 7
> Gen4 type B    11708L0420 11712L0751
> 11712L0754 11801L0419 HS/HM    0 1 3
> 24.667702
> 25.3371
> 25.9353
> 26.100883
> 19.3247
> 19.4779
> 24.14
> 24.6677
> 25.09
> 25.1318
> 25.3371
> 25.38
> 25.84
> 26.1009
> 29.12

woodcock · ‎04-23-2019

Try this:

index=pj 
Head Optimization Wizard (was selected) OR (num of missing) OR (Save voltage) OR (Support voltage) OR (Model voltage) OR (Serial Number) OR (Head Type)
| rex "Serial Number = (?<LogSerialNumber>\w+)" 
| rex "Head Type = (?<HeadType>\S+)" 
| rex "Head Optimization Wizard: (?<HeadOptimizationWizard>\S+) was selected" 
| rex "Nozzles for head Head (?<head>\d+) is (?<missing_nozzles>\d+)" 
| rex "Save voltage (?<SaveVoltage>[\d+\.]+) for head Head (?<head>\d+)" 
| rex "Support voltage (?<SupportVoltage>[\d+\.]+) for head Head (?<head>\d+)" 
| rex "Model voltage (?<ModelVoltage>[\d+\.]+) for head Head (?<head>\d+)" 
| table _time  mode HeadType LogSerialNumber head missing_nozzles SaveVoltage CurrentVoltage ModelVoltage
| sort 0 _time 
| stats values(*) AS * values(_time) AS time BY head

sarit_s · ‎04-24-2019

this is the error msg im getting:

Error in 'rex' command: Encountered the following error while compiling the regex 'Serial Number = (?\S+)': Regex: unrecognized character after (? or (?-.

can you explain the meaning of (?\S+)

woodcock · ‎04-25-2019

It got clipped somehow. Try edited answer.

woodcock · ‎04-22-2019

Try this:

(index="pj" AND (serial OR "Head Type" OR voltage OR "Head Optimization Wizard" OR "was selected") AND NOT (Disabling OR Enabling)) OR (index=YouShouldAlwaysSpeciryAnIndex AND ("Next Voltage" OR "save Voltage"))
| rex "Serial Number = (?<LogSerialNumber>\S+)"
| eval HeadType = coalesce(HeadType, Type)
| rex "Next Voltage = (?<NextVoltage>\S+)"
| rex "save Voltage = (?<SaveVoltage>\S+) "
| table _time  LogSerialNumber HeadType NextVoltage SaveVoltage

sarit_s · ‎04-22-2019

Next Voltage and Save Voltage still empty

woodcock · ‎04-23-2019

Your logs have "Save voltage" but your rex has "save Voltage". Maybe that is the main problem; make it match.

sarit_s · ‎04-23-2019

i noticed and changed it but still the same

sarit_s · ‎04-23-2019

i found the problem just don't know how to deal with it

i have values in each of the columns but since its coming from different events i see it in separate rows. how can i merge them ?

so for example this is how my table looks like:

mode    HeadType    LogSerialNumber head    missing_nozzles SaveVoltage CurrentVoltage  ModelVoltage
_time   mode    HeadType    LogSerialNumber head    missing_nozzles SaveVoltage CurrentVoltage  ModelVoltage
2019-04-09 13:26:34 HS/HM                            
2019-04-09 15:16:07 HS/HM                            
2019-04-09 16:25:38 HS/HM                            
2019-04-09 16:33:27 HS/HM                            
2019-04-09 16:33:27 HS/HM                            
2019-04-11 07:34:39 HS/HM                            
2019-04-09 13:18:29                              
2019-04-09 13:39:58             7   1            
2019-04-09 13:39:58             6   5            
2019-04-09 13:39:58             5   0            
2019-04-09 13:39:58             4   1            
2019-04-09 13:39:58             3   3            
2019-04-09 13:39:58             2   2            
2019-04-09 13:39:58             1   0            
2019-04-09 13:39:58             0   0            
2019-04-09 13:42:02             7               27.2549
2019-04-09 13:42:02             6               28.7302
2019-04-09 13:42:02             5               27.9447
2019-04-09 13:42:02             4               26.9673
2019-04-09 13:42:02             3               26.6693

this is my last query:

index=pj 
Head Optimization Wizard (was selected) OR (num of missing) OR (Save voltage) OR (Support voltage) OR (Model voltage) OR (Serial Number) OR (Head Type)
| rex "Serial Number = (?\S+)"
| rex "Head Type = (?\S+)"
| rex "Head Optimization Wizard: (?\S+) was selected"
| rex "Nozzles for head Head (?\d+) is (?\d+)"
| rex "Save voltage (?[\d+\.]+) for head Head (?\d+)"
| rex "Support voltage (?[\d+\.]+) for head Head (?\d+)"
| rex "Model voltage (?[\d+\.]+) for head Head (?\d+)"
| table _time  mode HeadType LogSerialNumber head missing_nozzles SaveVoltage CurrentVoltage ModelVoltage
| sort 0 _time 
| streamstats values(mode) as mode by head reset_before="isnull(head)"

sarit_s · ‎04-22-2019

and another thing.. if possible..
LogSerialNumber returns with comma
for example :

11809L0606,
how can i remove the comma ?

thanks !

somesoni2 · ‎04-22-2019

Give this a try

 index="pj"  serial OR "Head Type" OR voltage  OR "Head Optimization Wizard" OR "was selected" "Next Voltage" OR "save Voltage" NOT (Disabling OR Enabling)
| rex "Serial Number = (?<LogSerialNumber>\S+) " 
| rex "Next Voltage = (?<NextVoltage>\S+)" | rex "save Voltage = (?<SaveVoltage>\S+) "
| rename Type AS HeadType  
|table _time  LogSerialNumber HeadType NextVoltage SaveVoltage

sarit_s · ‎04-22-2019

returns no results at all .. 😞

Why is the search not showing all the expected results?

Introducing the 2024 SplunkTrust!

Introducing the 2024 Splunk MVPs!

Splunk Custom Visualizations App End of Life