Hi Team,
I m planning to collect the highlited text from the raw data as below
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_generic_reformat_control_file_2019-04-10-06-35-06_**10471**.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-35-11_11311.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./aiblk_linear_framework_us__msa104_gl_txn__feed_2019-04-10-06-35-58_18695.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_process_acqit_log_files_2019-04-10-06-43-49_4398.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-44-21_8468.log
Can you please help me wit regex expression for the same . thank you
@vnravikumar @jkat54
Try this:
|makeresults | eval _raw="apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_generic_reformat_control_file_2019-04-10-06-35-06_10471.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-35-11_11311.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./aiblk_linear_framework_us_msa104_gl_txn_feed_2019-04-10-06-35-58_18695.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_process_acqit_log_files_2019-04-10-06-43-49_4398.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-44-21_8468.log"
| rex max_match=1 "_\d{4}(?:-\d{2}){5}_(?<log_number>\d+)\.log"
thanks @woodcock ...but i cant use makeresults command in my query...do you have any alternative way to get this
The makeresults
was to generate fake events to test your solution, which is only the last line.
Hi,
Please try below regex, it will extract highlighted value in new field called ext_value
<yourBaseSearch> | rex field=_raw "_\d{4}-\d{2}-\d{2}-\d{2}-\d{2}-\d{2}_(?<ext_value>\d+)\.log"
thanks for the resply @harsmarvania57 ....its matching with all the rows , but i need to extract the value only from first row.
Sample data which you have provided is single event only or those are different events ?
its from single event.
Try this | rex field=_raw max_match=1 "^(?s)(?:[^\/]*[\/]){11}([^\d]*)\d{4}-\d{2}-\d{2}-\d{2}-\d{2}-\d{2}_(?<ext_value>\d+)\.log"
its not working @harsmarvania57 marvania...actually my raw data is like this and its coming as single event... i need to extract the hightlighted value
[2019-04-15 06:12:26] Plan File: /apps/src/aasconap/prod/abinitio/cnapp/cnapp_src/cnapp_src_msp/pset/planpset/processing_plan.msp_master_708_936.pset
[2019-04-15 06:12:26] Recovery File: /apps_run_aasconap/prod/processing_plan.mspmaster_708_936.rec
[2019-04-15 06:12:26] Beginning plan '/'
[2019-04-15 06:12:28] Method '/Get RUN_ID/perform' changed parameter 'RUN_ID' from '' to '28090'
[2019-04-15 06:12:43] Standard Output for '/Standardize control file/perform':
info : ++++ STARTED ++++ Job cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803
info : Central logging to /apps/dat/aasconap/prod/admin/log/environment_operations_2019_04.log
info : Raw tracking to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/tracking/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.tracking
info : Input pset archived to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/parameter/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.pset
info : Summary is not being collected
info : Error logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/error/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.err
info : Duplicating stderr
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/log/./cnapp_generic_reformat_control_file_2019-04-15-06-12-437803.log
[2019-04-15 06:12:46] Standard Output for '/Standardize control file/perform':
info : ++++ COMPLETED ++++ Job cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803
[2019-04-15 06:12:47] Method '/Set dynamic plan variables from control file/perform' changed parameter 'EFF_DATE' from '2019-04-14' to '2019-04-14'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'DATA_READ_LOCATION' from '' to 'hdfs:/datalake/consumer/msp/raw/tmp/MSP_DELTA_PR708_936_MASTER_190414'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'REC_CNT' from '' to '580157'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'CNAPP_PUB_KEY_REG_PG' from 'PG777' to '708_936'
[2019-04-15 06:12:48] Standard Output for '/Set dynamic plan variables from control file/perform':
Successfully validated effective date format from control file (value = 2019-04-14)
[2019-04-15 06:12:51] Standard Output for '/Publish module start metadata/perform':
info : ++++ STARTED ++++ Job eapp_generic_publish_status_2019-04-15-06-12-50_8647
info : Central logging to /apps/dat/aasconap/prod/admin/log/environment_operations_2019_04.log
info : Raw tracking to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/tracking/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.tracking
info : Input pset archived to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/parameter/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.pset
info : Summary is not being collected
info : Error logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/error/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.err
info : Duplicating stderr
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/log/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.log
@FrankVl can you help me here.