it fixed the problem.......
rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*
it fixed the problem.......
rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*
This rex
does not work with the above data.
It can not be complete here?
PS you do not need to specify field=_raw
, if omitted, _raw
is used by default.
I have tried both the rex and both seems to be working fine.
(?P..csv)
this does not extract anything. If it was more like (?<CSVFiles>\w+\.csv)
it will work.
It's better if you past the text in stead of a picture of the text.
Below are the logs. Need to list down all files names ending with .csv. e.g.:
1. adn_attribute_set.csv
2. adn_navigation_attributes.csv
host=mdc1vr1002 sourcetype=MCOM_ETL_OUT
2016-11-06 19:42:35,800 | DEBUG | main:ConcatNCopy | Appending smaller file: adn_attribute_set.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adn_attribute_set.csv to /opt/pim/ETL/MCOM/etlc/output/site/adn_attribute_set.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Appending smaller file: adn_navigation_attributes.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adn_navigation_attributes.csv to /opt/pim/ETL/MCOM/etlc/output/site/adn_navigation_attributes.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:35,809 | DEBUG | main:ConcatNCopy | Appending smaller file: archived_products.csv
2016-11-06 19:42:35,830 | DEBUG | main:ConcatNCopy | Copy: archived_products.csv to /opt/pim/ETL/MCOM/etlc/output/site/archived_products.csv, size: 2768026, elapsed ms: 21
2016-11-06 19:42:35,853 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_cat.csv
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Copy: attr_cat.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_cat.csv, size: 201223799, elapsed ms: 190
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_category_exclusion.csv
2016-11-06 19:42:36,044 | DEBUG | main:ConcatNCopy | Copy: attr_category_exclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_category_exclusion.csv, size: 16705, elapsed ms: 1
2016-11-06 19:42:36,045 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_page_media.csv
2016-11-06 19:42:36,079 | DEBUG | main:ConcatNCopy | Copy: attr_page_media.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_page_media.csv, size: 38563205, elapsed ms: 34
2016-11-06 19:42:36,125 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_brand_ship.csv
2016-11-06 19:42:36,169 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_copy_reg.csv
2016-11-06 19:42:36,359 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_data_source.csv
2016-11-06 19:42:36,366 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_forced_new.csv
2016-11-06 19:42:36,422 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_site_search.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_trigger_data.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_tuple_data.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod_brand_ship.csv attr_prod_copy_reg.csv attr_prod_data_source.csv attr_prod_forced_new.csv attr_prod_site_search.csv attr_prod_trigger_data.csv attr_prod_tuple_data.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod.csv, size: 876915462, elapsed ms: 852
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod2.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod2.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod2.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod3.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod3.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod3.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod4.csv
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Copy: attr_prod4.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod4.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod5.csv
2016-11-06 19:42:36,984 | DEBUG | main:ConcatNCopy | Copy: attr_prod5.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod5.csv, size: 6903099, elapsed ms: 6
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod7.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod7.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod7.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod8.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod8.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod8.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_colorway.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod_colorway.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod_colorway.csv, size: 11624, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_product_exclusion.csv
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Copy: attr_product_exclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_product_exclusion.csv, size: 5340406, elapsed ms: 6
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_promo.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attr_promo.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_promo.csv, size: 577403, elapsed ms: 1
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_req.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attr_req.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_req.csv, size: 52738, elapsed ms: 0
2016-11-06 19:42:37,010 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_upc.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attr_upc.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_upc.csv, size: 478738319, elapsed ms: 485
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_vl.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attr_vl.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_vl.csv, size: 21284, elapsed ms: 0
2016-11-06 19:42:37,496 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_vlitems.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attr_vlitems.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_vlitems.csv, size: 1181621, elapsed ms: 1
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: attribute.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attribute.csv to /opt/pim/ETL/MCOM/etlc/output/site/attribute.csv, size: 173351, elapsed ms: 0
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: brand.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand.csv, size: 117929, elapsed ms: 1
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brand_constraint.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand_constraint.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand_constraint.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brand_constraint_val.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand_constraint_val.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand_constraint_val.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: cat_pools.csv
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Copy: cat_pools.csv to /opt/pim/ETL/MCOM/etlc/output/site/cat_pools.csv, size: 4362512, elapsed ms: 4
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Appending smaller file: cat_prod.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: cat_prod.csv to /opt/pim/ETL/MCOM/etlc/output/site/cat_prod.csv, size: 54444, elapsed ms: 1
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalog.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalog.csv, size: 9310, elapsed ms: 0
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog_context.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalog_context.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalog_context.csv, size: 31, elapsed ms: 0
2016-11-06 19:42:37,504 | DEBUG | main:ConcatNCopy | Appending smaller file: category.csv
2016-11-06 19:42:37,512 | DEBUG | main:ConcatNCopy | Copy: category.csv to /opt/pim/ETL/MCOM/etlc/output/site/category.csv, size: 9478833, elapsed ms: 8
2016-11-06 19:42:37,513 | DEBUG | main:ConcatNCopy | Appending smaller file: category_facet.csv
2016-11-06 19:42:37,543 | DEBUG | main:ConcatNCopy | Copy: category_facet.csv to /opt/pim/ETL/MCOM/etlc/output/site/category_facet.csv, size: 36649061, elapsed ms: 30
2016-11-06 19:42:37,544 | DEBUG | main:ConcatNCopy | Appending smaller file: contextual_category.csv
2016-11-06 19:42:37,547 | DEBUG | main:ConcatNCopy | Copy: contextual_category.csv to /opt/pim/ETL/MCOM/etlc/output/site/contextual_category.csv, size: 2614776, elapsed ms: 3
Then this should do:
your search | rex "(?<file>\w+\.csv)"
Sames as rich7177
posted. So if this works, accept his answer.
PS some lines have more than one file name, this rex
gets them all.
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod_brand_ship.csv attr_prod_copy_reg.csv attr_prod_data_source.csv attr_prod_forced_new.csv attr_prod_site_search.csv attr_prod_trigger_data.csv attr_prod_tuple_data.csv to
Try..
... | rex "(?<MyCSVFile>\w+\.csv)"
Change the name MyCSVFile
to whatever you want to call it. Here you can see it in regex101.com.