Security
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question
Solved! Jump to solution

How to extract all sub strings ends with .csv in a string?

SumitPan
Explorer
‎11-06-2016 02:36 PM

Sorry I'm new to regex. I'm trying to get some meaning full data from the log files.

I want all the sub-strings ending with .csv in my log file at any given point of time. Below is the the log file preview. Any leads would be highly appreciated.

alt text

Tags (2)
Preview file
1 KB
0 Karma
Reply
1 Solution
Solved! Jump to solution
Solution

SumitPan
Explorer
‎11-11-2016 09:13 AM

it fixed the problem.......

rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*

View solution in original post

0 Karma
Reply
Solved! Jump to solution
Solution

SumitPan
Explorer
‎11-11-2016 09:13 AM

it fixed the problem.......

rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*
0 Karma
Reply
Solved! Jump to solution

lakromani
Builder
‎11-12-2016 02:19 PM

This rex does not work with the above data.
It can not be complete here?

PS you do not need to specify field=_raw, if omitted, _raw is used by default.

0 Karma
Reply
Solved! Jump to solution

SumitPan
Explorer
‎11-12-2016 03:14 PM

I have tried both the rex and both seems to be working fine.

0 Karma
Reply
Solved! Jump to solution

lakromani
Builder
‎11-13-2016 12:02 AM

(?P..csv) this does not extract anything. If it was more like (?<CSVFiles>\w+\.csv) it will work.

0 Karma
Reply
Solved! Jump to solution

lakromani
Builder
‎11-06-2016 10:51 PM

It's better if you past the text in stead of a picture of the text.

0 Karma
Reply
Solved! Jump to solution

SumitPan
Explorer
‎11-07-2016 07:35 AM

Below are the logs. Need to list down all files names ending with .csv. e.g.:
1. adn_attribute_set.csv
2. adn_navigation_attributes.csv

host=mdc1vr1002 sourcetype=MCOM_ETL_OUT

2016-11-06 19:42:35,800 | DEBUG | main:ConcatNCopy | Appending smaller file: adn_attribute_set.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adn_attribute_set.csv to /opt/pim/ETL/MCOM/etlc/output/site/adn_attribute_set.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Appending smaller file: adn_navigation_attributes.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adn_navigation_attributes.csv to /opt/pim/ETL/MCOM/etlc/output/site/adn_navigation_attributes.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:35,809 | DEBUG | main:ConcatNCopy | Appending smaller file: archived_products.csv
2016-11-06 19:42:35,830 | DEBUG | main:ConcatNCopy | Copy: archived_products.csv to /opt/pim/ETL/MCOM/etlc/output/site/archived_products.csv, size: 2768026, elapsed ms: 21
2016-11-06 19:42:35,853 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_cat.csv
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Copy: attr_cat.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_cat.csv, size: 201223799, elapsed ms: 190
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_category_exclusion.csv
2016-11-06 19:42:36,044 | DEBUG | main:ConcatNCopy | Copy: attr_category_exclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_category_exclusion.csv, size: 16705, elapsed ms: 1
2016-11-06 19:42:36,045 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_page_media.csv
2016-11-06 19:42:36,079 | DEBUG | main:ConcatNCopy | Copy: attr_page_media.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_page_media.csv, size: 38563205, elapsed ms: 34
2016-11-06 19:42:36,125 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_brand_ship.csv
2016-11-06 19:42:36,169 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_copy_reg.csv
2016-11-06 19:42:36,359 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_data_source.csv
2016-11-06 19:42:36,366 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_forced_new.csv
2016-11-06 19:42:36,422 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_site_search.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_trigger_data.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_tuple_data.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod_brand_ship.csv attr_prod_copy_reg.csv attr_prod_data_source.csv attr_prod_forced_new.csv attr_prod_site_search.csv attr_prod_trigger_data.csv attr_prod_tuple_data.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod.csv, size: 876915462, elapsed ms: 852
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod2.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod2.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod2.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod3.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod3.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod3.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod4.csv
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Copy: attr_prod4.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod4.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod5.csv
2016-11-06 19:42:36,984 | DEBUG | main:ConcatNCopy | Copy: attr_prod5.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod5.csv, size: 6903099, elapsed ms: 6
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod7.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod7.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod7.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod8.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod8.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod8.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_prod_colorway.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr_prod_colorway.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_prod_colorway.csv, size: 11624, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_product_exclusion.csv
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Copy: attr_product_exclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_product_exclusion.csv, size: 5340406, elapsed ms: 6
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_promo.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attr_promo.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_promo.csv, size: 577403, elapsed ms: 1
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_req.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attr_req.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_req.csv, size: 52738, elapsed ms: 0
2016-11-06 19:42:37,010 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_upc.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attr_upc.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_upc.csv, size: 478738319, elapsed ms: 485
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_vl.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attr_vl.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_vl.csv, size: 21284, elapsed ms: 0
2016-11-06 19:42:37,496 | DEBUG | main:ConcatNCopy | Appending smaller file: attr_vlitems.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attr_vlitems.csv to /opt/pim/ETL/MCOM/etlc/output/site/attr_vlitems.csv, size: 1181621, elapsed ms: 1
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: attribute.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attribute.csv to /opt/pim/ETL/MCOM/etlc/output/site/attribute.csv, size: 173351, elapsed ms: 0
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: brand.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand.csv, size: 117929, elapsed ms: 1
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brand_constraint.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand_constraint.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand_constraint.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brand_constraint_val.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand_constraint_val.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand_constraint_val.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: cat_pools.csv
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Copy: cat_pools.csv to /opt/pim/ETL/MCOM/etlc/output/site/cat_pools.csv, size: 4362512, elapsed ms: 4
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Appending smaller file: cat_prod.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: cat_prod.csv to /opt/pim/ETL/MCOM/etlc/output/site/cat_prod.csv, size: 54444, elapsed ms: 1
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalog.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalog.csv, size: 9310, elapsed ms: 0
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog_context.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalog_context.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalog_context.csv, size: 31, elapsed ms: 0
2016-11-06 19:42:37,504 | DEBUG | main:ConcatNCopy | Appending smaller file: category.csv
2016-11-06 19:42:37,512 | DEBUG | main:ConcatNCopy | Copy: category.csv to /opt/pim/ETL/MCOM/etlc/output/site/category.csv, size: 9478833, elapsed ms: 8
2016-11-06 19:42:37,513 | DEBUG | main:ConcatNCopy | Appending smaller file: category_facet.csv
2016-11-06 19:42:37,543 | DEBUG | main:ConcatNCopy | Copy: category_facet.csv to /opt/pim/ETL/MCOM/etlc/output/site/category_facet.csv, size: 36649061, elapsed ms: 30
2016-11-06 19:42:37,544 | DEBUG | main:ConcatNCopy | Appending smaller file: contextual_category.csv
2016-11-06 19:42:37,547 | DEBUG | main:ConcatNCopy | Copy: contextual_category.csv to /opt/pim/ETL/MCOM/etlc/output/site/contextual_category.csv, size: 2614776, elapsed ms: 3

0 Karma
Reply
Solved! Jump to solution

lakromani
Builder
‎11-11-2016 01:37 PM

Then this should do:
your search | rex "(?<file>\w+\.csv)"

Sames as rich7177 posted. So if this works, accept his answer.

PS some lines have more than one file name, this rex gets them all.

2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod_brand_ship.csv attr_prod_copy_reg.csv attr_prod_data_source.csv attr_prod_forced_new.csv attr_prod_site_search.csv attr_prod_trigger_data.csv attr_prod_tuple_data.csv to
0 Karma
Reply
Solved! Jump to solution

Richfez
SplunkTrust
SplunkTrust
‎11-06-2016 05:19 PM

Try..

 ... | rex "(?<MyCSVFile>\w+\.csv)"

Change the name MyCSVFile to whatever you want to call it. Here you can see it in regex101.com.

Reply
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...