I have a set of keywords which I need to search and generate the output similar to
https://answers.splunk.com/answers/588828/how-do-you-display-the-of-requests-per-24-hours-of.html#co...
keywords:
DCBClaimSearch
CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=
etc
log format:
1x.xx.xxx.xxx - - 1xxxx4 [04/Oct/2017:00:03:44 -0400] - /xxx/Gxxxt/ibxx_xxxxxxxxxxxxxxxxxxxxxxx[[/!TABTHREAD1 HTTP/1.1 xxxxxxxxxxx.net TIME:0/49626 "POST /pxxxb/Gxxxxt/ixxxU_xxxxxxxxxxxxxxxxxxxxxxxxx/!TABTHREAD1?pyActivity=xxxxxxxxxxxxxxxxxxxxxxxx%20&Request_Type=&xxxxxxxxxxxx_CD=Nxxx_xxxxEL&Exxxxxxxxx_CD=&Lxxxxxxxxxe=&CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count= HTTP/1.1" 200 737
?pyActivity=...............................................PreActivity=DCBClaimSearch&HeaderButtonSectionName.................HTTP/1.1" 200 4502
?pyActivity=...........................LanguageCode=&CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=..........HTTP/1.1" 200 3402
Every keyword has different pattern and it lies in middle of requests but all will start with ?pyActivity=
For example:
?pyActivity=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx%20&Request_Type=&xxxxxxxxxxxxxx_xxxxxxxxxxxxxxx&ELEMENT_CD=&LanguageCode=&CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count= HTTP/1.1" 200 5092
Hi @saifullakhalid,
Can you please try this search?
YOUR_SEARCH
| rex field=_raw Count=(.*)\s.*\s(?<COUNT>.*)
| stats count AS "Count" sum(COUNT) as "Sum of COUNT" min(_time) AS start_time max(_time) as end_time by date_month,date_mday,date_hour
| eval "Start Time"=strftime(start_time,"%d/%m/%Y %I:%M:%S:%p")
| eval "End Time"=strftime(end_time,"%d/%m/%Y %I:%M:%S:%p")
| table "Start Time" "End Time" "Count" "Sum of COUNT"
Thanks
This is displaying me start time , end time Count and total count, How should I display the keys words below along with these. I tried adding COUNT, but it was empty. I tried adding like below but it is displaying the full request, I just wanted keywords to display.
index="prod_4oct_508" pyActivity="" | rex field=_raw Count=(.)\s.\s(?.)
| stats count AS "Count" sum(COUNT) as "Sum of COUNT" min(_time) AS start_time max(_time) as end_time by date_month,date_mday,date_hour,pyActivity
| eval "Start Time"=strftime(start_time,"%d/%m/%Y %I:%M:%S:%p")
| eval "End Time"=strftime(end_time,"%d/%m/%Y %I:%M:%S:%p") | dedup pyActivity | sort –count
| table "Start Time" "End Time" "pyActivity" "Count" "Sum of COUNT"
keywords:
DCBClaimSearch
CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=
Hi @saifullakhalid,
Do you want to extract these keywords?
DCBClaimSearch
CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=
Below search will extract "CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=
"
index="prod_4oct_508" pyActivity=""
| rex field=_raw "&CountryCode=(?<FIELD1>.*)\s(.*)\s(.*)\s(?<COUNT>.*)"
| table _time FIELD1 COUNT
Can you please let me know the pattern of keyword "DCBClaimSearch
" ?
Thanks
I have written a java program which reads the list of keywords (61 keywords )from csv and parse the logs and generate 24 hours reports like below for each keyword. Can we do this in splunk if so then please suggest.
First report format:
User Time Protocol Url Elapsed Time (Seconds) call Status Size logname
1432029 17/Oct/04 05:19:05 HTTP/1.1 xxxxxxxxxxx.net 0.150266 POST /xxxx/xxxxt/xxxxxxxxxxxxxx*/!TABTHREAD4?pyActivity=Reloadxxxx&pzxxxxxxxx=xxxxxxxxx&pzFromFrame=xxxxxxxxx&pzxxxxe=pyxxxe&pzxxxxx=false&StxxxxName=xxxxxe&xxxx&inStandardsMode=false&AJAXTrackID=3&pzHarnessID=xxx36749 HTTP/1.1 200 2207 \507\access_log_10_04_2017
second report format:
Transaction keyword Start Time End Time Total # of executions Avg # of Executions per Hour Min Resp Time Max Resp Time Avg Resp Time 90th percentile Resp Time Std Dev Of Resp Time Min Size of Response Max Size of Response Avg Size of Response 90th percentile Size of Response Std Dev Of Size of Response
TC01 DCBClaimSearch 17/Oct/04 00:11:46 17/Oct/04 23:24:05 2398 104 0.029761 27.35804 0.108725955 0.1099671 0.594170951 82 10342 4302.940784 4543 424.216474
3rd report:
Transaction keyword Start Time End Time Total # of executions Avg # of Executions per Hour Min Resp Time Max Resp Time Avg Resp Time 90th percentile Resp Time Std Dev Of Resp Time Min Size of Response Max Size of Response Avg Size of Response 90th percentile Size of Response Std Dev Of Size of Response
TC01 DCBClaimSearch 17/Oct/04 00:00:00 17/Oct/04 00:59:59 4 4 0.056125 0.070999 0.0613225 0.070999 0.00671778 3617 4533 3886.75 4533 437.5083809
TC01 DCBClaimSearch 17/Oct/04 01:00:00 17/Oct/04 01:59:59 3 3 0.058215 0.080105 0.066264 0.080105 0.012039662 3780 4548 4036 4548 443.4050067
Hi @saifullakhalid,
Can you please try this?
index="prod_4oct_508" pyActivity="*"
| rex field=_raw "&CountryCode=(?<FIELD1>.*)\s(.*)\s(.*)\s(?<COUNT>.*)"
| rex field=_raw "pyActivity=(.*)PreActivity=(?<PreActivity>.*)&"
| table _time PreActivity _time FIELD1 COUNT
looks like it is extracting values which has the pattern "&CountryCode" and c
But is it possible to read these keywords values which needs to be extracted using csv. because as I said the pattern is every line in the code starts with ?pyActivity= , but the keywords falls in middle of this request. like in above comment "CountryCode=&PRODUCT_LINE_CD=®ION_CD=&LOB=&LOB_SUB_CD=&Count=
" is part of the line ?pyActivity=
I have created a different question for reports mentioned below https://answers.splunk.com/answers/590577/how-do-you-create-the-report-for-the-log-access-be.html?mi...