Hi Splunkers,   I am facing a strange issue like the splunk forwarder stopped forwarding data. I see the forwarder is working fine and as per the splunk logs I see The monitor input cannot produce data because splunkd's processing queues are full. This will be caused by inadequate indexing or forwarding rate, or a sudden burst of incoming data I tried restarting the indexers and forwarder. It works for a while and after that it stops.  Could you please advice. Thanks, Amit ... View more

Hi All,   I have a query to extract the from the xml log data -------------- 2020-10-22 11:29:23,712 INFO (default-workqueue-73) [com.xyz.uyt.eds.building.documents.btw.house] ID-apprdhgr001-co-dmz-33780-17w876834676-5-w87434 building-documents-vic:pull-order-orchestration-v1 pull Order Response : <v1:SubmitOrderResponse xmlns:v1="http://www.xyz.com/services/building/documents/btw/v1" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <v1:context> <v1:chnnlcd>06</v1:chnnlcd> <v1:user> <v1:memCode>5436</v1:memCode> <v1:brCode>7654</v1:brCode> <v1:usrCode>7634B</v1:usrCode> </v1:user> <v1:clientReferences> <v1:clientReference type="type0">1234 - amit</v1:clientReference> </v1:clientReferences> <v1:transaction> <v1:transactionDateTime>2020-10-22T11:29:00Z</v1:transactionDateTime> </v1:transaction> <v1:conversationId>2</v1:conversationId> </v1:context> <v1:documents> <v1:document sequenceNo="1"> <v1:identifer>67</v1:identifer> <v1:name>Reg Src Statement (Title)</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>320</v1:identifier> <v1:name/> </v1:authority> </v1:organisations> <v1:fee>5.9</v1:fee> <v1:gst>0</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>5.9</v1:recoupmentFee> <v1:totalFeeExclGst>15.9</v1:totalFeeExclGst> <v1:totalGst>1</v1:totalGst> </v1:document> <v1:document sequenceNo="8"> <v1:identifer>75</v1:identifer> <v1:name>Index Search</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>320</v1:identifier> <v1:name>Index Registry</v1:name> </v1:authority> </v1:organisations> <v1:fee>6.91</v1:fee> <v1:gst>0.63</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>0</v1:recoupmentFee> <v1:totalFeeExclGst>16.28</v1:totalFeeExclGst> <v1:totalGst>1.63</v1:totalGst> </v1:document> <v1:document sequenceNo="9"> <v1:identifer>238</v1:identifer> <v1:name>Plan Copy</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>320</v1:identifier> <v1:name>Plan 8T6543</v1:name> </v1:authority> </v1:organisations> <v1:fee>5.84</v1:fee> <v1:gst>0</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>5.84</v1:recoupmentFee> <v1:totalFeeExclGst>15.84</v1:totalFeeExclGst> <v1:totalGst>1</v1:totalGst> </v1:document> <v1:document sequenceNo="10"> <v1:identifer>1</v1:identifer> <v1:name>=Tax Certificate</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>34</v1:identifier> <v1:name>Revenue Cert</v1:name> </v1:authority> </v1:organisations> <v1:fee>20.27</v1:fee> <v1:gst>1.84</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>0</v1:recoupmentFee> <v1:totalFeeExclGst>28.43</v1:totalFeeExclGst> <v1:totalGst>2.84</v1:totalGst> <v1:serviceLevel> <v1:period>10</v1:period> </v1:serviceLevel> </v1:document> <v1:document sequenceNo="89"> <v1:identifer>80</v1:identifer> <v1:name>Information Certificate</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>249</v1:identifier> <v1:name>Check</v1:name> </v1:authority> </v1:organisations> <v1:fee>35.690002</v1:fee> <v1:gst>3.24</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>0</v1:recoupmentFee> <v1:totalFeeExclGst>42.45</v1:totalFeeExclGst> <v1:totalGst>4.24</v1:totalGst> <v1:serviceLevel> <v1:period>5</v1:period> </v1:serviceLevel> </v1:document> <v1:document sequenceNo="26"> <v1:identifer>23</v1:identifer> <v1:name>Other Information</v1:name> <v1:description/> <v1:organisations> <v1:authority> <v1:identifier>698</v1:identifier> <v1:name>RIVER WATER</v1:name> </v1:authority> </v1:organisations> <v1:fee>87.04</v1:fee> <v1:gst>7.91</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>0</v1:recoupmentFee> <v1:totalFeeExclGst>89.13</v1:totalFeeExclGst> <v1:totalGst>8.91</v1:totalGst> <v1:serviceLevel> <v1:period>7</v1:period> </v1:serviceLevel> </v1:document> </v1:documents> <v1:details> </v1:details> </v1:SubmitOrderResponse> ----------------- Similar lines are there in the log files but the entries vary as per the record. Hence, I need to output to extract <v1:fee>97.99</v1:fee> <v1:gst>8.9</v1:gst> <v1:deliveryFee>11</v1:deliveryFee> <v1:deliveryGst>1</v1:deliveryGst> <v1:recoupmentFee>0</v1:recoupmentFee> <v1:totalFeeExclGst>99.09</v1:totalFeeExclGst> <v1:totalGst>9.9</v1:totalGst>   And show them in tabular format for each memcode, brcode and usrcode. I tried using spath and xpath and xmlkv but none of them are working as expected. With xmlkv, it shows only the last value or one value. The xpath and spath are not extracting any data as shown in my below queries   <main search> | xpath outfield=memCode "/v1:SubmitOrderResponse/v1:context/v1:user/v1:memCode" | table outfield, _time <main search> | spath outfield=v1:memCode path=v1:SubmitOrderResponse.v1:context.v1:user.v1:memCode | xmlkv| table outfield, _time   Can you please advise.   Thanks, Amit     ... View more

Hi Splunkers, I have a splunk search query  index="xyz" source="/var/log/production.log" sourcetype="xyzlogs" type="report" | dedup uuid | stats count(uuid) as TOTAL | append [ search index="xyz" sourcetype=abclogs NOT host="xyte150.com.dmz" "<vv:general-messages>" ("conditions1"  "conditions2" | dedup uuid | stats count(uuid) as FAIL] | eval SUCCESS=TOTAL - FAIL |stats list(TOTAL) as TotalTransactions, values(SUCCESS) as PASSED, list(FAIL) as FAILED | eval Availability=round((PASSED*100)/TotalTransactions,2)   I cannot see any value in SUCCESS and due to this no Availability. Somehow the subtraction is not working. My end goal is display a table to show the below TOTAL PASSED FAIL Availability   Can you please suggest why is not working? Thanks, Amit ... View more

Hi Splunkers,   I have set up a field extractor and it does not work when the log entry is empty. For e.g Field extraction syntax is --------------------------- (?:[^=\n]*=){9}"(?P<frontEndLatency>\d+)"\s+\w+="(?P<backEndLatency>\d+) -------------------------- Log messages --------------------- blah blah contentType="text/xml" frontEndLatency="587" backEndLatency="391" messages= blah --------------- It extracts correctly  frontEndLatency="587" and backEndLatency="391" If somehow in the log file one of the field is empty, it does not extracts properly Log Messages -------------  blah blah contentType="text/xml" frontEndLatency="1795" backEndLatency="" messages= blah blah -------------- How to set this up or handle it via field extraction? Your help is much appreciated. Thanks,   ... View more

Hi Splukers, I have a requirement to search for some filenames and display the missing files as per the date. Thus, i made up a query to look like index=123 host=htrstef87 "string_1," "created" NOT client_ip="192.168.17.5" "String_examp_*" "xml.7z.pgp" | eval keyword=case(searchmatch("string_1L_2"),"string_1L_2",searchmatch("string_1L_21"),"string_1L_21",searchmatch("string_1L_22"),"string_1L_22",searchmatch("string_1L_23"),"string_1L_23",searchmatch("string_1L_24"),"string_1L_24") | eval Filestatus=if(like(keyword, "string_1L%"), "fileFound", "Filenotfound") |eval DateReport= date_month."-".date_year| stats values(keyword), values(FileName), values(Filesize) by DateReport | where Filesize>0 This displays all the filenames with all the data. But the requirement is to match the keyword and check them every month at certain date and send them if any files are missing or no bytes (filesize). Any help is much appreciated. note: I am running splunk 6.5.3 and thus queries like where(in) does not work for me. Thanks, Amit ... View more