thanks for giving alternate, machine learning in splunk seems better. I know i need to do more than just an average + standard deviation (unless login volume really does follow a Gaussian distribution), so need to figure out a good method for baselining so for this probably need to do some basic five-number summary things to explore (min, max, median, first quartile, third quartile); keeping in mind the cyclic nature of the data (e.g., business hours + work days vs. off-hours and weekends). So not sure how to approach in machine learning, which algorithm or method to start and also what search query to use to fit in algorithm for my use case. Can u help in proceeding for this ? ... View more

Hi , My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name log_modified_date which i want to be extracted i have written below props.conf: [_json] INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = last_modified_date TIME_FORMAT = %Y-%m-%dT%H:%M:%S+%2N:%2N MAX_TIMESTAMP_LOOKAHEAD = 25 and getting time extracted as :  7/2/20 6:00:18.020 PM   but I want the time field extracted in same way as in data with + value as well like:   7/2/20 6:00:18+02:00  Please let me know what i am doing wrong as i am not getting expected output with + value. Note： this +02：00 value is fixed with every timestamp in data . Here’s my sample log data： {"_timestamp":"2020-07-02 18:00:46","_ver":"2","asset_name":"","assigned_group":"Troubleshooting - Tier 2","assignee":"Buhle Mahlaba","ci":"","cause":"","city":"","client_type":"","closed_date":"","closure_source":"","company":"MTN BUSINESS","contact_phone":"","contact_site":"","country":"","created_from_template":"","customer_phone":"###","customer_site":"INTERNET SOLUTIONS(PTY) LTD","debtor_code":"MTN000","direct_contact_city":"","direct_contact_company":"","direct_contact_corporate_id":"","direct_contact_country":"","direct_contact_country_code":"","direct_contact_department":"","direct_contact_desk_location":"","direct_contact_extension":"","direct_contact_first_name":"","direct_contact_internet_email":"","direct_contact_last_name":"","direct_contact_local_number":"","direct_contact_location_details":"","direct_contact_middle_initial":"","direct_contact_organization":"","direct_contact_region":"","direct_contact_site_group":"","direct_contact_state_province":"","direct_contact_street":"","direct_contact_time_zone":"","direct_contact_zip_postal_code":"","first_name":"Melvern","impact":"2-Significant\/Large","incident_id":"MTNB00001289400","incident_type":"User Service Restoration","last_acknowledged_date":"","last_modified_by":"412877","last_modified_date":"2020-07-02T18:00:44+02:00","last_name":"Banoo","last_resolved_date":"","middle_name":"","notes":"HI Team\n\nThe mentioned link is down ,Please investigate and advise.\n\n\nRP\/0\/RSP0\/CPU0:mi-za-bry-mspe4#sho log | inc BVI906\nRP\/0\/RSP0\/CPU0:Jul  2 14:43:49.894 SAST: mpls_ldp[1204]: %ROUTING-LDP-5-HELLO_ADJ_CHANGE : VRF 'default' (0x60000000), Link hello adja...","operational_categorization_tier_1":"TES_Link","operational_categorization_tier_2":"Microwave PTP","operational_categorization_tier_3":"Link Down","owner_group":"General Support","priority":"Critical","product_categorization_tier_1":"TES_Managed Networks","product_categorization_tier_2":"Access Service","product_categorization_tier_3":"Cloud Connect","product_name":"","region":"","reported_date":"2020-07-02T16:36:04+02:00","reported_source":"Email","resolution":"","resolution_categorization_tier_1":"","resolution_categorization_tier_2":"","resolution_categorization_tier_3":"","resolution_product_categorization_tier_1":"","resolution_product_categorization_tier_2":"","resolution_product_categorization_tier_3":"","responded_date":"2020-07-02T18:00:43+02:00","slm_real_time_status":"Within the Service Target","satisfaction_rating":"","service_manager":"","service_request_id":"","site_group":"","state_province":"","status":"In Progress","status_reason_hidden":"","street":"","submit_date":"2020-07-02T16:36:04+02:00","submitter":"AR_ESCALATOR","summary":"INC000147465| me-za-gp80-hoedspru-bry-1 | | E2379","time_zone":"","urgency":"1-Critical","vendor_group":"","vendor_name":"","vendor_ticket_number":"","zip_postal_code":""} ... View more

@oscar84x   Hi , My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name last_modified_date which i want to be extracted i have written below props.conf: [_json] INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = last_modified_date TIME_FORMAT = %Y-%m-%dT%H:%M:%S+%2N:%2N MAX_TIMESTAMP_LOOKAHEAD = 25 and getting time extracted as :  7/2/20 6:00:18.020 PM   but I want the time field extracted in same way as in data with + value as well like:   7/2/20 6:00:18+02:00 PM Please let me know what i am doing wrong as i am not getting expected output with + value. Note： this +02：00 value is fixed with every timestamp in data . Here’s my sample log data： {"_timestamp":"2020-07-02 18:00:46","_ver":"2","asset_name":"","assigned_group":"Troubleshooting - Tier 2","assignee":"Buhle Mahlaba","ci":"","cause":"","city":"","client_type":"","closed_date":"","closure_source":"","company":"MTN BUSINESS","contact_phone":"","contact_site":"","country":"","created_from_template":"","customer_phone":"###","customer_site":"INTERNET SOLUTIONS(PTY) LTD","debtor_code":"MTN000","direct_contact_city":"","direct_contact_company":"","direct_contact_corporate_id":"","direct_contact_country":"","direct_contact_country_code":"","direct_contact_department":"","direct_contact_desk_location":"","direct_contact_extension":"","direct_contact_first_name":"","direct_contact_internet_email":"","direct_contact_last_name":"","direct_contact_local_number":"","direct_contact_location_details":"","direct_contact_middle_initial":"","direct_contact_organization":"","direct_contact_region":"","direct_contact_site_group":"","direct_contact_state_province":"","direct_contact_street":"","direct_contact_time_zone":"","direct_contact_zip_postal_code":"","first_name":"Melvern","impact":"2-Significant\/Large","incident_id":"MTNB00001289400","incident_type":"User Service Restoration","last_acknowledged_date":"","last_modified_by":"412877","last_modified_date":"2020-07-02T18:00:44+02:00","last_name":"Banoo","last_resolved_date":"","middle_name":"","notes":"HI Team\n\nThe mentioned link is down ,Please investigate and advise.\n\n\nRP\/0\/RSP0\/CPU0:mi-za-bry-mspe4#sho log | inc BVI906\nRP\/0\/RSP0\/CPU0:Jul  2 14:43:49.894 SAST: mpls_ldp[1204]: %ROUTING-LDP-5-HELLO_ADJ_CHANGE : VRF 'default' (0x60000000), Link hello adja...","operational_categorization_tier_1":"TES_Link","operational_categorization_tier_2":"Microwave PTP","operational_categorization_tier_3":"Link Down","owner_group":"General Support","priority":"Critical","product_categorization_tier_1":"TES_Managed Networks","product_categorization_tier_2":"Access Service","product_categorization_tier_3":"Cloud Connect","product_name":"","region":"","reported_date":"2020-07-02T16:36:04+02:00","reported_source":"Email","resolution":"","resolution_categorization_tier_1":"","resolution_categorization_tier_2":"","resolution_categorization_tier_3":"","resolution_product_categorization_tier_1":"","resolution_product_categorization_tier_2":"","resolution_product_categorization_tier_3":"","responded_date":"2020-07-02T18:00:43+02:00","slm_real_time_status":"Within the Service Target","satisfaction_rating":"","service_manager":"","service_request_id":"","site_group":"","state_province":"","status":"In Progress","status_reason_hidden":"","street":"","submit_date":"2020-07-02T16:36:04+02:00","submitter":"AR_ESCALATOR","summary":"INC000147465| me-za-gp80-hoedspru-bry-1 | | E2379","time_zone":"","urgency":"1-Critical","vendor_group":"","vendor_name":"","vendor_ticket_number":"","zip_postal_code":""} ... View more

Sorry that doesn't work I changed user preference and time zone (+2) but that too will change my time according to that time zone. This is not what I want which i told several times. I want +2:00 value included in my _time field that is same as log_modified_date field. Hope now you understand what I want . There is definitely some way through time format or transforms we can achieve this : 7/2/20 6:00:44+02:00 PM   Can anyone help me on this? ... View more

Ahh that might work. Could you please elaborate how and where to change in user preference and timezone(+02:00) ?   Thanks ... View more

Hi @to4kawa  , is 2020-07-02T18:02:18 ? - No it doesn't mean I want +02:00 value separately only along with time: 7/2/20 6:00:44+02:00 PM             like this i want in _time. Please help me getting this and as I told earlier that +02:00 value is fixed with each timestamp so you can leverage of adding hardcore as well, i won't mind just output should be same. Thanks ... View more

Hi , Thanks for your quick response but it is also not giving output as expected. using transforms.conf , you are adding +2:00 value in time and thats why when running it is giving output: 7/2/20 6:02:44.000 PM But as I told I want : 7/2/20 6:00:44+02:00 PM Exactly the same way as in sample log last_modified_date has value. Please try to understand I don't want to add that value in time, i want in same format as mentioned .   ... View more

This timeformat will not work , with adding %：z it will just convert time as per timezone . I hv already tried what you suggested. As i said i want time as it is with + value mentioned. Note +02：00 is fixed with each timestamp in events. So in case if we can’t get in timeformat ,can we add explicitly at index time. Pls suggest. And i have to use time format because there is 1 more time field in data which splunk detecting automatically. ... View more

Hi ,   I am able to get at search time what I want but unable to achieve at index time My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name log_modified_date. i have written below props.conf: [_json] INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = last_modified_date TIME_FORMAT = %Y-%m-%dT%H:%M:%S+%2N:%2N MAX_TIMESTAMP_LOOKAHEAD = 25     and getting time extracted as :  7/2/20 6:00:18.020 PM     I want the time field extracted in same way as in data with + value as well like:   7/2/20 6:00:18+02:00 PM    something like this Please let me know what i am doing wrong as i am not getting expected output. ... View more

Hi ,   The TIME_FORMAT you provided doesn't work for me. My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name log_modified_date. i have written below props.conf: [_json] INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = last_modified_date TIME_FORMAT = %Y-%m-%dT%H:%M:%S+%2N:%2N MAX_TIMESTAMP_LOOKAHEAD = 25     and getting time extracted as :  7/2/20 6:00:18.020 PM     I want the time field extracted in same way as in data with + value as well like:   7/2/20 6:00:18+02:00 PM    something like this Please let me know what i am doing wrong as i am not getting expected output. ... View more