what if 1 field  with string "A" is the substring  of flied "B"? |where B=*A* ,  how can we find out that? ... View more

How UF will be installed on newer version of Splunk with no default password "changeme"??   ... View more

But nothing have changed, the configurations are same in btool. ... View more

Yes have restarted the splunk service after applying changes to the conf files.  Actually it did worked on Friday ,the day changes were applied but it is not working after that . ... View more

https://community.splunk.com/t5/Getting-Data-In/Masking-sensitive-information-from-event/m-p/519664#M87857 ... View more

Hi @lguinn2 , I am trying to remove some of the sensitive information to be indexed by Splunk. But these configurations are not working ,even after getting the configuration reflected over btool and validating the regex over SPL. Can you please have a look on it? props.conf [o365:management:activity] TRANSFORMS-anonymize = info-anonymizer KV_MODE = json TRUNCATE = 10485760 transforms.conf [info-anonymizer] DEST_KEY = _raw FORMAT = $1$2 REGEX = (.*\"SensitiveInformationDetections\"\:\s\{)\"DetectedValues\"\:\s\[.*\]\,\s(\"ResultsTruncated\"\:.*) Have already Validated regex over SPL, It is working fine. |regex _raw="(.*\"SensitiveInformationDetections\"\:\s\{)\"DetectedValues\"\:\s\[.*\]\,\s(\"ResultsTruncated\"\:.*)" and |rex field=_raw "(?<before>.*\"SensitiveInformationDetections\"\:\s\{)\"DetectedValues\"\:\s\[.*\]\,\s(?<after>\"ResultsTruncated\"\:.*)" |eval _raw=before+""+after ... View more

Hi , I know about the internal forwarding but I was experimenting with the REST API Modular Inputs..and I came across this question ... View more

@jkat54 @dpanych Hi, I have checked each line and could not find any syntax errors or problem with indents. I modified the script, but am still getting same exit code. Also, I am using query as search * and I am putting workspace name instead of workspaceID in inputs.conf. Could you please help on this. Any definitive date you are going to repackage app. import os import sys import time import datetime def validate_input(helper, definition): inputs=helper.get_input_stanza() for input_name, input_item in inputs.iteritems(): max_count = str(input_item["max_count"]) start_date = str(input_item["start_date"]) if int(max_count) <= 0: helper.log_error("Max count must be greater than zero (0); found max_count=" + str(max_count)) try: valid_date = datetime.datetime.strptime(start_date, '%d/%m/%Y %H:%M:%S') except ValueError: helper.log_error("Start date must be in the format of dd/mm/yyyy hh:mm:ss") pass def collect_events(helper, ew): import adal import json import requests # Go through each input for this modular input inputs=helper.get_input_stanza() for input_name, input_item in inputs.iteritems(): # Get the values, cast them as floats resource_group = str(input_item["resource_group"]) workspace = str(input_item["workspace_id"]) query = str(input_item["oms_query"]) max_count = str(input_item["max_count"]) subscription_id = str(input_item["subscription_id"]) tenant_id = str(input_item["tenant_id"]) application_id = str(input_item["application_id"]) application_key = str(input_item["application_key"]) # Date and delta if helper.get_check_point('last_date'): start_datetime = datetime.datetime.strptime(helper.get_check_point('last_date'),'%d/%m/%Y %H:%M:%S') else: start_datetime = datetime.datetime.strptime(str(input_item['start_date']),'%d/%m/%Y %H:%M:%S') now = datetime.datetime.now() now_dt = now.replace(microsecond=0) # URLs for authentication authentication_endpoint = 'https://login.microsoftonline.com/' resource = 'https://management.azure.com/' # Get access token context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id) token_response = context.acquire_token_with_client_credentials('https://management.azure.com/', application_id, application_key) access_token = token_response.get('accessToken') # Add token to header headers = { "Authorization": 'Bearer ' + access_token, "Content-Type":'application/json' } # URLs for retrieving data uri_base = 'https://management.azure.com' uri_api = 'api-version=2017-10-01-preview' uri_subscription = 'https://management.azure.com/subscriptions/' + subscription_id uri_resourcegroup = uri_subscription + '/resourceGroups/'+ resource_group uri_workspace = uri_resourcegroup + '/providers/Microsoft.OperationalInsights/workspaces/' + workspace #uri_search = uri_workspace + '/search' # Build search parameters from query details search_params = { "query": query, "top": max_count, "start": start_datetime.strftime('%Y-%m-%dT%H:%M:%S'), "end": now_dt.strftime('%Y-%m-%dT%H:%M:%S') } # Build URL and send post request #uri = uri_search + '?' + uri_api uri = uri_workspace + '?' + uri_api response = requests.post(uri,json=search_params,headers=headers) # Response of 200 if successful if response.status_code == 200: # If debug, log event helper.log_debug('OMSInputName="' + str(input_name) + '" status="' + str(response.status_code) + '" step="Post Query" search_params="' + str(search_params) + "'") # Parse the response to get the ID and status data = response.json() ''' search_id = data["id"].split("/") id = search_id[len(search_id)-1] status = data["__metadata"]["Status"] # If status is pending, then keep checking until complete while status == "Pending": # Build URL to get search from ID and send request uri_search = uri_search + '/' + id uri = uri_search + '?' + uri_api response = requests.get(uri,headers=headers) # Parse the response to get the status data = response.json() status = data["__metadata"]["Status"] ''' else: # Request failed helper.log_error('OMSInputName="' + str(input_name) + '" status="' + str(response.status_code) + '" step="Post Query" response="' + str(response.text) + '"') # Print the results of the search to std_out for i in range(len(data["tables"][0]["rows"])): data1 = "{" #This nested loop goes through each field, in each event, and concatenates the field name to the field value for n in range(len(data["tables"][0]["rows"][i])): field = str(data["tables"][0]["columns"][n]["name"]) value = str(data["tables"][0]["rows"][i][n]).replace('"',"'").replace("\\", "\\\\").replace("None", "") if value == "": continue else: data1 += '"%s":"%s",' % (field, value) data1 += "}" data1 = data1.replace(",}", "}") event = Event() event.stanza = input_name event.data = data1 ew.write_event(event) #Delta state = now_dt.strftime("%d/%m/%Y %H:%M:%S") helper.save_check_point('last_date', state) ****** ... View more