I'd like to help you. Can you post a sample of the syntax you're trying to use using? Please be mindful not to post sensitive information. I think you just want to do ("BigTechU" OR "error") but I'm not totally clear on what you're trying to do. Do you want to look for events that contain both keywords or events that contain at least one of them? ... View more

I wanted to add: You generally will want "app" to be the application that the user is logging into. You can use the authentication mechanism (ldap, etc...) in the metadata fields like source or sourcetype. ... View more

There are a few ways to go about this and the optimal solution depends on specifics of your data. Do the events come in every five minutes or is that just an example? ... View more

It sounds like you want to do a field extraction for the log level. You can do this through the user interface by dropping down "Settings" then going to "Fields" then going to "Field Extractions" and using the wizard to create the extraction. You can also do this through the props.conf file directly if you're familiar with that syntax. https://docs.splunk.com/Documentation/Splunk/7.2.3/Knowledge/ExtractfieldsinteractivelywithIFX https://docs.splunk.com/Splexicon:Fieldextraction ... View more

Are you wanting to do this in a dashboard on inline in a search? ... View more

Thanks for accepting my answer, but feel free to accept Rich's answer if you can. Sharing knowledge with and helping other Splunk users is what's important to me. I don't pay attention to the Internet points. ... View more

do you still use the AWS app, or do you mainly use your own app that you ported Both, but for different purposes. The AWS app is great for getting the data into Splunk in a clean way. Specifically, "inputs.conf", "aws_description_tasks.conf", "aws_account_ext.conf", and the other data ingestion configs are what we use. We created our own app for reports, dashboards, field aliasing, and scheduled searches. Each AWS customer has a different setup and different things they want to do with the data, so it's hard for Splunk to develop one central app that can account for everyone's needs. As you astutely pointed out, the AWS app comes with a lot of content and not all of it is needed for you. For a different customer, they might want to use the stuff that you don't want to use. I think that's part of @richgalloway's logic for creating your own app for that stuff. we'd like to just have our own non-cluttered app. Exactly. You should go make your own that has exactly what you need / want. Following that advice, I'm going to create a company app where the general searching will be mandated This is a great idea. We do that too. We actually have two different company apps: One for the operations team (support, developers, infrastructure admins, etc...) and one for the security team. This lets us segregate content out in a clean way. should we create another company app for AWS? I would recommend that. Think of it like this, you can use the Splunk AWS app to handle ingestion of the data, create your company app to handle user knowledge objects like scheduled searches, dashboards, and reports, and your own company AWS app to handle any field aliasing or other data normalization you might want to do that meets your specific needs. You just have to make sure the permissions are setup correctly that the apps can all use the necessary resources across apps. One major benefit of this setup is that it creates a blast radius between configurations and knowledge objects for each app. Say if you pivot away from AWS and start using Azure instead, you can just disable / delete / archive the AWS apps and not worry about having to surgically remove that content from your company app. Among other benefits... 🙂 What do you name your AWS Indexes? We do something similar to what you are describing. We don't aggregate CloudTrail into a security index though. We just send those to the appropriate prod / QA / dev indexes, and the security team has access to those indexes. I don't consider CloudTrail to be security content per se, so there's no need for us to put that in the security index. I think that answers all of your questions, but if I've missed something, or you have additional questions, please let me know. Edit to add: Depending on your needs, make use of data models and/or accelerated reports for high-volume CloudTrail and ELB access log data. This might require a some tweaking to get exactly what you need, but it's worth it. ... View more

I work with AWS data in Splunk, and I have a few tips. First, I agree with everything that Rich Galloway said. That's great advice for all Splunk apps. AWS can be a massive amount of data depending on how many accounts you have, what services you use, and other specifics for your use case. Here are my suggestions: Account Lookup Table One thing that I've found incredibly useful is to use a lookup table for human-friendly names for the accounts. I don't know what account ID 123456789 is just by the number, so I use the lookup table to enrich the account ID with something like "prod_application_stack." This is very useful for creating metrics and dashboards that you can present to people, but mostly my infrastructure admins are more responsive to a human-friendly name than an account ID. AWS Description Data When you setup your ingestion for AWS Description (sourcetype=aws:description) make sure to configure aws_description_tasks.conf to pull for ALL available services for ALL available regions, and not just the ones you think you use. I've identified instances where developers forgot to be in the correct region, and ended up putting resources somewhere they shouldn't. You could also detect account compromise this way (if you have a security inclination) because bad guys will sometimes create resources where they think you aren't looking. I think the default ingestion rate for description data is five minutes; which we throttled back to reduce Splunk license utilization. We just don't need the data updated that often. Figure out what's right for you to balance your license usage. If you wanted it to pull every 15 minutes, your stanza should look like this: [desc:account_id] account = Instance profile for your heavy-weight forwarder aws_iam_role = IAM role that has access to the account (if using multiple accounts) apis = ec2_instances/900,ec2_reserved_instances/900,ebs_snapshots/900,ec2_volumes/900,ec2_security_groups/900,ec2_key_pairs/900,ec2_images/900,ec2_addresses/900,elastic_load_balancers/900,classic_load_balancers/900,application_load_balancers/900,vpcs/900,vpc_subnets/900,vpc_network_acls/900,cloudfront_distributions/900,rds_instance/900,lambda_functions/900,s3_buckets/900,iam_users/900 index = AWS INDEX regions = us-west-2,us-west-1,us-east-1,us-east-2,eu-central-1,ap-northeast-1,ap-northeast-2,ap-northeast-3,ap-south-1,ap-southeast-1,ap-southeast-2,ca-central-1,cn-north-1,cn-northwest-1,eu-west-1,eu-west-2,eu-west-3,sa-east-1 sourcetype = aws:description Field Parsing The field parsing out of the box with the app is okay, but I've had to customize it a bit for my purposes. I would validate that the fields work the way you want them to work. If they don't, there's no real harm in creating your own props.conf file and sticking it in a "local" directory in the AWS app. Guard Duty If you're not planning to use AWS Guard Duty, you should. AWS can detect things that you can't because they have access to underlying data that they don't expose to you. This will be a real lifesaver if anything funky happens. Trust me. CloudTrail CloudTrail is awesome, but it's also riddled with lots of nested JSON. This is one of the big areas that I forked from the main AWS app. I don't like working with dot notation in my field names, and I don't want to impose that on my users. It makes SPL unnecessarily long and ugly. At a minimum, I recommend using props.conf to field alias out of the JSON dot notation if you can. Miscellany I've found creating asset inventories to be invaluable. I'm currently using a method where I have a scheduled search that runs every few hours, and outputs my EC2 inventory data to a lookup table. I'm in the process of re-writing that to utilize the Inventory datamodel instead. I recommend you do something similar. If you know for a fact that there are only certain regions you use, create some dashboards (or even better scheduled searches that notify someone) if new resources or CloudTrail API activity occurs in the regions you don’t use. There are things that happen by default, but you should be able to tune out the noise and identify when something that shouldn't be there gets created. Audit your security groups. It's kind of a pain because of the nested JSON. This will give you a good view of your ingress security group rules: index=* sourcetype=aws:description source=*ec2_security_groups | spath path=rules{} output=a | table account_id, description, id, name, owner_id, region, vpc_id, a | fields - _raw | mvexpand a | spath input=a | where from_port!="null" AND to_port!="null" AND 'grants{}.cidr_ip'!="null" | fields - a This will give you the same view of your egress security group rules: index=* sourcetype=aws:description source=*ec2_security_groups | spath path=rules_egress{} output=a | table account_id, description, id, name, owner_id, region, vpc_id, a | fields - _raw | mvexpand a | spath input=a | where from_port!="null" AND to_port!="null" AND 'grants{}.cidr_ip'!="null" | fields - a And if you want to specifically look for potentially insecure rules, I'd add a match for "/0" to the final where clause: index=* sourcetype=aws:description source=*ec2_security_groups | spath path=rules_egress{} output=a | table account_id, description, id, name, owner_id, region, vpc_id, a | fields - _raw | mvexpand a | spath input=a | where from_port!="null" AND to_port!="null" AND 'grants{}.cidr_ip'!="null" AND match('grants{}.cidr_ip', "/0") | fields - a That will check for rules that are set with "0.0.0.0/0"; which means any IP address. If you see that and it's not something like port 443 or port 80, you might want to investigate why that rule exists. That's off the top of my head. I'll comment again if I think of anything else. ... View more

Hello, I see that you already accepted an answer, but I wanted to inform you of the CIM Validator app. It has a csv lookup table of all of the CIM fields and their data model mappings. The original app is here: https://github.com/hire-vladimir/SA-cim_vladiator I forked it and updated the dictionary to be CIM 4.12.0 (current) compliant: https://github.com/zoneice/SA-cim_vladiator Here's a link directly to the csv if you aren't interested in the full app: https://github.com/zoneice/SA-cim_vladiator/blob/master/lookups/cim_dictionary.csv ... View more

I understand now. Thank you for the clarification. I think your best bet is to feed events into the data model and then create a separate search or report that pulls out the latest scan events. I'll try a few things to see if I can get it to work and let you know. ... View more

I'd like to help you, but I need more information. There are ways to search data models using pipes. For example, we can use tstats to search the authentication data model and use trailing pipe commands: | tstats `summariesonly` values(Authentication.app) AS app count from datamodel=Authentication.Authentication WHERE Authentication.user!="unknown" by Authentication.action,Authentication.user | `drop_dm_object_name("Authentication")` | eval success=if(action="success",count,0),failure=if(action="failure",count,0) | stats values(app) as app,sum(failure) as failure,sum(success) as success by user | where success > 0 | xswhere failure from failures_by_src_count_1d in authentication is above medium | sort - failure | eval failure = tostring('failure',"commas"), success = tostring('success',"commas") We can also use the "|from datamodel" syntax: | from datamodel:"Authentication"."Authentication" | search user="malicious_user" errorCode="AccessDenied" | stats count by app Can you provide some additional info about what data model you're working with, and maybe a sample of the search you have so far? ... View more

Data models can be interacted with in multiple ways. Can you provide the search that you're working on? Also, is the data model accelerated? ... View more

You have to change "fieldForLabel" and "fieldForValue" to whatever you're passing for your token. Essentially, they should both be set to whatever you rename your field to in the dynamic search query. If you want it to be Choice4 then it should look like this: <init> <set token="choice_four"> "4" </set></init> ... <input type="multiselect" token="field1"> <label>field1</label> <choice value="1">Choice1</choice> <choice value="2">Choice2</choice> <choice value="3">Choice3</choice> <fieldForLabel>Choice4</fieldForLabel> <fieldForValue>Choice4</fieldForValue> <search> <query>|localop| stats count | eval count = $choice_four$ | rename count AS Choice4</query> <earliest>-15m</earliest> <latest>now</latest> </search> </input> ... View more

Do you need it to be part of the static options? You can use a token in the dynamic portion of the multi-select options: <init> <set token="token_test1"> "tester1" </set></init> ... <input type="multiselect" token="field1"> <label>field1</label> <choice value="1">aaa</choice> <choice value="2">bbb</choice> <choice value="3">ccc</choice> <fieldForLabel>token_value</fieldForLabel> <fieldForValue>token_value</fieldForValue> <search> <query>|localop| stats count | eval count = $token_test1$ | rename count AS token_value</query> <earliest>-15m</earliest> <latest>now</latest> </search> </input> This populates with the token and token value. ... View more

Hi, I spent some time figuring out a way to get it to filter out Mondays without having to update the search every Monday. | eval date_filter = strftime('_time',"%a") | eval todays_date=strftime('_time',"%Y/%m/%d") | eval start_time_filter = strptime("06:00:00 AM","%H:%M:%S %p") | eval start_time_filter=strftime('start_time_filter',"%H:%M:%S %p") | eval start_time='todays_date' + " " + 'start_time_filter' | eval start_time=strptime('start_time',"%Y/%m/%d %H:%M:%S %p") | eval end_time_filter = strptime("11:00:00 AM","%H:%M:%S %p") | eval end_time_filter=strftime('end_time_filter',"%H:%M:%S %p") | eval end_time='todays_date' + " " + 'end_time_filter' | eval end_time=strptime('end_time',"%Y/%m/%d %H:%M:%S %p") | eval fake_start_time=strptime("1990/09/24 06:00:00 AM","%Y/%m/%d %H:%M:%S %p") | eval fake_end_time=strptime("1990/09/24 11:00:00 AM","%Y/%m/%d %H:%M:%S %p") | eval start_time = if('date_filter'="Mon",'start_time','fake_start_time') | eval end_time = if('date_filter'="Mon",'end_time','fake_end_time') | where _time > end_time OR _time < start_time You could shorten this by putting some of the eval statements together, but I left them like this for ease of reading. This creates logic to check if today's day is Monday and if it is then create a filter for today's date from 6 to 11 am. If it's not Monday then we set the filter to 1990 which will always be greater than the end_time in our where statement. This uses the event time stamp itself vs. the other method of hard-coding a date. I think this is what you're looking for exactly, but let me know if you need anything else. ... View more

I have two solutions for you. If we stick with the way you were doing it then it could look like this: | eval weekDay = strftime(_time,"%a") | eval HourOfDay = strftime(_time,"%H") | where weekDay!="Mon" OR (HourOfDay!="06" AND HourOfDay!="07" AND HourOfDay!="08" AND HourOfDay!="09" AND HourOfDay!="10" AND HourOfDay!="11") It's not the prettiest but it works. Another way to do it is by creating an explicit time range and filtering that out with a where clause: | eval start_time=strptime("2018/09/24 06:00:00 AM","%Y/%m/%d %H:%M:%S %p") | eval end_time=strptime("2018/09/24 11:00:00 AM","%Y/%m/%d %H:%M:%S %p") | where _time > end_time OR _time < start_time The downside of this method is that you have to update the date to reflect the latest Monday. ... View more

I was just playing around with it and it turns out you don't even need the timechart. You can get it done just by adding the _time field to your by clause: | tstats summariesonly=true count from datamodel=Authentication by "Authentication.action",_time | search Authentication.action!="unknown" | trendline sma5(count) AS trend ... View more

Hi, I think you need to add the _time field to your by clause and then use timechart followed by the trendline function: | tstats summariesonly=true count from datamodel=Authentication by "Authentication.action",_time | search "Authentication.action"!="unknown" | timechart count | trendline sma5(count) AS trend That worked for me. Edit: Actually, that didn't work. I just realized it was going off the count of the number of "count" rather than the values you were looking for... I think this works. | tstats summariesonly=true count from datamodel=Authentication by "Authentication.action",_time |rename Authentication.action AS action | search action!="unknown" | timechart sum(count) AS count by action | trendline sma5(count) AS trend I renamed the action field to something easier to use and then did a sum of the count from tstats by action. This looked right to me based on the trendline before and after adding the timechart. ... View more

Oh, I just realized something. If we change the AND statement in my first comment to an OR statement then it does exactly what you're asking to do: | eval weekDay = strftime(_time,"%a") | eval HourOfDay = strftime(_time,"%H") | where weekDay!="Mon" OR HourOfDay!="06" ... View more

Apologies! It's the way the AND statement is getting processed in the search. One solution is to combine the two fields and filter based on the combined field: | eval weekDay = strftime(_time,"%a") | eval HourOfDay = strftime(_time,"%H") | eval date_time_filter = 'weekDay' + "-" + 'HourOfDay' | where date_time_filter!="Mon-06" I tested this and it works for me. Let me know if it works for you. ... View more