About jlundtristate

jlundtristate

Is splunk laughing even harder, does the second query - the one inside the join run before the query outside the join?

jlundtristate

I am needing to find earlier version number of linux patches. I have to compare many patches, so I was wanting to use a join for two queries (assuming patching happens once a month, but not all packages have an update every month). The first query would get the latest packages patched (with in the last 30 days) - depending on what day of the month the patching occurred - I would like to pass the earliest datetime stamp found minus X seconds (as MaxTime) to the second query. So, the second query could use the same index, source, sourcetype but where latest=MaxTime. Don't try this at home, putting latest=MaxTime-10 in the second query caused Splunk to laugh at me and return "Invalid value 'MaxTime-10' for time term 'latest'"...no hard feelings, Splunk laughs at me often. Thanks for any assistance in advance. JLund

jlundtristate · ‎09-13-2022

If I have a simple dashboard with a time range picker input, how can I add source code to convert the picker selection to a StartDate and EndDate token. StartDate = strftime(earliest, %m/%d/%Y %H:%M:%S), EndDate=strftime(latest, %m/%d/%Y %H:%M:%S) { "visualizations": { "viz_ZgRiQCoQ": { "type": "viz.column", "options": {}, "dataSources": { "primary": "ds_GHdtwfg5" } } }, "dataSources": { "ds_GHdtwfg5": { "type": "ds.search", "options": { "query": "index=_internal \n| top 100 sourcetype" }, "name": "Search_1" } }, "defaults": { "dataSources": { "global": { "options": { "queryParameters": { "latest": "$global_time.latest$", "earliest": "$global_time.earliest$" } } } }, "visualizations": { "global": { "showLastUpdated": true } } }, "inputs": { "input_global_trp": { "type": "input.timerange", "options": { "token": "global_time", "defaultValue": "-24h@h,now" }, "title": "Global Time Range" } }, "layout": { "type": "absolute", "options": {}, "structure": [ { "item": "viz_ZgRiQCoQ", "type": "block", "position": { "x": 0, "y": 0, "w": 300, "h": 300 } } ], "globalInputs": [ "input_global_trp" ] }, "title": "Global time range picker", "description": "" } And then be able to display StartDate and EndDate on the top of the dashboard, so if the user selects Last 24 hrs or Last 30 days - it can be displayed as 09/12/2022 - 09/13/2022 or 08/14/2022 - 09/13/2022?

jlundtristate · ‎03-01-2019

lookup doesn't work because there could be multiple allowedrules for the same source/destination/protocol combination - then the Port value(s) returned from the lookup may have 123 580-590 6000-6500 49152-65535 also, this works for events where the Port falls within the range, but for those that fail the where, how do you still capture the original source, destination, protocol and destinationport and indicate "needs reviewed" example: we need to verify each event in the table and add a field that indicates the status - communication is allowed or is not found in our allowed rules, so it should be reviewed.

jlundtristate · ‎03-01-2019

In my previous question I didn't think a join would work, but somesoni2, proved that it would work. The only problem was it didn't scale. search netactivity | stats count by source, destination, protocol, destinationport 2. | join type=left source, destination, protocol, destinationport 3. [ | inputlookup allowedrules | eval Port=split(Port,"-") 4. | eval s=tonumber(mvindex(Port,0))| eval e=tonumber(mvindex(Port,1)) 5. | eval destinationport=mvrange(s,e+1,1) | mvexpand destinationport 6. | table Source Destination Protocol destinationport ApprovedBy 7. | rename Source as source Destination as destination Protocol as protocol ] As the allowedrules table was being updated, we found the need to have a number of entries between distinct sources and destinations for high ports (49152-65535) over 16,000 ports per source/destination combination. Quickly we found that the mvrange and mvexpand option above, grew the allowedrules table too large to be effective (actually, there must be a limit to an inputlookup or our limits.conf needed to be re-configured because we would find that the table would not grow larger that 12,000 entries. So I am back trying to come up with a better option. I have tried a map search looking for the source, destination and protocol, then parsing the Port (if it contains a "-") to then compare the port (if startport <= Port AND endport >= Port). for some reason search netactivity | stats count by source, destination, protocol, destinationport | map search="| inputlookup allowedrules | search Src=$source$ Dest=$destination$ Proto=$protocol$ | eval Destport=$destinationport$ | table Src, Dest, Proto, Destport" pulls all the allowedrules for the source/destination/protocol combination, but as soon as I add the logic to test if the destinationport falls within the port range, no matches are found: search netactivity | stats count by source, destination, protocol, destinationport | map search="| inputlookup allowedrules | search Src=$source$ Dest=$destination$ Proto=$protocol$ | eval Destport=$destinationport$ | eval tmpPort=split(Port,"-") | eval startport=tonumber(mvindex(tmpPort,0)) | eval endport=if(mvcount(tmpPort)>0,tonumber(mvindex(tmpPort,1)), startport) | search startport<=Destport AND endport>=Destport | eval Status=if(isnotNull(ApproveBy), ApproveBy, "Not Approved") | table Src, Dest, Proto, Destport, Status" Thanks in advance for any help. Jason

jlundtristate · ‎02-26-2019

somesoni2, is there a limit to the number of entries in the join? for the high ports range 49152-65535, because we have a number of source and destination types. I'm afraid that the join/inputlookup is growing too large for the join to match in some instances. Our inputlookup allowedrules file has over 10,000 entries... so is there a way to pass and extra filter or two from the netactivity search to the inputlookup allowed rules something like [| inputlookup allowed rules | search Port=(destinationport from the netactivity search) |...

jlundtristate · ‎02-21-2019

Thanks somesoni2, that worked great, I didn't think join was the answer, but you proved it is... :-}

jlundtristate · ‎02-21-2019

Here is the example in the Splunk documentation: specific.server | stats dc(userID) as totalUsers | appendcols [ search specific.server AND "text" | addinfo | where _time >= info_min_time AND _time <=info_max_time | stats count(field) as variableA ] | eval variableB = exact(variableA/totalUsers) My need has a difference where the (appendcols [ search) is on a different target and the target is a lookup or inputlookup... First search is the source, destination, protocol and destinationport seen in a given time period, the second search is a lookup table that has allowed traffic rules (source, destination, protocol, allowedport(s)), if the allowedport(s) is a range it can be 580-590. search netactivity | stats count by source, destination, protocol, destinationport | appendcols [search inputlookup allowedrules | where Source=source and Destination=destination and Protocol=protocol | eval tmpport=(Port,"-"), portcnt=mvcount(tmpport) | eval startport=mvindex(tmpport,0), endport=if(portcnt>1, mvindex(tmpport,1), mvindex(tmpport,0)) | where startport<= destinationport AND endport>=destinationport | (table/stats/fields) Source, Destination, Protocol, Port, ApprovedBy ] | table source, destination, protocol, destinationport, Source, Destination, Protocol, Port, ApprovedBy For the second search, I am trying to return the ApprovedBy field most importantly, but for validation a testing purposes, having the information from the rule that is being found, is beneficial. So I have tried the table, stats and fields clauses none of which has returned any values. My results are just the fields from the initial netactivity file. I settled on trying to get the appendcols to work as I read the documentation, I believe it is the correct option. Lookup tables don't let me do a where clause and if all the allowed rules were a 1-1 relationship on the port (instead of ranges) maybe that would work better, but the port ranges rule out that option. Even if the allowed rules table was recreated to have a start and end port, lookup doesn't all for <= or >= in the clause. I also looked at join, but again the port range being a single port or a range of ports makes joining by an individual field impossible. I figured this would have been a easy search, but I didn't find an example of anyone doing this. If anyone has implemented something along these lines, I would appreciated their insight. Thanks in advance for any assistance. Jason

jlundtristate · ‎02-21-2019

Here is the example in the Splunk documentation: specific.server | stats dc(userID) as totalUsers | appendcols [ search specific.server AND "text" | addinfo | where _time >= info_min_time AND _time <=info_max_time | stats count(field) as variableA ] | eval variableB = exact(variableA/totalUsers) My need has a difference where the (appendcols [ search) is on a different target and the target is a lookup or inputlookup... First search is the source, destination, protocol and destinationport seen in a given time period, the second search is a lookup table that has allowed traffic rules (source, destination, protocol, allowedport(s)), if the allowedport(s) is a range it can be 580-590. search netactivity | stats count by source, destination, protocol, destinationport | appendcols [search inputlookup allowedrules | where Source=source and Destination=destination and Protocol=protocol | eval tmpport=(Port,"-"), portcnt=mvcount(tmpport) | eval startport=mvindex(tmpport,0), endport=if(portcnt>1, mvindex(tmpport,1), mvindex(tmpport,0)) | where startport<= destinationport AND endport>=destinationport | (table/stats/fields) Source, Destination, Protocol, Port, ApprovedBy ] | table source, destination, protocol, destinationport, Source, Destination, Protocol, Port, ApprovedBy For the second search, I am trying to return the ApprovedBy field most importantly, but for validation a testing purposes, having the information from the rule that is being found, is beneficial. So I have tried the table, stats and fields clauses none of which has returned any values. My results are just the fields from the initial netactivity file. I settled on trying to get the appendcols to work as I read the documentation, I believe it is the correct option. Lookup tables don't let me do a where clause and if all the allowed rules were a 1-1 relationship on the port (instead of ranges) maybe that would work better, but the port ranges rule out that option. Even if the allowed rules table was recreated to have a start and end port, lookup doesn't all for <= or >= in the clause. I also looked at join, but again the port range being a single port or a range of ports makes joining by an individual field impossible. I figured this would have been a easy search, but I didn't find an example of anyone doing this. If anyone has implemented something along these lines, I would appreciated their insight. Thanks in advance for any assistance. Jason

Posts	9
Solutions	0
Karma Given	0
Karma Received	0
Member Since	‎02-21-2019

Online Status	Online
Date Last Visited	9 hours ago

Can a calculated value be used for the latest para...

How convert input date time to token values and di...

Follow-Up: Appendcols or other options for searchi...

Appendcols or other options for searching a sub-se...

Trouble appending columns for a second search

Re: Can a calculated value be used for the latest ...