About jeffrey_berry

jeffrey_berry · ‎06-20-2022

In our environment (Phantom version 4.10.3.x), the HEC (HTTP Event Collector) server name that is used as an "Indexer Host" (i.e. Phantom UI field label for the HEC server for a "Distributed Splunk Enterprise Deployment" ) was changed recently. The new server name was entered into the "Indexer Host" field, "Test Connection" was successful, and "reindex" was successful. However later it was noticed that no new event data was being ingested into the Splunk Enterprise phantom* indexes. The resolution was to restart Phantom and "reindex " again for the missing events in the phantom* indexes. It is suspected that the "process" for ingesting new events into the phantom* indexes is not updated with the changes to the "Indexer Host" field until Phantom is restarted; however, the "processes" for "Test Connection" and "reindex" appeared to work without a Phantom restart. No references that a Phantom restart is required was found in the online documentation. Does anyone have more on this issue/bug/phenomenon and/or has anyone else experienced this issue/bug/phenomenon?

jeffrey_berry · ‎03-02-2022

@phanTom In a large organization as you may know, the focus and expectations of users can be vary widely. It sounds like your energy may be low for a feature request to allow customization of the Phantom dashboard time filters. With one of my many roles being a Phantom admin, I do not desire to understand and maintain an additional setting that someone suggests is value added to Phantom unless several folks agree with it. What are your thoughts on a feature request for text next to the time filter dropdown that communicates the create time of the oldest container record or something similar to give the user more info about the data in the Phantom dashboard (see1st screenshot below)? Do you think that the Phantom community would be more supportive of it? As a comparison, the Splunk Enterprise search dashboard has a graph to indicate that amount of data available (see second screenshot below). Custom Splunk Enterprise dashboards much more capabilities to fulfill user requests like this one than the Phantom product. For a more direct comparison, a Splunk Enterprise user can select their own home dashboard (custom or not) in the Search app (see third screenshot below).

jeffrey_berry · ‎03-02-2022

How to customize the Phantom dashboard time filters dropdown box (see screenshot below)? For a Phantom instance, we have started exploring using the data retention features of Splunk Phantom keeping less than 1 year of Phantom data. It is desired to have a maximum filter equal to the current number of days for data retention. Otherwise, users are misled by time filters that are more than current number of days for data retention. A feature that might nice to have is a way to tie the Phantom dashboard time filters dropdown box to the days of data retention.

jeffrey_berry · ‎08-06-2020

What is best practice for the HEC endpoint(s) for the "Phantom Remote Search" app in a clustered environment? Per the instructions in the url below for configuring the "Phantom Remote Search" app in a distributed environment, the HEC endpoint(s) are implied to be indexer server(s). https://docs.splunk.com/Documentation/PhantomRemoteSearch/1.0.14/PhantomRemoteSearch/Connecttodistributedsplunk Our environment uses clustered indexers. Can a heavy forwarder with a HEC endpoint be used to externalize search of a Phantom environment instead of the HEC endpoint(s) being on the indexer(s)?

jeffrey_berry · ‎01-21-2020

Thanks @darrenk_splunk . Based on the info that you provided, I agree...it is not a bug. The Linux "cat" and grep commands help explain the unexpected output also. [root@localhost bin]$ cat /opt/splunk/etc/system/default/inputs.conf | grep "monitor://" [monitor://$SPLUNK_HOME/var/log/splunk] [monitor://$SPLUNK_HOME/var/log/watchdog/watchdog.log*] [monitor://$SPLUNK_HOME/var/log/splunk/license_usage_summary.log] [monitor://$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*] [monitor://$SPLUNK_HOME/etc/splunk.version] [root@localhost bin]$

jeffrey_berry · ‎01-07-2020

@rob_jordan With your confirmation and confirmation from other users, I am just reporting a possible bug (update 1/21/2020: not a bug...see answer below), and spreading awareness that the btool may not return the expected output for certain input. My question here was worded in the off-chance that it is not a bug, and the output could be explained. Per a recent Data Admin training class, the entire stanza name can be included in the btool command. For another example, the "monitor:///var/log" stanza (i.e. entire stanza name) returns the expected output (see below). However, certain stanzas in the default inputs.conf file do not return the expected output. I am aware that it appears that using grep is a work around. However, I would think that you would agree that It is inconsistent behavior of the btool command, and Splunk users should be aware of it. [root@localhost bin]$ ./splunk btool inputs list monitor:///var/log [monitor:///var/log] _rcvbuf = 1572864 blacklist = (lastlog|anaconda.syslog) disabled = false host = localhost.localdomain index = default whitelist = (.log|log$|messages|secure|auth|mesg$|cron$|acpid$|.out) [root@localhost bin]$

jeffrey_berry · ‎01-06-2020

@BainM Mike, I am glad that you were able to resolve the issue(s) with your custom index. No credentials for the network node(s) in your Node List are required for the "Check ssh Port Open" Command Type. The "Check ssh Port Open" Command Type uses the Python socket library to check if the ssh port is open which does not require logging into the ssh server of the network node(s). Regards, Jeff

jeffrey_berry · ‎01-06-2020

@rob_jordan To re-create the issue, you have to include the name of the stanza with the btool command. [splunk@localhost bin]$ ./splunk btool inputs list monitor:///opt/splunk/var/log/splunk [splunk@localhost bin]$ Based on the output in your post, the stanza name would be monitor:///opt/spl/splunk/var/log/splunk for your Splunk instance.

jeffrey_berry · ‎01-03-2020

@BainM I just created an index named cliautoIdx using the Splunk Web GUI. Splunk changed the uppercase "I" to a lowercase "i"; so the uppercase "I" could be causing the error(s).

jeffrey_berry · ‎01-03-2020

@BainM Based on the info provided, the error(s) appears to occur on the following line of code in the cliauto_index.py file which retrieves the index object using the index name in the objcfg.index string variable using the Splunk Python SDK. I would guess the CLI Auto for Splunk app (or the user using the app) does not have proper permissions to the cliautoIdx index. # Retrieve the index for the data self.myindex = self.service.indexes[objcfg.index] You might try searching the cliautoIdx index from the Search dashboard in the CLI Auto for Splunk app using the same user (i.e. to mimic the same app and user context that generated the error(s)) to verify read permissions to the cliautoIdx index. The app (and user) will need write permissions to the cliautoIdx index also to submit and execute a job successfully in the app.

jeffrey_berry · ‎01-03-2020

@BainM Splunk Answers website removed the backslash character in the second btool comand. I was able to edit my question, and add another backslash to get one backslash to appear. Adding the backslash should fix the regex error. I am using a CentOS VM. I agree...it is an odd response from btool. Other stanzas with a /opt/splunk/var/log parent path are not returned by "splunk btool inputs list" command also using the entire stanza name.

jeffrey_berry · ‎01-03-2020

Does a reason exist that btool does not return the inputs.conf stanza for the Splunk log folder (/opt/splunk/var/log/splunk)? [splunk@localhost bin]$ ./splunk btool inputs list monitor:///opt/splunk/var/log/splunk [splunk@localhost bin]$ It looks like a bug to me since the using the keyword "monitor" returns the stanza for it. [splunk@localhost bin]$ ./splunk btool inputs list monitor | grep "\[" [monitor:///Library/Logs] [monitor:///etc] [monitor:///home/*/.bash_history] [monitor:///opt/splunk/etc/splunk.version] [monitor:///opt/splunk/var/log/introspection] [monitor:///opt/splunk/var/log/splunk] [monitor:///opt/splunk/var/log/splunk/license_usage_summary.log] [monitor:///root/.bash_history] [monitor:///var/adm] [monitor:///var/log] [splunk@localhost bin]$

jeffrey_berry · ‎01-03-2020

@BainM Question #1 answer- Yes, up to 4 variables (var1, var2, var3, and var4), 3 passwords (pw1, pw2, & pw3), and special variable drop1 can be configured in a stanza of the cliauto_cmds.conf file to be passed via the custom Splunk REST API endpoint from the cliauto dashboard to the server side script engine. An example is the cli_cpfwguds stanza (CPFW Get User's Default Shell) in the cliauto_cmds.conf file of the default folder that uses variable (var1) to pass an username to get it's default shell. The var1 variable is referenced by adding @ characters (e.g. @ var1@ ) in the command line string (see key cmd04 in the stanza). Question #2 answer- To post server script execution results to a dashboard without indexing the results is not possible without a code change and/or redesign of the app. Our use cases required the results data to be logged; so the time to add this feature cannot be justified. and it would make the app more complex to add it as a feature. Also, some of the CLI Auto Command Types (i.e. a set of CLI commands defined by a cliauto_cmds.conf file stanza) can produce a large amount result data that Splunk Web and Splunk Search helps the user analyze the results data. I would think that Splunk favors apps that ingest data due to licensing. LOL! I have not used the app for this use case. Please let me know if it works for you. Regards, Jeff

jeffrey_berry · ‎12-19-2019

@psychogyiokostas Based on your responses and other answers here, I would guess that both methods for ingesting the data (CLI and UI) is indexing all of the data, but during the indexing process, the data is ingested with the event break configuration for the sourcetype being different due to the process performing the ingestion using a different context. The props.conf file(s) controls the event breaks, but there are (can be) several props.conf files in a Splunk environment. See https://docs.splunk.com/Documentation/Splunk/8.0.0/Admin/Wheretofindtheconfigurationfiles which explains the configuration (conf) file precedence. Also, the existence of multiple contexts (i.e global and app contexts) which follow different configuration (conf) file precedence does not make it easier. At the CLI, you can execute the following splunk command to view the props.conf settings being used for the context at command prompt, but you will need to know which stanza applies for ingesting your data. Unfortunately, the btool command is not supported in the UI, but Splunkbase contains some "btool" apps that may be helpful. A Splunk feature request could be to ask for better tools to view configuration (conf) file precedence based on an user chosen context. ./splunk btool props list --debug P.S. The Splunk Answers tag "CLI Auto for Splunk" is for an app that I developed and shared on Splunkbase. While I appreciate the visibility for the app, your question does not seem related to this app. The "CLI Auto for Splunk" app is an automation tool for executing CLI commands on any network device with a CLI (ssh) interface. You might try searching for other Splunk Answers tags that might be more related to your question.

jeffrey_berry · ‎10-16-2019

We ended up using the "Splunk DB Connect" app which query's the Solarwinds tables for the needed data. It requires a bit of knowledge about the Solarwinds DB schema and "checkpointing" a Splunk data source for Solarwinds logging tables. Also, we replicated the Solarwinds NPM CustomPollerStatistics_Detail table (and added an identity column to this replicated table) using MS SQL server replication since the DateTime field was not a good checkpoint field for the "Splunk DB Connect" app. Also, see these Solarwinds Thwack and Splunk answer posts https://thwack.solarwinds.com/thread/120056 and https://answers.splunk.com/answers/596395/what-are-the-steps-to-configure-an-incremental-pol.html .

jeffrey_berry · ‎05-22-2019

Users of a REST API can "ingest ANY data for ANY sources" also. As an example, a REST API could have an input parameter for a query to a third party database, log files, and/or some other data source(s) which has different data and/or sources. I am not familiar with the JIRA process for the "Splunk Add-on Builder" app. Do you have url link for it?

jeffrey_berry · ‎05-21-2019

Other Splunk supported apps expose the source and source type that allows end users to set the values as needed. For example, the Splunk DB Connect app (see the doc at https://docs.splunk.com/Documentation/DBX/3.1.4/DeployDBX/Createandmanagedatabaseinputs ). Are you aware of a way to expose the source and source type for a created add-on app for a REST API data input? The data can ingested by a REST API data input without writing Python code for it.

jeffrey_berry · ‎05-21-2019

I agree with you that an "unauthorized" end user of a Production environment should not be granted rights to change settings like source and source type. However, administrator end users should have the ability to change the source and source type of a created add-on app without the need of the developer to change the settings in the "Splunk Add-on Builder" app. I do not follow how the url link that you provided assists with adding source and source type fields to a created add-on app. The "Splunk Add-on Builder" app allows an end user of created add-on app to change/manage the index without any additional configuration (see https://docs.splunk.com/Documentation/AddonBuilder/2.2.0/UserGuide/Usetheaddon ), but not the source and source type. If the index is allowed to be managed by an end user of a created add-on app, why is it not allowed for the source and source type to be managed by an end user ? Can you provide more details to add the source and source type input fields to a created add-on app to manage the source and source type of the ingested data by the created add-on?

jeffrey_berry · ‎05-21-2019

I have recently been investigating using the "Splunk Add-on Builder" to ingest data from a REST API, and I had with a similar issue diagnosing the http requests sent from the created add-on app. The Burp Suite Community Edition app (Burp Proxy manual tool) helped troubleshoot the https request from the created add-on app and the https response from the REST API. The created add-on was configured to use the Burp Suite app as a proxy. The https requests from the created add-on are logged in the Burp Suite app, and the responses from the REST API are logged also. Also, the Burp Suite allows the user to intercept the https requests from the created add-on app, and the user can manually modify before forwarding to the REST API. The Burp Suite Community Edition app is free to download at the https://portswigger.net/burp url.

jeffrey_berry · ‎05-21-2019

The above links to docs give instructions for managing the source and source type within the "Splunk Add-on Builder" app which is not the question. I will re-phrase...How can an user within a created add-on app by given an input screen to manage the source and source type? For example, a data input parameter can be created in the "Splunk Add-on Builder" to allow the user of the add-on app to customize the values sent for a REST API, but a data input parameter cannot be created for the source or source type.

jeffrey_berry · ‎05-21-2019

How do you allow an user of an add-on created by the "Splunk Add-on Builder" app to configure the source and source type of the input data? It appears that this feature does not exist. If the feature does not exist, has anyone been able to modify the config files of an add-on built using the Add-on Builder app to change the source and source type?

jeffrey_berry · ‎02-04-2019

Can someone point me to the best practice for providing full CRUD (Create, Read, Update, & Delete) UI features ffor Splunk conf files with a third party application for Splunk? (with emphasis on the Delete part of the CRUD acronym) I have searched the Splunk websites (answers, docs, etc.) and reviewed apps from splunkbase, but the design model that allows full CRUD control of a conf file, stanza, and key/value pair with the REST API alludes me (or some needed REST API features do not exist). Based on the REST API reference for Configuration endpoints, endpoints exist to create, read, and update a conf file, stanza, and key/value pair local folder conf file, but no endpoints exist to delete a conf file or key/value pair in the local folder. The endpoint to delete a stanza works for a local folder stanza created through the REST API, but a local folder stanza cannot be deleted if it exists in the conf file of the default folder also (see error below). Error received to delete a stanza that exists in the local and default conf files Object id=stanza_name cannot be deleted in config=conf_file_name. The documentation in the following urls were used to verify that fully functional configuration deletion endpoints do not exist REST API Reference Manual - Configuration endpoint descriptions https://docs.splunk.com/Documentation/Splunk/7.2.3/RESTREF/RESTconf REST API Tutorials - Accessing and updating Splunk Enterprise configurations https://docs.splunk.com/Documentation/Splunk/latest/RESTTUT/RESTconfigurations With the Splunk configuration file precedence concept, I would guess that fully functional delete endpoints should exist to allow deletion of a conf file, stanza, and key/value pair in a local folder. If deletion is considered dangerous for the local folders of the Splunk application itself, deletion endpoints should exist for the local folder of a third party app to allow an app to conform to the configuration file precedence concept. An example use case is a button on an app's UI to delete a local folder conf file which would revert the conf file settings to the conf file in the default folder. Or maybe I am just missing a keypoint(s) on managing the configuration within Splunk with a CRUD UI.

jeffrey_berry · ‎01-16-2019

Yes...the "CLI Auto for Splunk" app can be configured by adding a Command Type (a set CLI commands added to the cliauto_cmds.conf file) or using the Custom Command Type (adhoc CLI commands) to run CLI commands against the local machine assuming that a ssh server is active on the machine. Also, I assuming that you referring to the Splunk server hosting the "CLI Auto for Splunk" app as the "local machine". However, it should be highlighted that the "CLI Auto for Splunk" has no hard coded host names/hard code ip addresses (e.g. localhost, 127.0.0.1) or a default node list with the installation files. The user of the app must supply a node list (i.e. host name/ip address list) and proper credentials for the local machine. To assist with the "failed publishing" and not able to "pass appInspect" issues of your app, more details may be helpful. Are you getting an error upon uploading your app to splunkbase? If so, please share the error message. If you have successfully uploaded your app to splunkbase, are you able to see the "APPINSPECT STATUS" column in splunkbase (see attached screenshot). The "Details" should provide you the info for inspection failure. If splunkbase is not publishing the app (i.e. not allowing other users to see the app on splunkbase) after you scheduled it to be published (I do not recall to details to schedule it for publishing), you may wait a few days for the splunkbase team to approve your app. The automated email from splunkbase that I received for my app stated that it takes about 3 business days for approval. Based on my experience, the splunkbase team does not send an email confirming approval/disapproval for the app or provide a status of approval within your account on splunkbase. You may need to contact the splunkbase team if the app is not published in a few days after scheduling it. As a possible alternative if the "CLI Auto for Splunk" does not meet your use case(s), the "Web Terminal for Splunk" app may be helpful to you since it sounds like your app connects to the Splunk server).

jeffrey_berry · ‎05-04-2018

We had the same issue with DBX on the free version. Upon asking about additional licensing options, the "Splunk Developer License" or the "Personalized Dev/Test Licenses for Splunk Customers" were discussed. As of the time of this post, the "Splunk Developer License" and "Personalized Dev/Test Licenses for Splunk Customers" are free licenses for development/testing purposes. The info that I received is the "Splunk Developer License" is limited to 10GB per day, and no requirement exists to be a Splunk customer. The "Personalized Dev/Test Licenses for Splunk Customers" license is limited to 50GB per day (urls below). "Splunk Developer License" - for non-customers http://dev.splunk.com/page/developer_license_sign_up "Personalized Dev/Test Licenses for Splunk Customers" https://www.splunk.com/en_us/resources/personalized-dev-test-licenses.html The following url does not mention any additional limitations like disabled features like authentication capabilities, but Splunk version 6.5.0 or later is listed as a requirement. http://docs.splunk.com/Documentation/Splunk/7.1.0/Admin/TypesofSplunklicenses

jeffrey_berry · ‎01-25-2018

Thanks rich7177 for your response... After some research, we used the "Splunk DB Connect" app (similar to the "Splunk Add-on for Microsoft SQL Server" app") to get data from our Solarwinds installation. "Splunk DB Connect" app offers significantly more features than the "SolarWinds Add-on for Splunk" app. However as you alluded, it requires some level of knowledge of the Solarwinds DB schema to SQL query the Solarwinds DB effectively. The SWQL API can be easier at first look because data is organized into single "SWQL tables". MS SQL Server has tools like SQL Profiler to "sniff" the SQL queries that the Solarwinds application and/or SDK executes to gain an understanding of the Solarwinds DB schema. Also, the SWQL queries (aka Solarwinds API function calls) used by the "SolarWinds Add-on for Splunk "app are logged to the "_internal" index of Splunk in an http url encoded string which can be translated into SQL queries if you have enough knowledge of the Solarwinds DB schema and http url encoding. An example Splunk search query to help find the SWQL queries of the "SolarWinds Add-on for Splunk" app logged to the Splunk _internal index is below. index="_internal" nodes source="*solarwinds*" Our biggest issue for adding new capabilities related to Solarwinds at this time is our version of Solarwinds NPM is several years old. The "SolarWinds Add-on for Splunk" app is not compatible with older versions of Solarwinds NPM and community supported.

Posts	27
Solutions	3
Karma Given	0
Karma Received	5
Member Since	‎12-04-2017

Online Status	Offline
Date Last Visited	‎11-28-2023 03:58 PM

"Search Settings" Indexer Host change requires Pha...

How to customize the Phantom dashboard time filter...

What is best practice for the HEC endpoint(s) for ...

Does reason exist that btool does not return the i...

How do you allow a user of an add-on app created b...

What is the best practice for providing full CRUD ...

What are the steps to configure an incremental pol...

"Search Settings" Indexer Host change requires Pha...

Re: How to customize the Phantom dashboard time fi...

How to customize the Phantom dashboard time filter...

What is best practice for the HEC endpoint(s) for ...

Re: Does reason exist that btool does not return t...

Re: Does reason exist that btool does not return t...

Re: Cannot POST to Custom Index

Re: Does reason exist that btool does not return t...

Re: Cannot POST to Custom Index

Re: Cannot POST to Custom Index

Re: Does reason exist that btool does not return t...

Does reason exist that btool does not return the i...

Re: CLIAuto - Pass Variables to Remote Scripts and...

Re: Splunk CLI and UI give different search result...

Re: Solarwinds Orion and Splunk

Re: How do you allow a user of an add-on app creat...

Re: How do you allow a user of an add-on app creat...

Re: How do you allow a user of an add-on app creat...

Re: Enable REST API response logging in the Splunk...

Re: How do you allow a user of an add-on app creat...

How do you allow a user of an add-on app created b...

What is the best practice for providing full CRUD ...

Re: commands against localhost?

Re: Splunk DB Connect: Splunkd error "HTTP 402 – R...

Re: What are the steps to configure an incremental...