Hi Folks,
we are ingested the aws vpc flow logs in splunk and able to see the data while searching with index but while searching with source it is not showing any data for that particular source.
for example: while using below search it is showing data.
1. index=* and able to see that source(aws:cloudwatch:vpclogs)
2. index=* | stats count by source , it is display the source(aws:cloudwatch:vpclogs) with 86 event counts.
but while searching with below command.
index=* source=aws:cloudwatch:vpclogs it is not displaying any data.
Thanks,
Sridhar
Either on $SPLUNK_HOME/etc/apps/{App where the source comes from}/fields.conf or $SPLUNK_HOME/etc/system/local/fields.conf, set the following configuration:
[source]
INDEXED = False
Restart Splunk.
NOTE: For Cloud users, have Support do this for you.
To my understanding, this is the reasoning behind this:
INDEXED=True
Setting this attribute to True, tells Splunk that the source has already been extracted during index-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" as their source (metadata).
INDEXED=False
Setting this attribute to False, tells Splunk that the source will need to be extracted during search-time. When we run a search with that source, Splunk will look for events that have "aws.guardduty" in their events (raw data). Otherwise, it will not find anything because the source was not extracted during index-time.
Find documentation here: https://docs.splunk.com/Documentation/Splunk/8.0.3/Admin/Fieldsconf
Then you need to check the default search indexes that are assigned to your user role. Probably the specific index you put the vpc logs into, is not included in the indexes searched by default (ie. when using index=*).
@renjith, I used the quotes but it is not working. index=test source="aws:guraddutty".
@Frank, I have access for that index and able to search other sources from that index.
It's not about whether you have access. There are 2 settings related to indexes in a user role:
I would check that second setting. Quite possible this is set to the default setting of 'main' (if I'm not mistaken), which means index=*
will only look in the main index, while your data is elsewhere. If you want to be able to search using index=*
, you need to make sure the Indexes searched by default are set to contain the indexes you want to search through.
I have cross checked that the particular source is storing the data in test index. but it is not working while searching data with that sources
I tried these command and it is not working.
index=* source=aws:guarddutty
index=test source=aws:guarddutty
source=aws:guarddutty.
Ok, now I'm getting confused.
In your question, you mentioned source=aws:cloudwatch:vpclogs
, now you are searching for another source?
Also: I think it should be source=aws:guardduty
, so with a single 't' in duty?
I given example. the issue is not able to search data with source.
Just to prevent confusion over typo's in examples, can you provide a screenshot, showing the indexed data, showing the value of the source field. And then a second screenshot showing the search you are trying but fails?
Feel free to hide/obfuscate any sensitive info of course.
I could not able to upload the files here.
Just put the screenshots on some online image host and then post the url here.
Try putting the source in quotes (")