Using the REST api, I am currently retrieving a set of events from Splunk and extracting all of the field names and log sources, simultaneously building a map of log sources and fields belonging to them. Is there any way that I can retrieve this data with a minimal payload? For example, if I pull back 1 record that is from LogSource1 and has Property1 equal to [some really long string], I really don't want that whole string back. I just need to consume LogSource1 and Property1. I'm open to any ideas.
source="A" |table * | foreach * [ eval <<FIELD>>="sourceA" ] |append [ search source="B" | table * | foreach * [ eval <<FIELD>>="sourceB" ] ] |stats values(*) as * | transpose 0 | where mvcount('row 1')=1
This query shows the fields from only one source. How about this?
You are making this impossible. You need to back all the way and explain the problem FULLY and clearly.
Where is Splunk in this? The source of the data? The destination of the data? You have told us almost nothing. You need to try again and give ALL the details.
...it is the splunk REST api, sir. That is where the events are located.
That is not all the details; that is just one.
I'm looking for suggestions to optimally retrieve event data via splunk's API aside from loading the entire event. I currently send basic SPL queries with a time range and pull out the fields and sources I see.. that results in gigantic payloads which I extract only those 2 pieces of data. I'm not sure what else needs to be clarified. I know about the field summary option, but that doesn't give me the log sources used for each field.
You can always end your SPL
with | table Just the fields I need
.
The issue is that I don't know what fields are available since we have several log sources.
your search
| fieldsummary
try this and check your fields.
@to4kawa I've alluded to that already. The issue is that it won't indicate which sources contained the fields.
after searching, select source
from left side extract fields
and then, check your fields again.
Thanks @to4kawa - I'm not sure what the SPL looks like for this but I'll try to play around with this. In the end, I want to be able to tell senior mgmt "here are the 10 fields we have, and these 2 are from source 1 while these 2 come from source 2" for today" so this seems to be closer to what I'm looking for.