Splunk Search

How append will act if the first search return nothing?

bagir32
Explorer

I want to search for a phone number among multiple indexes and I use append to combined the result together but what I found when the first search has no events the second search will not append its result.

the format I use:
search 1 alone returns no events
search 2 alone returns 6 events
search 1 | append [search 2] returns no events

Thanks

Tags (1)
0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

Hmm. NO, it's not supposed to do that. Probably what is happening is that you are running out of time on the search.

Check the "inspect job" screen for messages and errors.


we'll go ahead and give you some more stuff to check, just in case that guess was wrong. When testing these, make sure you use a fixed time period, like earliest=-1d@d latest=@d.

Try these...

your search 1
| append [your search 2]

your search 2
| append [your search 1]

| multisearch  
    [your search 1]
    [your search 2]

In the last case, every command in the search needs to be a distributed streaming command.


If none of that helps you solve the issue, please post your actual searches, minus any confidential data, and we'll see what we can come up with.

View solution in original post

DalJeanis
SplunkTrust
SplunkTrust

Hmm. NO, it's not supposed to do that. Probably what is happening is that you are running out of time on the search.

Check the "inspect job" screen for messages and errors.


we'll go ahead and give you some more stuff to check, just in case that guess was wrong. When testing these, make sure you use a fixed time period, like earliest=-1d@d latest=@d.

Try these...

your search 1
| append [your search 2]

your search 2
| append [your search 1]

| multisearch  
    [your search 1]
    [your search 2]

In the last case, every command in the search needs to be a distributed streaming command.


If none of that helps you solve the issue, please post your actual searches, minus any confidential data, and we'll see what we can come up with.

bagir32
Explorer

Thanks, DalJeanis, you are right the search was timed out, I used maxtime option after append command and I start to see second search results

| append maxtime=600 []

if there are any draw backs of this approach please advise

regards,

0 Karma

cmerriman
Super Champion

is there a way that you can combine these two searches together with a common field using stats instead? can you give us the search stripped of any confidential information so we can better help build a search out?

my suggestion otherwise is to use a quick search1|appendpipe [stats count as testCount] to add a field to search1 called testCount with the count of events from search1. and then you can append search2

0 Karma

bagir32
Explorer

here the search note that rename and eval in the search I used to make sure the two data set has same fieds name

index=indexA earliest=07/17/2017:17:00:00 latest=07/18/2017:22:00:00| rename event_time AS Timestamp, preview AS Content, sender AS SenderNumber, original_recipients AS original_recipients, URL AS URLCTA, domain AS DomainCTA, phone AS PhoneCTA, email AS EmailCTA | eval subject="" | eval Size="" | eval Headers="" | eval Sender_ip_address="" | eval ReporterNumber="" | eval IMEI_Sender="" | eval Type="SMS" | eval Direction="Outgoing" | search original_recipients=* SenderNumber=1234567890
| table Timestamp, subject, Content, Size, Headers, SenderNumber, Sender_ip_address, ReporterNumber, original_recipients, URLCTA, DomainCTA, PhoneCTA, EmailCTA, IMEI_Sender, Type, Direction
| append[search index=indexB  earliest=06/25/2017:17:00:00 latest=07/01/2017:10:00:00 | rename Timestamp AS Timestamp, Content AS Content, SenderNumber AS SenderNumber, ReporterNumber AS ReporterNumber, URLCTA AS URLCTA, DomainCTA AS DomainCTA, PhoneCTA AS PhoneCTA, EmailCTA AS EmailCTA 
| eval subject="" | eval Size="" | eval Headers="" | eval Sender_ip_address="" | eval original_recipients=7726 | eval IMEI_Sender="" | eval Type="New_7726" | eval Direction="Incoming"
| search ReporterNumber=1234567890
| table Timestamp, subject, Content, Size, Headers, SenderNumber, Sender_ip_address, ReporterNumber, original_recipients, URLCTA, DomainCTA, PhoneCTA, EmailCTA, IMEI_Sender, Type, Direction]

thanks and regards,

0 Karma

cmerriman
Super Champion

perhaps try something like this. i'm evaluating a bunch of your fields together for each index and then doing a stats to get the values of them by the common fields.

(index=indexA earliest=07/17/2017:17:00:00 latest=07/18/2017:22:00:00 ) OR (index=indexB earliest=06/25/2017:17:00:00 latest=07/01/2017:10:00:00 )
|eval Timestamp=coalesce(event_time,Timestamp)|eval Content=coalesce(preview,Content)|eval SenderNumber=coalesce(sender,SenderNumber)|eval URLCTA=coalesce(URL,URLCTA)|eval DomainCTA=coalesce(domain,DomainCTA)|eval PhoneCTA=coalesce(phone,PhoneCTA) |eval EmailCTA=coalesce(email,EmailCTA)
| eval subject="" 
| eval Size="" 
| eval Headers="" 
| eval Sender_ip_address="" 
| eval ReporterNumber="" 
| eval IMEI_Sender="" 
| eval original_recipients=if(index="indexA",original_recipients,7726)
| eval Type=if(index="indexA","SMS","New_7726")
| eval Direction=if(index="indexA","Outgoing" ,"Incoming")
|stats values(*) as * by Timestamp, Content, SenderNumber, URLCTA, DomainCTA, PhoneCTA, EmailCTA, IMEI_Sender, Type, Direction
| search original_recipients=* SenderNumber=1234567890 ReporterNumber=1234567890 
| table Timestamp, subject, Content, Size, Headers, SenderNumber, Sender_ip_address, ReporterNumber, original_recipients, URLCTA, DomainCTA, PhoneCTA, EmailCTA, IMEI_Sender, Type, Direction 
0 Karma

sideview
SplunkTrust
SplunkTrust

This really shouldn't be the case, at least I've never seen it happen. Are you sure the searchterms in the second search aren't also changing so that it happens to also have zero results?

Here is a search that will always not return any events in the first search, and that (if you have any indexed data in the timerange) will always have some results in the second.

 foo NOT foo | append [search index=* | head 100 | stats count by sourcetype index]

bagir32
Explorer

thanks, this response kills my doubts

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...