search not returning after map command

[brdr]

I'm writing a search that extracts data from 2 indexes. I have 3 searches that tries to accomplish this.

1st search is against index1 as is the second search. This index has events for actions for 'adding' a user to a host and an action for 'removing' a user from the host. My first search gets the 'adding' action for the host. Then I use map command to find the 'removing action' based on a date that these 2 actions have in common. This all works fine.

I table the results of the above and then use another map command (against a different index) based on the date field from the results above. This map command works well until I use an 'eval' command just after the map which then causes the search to hang. If I don't use an eval command I can see fields from all the searches that I care about via table command.

If there is another way to accomplish what I'm trying to do I'm all ears.

Thanks for your help.

[woodcock]

You probably should back all the way up and reconsider your overall approach but for now, I will answer your specific question (now that you have added enough additional detail to do so). Your last line is this:

| table Account, Name, "Checked Out", "Host Targeted", Add, Remove

This throws away the _time field; that is the key problem because then you are referencing it with this:

| eval check_out_time_ep=relative_time(_time, "+4h") ...

So change your last 3 lines to this:

| table _time, Account, Name, "Checked Out", "Host Targeted", Add, Remove
| eval check_out_time_ep=relative_time(_time, "+4h")
| fieldformat check_out_time = strftime(check_out_time_ep, "%Y-%m-%d %H:%M:%S")
| fields - _time

[DalJeanis]

Okay, map is almost always the wrong method. In this case, your first and second searches are in the same index and appear to be linked by host and created. It looks like the only information coming from the second record is remove_time. So, your first set of maps can be refactored this way...

  index=myindex1 (message=”add” OR message="remove")
     | fields _time host created request.created add_time remove_time
     | eval early_time=case(message="add",_time)
     | eval late_time=case(message="add",_time - 3600)
     | eval created=coalesce(created,request.created)
     | fields - request.created
     | stats values(*) as * by host created

That should get you a record that looks something like this

| table host created add_time remove_time early_time late_time 

Now, your next map command doesn't make any sense at all, because you are going against another index and returning no new information.

Additionally, you are testing for _time > early_time OR _time > late_time, and with OR and those times, the late_time test is redundant.

Finally, these fields used later don't appear to come from anywhere. (src_user, src_user_name, time) They are probably on that second index.

I suspect that, if you filled out the third search manually and gave us the search wording and sample results, JUST FROM THAT INDEX, along with what keys are to be matched to the other records, then we would be able to collapse it the same way we collapsed the first two searches together, and eliminate the map verb altogether.

[brdr]

First all, thank you DalJeanis for responding back. And yes, lets remove the map verb please.

Question. In Line 2 you have 2 fields; ‘created’ and ‘request.created’. They are actually the same field. So when I coalesce I get no events.

NOTE: For clarity, the add_time is a timestamp (actually called timestamp) field shared by all events (i.e. add and remove). In the original second map I’m only interested in a) capturing the remove timestamp and b) carrying the other fields from the original search into the final map call. You are correct the fields you see at the bottom is from myindex2.

Sample data from myindex1 looks like this:

_time, “add”, timestamp, host1
_time, “remove”, timestamp, host1
_time, “add”, timestamp, host2
_time, “remove”, timestamp, host2

Sample data from myindex2 looks like this:

_time, src_user, src_user_name

The early and late time (add_time and the offset of add_time) gotten in the first search will be used to find the appropriate event in myindex2 by comparing its _time field.

In the end, after some renaming, it is this:

| table Account, Name, “Checked Out (_time_from myindex2)”, “Host Targeted”, “Add Time”, “Remove Time”

Please let me know if you have any questions. Thank you.
brdr

[brdr]

Dal Jeanis... could I entice you to help me further for 25 Karma points? 🙂

[brdr]

Hi Dal Jeanis,

I was able to push through my issue using map. going to close this one.

[cmerriman]

can you share the search as well as sample data and desired output? as well as maybe what the output looks like at the last step where the search works? removing and confidential information, of course. This should make it easier to assist in any reworking of the syntax that might need to happen.

[brdr]

here is where the search just goes away....

After the last map (going against myindex2) I have the following 2 commands just prior to rename and table'ing the final results results....

| eval check_out_time_ep=relative_time(_time, "+4h")
| convert timeformat="%Y-%m-%d %H:%M:%S" ctime(check_out_time_ep) AS check_out_time

If I remove these 2 lines the search runs as expected.

[brdr]

Sure. here is the essential logic for the search(es):

    index=myindex1 message=”add”
    | eval early_time=_time
    | eval late_time=relative_time(_time, "-60m")
    | sort _time
    | table early_time late_time created host add_time

    | map search="search myindex1 “request.created"=$created$ message=”remove”| eval created=$created$, host=$host$, add_time=$add_time$, early_time=$early_time$ late_time=$late_time$ | head 1" maxsearches=100
    | sort _time
    | table add_time remove_time early_time latest_time  host 

    | map search="search index=myindex2 (_time > $early_time$ OR _time > $late_time$) | sort _time | eval  host=$host$, add_time=$add_time$, remove_time=$remove_time$ | head 1" maxsearches=100
    | table add_time remove_time early_time latest_time  host 

    | convert ctime(_time) as time
    | rename src_user as Account, src_user_name as "Name", time as "Checked Out",  host as "Host Targeted",  add_time as "Add", remove_time as "Remove"
    | table Account, Name, "Checked Out", "Host Targeted", "Add", "Remove"

In terms of the output it is straight forward. Just have account info, date account checked out, host, and add and remove times.