Trying to slip SNMP data on index or through rex

[srobinsonxtl]

All,

I have the following Data: (192 of these) and trying to split the data into a multi-lined event, to extract the last number as 1.1.3.1.3.2.1.25.1 and the number after the = sign as the .

11/29/17

11:04:30.000 AM
SNMPv2-SMI::enterprises."1.1.3.1.3.2.1.25.1" = "1162" SNMPv2-SMI::enterprises."1.1.3.1.3.2.1.25.2" = "0"

I am trying to do this on index but can't seem to get it to work, or I can't get it to work using rex. Any help will be much appreciated.

Thanks,

Stephen Robinson

[richgalloway · Accepted solution]

It's seems your LINE_BREAKER setting will cut off the timestamp from the event. Is that really what you want?

To extract the "id" and "Reading" fields from the event, this regex works on regex101.com with your sample event: (?<id>\d+)".*=\s"(?<Reading>\d+)". For example:

<your basic search> | rex "(?<id>\d+)\".*=\s\"(?<Reading>\d+)" | table id Reading

[srobinsonxtl]

Thank you for responding to my questions, this works, but just returns the first id and Reading, the end goal is to extract all 192 values that are on a single line, and output the id and reading into a multi-lined event. I would like to do it at index time, but it doesn't seem to be working using the props.conf I put in place.

[richgalloway]

Thanks for clarifying. Here is an updated query for search-time extraction.

<your basic search> | rex max_match=0 "(?<id>\d+)\"\s+=\s\"(?<Reading>\d+)" | eval fields=mvzip(id, Reading) | mvexpand fields | rex field=fields "(?<id>\d+),(?<Reading>\d+)" | table id Reading

At index time you should be able to use the same regex string ("(?\d+)"\s+=\s"(?\d+)"), but be sure to include the mv_add = true option.

[srobinsonxtl]

Just to let you know and I took you example, and did the following in less steps: | rex max_match=0 "(?\d+)\"\s+=\s\"(?\d+)" | stats list(id),list(Reading) by _time,host

[richgalloway]

Is it currently a multi-line event (it looks like one in the question)?
What are your current props.conf/transforms.conf settings for that sourcetype?
What will the number after the = sign be extracted as?

[srobinsonxtl]

No it's not a multi-lined event at least I don't believe so. Here is my props.conf information and I don't have anything in Transforms for this sourcetype

[sourcetype_test]
LINE_BREAKER=([\r\n]+\s*)SNMPv2-SMI
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false

1.1.3.1.3.2.1.25.{1} that value should be {id} and "1.1.3.1.3.2.1.25.1" = <"1162"> this value should be <"Reading">

I am leveraging the SNMP Modular Input application. I really appreciate you getting back to me so quickly.

THanks,

Stephen Robinson

[srobinsonxtl]

At the end of the day, I would like to see the {id} as the field, and as the value, so I would have 192 fields with 192 values.

1 | 2 | 3 | 4| .....192
20| 43 | 80 | 100 | ..... 0

If this make sense.

Thanks,

Stephen Robinson

[harsmarvania57]

Still I am not clear how many SNMPv2-SMI in single line and do you want to extract all MIB values from same line If so then you can try something like this

    | makeresults
    | eval raw="11/29/17 11:04:30.000 AM SNMPv2-SMI::enterprises.\"1.1.3.1.3.2.1.25.1\" = \"1162\" SNMPv2-SMI::enterprises.\"1.1.3.1.3.2.1.25.2\" = \"0\""
    | makemv delim="::" raw
    | mvexpand raw
    | rex field=raw "enterprises\.\"\d+\.\d+\.\d+\.\d+\.\d+\.\d+\.\d+\.\d+\.(?<id>\d+)\"\s\=\s\"(?<Reading>\d+)\""

[srobinsonxtl]

Thank you for your response. I tried this and it doesn't seem to work. I have 192 SNMPv2-SMI that comes in on the single line for each poll.

Working through some trial and errors yesterday, I came up with the following but its not vary efficient.

index="dev" | eval _raw = split(_raw, "SNMPv2-SMI::enterprises.") | rex field=_raw "10381.1.3.1.3.2.1.1.?(?\d+)[\"]\s=\s[\"]?(?\d+)[\"]" | table _time,cid,vid | eval reading=mvzip(cid, vid) | fields - cid,vid| mvexpand reading | eval final=mvzip(reading, _time) | mvexpand final | makemv final delim="," | fields - _time,reading | eval time=mvindex(final, 2) | eval device=mvindex(final, 0) | eval data=mvindex(final, 1) | fields - final | table time,device,data | convert timeformat="%Y/%m/%d %T" mktime(time) as _time | fields - time | eval {device}=data | fields - device,data | fillnull | timechart sum() as (Device) usenull=f useother=f | addtotals as Total