Hi All, I'm struggling with a data input from the EMC Recoverpoint devices. I may be making things hard for myself, but I'm sure someone will have come across this scenario on their travels 😉
An event looks like this:
SITE_VPLEX: RPAs: RPA 1: Version: 4.1.SP1.P1(h.167) WAN IP: 000.000.000.000 RPA LAN IPv4: 000.000.000.000 RPA LAN IPv6:N/A iSCSI interface IPs: None Interfaces: Type: FC Initiator ID: 50012481006bexxx Type: FC Initiator ID: 50012481006bexxx Type: FC Initiator ID: 50012481006bexxx Type: FC Initiator ID: 50012481006bexxx Hardware details: Hardware type: Intel Corporation S2600GZ GEN5 Adapter type: 2564 Vendor: Intel Corporation Hardware Serial ID: FC6RP133000229_00000000002_FFF Hardware Platform: Intel Corporation S2600GZ GEN5 Amount of memory: 16269416 KB Number of CPUs: 12 RPA 2: Version: 4.1.SP1.P1(h.167) WAN IP: 000.000.000.000 RPA LAN IPv4: 000.000.000.000 RPA LAN IPv6:N/A iSCSI interface IPs: None Interfaces: Type: FC Initiator ID: 50012481006bdxxx Type: FC Initiator ID: 50012481006bdxxx Type: FC Initiator ID: 50012481006bdxxx Type: FC Initiator ID: 50012481006bdxxx Hardware details: Hardware type: Intel Corporation S2600GZ GEN5 Adapter type: 2564 Vendor: Intel Corporation Hardware Serial ID: FC6RP133000135_0000000000_FFF Hardware Platform: Intel Corporation S2600GZ GEN5 Amount of memory: 16269416 KB Number of CPUs: 12
I have the fields extracting without issue, but the issue I have is each field needs to belong to either RPA1 or RPA2.
Thanks in advance
Steve
I would handle this by splitting each one of these into 2 events on the way in.
I would handle this by splitting each one of these into 2 events on the way in.
Hi,
Thanks. I did consider doing this before I posted. However, it does seem to fit better and is simple to implement. I've had to apply a little extra config to get the data how it's needed. But it's all working 🙂 So the props.conf I'm now using on the heavyforwarder is below, the props above lives on the search head for search time extraction:
[emc_sourcetype] DATETIME_CONFIG=CURRENT SHOULD_LINEMERGE=true BREAK_ONLY_BEFORE=^\s+RPA\s\d: NO_BINARY_CHECK=true disabled=false TRUNCATE = 999999 MAX_EVENTS = 15000 SEDCMD-topline1 = s/string//g SEDCMD-topline2 = s/string//g SEDCMD-topline3 = s/string//g SEDCMD-topline4 = s/string//g SEDCMD-RPA1 = s/RPA 1:/RPA: 1/g SEDCMD-RPA2 = s/RPA 2:/RPA: 2/gSo this splits the event at RPA X:, removes the top 4 lines as these are now nolonger required and then replaces the RPA id for auto extraction. Job done 🙂
Thanks to you both for your help.
Steve
You should actually get rid of this:
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^\s+RPA\s\d:
And use this instead:
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n\s]+)RPA\s*\d:
The latter is FAR more efficient.
Hi cdstealer,
you should try with two regex like the following (for Initiator_ID field both in RPA1 and RPA2):
(?ms)(RPA\s1:).+?Initiator\sID:\s(?<RPA1Initiator>[^ ]*)
(?ms)(RPA\s2:).+?Initiator\sID:\s(?<RPA2Initiator>[^ ]*)
see an example in https://regex101.com/r/OY794T/1
Bye.
Giuseppe
Hi Giuseppe,
Thanks for the reply. Apologies, I probably should have stated how I'm extracting the fields.
transforms:
[emc_sourcetype] DELIMS = "\"{\n}\"", ":" FORMAT = $1::$2 MV_ADD = true
props:
[emc_sourcetype] KV_MODE = none KV_TRIM_SPACES = true REPORT-extractions = emc_sourcetype
Thanks
Sorry I didn't understand your need: can you explain again?
with the regex I gave, you can extract both the Initiator_ID field for RPA1 and RPA2.
Bye.
Giuseppe
Hi, no problem 🙂 So I'm trying to make this dynamic so that if the data changes, the extractions still work and also any new fields get extracted without having to manually create them. So the props/transforms I gave extract all the fields almost as I need. This also includes new fields. So what I need is a way to get splunk to enrich the fields so that we know which field belongs to which RPA.
In the example above, I have 8 "Initiator ID" values which anyone searching this data will have no idea what RPA they belong to. So I want splunk to assign the 4 under the RPA1 heading and the second 4 to belong to RPA2. So if a user searches for RPA1, they only get the fields extracted under the RPA1 heading. I hope I'm explaining this ok.
Thanks
Steve
Ok, Using a regex like the one I sent, you'll have different Initiator_ID field for RPA1 and RPA2.
If you need all the four values of Initiator_ID both for RPA1 and RPA2, you could use a regex like this:
(?ms)(RPA\s1:).+?Initiator\sID:\s(?<RPA1Initiator1>[^ ]*)\s+Type:\s.+?Initiator\sID:\s(?<RPA1Initiator2>[^ ]*)\s+Type:\s.+?Initiator\sID:\s(?<RPA1Initiator3>[^ ]*)\s+Type:\s.+?Initiator\sID:\s(?<RPA1Initiator4>[^ ]*)
Bye.
Giuseppe
Hi, Thanks. The regex way is what I was wanting to avoid as it means any changes to the event will break it. But if I have no other option, then I'll have to go down that road 😞
Thanks again for your help.
Steve