Getting Data In

Trouble setting host from a hostname field in a json datastructure.

gryz
Explorer

Here is our props.conf:

[aristajson]
TIME_PREFIX = hosttime": "
MAX_TIMESTAMP_LOOKAHEAD = 22
BREAK_ONLY_BEFORE = {{"hostname
KV_MODE = json
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 90000
pulldown_type = 1
TRANSFORMS-larry = aristahostname

Here is our transforms.conf:

I've tried it with and without the host in <> . I've also tried to indicate the space after the : with a \s

[aristahostname]
REGEX = "hostname": "([a-zA-Z0-9-_]+)"
FORMAT = host::$1
DEST_KEY = MetaData:Host

Here is a snippet of our data that comes in via tcp. There is only one cr at the very end of the event:

{{"hostname": "nyaristalab-2"}{"hosttime": "2012-09-19 18:58:58"}{"neighbors": {"Ethernet3": {"2": {"sysName": "nyaristalab-1", ..... }

Here is what it kinda looks like in search. Each of the + are drill downable. So it is all good except for it not using what is in hostname as the host field.

1 » 9/25/12
10:25:59.000 PM
{[-]
hostname : "nyaristalab-2",
hosttime : "2012-09-25 22:25:59",
interfaces : {[+]},
neighbors : {[+]},
routing : {[+]}
}

Thanks for looking!

0 Karma

kristian_kolb
Ultra Champion

In transforms.conf you should not specify <host> inside the capturing group. Have you tried;

[aristahome]
REGEX = \"hostname\":\s+\"([^"]+)
FORMAT = host::$1
DEST_KEY = MetaData:Host

On a side note, your combination of SHOULD_LINEMERGE=false and BREAK_ONLY_BEFORE=... is invalid. BREAK_ONLY_BEFORE/AFTER, MUST_(NOT_)BREAK_BEFORE/AFTER etc will only work with SHOULD_LINEMERGE=true

If you have SHOULD_LINEMERGE=false, you set your event splitting with LINE_BREAKER= regex. The default value for LINE_BREAKER is one or more newlines, so for single-line events, you do not need to specify this.

If your event is multilined, you should probably set SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true. If this breaks events in a strange manner, because there are several date/timestamps in the events, you might have to set SHOULD_LINEMERGE = false, and use a LINE_BREAKER regex like

([\r\n]+)\d+/\d+/\d+\s+\d+:\d+:\d+

which should match a m/d/y h:m:s style timestamp after a newline

Hope this helps,

Kristian

0 Karma

gryz
Explorer

Thanks for the clearer explanation of the SHOULD_LINEMERGE and BREAK_ONLY_BEFORE. I got rid of those.

However the regex suggestions did not work. As I had noted in my original post, I had tried without the .

I checked both your and my regexes with various regex checkers and they both should work.

Any other ideas?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...