Getting Data In

How can I get my TCP data into the metrics index?

walkerhound
Path Finder

I am trying to pull incoming tcp data into the Metrics Store using this information:

http://docs.splunk.com/Documentation/Splunk/7.0.0/Metrics/GetMetricsInOther#Get_metrics_in_from_clie...

Currently the data is coming in to an event index and the _raw field looks like this:
2017-11-29 18:43:22.807 +0000 Tag="Channel1.Device1.Tag1" Value="1523" Quality="good"

We would like to pull this into the Metrics Store with these fields
metric_name Tag
_value Value
_time _time

(We also want the Quality, but I'm ignoring that until I get the basics.)

Created a new port for the data to come in.

Source Sourcexxxxx
Source Type Typexxxxx
index indexxxxxx

The index (indexxxxx) is a metrics index

I edit configuration files as below and restart Splunk. I get two errors:

Index Processor: Metric value= is not valid for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=index51115. Metric event data with an invalid metric value would not be indexed. Ensure the input metric data is not malformed.

Index Processor: Metric name is missing for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=indexxxxxx. Metric event data without metric name is invalid and would not be indexed. Ensure the input metric data is not malformed.

What am I doing wrong? Thanks in advance for your help.

props.conf

[Typexxxxx]
    TIME_FORMAT = %F %H:%M:%S.%Q %z
    TRANSFORMS-test_metric_name = test_name
    TRANSFORMS-test_metric_value = test_value
    NO_BINARY_CHECK = true
    SHOULD_LINEMERGE = false
    pulldown_type = 1
    category = Metrics

transforms.conf
[test_name]
REGEX = "Channel1.Device1.(\S+)"
FORMAT = metric_name::$1
WRITE_META = true

[test_value]
    REGEX = Value="(\S+)"
    FORMAT = _value::$1
    WRITE_META = true

fields.conf
[metric_name]
INDEXED=true

    [_value]
    INDEXED=true
0 Karma

MuS
SplunkTrust
SplunkTrust

Hi walkerhound,

One thing I spot is that the docs mentions this value for the metric_name FORMAT to be FORMAT = metric_name::graphite.$1 maybe try to use FORMAT = metric_name::channel1.$1 in your case.

For the _value, try this regex Value=\"([^\"]+)\" or if you really have literary Value="1234" in your logs try this:

 [test_value]
 REGEX = (.+)
 SOURCE_KEY = Value
 FORMAT = _value::$1
 WRITE_META = true

Hope this helps ...

cheers, MuS

0 Karma

walkerhound
Path Finder

It turns out that my problem had to do with permissions. Splunk could not read the fields.conf file. When I corrected that problem, this worked.

0 Karma

MuS
SplunkTrust
SplunkTrust

Nice finding! How about writing that up as new answer and accept it so it can help other finding the right solution? I suspect you were doing this on a Windows Splunk instance?

0 Karma

walkerhound
Path Finder

Thank you for the suggestions. I experimented with several varieties of your ideas but I continue to get the same two errors indicating that the metric name and metric values are missing or unset.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...