I have single event looking like below and trying to figure the best way for Splunk to recognize the key-value pairs. Ideally would have each line as a separate event.
{
"compsModelObjectName": "Desktop",
"compsObjectList": [
{
"buildGUID": "8D36EF88-3319-4770-BDD3-DCDA614C40DB",
"buildType": "ONEDESK - FULLBUILD",
"buildVersion": "2.22.080214-1002",
"description": "MY TEXT IN HERE",
"purpose": "Normal",
"lastScanDate": "Apr 29, 2010",
"assetName": "WLDNETSBWGS41J",
"dateModified": "Mar 27, 2013",
"dateInstalled": "Dec 17, 2009",
"invNo": "DIMS-1268745",
"serialNo": "BWGS41J",
"manufacturer": "UNKNOWN",
"model": "UNKNOWN PC",
"assetTag": "Z00880152",
"status": "INAC",
"productClass": "UNKNOWN PC",
"productType": "UNKNOWN",
"owner": "X1111111",
"subStatus": "Disposal",
"compsIdentifier": "DIMS-1268745"
},
{
"buildGUID": "JENYX1111111XP",
"buildType": "JENY",
"description": "Unknown Class",
"purpose": "Normal",
"lastScanDate": "Nov 1, 2010",
"assetName": "JENYX1111111XP",
"dateModified": "Mar 31, 2011",
"dateInstalled": "Jan 1, 1970",
"invNo": "TEXTTEXT",
"serialNo": "JENYX1111111XP",
"manufacturer": "JENY",
"model": "JENY",
"assetTag": "D04936865",
"status": "INAC",
"productClass": "JENY",
"productType": "JENY",
"owner": "X1111111",
"subStatus": "Disposal",
"compsIdentifier": "DIMS-4182421"
},
{
"buildGUID": "JENYX1111111",
"buildType": "JENY",
"description": "Unknown Class",
"purpose": "Normal",
"lastScanDate": "Nov 21, 2011",
"assetName": "JENYX1111111",
"dateModified": "Nov 20, 2011",
"dateInstalled": "Jan 1, 1970",
"invNo": "DIMS-4827747",
"serialNo": "JENYX1111111",
"manufacturer": "JENY",
"model": "JENY",
"assetTag": "D06722795",
"status": "INAC",
"productClass": "JENY",
"productType": "JENY",
"owner": "X1111111",
"subStatus": "Disposal",
"compsIdentifier": "DIMS-4827747"
},
{
"buildGUID": "2DB77FB4-C1D2-4AD4-9453-4A06D4017076",
"buildType": "xSPACE - FULLBUILD",
"buildVersion": "4.12",
"description": "Business Basic PC",
"domain": "EMEA",
"purpose": "Normal",
"lastScanDate": "Aug 30, 2016",
"assetName": "WC2291Y7F",
"dateModified": "Aug 31, 2016",
"dateInstalled": "Jun 24, 2013",
"invNo": "DIMS-5916063",
"serialNo": "CZC2291Y7F",
"manufacturer": "DP",
"model": "Z611",
"assetTag": "08192",
"status": "AC",
"productClass": "Desktop",
"productType": "DESKTOP",
"owner": "X1111111",
"subStatus": "CONFIGURED",
"compsIdentifier": "DIMS-5916063"
}
],
"statusCode": 200
}
Not sure why i cant add to thread but this works..
This is not working. I have tried adding as a flat file and changing sourcetype to _json but still the events dont break as they do in data preview.
@smudge797 I am getting email notifications that you are replying, but nothing is displaying on the page here. Are you seeing the same thing?
When treads get to long they dont display correctly start a new thread.
Depending on what you want do to you can use mvexpand.
...| mvexpand compsObjectList
OR
...| mvexpand compsObjectList | spath
Hope this helps
That looks like JSON format. What is your sourcetype set to? Setting it to _json seems like it would do the trick. You can test this with a sample of the data using Settings > Add Data and uploading a sample. You don't have to go through the entire process, but it will show you how the extraction would look when the sourcetype is set to _json.
Looks much better in the previewer but do i have to use _json sourcetype name? Thanks!
If you want Splunk to understand how to work with it, the best option is to mark it with the correct sourcetype. Are you wanting to configure your own sourcetype to automatically extract these fields? Or do you want to do something manually at search time?
Yes i want to use custom sourcetype to extract.
Ok, if you want to keep your custom sourcetype name, then you'll need to add a props.conf entry for it. You can add it to $SPLUNK_HOME/etc/system/local/props.conf
, but be sure it isn't going to conflict with an existing sourcetype:
[your_custom_sourcetype_name]
INDEXED_EXTRACTIONS = json
KV_MODE = none
Restart Splunk after adding it, and you should be good to go.
Using props on indexers the events don't look the same as preview and are breaking on the [ ...]
If each event is not on a single line, you will need to work on your line breaking configuration so it breaks where you expect. How is this data coming into Splunk?
a single event is rendering like this:
{
"compsModelObjectName": "Desktop",
"compsObjectList": [
{
"buildGUID": "8D36EF88-3319-4770-BDD3-DCDA614C40DB",
"buildType": "ONEDESK - FULLBUILD",
"buildVersion": "2.22.080214-1002",
....
"subStatus": "CONFIGURED",
"compsIdentifier": "DIMS-5916063"
}
],
"statusCode": 200
}
Are these being written to a file or being ingested another way?
inserted via curl, that make a difference?
Yes, but just in the way you tell Splunk how to break events. So right now it is breaking them on the opening of an array [
so we need to tell it not to.
As a test, have you tried sending an event using the sourcetype _json to see if that performs as expected? If it does, you might try adding this to your custom sourcetype stanza:
BREAK_ONLY_BEFORE = ^{
DATETIME_CONFIG = CURRENT
I tried adding as a flat file and still not breaking as it looks in previewer?
Tried changing to monitoring a flat file and still no change.
The BREAK_ONLY_BEFORE made no difference..?
I set that in props.conf system/local on indexer and still not working?
What is the name of your custom sourcetype? Have you tried setting it for a test to _json and see if it works?