Getting Data In

Why are all of our JSON fields alphanumeric strings?

cdstealer
Contributor

Hi,
I'm ingesting data in pure json and all fields are being extracted. However, all fields are strings regardless of whether the field contains a float or an integer. Trying to convert the required fields in search has no effect, so we are unable to execute any arithmetic or numeric comparisons. Is there anything that can be done to have the correct type at indextime?

Cheers
Steve

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Please post your .conf stanzas for that sourcetype. Your JSON works just fine in spath. Run anywhere code follows.

| makeresults | eval _raw="[{\"total\": 40848933, \"name\": \"DR_SP_OBOFF_IFX_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 40848933, \"data_reduction\": 5.5917148342819205, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629603181566511, \"total_reduction\": 150.9655201124792}, {\"total\": 78303476810, \"name\": \"DR_SP_OBOFF_IFX_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 78303476810, \"data_reduction\": 3.0048772949588285, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.13935706065251277, \"total_reduction\": 3.4914331572127146}, {\"total\": 1151894, \"name\": \"DR_SP_OBOFF_IFX_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1151894, \"data_reduction\": 5.826071620538336, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769851684570313, \"total_reduction\": 6.67896562490172}, {\"total\": 4194101563, \"name\": \"DR_SP_OBOFF_IFX_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 4194101563, \"data_reduction\": 5.762749779233561, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.3507057993752616, \"total_reduction\": 8.87540620829318}, {\"total\": 30325162691, \"name\": \"DR_SP_OBOFF_IFX_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 30325162691, \"data_reduction\": 5.7856689710564675, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.4147625645622611, \"total_reduction\": 9.886019965091556}, {\"total\": 26692006101, \"name\": \"DR_SP_OBOFF_IFX_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 26692006101, \"data_reduction\": 5.713406580985547, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37600740242004393, \"total_reduction\": 9.156208908797916}, {\"total\": 74914950529, \"name\": \"DR_SP_OBOFF_IFX_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 74914950529, \"data_reduction\": 5.810545839529534, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.44200734570622446, \"total_reduction\": 10.413301671298274}, {\"total\": 1150454538, \"name\": \"DR_SP_OBOFF_IFX_MGS-APP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1150454538, \"data_reduction\": 2.8930656374658144, \"size\": 36507222016, \"shared_space\": null, \"thin_provisioning\": 0.9088305866017061, \"total_reduction\": 31.732853482634717}, {\"total\": 635496861, \"name\": \"DR_SP_OBOFF_IFX_MGS-AUD1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 635496861, \"data_reduction\": 6.0887140231403905, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9477627519248188, \"total_reduction\": 116.55885881234325}, {\"total\": 3419466396, \"name\": \"DR_SP_OBOFF_IFX_MGS-FUN1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 3419466396, \"data_reduction\": 2.808043834085122, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 0.8959998640903207, \"total_reduction\": 27.00038619683935}, {\"total\": 36900508384, \"name\": \"DR_SP_OBOFF_IFX_MGS-SYS1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 36900508384, \"data_reduction\": 3.252184366603066, \"size\": 148176371712, \"shared_space\": null, \"thin_provisioning\": 0.15868666552115177, \"total_reduction\": 3.8656042087073685}, {\"total\": 560836420, \"name\": \"DR_SP_OBOFF_IFX_MGS-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 560836420, \"data_reduction\": 5.952361806622659, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9549372023430424, \"total_reduction\": 132.09037423586636}, {\"total\": 1128669697, \"name\": \"DR_SP_OBOFF_IFX_MGS-VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 1128669697, \"data_reduction\": 5.370497559256691, \"size\": 55834574848, \"shared_space\": null, \"thin_provisioning\": 0.24247801303863525, \"total_reduction\": 7.089559975413095}, {\"total\": 384092766229, \"name\": \"DR_SP_OBOFF_IFX_MGS-VLAT1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 384092766229, \"data_reduction\": 2.7056223050113544, \"size\": 1884416901120, \"shared_space\": null, \"thin_provisioning\": 0.44581108854027557, \"total_reduction\": 4.882129990448185}, {\"total\": 14591737, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 14591737, \"data_reduction\": 5.825555352683882, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.4441834176890552, \"total_reduction\": 10.481075120973713}, {\"total\": 12940424, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 12940424, \"data_reduction\": 5.769282773109169, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629402092524937, \"total_reduction\": 155.6749959117721}, {\"total\": 41327630331, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 41327630331, \"data_reduction\": 4.196256297529051, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.1393570459806002, \"total_reduction\": 4.875722595452125}, {\"total\": 2093944, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 2093944, \"data_reduction\": 5.82486857314305, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769844748757103, \"total_reduction\": 6.67758592927651}, {\"total\": 110567079, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 110567079, \"data_reduction\": 5.8216771539896355, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.35301270621163505, \"total_reduction\": 8.998132126987265}, {\"total\": 11450502587, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 11450502587, \"data_reduction\": 5.717953722895729, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37651851806640624, \"total_reduction\": 9.171008103019716}, {\"total\": 8724590790, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 8724590790, \"data_reduction\": 5.815905885628052, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.41740459154049553, \"total_reduction\": 9.982752697976865}, {\"total\": 187692578634, \"name\": \"PRC_CORP_UIM_MYSQL_DB1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 187692578634, \"data_reduction\": 2.8722118919761837, \"size\": 6597069766656, \"shared_space\": null, \"thin_provisioning\": 0.9182717454774926, \"total_reduction\": 35.14343856671008}, {\"total\": 55128374940, \"name\": \"PRN_MS_MARS_MYSQL_DATA-SLAVE1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 55128374940, \"data_reduction\": 2.887891129235254, \"size\": 187904819200, \"shared_space\": null, \"thin_provisioning\": 0.1473721149989537, \"total_reduction\": 3.387047479958634}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-BACKUP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-DATA1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-LOGS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-QUORUM\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 1073741824, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 37580963840, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}]"
| spath
| rename {}.data_reduction as DR
| table DR
| mvexpand DR 
| eval DR1 = DR + 1
0 Karma

niketn
Legend

@cdstealer, can you add a mocked up sample of the JSON data being idexed? Ideally numeric fields should be treated as numbers. Try trim() function in case numbers in JSON have spaces before or after i.e. " 30 "

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

cdstealer
Contributor

Hi @niketnilay, I've attached an image and sample data to the thread. Thanks for your help.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...