Getting Data In

Why are my props.conf settings not being applied to my data

efrenette11
Path Finder

Here's my local props.conf.

[tmweb@app1.splunkdev.jetdev2.syseng.tmcs ~]$ cat /opt/splunk-efr/splunk/etc/system/local/props.conf
[default]
TRUNCATE=100
TIME_PREFIX = datetime=
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ
SHOULD_LINEMERGE = False
TZ=Canada/Eastern

I don't understand why those settings are not considered. No matter what I'm doing, Splunk does not consider my setting.

Please, I'm really upset about this. I'm using a new version of splunk, and it was working very nicely before. Now, sometime it's works, sometimes not.

Here's a result in Splunk. As you see, date time is not considered, nor truncation.

08/12/2015 10:23:15.000 
severity="INFO" host="app1.jdas.jetdev2.syseng.tmcs" service_version="1.4.47" client_host="app2.jdat.jetdev2.syseng.tmcs" client_version="1.5.97" Correlation-ID="a3b3fde3-e657-42bc-804e-4677ce4de1a7" client_rid="73891577-0da4-4f7f-ac07-b1497ab68246" rid="249cce85-4b42-43e7-8ea6-7e7b951993fc" sid="8d2c2fc8-9981-4b90-a6e1-b5190c1874d7_143" thread="http-8080-2" category="com.ticketmaster.platform.bam.strategies.PayloadBAMStrategy" datetime="2015-12-08T15:23:12.805Z" bam="payload" appCode="outbound.response.rest" activity="FindAttractions" seq="108697" payload="{'@type':'{http://www.ticketmaster.com/jdas/authoring/attraction/service/operations/1.0}FindAttractionResults','record':[{'@type':'{http://www.ticketmaster.com/jdas/authoring/attraction/service/operations/1.0}FindAttractionRecord','attractionVersion':{'@type':'{http://www.ticketmaster.com/jdas/authoring/attraction/model/1.0}AttractionVersion','attractionVersionId':'dataAdmin-attrVer-00000000027b9fe9','attractionId':'dataAdmin-attraction-000000000083d829'}}],'totalRecords':326,'nbRecords':1}" headers="{Response-Code=[200], TMPS-Request-Id=[249cce85-4b42-43e7-8ea6-7e7b951993fc], TMPS-Correlation-Id=[a3b3fde3-e657-42bc-804e-4677ce4de1a7], TMPS-Service-Version=[1.4.47], TMPS-Hostname=[app1.jdas.jetdev2.syseng.tmcs], Cache-Control=[no-cache], Date=[Tue, 08 Dec 2015 15:23:12 GMT]}" 
0 Karma

yannK
Splunk Employee
Splunk Employee

question : are you using an universal or lightweight forwarder, and are those props on he indexer or on the forwarder ?
Because in general, the events are parsed by the indexers, so you may want to put those props on :
- indexers
- intermediary heavy forwarder (if any)

woodcock
Esteemed Legend

First of all, you should not be modifying that file, but you should create a new file inside $SPLUNK_HOME/etc/apps/YourApp/default/props.conf. Second of all, you should not be using the [default] stanza header, but something that is specifically targetted to your events. Thirdly, I suspect that the Z means Zulu and is a TZ specifier and suspect that your TZ value is wrong. Lastly, I am highly skeptical that this was working just fine before (or at any time). In any case, make the packaging changes that I described and then try this:

[YourSourcetypeHere]
TIME_PREFIX = datetime="
MAX_TIMESTAMP_LOOKAHEAD = 11
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ
SHOULD_LINEMERGE = false
TZ=Canada/Eastern

Then deploy to your Indexers, restart all splunk instances there and verify events that are forwarders AFTER the restarts are OK.

efrenette11
Path Finder

MMM I understand skepticism. I'm not very good in splunk configuration. And also my english is not perfect to.

So your're saying that I should not modify that file $SPLUNK_HOME/etc/system/local/default/props.conf ? This is really strange because is what I took from this link:
https://answers.splunk.com/answers/44041/editing-props-conf-transforms-conf-configuration-file.html

By the way, I try to edit $SPLUNK_HOME/etc/apps/search/default/props.conf, but nothing has change. Looks like my configuration is not considered ???

Any other's idea ?

0 Karma

woodcock
Esteemed Legend

It is true that /local is almost always preferrable to /default, which is the point of the link that you referenced. It is also true that $SPLUNK_HOME/etc/apps/ is almost always better than $SPLUNK_HOME/etc/system. You should not make making a non-system-wide (global-scope) change inside of $SPLUNK_HOME/etc/system, which is my point. To make it worse, you are using the [defaiult] stanza which makes your local-scope configurations applied to everything everywhere, which is definitely a bad idea, which again is my point. Try to package your configurations in an app so that they apply ONLY to your input. It is MUCH easier to debug this way and you won't run the (HUGE) risk of breaking other stuff.

efrenette11
Path Finder

Ok, I've done everythings that you said, but nothing has changed.

From a brand new installation of indexer and heavy forwarder, I only add this as configuration of my indexer and splunk always merge line ???

/opt/splunk-efr/splunk/etc/apps/search/default/props.conf

[source::/app/local/log/payload_log]
SHOULD_LINEMERGE = False

Here's 2 lines merged as example and there are coming from heavy forwarder

06:58:30.000

severity="INFO" host="app1.cops.intqa102.syseng.tmcs" service_version="1.4.10" client_host="" client_version="2.0.9" Correlation-ID="20151217123221_f4f77d69-828e-41e2-acf9-eedf2d9d72ba" client_rid="" rid="10bfe7d0-3b39-436b-b612-824b630e1dc9" sid="" thread="http-8080-2" category="com.ticketmaster.platform.bam.strategies.PayloadBAMStrategy" datetime="2015-12-18T14:58:28.838Z" bam="payload" appCode="request.rest" activity="" seq="81289" payload="{'common':{'id':{'tmonline-us':{'id':'516110'},'primary':{'id':'dataAdmin-venue-000000000008d36a'},'jetson':{'id':'dataAdmin-venue-000000000008d36a'}},'name':{'es-us':'BUFFALO_ROSE ID = 516110 modified at 09:15','en-au':'BUFFALO_ROSE ID = 516110 modified at 09:15','es-mx':'BUFFALO_ROSE ID = 516110 modified at 09:15','fr-ca':'BUFFALO_ROSE ID = 516110 modifié a 13:31','en-nz':'BUFFALO_ROSE ID = 516110 modified at 09:15','en-ca':'BUFFALO_ROSE ID = 516110 modified at 09:15','en-us':'BUFFALO_ROSE ID = 516110 modified at 09:15','en-mx':'BUFFALO_ROSE ID = 516110 modified at 09:15'},'isTest':false,'source':'tmonline-us','image':[{'width':650,'height':366,'url':'http://qaip10.ticketmaster.net:8001/dbimages/17751v.jpg','path':'/dbimages/17751v.jpg','fallback':fa... Unidos','en-au':'United States Of America','es-mx':'Estados Unidos','fr-ca':'États-Unis','en-nz':'United States Of America','en-ca':'United States Of America','en-us':'United States Of America','en-mx':'United States Of America'},'countryCode':'US'},'state':{'name':{'es-us':'New York','en-au':'New York','es-mx':'New York','fr-ca':'New York','en-nz':'New York','en-ca':'New York','en-us':'New York','en-mx':'New York'},'stateCode':'NY'},'city':{'name':{'es-us':'new york','en-au':'new york','es-mx':'new york','fr-ca':'new york','en-nz':'new york','en-ca':'new york','en-us':'new york','se-eu':'new york','en-mx':'new york'}},'postalCode':'12345','zipCode':'12345','address':{'line1':{'es-us':'1495, du Golf','en-au':'1495, du Golf','es-mx':'1495, du Golf','fr-ca':'1495, du Golf','en-nz':'1495, du Golf','en-ca':'1495, du Golf','en-us':'1495, du Golf','se-eu':'1495, du Golf','en-mx':'1495, du Golf'},'line2':{'es-us':'Adress line 2','en-au':'Adress line 2','es-mx':'Adress line 2','fr-ca':'Adress line 2','en-nz':'Adress line 2','en-ca':'Adress line 2','en-us':'Adress line 2','se-eu':'Adress line 2','en-mx':'Adress line 2'}},'location':{'latitude':-72.12345675,'longitude':72.12345675},'timezone':'America/Phoenix','dma':[{'id':281}],'market':[{'id':{'tmonline-us':{'id':'23'},'primary':{'id':'23'}}}],'currency':'USD','isAvailableForNewEvents':true,'hostData':{'host':'PHX','venueNumber':14}}" headers="{Address=[http://app.cops.intqa102.syseng.tmcs:8080/rest/content-persistence-service/1-0/persist/unifiedVenue], Http-Method=[POST], Accept=[text/plain, application/json, application/+json, */], accept-charset=[big5, big5-hkscs, cesu-8, euc-jp, euc-kr, gb18030, gb2312, gbk, ibm-thai, ibm00858, ibm01140, ibm01141, ibm01142, ibm01143, ibm01144, ibm01145, ibm01146, ibm01147, ibm01148, ibm01149, ibm037, ibm1026, ibm1047, ibm273, ibm277, ibm278, ibm280, ibm284, ibm285, ibm290, ibm297, ibm420, ibm424, ibm437, ibm500, ibm775, ibm850, ibm852, ibm855, ibm857, ibm860, ibm861, ibm862, ibm863, ibm864, ibm865, ibm866, ibm868, ibm869, ibm870, ibm871, ibm918, iso-2022-cn, iso-2022-jp, iso-2022-jp-2, iso-2022-kr, iso-8859-1, iso-8859-13, iso-8859-15, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, jis_x0201, jis_x0212-1990, koi8-r, koi8-u, shift_jis, tis-620, us-ascii, utf-16, utf-16be, utf-16le, utf-32, utf-32be, utf-32le, utf-8, windows-1250, windows-1251, windows-1252, windows-1253, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, windows-31j, x-big5-hkscs-2001, x-big5-solaris, x-compound_text, x-euc-jp-linux, x-euc-tw, x-eucjp-open, x-ibm1006, x-ibm1025, x-ibm1046, x-ibm1097, x-ibm1098, x-ibm1112, x-ibm1122, x-ibm1123, x-ibm1124, x-ibm1364, x-ibm1381, x-ibm1383, x-ibm300, x-ibm33722, x-ibm737, x-ibm833, x-ibm834, x-ibm856, x-ibm874, x-ibm875, x-ibm921, x-ibm922, x-ibm930, x-ibm933, x-ibm935, x-ibm937, x-ibm939, x-ibm942, x-ibm942c, x-ibm943, x-ibm943c, x-ibm948, x-ibm949, x-ibm949c, x-ibm950, x-ibm964, x-ibm970, x-iscii91, x-iso-2022-cn-cns, x-iso-2022-cn-gb, x-iso-8859-11, x-jis0208, x-jisautodetect, x-johab, x-macarabic, x-maccentraleurope, x-maccroatian, x-maccyrillic, x-macdingbat, x-macgreek, x-machebrew, x-maciceland, x-macroman, x-macromania, x-macsymbol, x-macthai, x-macturkish, x-macukraine, x-ms932_0213, x-ms950-hkscs, x-ms950-hkscs-xp, x-mswin-936, x-pck, x-sjis_0213, x-utf-16le-bom, x-utf-32be-bom, x-utf-32le-bom, x-windows-50220, x-windows-50221, x-windows-874, x-windows-949, x-windows-950, x-windows-iso2022jp], accept-encoding=[gzip,deflate], accept_encoding=[gzip, deflate], connection=[Keep-Alive], Content-Length=[2229], content-type=[application/json;charset=UTF-8], host=[app.cops.intqa102.syseng.tmcs:8080], tmps-client-app-version=[2.0.9], tmps-client-hostname=[], tmps-correlation-id=[20151217123221_f4f77d69-828e-41e2-acf9-eedf2d9d72ba], tmps-request-id=[10bfe7d0-3b39-436b-b612-824b630e1dc9], user-agent=[Apache-HttpClient/4.3.5 (java 1.5)]}"

severity="INFO" host="app1.cops.intqa102.syseng.tmcs" service_version="1.4.10" client_host="" client_version="2.0.9" Correlation-ID="20151217123221_f4f77d69-828e-41e2-acf9-eedf2d9d72ba" client_rid="" rid="10bfe7d0-3b39-436b-b612-824b630e1dc9" sid="" thread="http-8080-2" category="com.ticketmaster.platform.bam.strategies.PayloadBAMStrategy" datetime="2015-12-18T14:58:28.859Z" bam="payload" appCode="outbound.response.rest" activity="PersistUnifiedVenue" seq="81296" payload="{'persistedId':'dataAdmin-venue-000000000008d36a','persistenceStatus':'Matched','modifiableStatus':false}" headers="{Response-Code=[200], Date=[Fri, 18 Dec 2015 14:58:28 GMT]}"

0 Karma

efrenette11
Path Finder

If I add the data from ui, really easy to configure. There's something that I really don't understand with props.conf files ?

0 Karma

ppablo
Retired

Hi @efrenette11

To provide more information for users to help you, can you share what version of Splunk you were using before and what version you upgraded to exactly? You mentioned your props.conf worked before, but after upgrading, that's when you started having an issue. Always include as much detail as you can in your post for users to have a full picture.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...