How to override and change incoming source and host names with JSON source and host fields?

[nfieglein]

I have JSON fields for source and host which I would like to use to override the incoming source and host. What is the easiest mechanism to accomplish this?

Thanks

[nfieglein · Accepted solution]

I actually created a calculated field for source and set it to the corresponding value from JSON. This seems to work correctly.

[somesoni2]

Could you post some sample value of raw data (or from "your base search | head 5 | table _raw" search query)?

[nfieglein]

,"SvctagSegmentGrp":{
"ModelNumber":"DZ","ServiceLevel":"CP","TechNameIssuedCall":"STC: SOME TECH","DspReplyCode":"0000","TechBadgeNumIssuedCall":659662,"ReasonCode":"SW3F","TechIdIssuedCall":"000251779","Buid":"000000707","ItemSubClass":"002","Svctag":"B375ST1","DpsNum":"000175479487","ProductDesc":"7737,NOTEBOOK,HADLEY 17FBTX","LineOfBusiness":55}
}
}
}
}
}

(Completed message)

[nfieglein]

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"FTF USED. ARTICLE NUMBER: non compliance","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Audibly obtained PN","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"No APN","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VDI Told cust about Refurb replacement parts","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VPKRS: No need for plastic replacement","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Media Check. Customer has: None","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided TAT: 1-2 BD","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"CRU/FRU part/s verified thru DTT/EducateDell","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided Moxie as a lifeline","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Sent summary email during call. Customer","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"confirmed e-mail is received.","CommentDate":20141031}
(again TBC)

[nfieglein]

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"SYSTEM: Inspiron 17","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"BTTR: Cx declined follow-up","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Consulted: CM POC Ice Bordeos","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Heat Check: SAT","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Captured, verified and updated Customer Name and","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Contact info in SR header.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Captured, keyspelled and updated email address","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"in SR header.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VA TOADE","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Cx agreed to 'One-Strike policy'","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided Ownership Spiel/s","CommentDate":20141031}
(again TBC)

[nfieglein]

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"D:Media Request","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"cx called in bec he would like to reinstall the","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"OS however he don't have the Media Recovery disc,","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"cx requesting for it, he would like to reinstall","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"the OS bec there's a lot of stuff or apps on","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"system and cx would like to refresh or to clean","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"it up, he wants to remove all the files and data.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"inform cx the this request is a one time deal","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"only, cx understand, inform also cx the warranty","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"details. done CDO - set to prio 4","CommentDate":20141031}
(again TBC)

[nfieglein]

Got it. I am working a chicken and egg problem, so I need to get back to DNA.
Here is an example message:
{
"Timestamp":"2014-11-03 14:34:55",
"Type":"INFO ",
"Class":"TopicListenner:?",
"LogEntry":
{
"source":"AMDPS",
"Content": {
"Amdps1204ipmCpy":{
"Dps1204Ipm":{
"CustomerSegmentGrp":{
"CustomerNumBuid":"000000707","CustomerNum":"7777777","DpsType":"EXG","CompanyNum":"02","RequestingService":"Safder Memon"}
,"OrderSegmentGrp":{
"ExgOrderNum":"000000000490644069","DomsStatusDate":20141103,"DspStatusCode":"0000","DomsStatusCode":"SC","PoNum":38045618}
,"PartsSegmentGrp":{
"PartsSegment":[{
"SkuMfgNbr":"89HT1","QuantityAtFsb":"0000000000","PartDescription":"KIT,MEDIA,DVD,RDVD,7737","Quantity":"000000001"}
,{
"SkuMfgNbr":"CY2KJ","QuantityAtFsb":"0000000000","PartDescription":"KIT,SW,W8H/P64,MUL24","Quantity":"000000001"}
]}
,"FsbLocationGrp":{
"FsbLocationSegment":[{
"VendorId":"0016","VendorInfo":[{
"MilesToFsb":"0007","FsbLocation":"WYOW1"}
,{
"MilesToFsb":"0092","FsbLocation":"WYUL1"}
,{
"MilesToFsb":"0092","FsbLocation":"WYGK1"}
]}
,"","",""]}
,"JobSegmentGrp":{
"JobIndx":0}
,"ContactSegmentGrp":{
"TelephoneNbr":5555555555,"ContactName":"NOONE SPECIAL","PhoneExtension":"00000"}
,"HeaderSegmentGrp":{
"CreationDate":"Mon Nov 03 14:34:54 CST 2014","OperationType":"TRN"}
,"CommentSegmentGrp":{
"CommentSegment":[{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"P:Media Request","CommentDate":20141031}
(to be continued)

[nfieglein]

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Gave service request number to customer.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"THIS IS AN ALABANG DISPATCH","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"S:dps media","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Created from DellServ Case #999999999","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"ETSB SOME OTHERTECH","DetailProblemDesc":"PART OK","CommentDate":20141101}
,{
"CommentType":"EXG","TechName":"ETSB SOME OTHERTECH","DetailProblemDesc":"RS APPROVED DPS","CommentDate":20141101}
]}
,"RaSegmentGrp":{
"RaFlag":"N"}
,"AddressSegmentGrp":{
"TimeZone":"EST","EdiRegionCode":"CA","StreetTwo":"Suite 3333","State":"ON","ZipCode":"999 999","City":"Gloucester","CountryCode":"CA","StreetOne":"9999 Duck St"}
,"StatusSegmentGrp":{
"StatusDate":"20141103:14:34:54.495440","RecordType":"C","StatusCode":"TTC"}
,"TranhistSegmentGrp":{
"EventDate":"20141103:14:34:54.448741","VendorNum":"0016","VendorDate":"20141104:16:00","EventCode":"VX","EventComments":"RTN WAYBILL#: KAM001231827"}
(again TBC)

[nfieglein]

Thanks. The issue is that I do not have a regex to populate the fields. They are automatically extracted in the data input because I am using JSON. I really just want to set the value of host and source using the extracted JSON field name if possible.

[aweitzman]

Except that by the time they are automatically extracted from your JSON you can't overwrite those fields anymore. At the point in the process where host and source get assigned, none of the fields have yet been extracted - it's just a big string. By the time the JSON parser gets hold of your input, it is too late, those fields have already been assigned. That's why you need to do it ahead of time, via a regex in props.conf/transforms.conf, as @somesoni2 has pointed out.

[somesoni2]

You can find the example here on how to use props.conf and transforms.conf to override Host and source (same way). Just replace the regex with you regex to find the host/source field from JSON.

http://answers.splunk.com/answers/1673/hostname-rename-using-transforms.html