How to include a unique ID to rsyslog client config?

[varunmalhotra1]

Is there a way to assign a unique id to each rsyslog client node. I'm trying to build a solution where multiple rsyslog clients would be sending their services logs to a centralized rsyslog server and from their those logs will be send to Splunk indexers via universal forwarder agent, the problem is a that customer can own 2-3 nodes and once those nodes logs are sent to rsyslog then how can I segeregate logs per customer. I am trying to find out if there a way to add a unique label like custumer_uid in rsyslog client nodes of each customer.

[FrankVl]

You could have the central rsyslog server write logs from different customers to separate folders, each with a unique folder name for that customer. That would then end up in the source field in Splunk (since that contains the file path).

Key point of course is: what do you use on the central rsyslog server to distinguish data from different customers. I can imagine if hostname data is reliably showing FQDN of sending host, you could filter by domains related to each customer. Alternatively, you could instruct each customer to send to a different port and have separate listeners (and associated rulesets) for each customer. If customers have trouble sending on custom ports, you could also run multiple rsyslog instances listening on separate virtual IPs and have customers send data to their own unique IP address.

[varunmalhotra1]

To write data to each unique folder per customer, I've to first find out a way to add the customer_ ID in the each nodes rsyslog client side. How can I add that.

[FrankVl]

Why do you need to add that ID? Do none of the other methods I mentioned work for distinguishing customers?

If the sending side is always an rsyslog daemon as well, you could configure that with a custom template that includes a unique ID. Careful where you put that though, as it easily messes up parsing by the receiving rsyslog daemon or Splunk down the line. Maybe add some prefix to the host name and then strip that off again in the receiving rsyslog daemon.

[varunmalhotra1]

Actually, our setup is bit complex, when a customer orders few boxes, we reserve those boxes and all of those boxes will have our management VM running(comprised of many small applications), none of the application that sends it's logs to Syslog Server writes customer-uid in it's log file. Our final goal is to be able to search all layers of logs that the request passes through in splunk based on customer ID, whenever a customer reports an issue. Like I said management VM runs many services and it's not possible to make code level at this stage to have applications write customer_id thats why I was thinking if at the time of assigning boxes we can just update rsyslog client config to just include customer_id, it will be easiest solution.

I tried to use '$InputFileTag' to define the customer_id but it doesn't accept more than 32 character and our customer ID has 36 character.