How's CRC key generated in the files less than 256 bytes ?

[sunrise]

Hi Splunkers,

I know that spunk creates a CRC key from initial 256 bytes of the monitoring file and memorize it,
so once splunk ingest some log file, splunk doesn't ingest the same data.

Now I convinced that how about the files less than 256 bytes.
It seems that splunk generates CRC keys from these files and CRC keys include file paths,
how does splunk generate CRC keys ?
Followings are suggested that two input log data with the same data have the different CRC keys.

$ ./splunk cmd btprobe -d /Applications/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /Applications/splunk/data/inputs.log
Using logging configuration at /Applications/splunk/etc/log-cmdline.cfg.
key=0x1870717c543a9e03 scrc=0xc6d8922272744c60 sptr=36 fcrc=0x1870717c543a9e03 flen=0 mdtm=1399168084 wrtm=1399168454

$ ./splunk cmd btprobe -d /Applications/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /Applications/splunk/data/inputs_1.log
Using logging configuration at /Applications/splunk/etc/log-cmdline.cfg.
key=0x70dd02e9d29906a5 scrc=0xc6d8922272744c60 sptr=36 fcrc=0x70dd02e9d29906a5 flen=0 mdtm=1399168084 wrtm=1399169085

Both files have same log data like followings.
inputs.log ＆ inputs_1.log

This is a test00.
This is a test01.

[sunrise · Accepted solution]

Although I don't get any splunk public documentation,
I got some points of CRC keys by testing some cases.

If we get something more than 256 bytes to Splunk, Splunk will generate CRC key from initial 256 bytes in that file.
The CRC key is the key to distinguish from other files and that key is different from fcrc here.
I don't know about "fcrc" in details, but it may seems to be CRC keys from file path.

# /opt/splunk610/bin/splunk cmd btprobe -d /opt/splunk610/var/lib/splunk/fishbucket/splunk_private_db --file /root/tutorialdata/www1/access.log
Using logging configuration at /opt/splunk610/etc/log-cmdline.cfg.
key=0x4e97d44b7327bf62 scrc=0x39e6880ad3d6050 sptr=4262086 fcrc=0x5fb99137ed3561c2 flen=0 mdtm=1399446902 wrtm=1399549867 

However, if we get something less than 256 bytes which is not enough to generate CRC keys, splunk assigns "fcrc" to the key. So "fcrc" is equal to key as follwoing.

# /opt/splunk610/bin/splunk cmd btprobe -d /opt/splunk610/var/lib/splunk/fishbucket/splunk_private_db --file /root/data/test02.log
Using logging configuration at /opt/splunk610/etc/log-cmdline.cfg.
key=0xb5d814cff824489b scrc=0x40262cd292160657 sptr=255 fcrc=0xb5d814cff824489b flen=0 mdtm=1399551711 wrtm=1399551777 

[lukejadamec]

Splunk only includes the path if you are using are crcsalt attribute.