About cvajs

cvajs · ‎09-20-2012

so, 2.4T raid-10 for hot buckets, and 5+P+cachecade (need one spare) 3T raid-5 for OS and cold buckets. problem is, i have syslog on same system and i expect 60/40 write:read ratio between syslog writing and Splunk reading those files, so i suspect syslog has to go on raid-10. 2.4T wont be enough for my syslog data. i also plan to do some ext4 filesystem tuning, like noatime, data=ordered, nouser_xattr, which saves some writes.

cvajs · ‎09-19-2012

if i run crude calc using 1/3 read and 2/3 writes in raid-5 w/ 1k IOPS as my target for operating IOPS, i get something like this: (1000*0.333)+(1000*0.666*4) 333+2664 = 2997 raw array IOPS to realize 1000 in raid-5 14*130*cachecade factor = 1820*3.5 = 6370 so, probably not 20k, but if the OS can "see" 6370 IOPS in a raid-5 split 33% read and 66% write, does my original statement still hold, does it really matter that its raid-5?

cvajs · ‎09-19-2012

in this specific case i am referencing Dell's R720 with one SSD CacheCade, tests ran in raid-10 using 15k drives yielded 32k+ IOPS. its a skewed # due to CacheCade and it all depends. 20k was my estimate based on same tests using 10k drives.

cvajs · ‎09-19-2012

thanks for the info, but i am not 100% convinced that raid-5 is a bad choice. raid-5 is certainly not better than a flavor of raid-10, but if a 14 spindle raid-5 array offers ~20000+ IOPS does it really matter that it's a raid-5? the cons for flavor of raid-10 is that you lose half the storage capacity. pros and cons to each, but i am not ready to say raid-5 is a bad choice.

cvajs · ‎09-19-2012

so, under a rhel6 and latest splunk, and likely sitting on 16 10k spindles raid-5, is there a filesystem best suited for the job given the fact that both syslog-ng files and splunk indexes are on the same filesystem?

cvajs · ‎09-05-2012

Some architecture Q's around Splunk. My customer is buying Splunk. They have global presence with a few data-centers. They want centralized syslog with Splunk. We are planning to put a indexer in two data-centers and use a single search head (typical distributed architecture). The indexers will sit atop of syslog-ng. Using metrics from another logging system we currently have approx 432kB/sec worth of syslog data. That said I need to know more about Splunk in virtual env and storage. Any issues running all of this in VMware ESX 4.1? Is there a Splunk doc which outlines the virtual guest requirements? The storage for indexers will either be a customer purchased NAS or hosted SAN. Any issues to be worried about there (i/o ability on NAS, SAN issues, etc), and if so is there a threshold to when those issues start to appear (like slow searching, delayed indexing, etc etc). Does the search head need big storage? good news is, my customer is already buying Splunk, i just need to properly spec out whats needed to make it work, etc.

cvajs · ‎04-17-2012

well, according the the stats for _internal index my daily indexing doesnt surpass 160MB. and, how could i exceed the quota if i had 50GB trial lic installed and then went to free, these messages came instantly right after the trial lic expired and i switched to free.

cvajs · ‎04-16-2012

v4.3.1 on sles 11.1 just finished a enterprise eval so when i logged in today i got some over quota messages. i made sure in Manager > Licensing that it is now Free lic model. my daily indexing volume is ~150MB, so why these warning messages? it has thus disabled searching per the Free model, why? Correct by midnight to avoid violation Learn more This pool contains slave(s) with 4 warnings myHost auto_generated_pool_free free pool_warning_count Apr 16, 2012 12:00:00 AM (11 hours ago) Indexing quota exceeded for this pool, poolsz=0 bytes myHost auto_generated_pool_enterprise enterprise license_window Apr 15, 2012 12:00:00 AM (1 day ago) Indexing quota exceeded for this pool, poolsz=0 bytes myHost auto_generated_pool_enterprise enterprise license_window Apr 14, 2012 12:00:00 AM (2 days ago) Indexing quota exceeded for this pool, poolsz=0 bytes myHost auto_generated_pool_enterprise enterprise license_window Apr 13, 2012 12:00:00 AM (3 days ago) Indexing quota exceeded for this pool, poolsz=0 bytes myHost auto_generated_pool_enterprise enterprise license_window

cvajs · ‎04-12-2012

copy out the data into a new file, then index that file into the same index, etc.

cvajs · ‎04-11-2012

i'll look into this further as suggested, but i did search all other transforms and props files looking for CSA related entries and only found related items in the CSA app folder, etc. and yes, the extractions i have do have "Wrong" and "INTEGER" included in the values, which leads me to believe i need to write my own transform regex/format to map the event data to variables. should not be too hard to do.

cvajs · ‎04-10-2012

sorry, i am on 4.3.1, my keyboard didnt get the .1 in there. and hmmm, i dont see any pipe chars in my raw data that can be used for the transform, however, when i do a open search on my csa index i get lots of fields on the left with data (albeit the values for each field have more data in them then needed, but it is extracting fields somehow). the delimiter in my case looks like it should be a tab \t or \s{2,10}

cvajs · ‎04-10-2012

but how does the regex actually group real matches for the FORMAT statement? the groupings have a \| between thme, but my data has no | char so how does the groupings fall into matches?

cvajs · ‎04-10-2012

v4.3 sles 11.1 can you explain for me this transform [csafields] REGEX = ^[^\|]+\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+) FORMAT = nbtname::$1 ip::$2 ruleid::$3 code::$4 remotetime::$5 alert::$6 i get the FORMAT part (which doesnt work correctly with my event data), but what is this regex? what does the \| mean? here's the raw event data 2012-04-05 15:38:29 syslog.prod.org [UDP: [10.219.0.134]:1086->[10.222.1.253]]: DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (124678725) 14 days, 10:19:47.25 SNMPv2-MIB::snmpTrapOID.0 = OID: CSAMC-SNMPv2-MIB::csaTraps.1 CSAMC-SNMPv2-MIB::eventID = INTEGER: 10329635 CSAMC-SNMPv2-MIB::ruleID = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::hostName = STRING: "hostB.prod.org" CSAMC-SNMPv2-MIB::eventTime = STRING: "2012-04-05 15:38:29.617" CSAMC-SNMPv2-MIB::severityCode = INTEGER: 2 CSAMC-SNMPv2-MIB::eventCode = INTEGER: 164 CSAMC-SNMPv2-MIB::processName = STRING: "<remote application>" CSAMC-SNMPv2-MIB::fileName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::sourceIPAddress = Wrong Type (should be IpAddress): NULL CSAMC-SNMPv2-MIB::destinationIPAddress = Wrong Type (should be IpAddress): NULL CSAMC-SNMPv2-MIB::eventText = STRING: "The process '<remote application>' has triggered too many log records in the last few minutes. Further messages will be logged at a decreased rate for 10 minutes." CSAMC-SNMPv2-MIB::hostID = INTEGER: 209 CSAMC-SNMPv2-MIB::currentHostIPAddress = Wrong Type (should be IpAddress): STRING: "10.132.194.158" CSAMC-SNMPv2-MIB::hostOSType = STRING: "W" CSAMC-SNMPv2-MIB::sourcePort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::destinationPort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::eventType = STRING: "Administrative" CSAMC-SNMPv2-MIB::ruleDescription = Wrong Type (should be OCTET STRING): NULLCSAMC-SNMPv2-MIB::ruleModuleID = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::ruleModuleName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::buttonCode = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::userName = STRING: "myDOMAIN\\WSecGat_Px" CSAMC-SNMPv2-MIB::flags = INTEGER: 0 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (124679186) 14 days, 10:19:51.86 SNMPv2-MIB::snmpTrapOID.0 = OID: CSAMC-SNMPv2-MIB::csaTraps.1 CSAMC-SNMPv2-MIB::eventID = INTEGER: 10329637 CSAMC-SNMPv2-MIB::ruleID = INTEGER: 1374 CSAMC-SNMPv2-MIB::hostName = STRING: "hostA.prod.org" CSAMC-SNMPv2-MIB::eventTime = STRING: "2012-04-05 15:38:29.999" CSAMC-SNMPv2-MIB::severityCode = INTEGER: 2 CSAMC-SNMPv2-MIB::eventCode = INTEGER: 179 CSAMC-SNMPv2-MIB::processName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::fileName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::sourceIPAddress = Wrong Type (should be IpAddress): NULL CSAMC-SNMPv2-MIB::destinationIPAddress = Wrong Type (should be IpAddress): NULL CSAMC-SNMPv2-MIB::eventText = STRING: "The 'Service Control Manager' service logged event code 7036 into the system event log: The LiveUpdate service entered the running state. " CSAMC-SNMPv2-MIB::hostID = INTEGER: 2206 CSAMC-SNMPv2-MIB::currentHostIPAddress = Wrong Type (should be IpAddress): STRING: "10.10.10.10" CSAMC-SNMPv2-MIB::hostOSType = STRING: "W" CSAMC-SNMPv2-MIB::sourcePort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::destinationPort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::eventType = STRING: "NT Event log"CSAMC-SNMPv2-MIB::ruleDescription = "" CSAMC-SNMPv2-MIB::ruleModuleID = INTEGER: 280 CSAMC-SNMPv2-MIB::ruleModuleName = STRING: "CSA Service Monitoring" CSAMC-SNMPv2-MIB::buttonCode = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::userName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::flags = INTEGER:

cvajs · ‎04-10-2012

i think the field extractions of the app are wrong for my version of CSA, and, possibly loading the CSA MIB is mucking up the event data. i cant find any info about using the app MIB with snmptrapd, etc. i would think using the MIB would be preferred. as example, the field "hostName" is pulling pulling out 'STRING: "hostname"' when i think it should just be 'hostname' etc

cvajs · ‎04-10-2012

ok, i copied props.conf from default to local in the CSA app. here's my props now in local dir [cisco_csa] TRANSFORMS = csa_hostoverride REPORT-extract = csafields SHOULD_LINEMERGE = true TRUNCATE = 0 BREAK_ONLY_BEFORE = ^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}) i cleaned the index to clear the junk, and now get events in full, but CSA app returns no data for any of its reports. whats weird is that splunk shows two eventtype fields: host=myHost |sourcetype=cisco_csa |eventtype=cisco_csa |eventtype=cisco_syslog_types cisco firewall network

cvajs · ‎04-10-2012

so i need to tell the preview that my event is "DATE \n EVENT_DATA \n" (not that way exactly, but you get my point). oddly, if i write snmptrapd out to syslog then Splunk will read those events from syslog file fine, but CSA app still doesnt work with that data.

cvajs · ‎04-10-2012

i dont quite understand. if i manually add the source and do a "Preview" of the log data it shows the events correctly, so why would it mess up event breaking after that?

cvajs · ‎04-10-2012

efelder0, what do you mean by "date". date as in "MM/DD/YYYY" or date as "MM/DD/YYYY HH:MM:SS (AM|PM)" i would opt to use [\d]{2}/[\d]{2}/[\d]{4} to grab MM/DD/YYYY just in case the space comes up missing (not likely, but you never know)

cvajs · ‎04-10-2012

anyone have an idea as to why splunk does this?

cvajs · ‎04-09-2012

i think SEDCMD is a better route

cvajs · ‎04-09-2012

you wrote "strftime" is that the right command? strftime takes (X) as epoch time and converts it to format Y you dont have epoch time anywhere in your syntax. epoch is # of sec since jan 1 1970 00:00:00 UTC http://docs.splunk.com/Documentation/Splunk/4.3.1/SearchReference/CommonEvalFunctions

cvajs · ‎04-09-2012

TRUNCATE=0 to default props in CSA app, but Splunk still shows: 2012-04-09 11:24:34 myHost.prod.org [UDP: [10.8.0.134]:1086->[10.1.1.53]]: props.conf [cisco_csa] TRANSFORMS = csa_hostoverride REPORT-extract = csafields SHOULD_LINEMERGE = false TRUNCATE=0 transfroms.conf [csafields] REGEX = ^[^\|]+\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+) FORMAT = nbtname::$1 ip::$2 ruleid::$3 code::$4 remotetime::$5 alert::$6 [csa_hostoverride] DEST_KEY = MetaData:Host REGEX = ^[^\|]+\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+) FORMAT = host::$1

cvajs · ‎04-06-2012

which props? i dont have a system/local props i dont have a local app props i have this props /opt/splunk/etc/apps/Splunk_CiscoClientSecurityAgent/default [cisco_csa] TRANSFORMS = csa_hostoverride REPORT-extract = csafields SHOULD_LINEMERGE = false

cvajs · ‎04-05-2012

then create separate charts (views) for only the timeframes that contain the successive days this way the chart(s) wont have breaks, etc. i dunno if you can tell stats timechart or chart to skip time.

cvajs · ‎04-05-2012

edit the source input path via Manager /logs/.../\w+\.\w+-\d+-\d+-\d+ or if this is too generic then /logs/.../application\.log-\d+-\d+-\d+

Posts	119
Solutions	1
Karma Given	4
Karma Received	29
Member Since	‎03-10-2012

Online Status	Offline
Date Last Visited	‎06-05-2020 02:04 AM

Filesystem for Splunk

Splunk in virtual environmnet

Indexing Quota with Free License

Explain this transform

PDF Report Server

Cisco CSA Indexing Issue

How-To Change Indexed Data?

Source using ISO Date

Whitelist for standard /var/log

Indexing hostname by segment issue

Re: Filesystem for Splunk

Re: Filesystem for Splunk

Re: Filesystem for Splunk

Re: Filesystem for Splunk

Filesystem for Splunk

Splunk in virtual environmnet

Re: Indexing Quota with Free License

Indexing Quota with Free License

Re: Gap in data after System event

Re: Explain this transform

Re: Explain this transform

Re: Explain this transform

Explain this transform

Re: Cisco CSA Indexing Issue

Re: Cisco CSA Indexing Issue

Re: Cisco CSA Indexing Issue

Re: Cisco CSA Indexing Issue

Re: Some RegEx help with date formatting

Re: Cisco CSA Indexing Issue

Re: Issue with strptime

Re: Issue with strptime

Re: Cisco CSA Indexing Issue

Re: Cisco CSA Indexing Issue

Re: How can you restrict timechart to display only...

Re: Input monitor wildcard and whitelist