Tag Archives: Performance

Poor Man’s vCPU & vRAM Right Size Recommendation Tool

VMware vCenter Operations Management Suite can be expensive. If you are like me and there is no budget for vCOPs, this script will give you a vCPU & vRAM recommendations based off of past virtual machine usage. The following script will connect to your vCenter, grab historical performance data and provide recommendations that were designed around two vKernel whitepapers. The following whitepapers are:

 

The script is simple to use only requiring the vCenter parameter to start with all defaults:

PoorMansRecommendations.ps1 -vCenter site1.local.domain

 

Specifies additional authentication information. Grabbing 60 days of past performance instead of the default 30 days:

PoorManRecommendations.ps1 -vCenter site1.local.domain -Username fred -Password root -PastDays 60

 

Specifies more samples for accuracy and using a larger ‘building block’ for memory recommendations:

PoorMansRecommendations.ps1 -vCenter site1.local.domain -PastDays 60 -MaxSamples 25000 -MemoryBuildingBlockMB 1024

 

When running the script interactively, a progress bar be displayed as it calculates recommendations per virtual machine:
Poor Man's Right Sizing

The results:

Poor Man's Recommendations Results

This should only be used as a guidance, point of reference, a conversation point or just a rough estimate. Each environment and workload characteristics are unique, please use your logic along with this data to come to a solution that is right for your environment.

Download the script: PoorMansRecommendations.ps1

Thanks for looking. Please leave any questions or comments below and have a great day!

Advertisements
Tagged , , , , , , ,

Cisco MAC Address Flapping Causing High CPU Utilization

A MAC flap is caused when a switch receives packets from two different physical/logical interfaces with the same source MAC address. The switch then learns where the MAC address is and puts in to a table. This table has the physical/logical interface and the MAC address. When flapping occurs it causes this table to be updated whenever a packet is sent/received. The more data flowing through the interface that is flapping the higher your CPU Utilization is going to be which can have serious potential negative side effects. Such as dropped packets, laggy terminal session and complete drop of network connectivity.

The following will give you the commands you need to help identify MAC Address Flapping and High CPU Utilization on Cisco Catalyst series switch. This was performed to troubleshoot CPU utilization issues on a Cisco Catalyst 4500 series switch but the same commands should be available to other Cisco switches which run the IOS firmware.

cisco4500#show processes cpu
CPU utilization for five seconds: 38%/1%; one minute: 32%; five minutes: 32%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
27         524    250268          2  0.00%  0.00%  0.00%   0 TTY Background
28         816    254843          3  0.00%  0.00%  0.00%   0 Per-Second Jobs
29      101100      5053      20007  0.00%  0.01%  0.00%   0 Per-minute Jobs
30    26057260  26720902        975 12.07% 11.41% 11.36%   0 Cat4k Mgmt HiPri
31    19482908  29413060        662 24.07% 19.32% 19.20%   0 Cat4k Mgmt LoPri
32        4468    162748         27  0.00%  0.00%  0.00%   0 Galios Reschedul

The following will give you a Target CPU percent and the Actual Percent. Look for percents that greatly exceed the Target CPU percent. This will help identify what is eating your processing power on your device. This is used to troubleshoot other items than MAC Address Flapping such as Routing Loops and other bad things that can bring your network to a halt.

cisco4500#show platform health
%CPU   %CPU    RunTimeMax   Priority  Average %CPU  Total
Target Actual Target Actual   Fg   Bg 5Sec Min Hour  CPU
Protocol-aging-revie   0.20   0.00      2      0  100  500    0   0    0  0:01
Acl-Flattener          1.00   0.00     10      5  100  500    0   0    0  0:04
KxAclPathMan create/   1.00   0.00     10      5  100  500    0   0    0  0:21
KxAclPathMan update    2.00   0.00     10      6  100  500    0   0    0  0:05
KxAclPathMan reprogr   1.00   0.00      2      1  100  500    0   0    0  0:00
TagMan-InformMtegRev   1.00   0.00      5      0  100  500    0   0    0  0:00
TagMan-RecreateMtegR   1.00   0.00     10     14  100  500    0   0    0  0:18
K2CpuMan Review       30.00  91.31     30     92  100  500  128 119   84  13039:02
K2AccelPacketMan: Tx  10.00   2.30     20      0  100  500    2   2    2  1345:30
K2AccelPacketMan: Au   0.10   0.00      0      0  100  500    0   0    0  0:00

First enter enabled mode then configure terminal mode. Issue the following command to ensure there is logging for mac-move which will identify MAC Address Flapping.

cisco4500(config)#mac address-table notification mac-move

After a period of time, view the log to identify the MAC address that is flapping.

cisco4500(config)#do show log
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
...
*Oct  3 08:51:28.149: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (10.10.10.236)
*Oct  3 09:43:46.437: %C4K_EBM-4-HOSTFLAPPING: Host 00:60:48:1B:01:15 in vlan 400 is moving from port Gi2/40 to port Gi2/30
*Oct  3 09:43:48.629: %C4K_EBM-4-HOSTFLAPPING: Host 00:60:48:1B:01:15 in vlan 400 is moving from port Gi2/30 to port Gi2/40
*Oct  3 09:43:48.717: %C4K_EBM-4-HOSTFLAPPING: Host 00:60:48:1B:01:15 in vlan 400 is moving from port Gi2/40 to port Gi2/30
*Oct  3 09:43:49.581: %C4K_EBM-4-HOSTFLAPPING: Host 00:60:48:1B:01:15 in vlan 400 is moving from port Gi2/30 to port Gi2/40

Furthermore, issue the following command at random periods of time to illustrate the MAC address bouncing between two different physical ports.

cisco4500#sh mac address-table address 00:60:48:1B:01:15

Unicast Entries
vlan   mac address     type        protocols               port
-------+---------------+--------+---------------------+--------------------
400    0060.481b.0115   dynamic ip                    GigabitEthernet2/30

cisco4500#sh mac address-table address 00:60:48:1B:01:15
Unicast Entries
vlan   mac address     type        protocols               port
-------+---------------+--------+---------------------+--------------------
400    0060.481b.0115   dynamic ip                    GigabitEthernet2/40

It depends on how your configuration is but it is generally a good idea to disable one of the two interfaces or fix an issue with a logical interface such as EtherChannel/LACP.

For furhter information and Cisco official documentation, http://www.cisco.com/c/en/us/support/docs/switches/catalyst-4000-series-switches/65591-cat4500-high-cpu.html

Tagged , , , , , , , ,

EMC Naviseccli View Service Processor Cache Utilization

After hooking up a host to an EMC SAN it is very helpful to have the host that is connected to the SAN to have Naviseccli installed along with the Host Agent. Naviseccli allows you to connect to the Service Processors to gather information among other things.

Dirty Cache Pages is data in the cache that is a solid state physical format waiting to be written to disk. You can monitor how full the cache is by the following commands on each Service Processor.

[root@localhost bin]# ./naviseccli -h <IP/Hostname SPA> getcache -pdp
[root@localhost bin]# ./naviseccli -h <IP/Hostname SPB> getcache -pdp

If you receive the following error, “Security file not found. Already removed or check -secfilepath option.” Issue to following commands to allow security to issue commands remotely.

[root@localhost bin]# ./naviseccli -User sysadmin -Password <password> -Scope 0 -h <IP/Hostname SPA> -AddUserSecurity
[root@localhost bin]# ./naviseccli -User sysadmin -Password <password> -Scope 0 -h <IP/Hostname SPB> -AddUserSecurity
Tagged , , , , , , , , , , ,