Thursday, October 29, 2009

bnx2x stuff

note to self

Experienced this issue on HP 460c G6 Blades.

Host Operating System RHEL5.4 is using Broadcom Corporation NetXtreme II driver module bnx2x in this setup.

Fully Virtualized Windows 2003 Server Guests could not transfer files over the network, while there is no drop of packets when tested with "ping" etc. and even the Remote Desktop Connections over RDP are working without any issue.

Additionally, the Host OS's (RHEL5.4 , RHEL5.3 etc. ) doesn't seem to have any problem either.
-before upgrading to the latest firmware

Finally, adding
options bnx2x disable_tpa=1
to /etc/modprobe.conf which disables LRO (or TPA) on the bnx2x-based netwrok cards resoled this issue.

All this above was done on HP NC-Series Broadcom 1Gb and 10GbE Multifunction Driver for Linux although it seemed to me as if the Broadcom Driver provided from RHEL5.4 it self should do the job with the above module options.

Got the solve through Red Hat Support and i quote,
FYI, LRO (Large Receive Offload) is a technique for increasing inbound throughput of high-bandwidth network connections by reducing CPU overhead. It works by aggregating multiple incoming packets from a single stream into a larger buffer before they are passed higher up the networking stack, thus reducing the number of packets that have to be processed.
Now the funny thing is, when i read the above it feels like this is something i would want to keep turned on and not* off! :-) Guess its gonna take me some googling to get that one figured out.


David Monro said...

Did you ever get any further with this? I am seeing something similar with ESX hosts at work...

Yajith Ajantha said...

Disabling TPA fixed the issue over here. According to the bnx2x driver README bridging can not be used with TPA on kernels older than 2.6.26.