Linux interface RX Packet Dropped with LACP and not doing loadbalacing
up vote
3
down vote
favorite
OS: CentOS 6.6 / 64bit / Kernel 2.6.32-504.30.3.el6.x86_64
I have bond0 interface with following configuration.
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 7
Number of ports: 2
Actor Key: 17
Partner Key: 3
Partner Mac Address: a4:56:30:c6:0d:00
Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f2
Aggregator ID: 7
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f4
Aggregator ID: 7
Slave queue ID: 0
modprobe.conf
alias bond0 bonding
options bond0 mode=4 miimon=100
Cisco switch config:
interface Port-channel1
description Linux-bond0
switchport access vlan 10
interface GigabitEthernet0/7
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
interface GigabitEthernet0/8
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
Issue
I am getting RX packet drop on bond0 and eth0 interface on Linux ( but not on eth1)
bond0 : RX packets:575214161 errors:0 dropped:6407 overruns:0 frame:0
eth0 : RX packets:573623915 errors:0 dropped:6410 overruns:0 frame:0
eth1 : RX packets:1590356 errors:0 dropped:0 overruns:0 frame:0
I have check MRTG to see traffic flow and its around ~30mbps
But interesting thing. I am seeing all 30mbps traffic on eth0, eth1 has few kbps traffic. That means my link is not doing load balancing right?
cacti reporting following data. LACP should share traffic on both link right?
eth0: 30mbps
eth1: 600kbps
linux ethernet bonding cisco
asked Dec 23 '15 at 17:18
Satish
63311131
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
up vote
3
down vote
favorite
OS: CentOS 6.6 / 64bit / Kernel 2.6.32-504.30.3.el6.x86_64
I have bond0 interface with following configuration.
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 7
Number of ports: 2
Actor Key: 17
Partner Key: 3
Partner Mac Address: a4:56:30:c6:0d:00
Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f2
Aggregator ID: 7
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f4
Aggregator ID: 7
Slave queue ID: 0
modprobe.conf
alias bond0 bonding
options bond0 mode=4 miimon=100
Cisco switch config:
interface Port-channel1
description Linux-bond0
switchport access vlan 10
interface GigabitEthernet0/7
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
interface GigabitEthernet0/8
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
Issue
I am getting RX packet drop on bond0 and eth0 interface on Linux ( but not on eth1)
bond0 : RX packets:575214161 errors:0 dropped:6407 overruns:0 frame:0
eth0 : RX packets:573623915 errors:0 dropped:6410 overruns:0 frame:0
eth1 : RX packets:1590356 errors:0 dropped:0 overruns:0 frame:0
I have check MRTG to see traffic flow and its around ~30mbps
But interesting thing. I am seeing all 30mbps traffic on eth0, eth1 has few kbps traffic. That means my link is not doing load balancing right?
cacti reporting following data. LACP should share traffic on both link right?
eth0: 30mbps
eth1: 600kbps
linux ethernet bonding cisco
asked Dec 23 '15 at 17:18
Satish
63311131
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
This server isfileserverjust like backup machine. multiple machine accessing this single Linux server
– Satish
Dec 23 '15 at 17:29
add a comment |
up vote
3
down vote
favorite
up vote
3
down vote
favorite
OS: CentOS 6.6 / 64bit / Kernel 2.6.32-504.30.3.el6.x86_64
I have bond0 interface with following configuration.
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 7
Number of ports: 2
Actor Key: 17
Partner Key: 3
Partner Mac Address: a4:56:30:c6:0d:00
Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f2
Aggregator ID: 7
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f4
Aggregator ID: 7
Slave queue ID: 0
modprobe.conf
alias bond0 bonding
options bond0 mode=4 miimon=100
Cisco switch config:
interface Port-channel1
description Linux-bond0
switchport access vlan 10
interface GigabitEthernet0/7
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
interface GigabitEthernet0/8
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
Issue
I am getting RX packet drop on bond0 and eth0 interface on Linux ( but not on eth1)
bond0 : RX packets:575214161 errors:0 dropped:6407 overruns:0 frame:0
eth0 : RX packets:573623915 errors:0 dropped:6410 overruns:0 frame:0
eth1 : RX packets:1590356 errors:0 dropped:0 overruns:0 frame:0
I have check MRTG to see traffic flow and its around ~30mbps
But interesting thing. I am seeing all 30mbps traffic on eth0, eth1 has few kbps traffic. That means my link is not doing load balancing right?
cacti reporting following data. LACP should share traffic on both link right?
eth0: 30mbps
eth1: 600kbps
linux ethernet bonding cisco
asked Dec 23 '15 at 17:18
Satish
63311131
OS: CentOS 6.6 / 64bit / Kernel 2.6.32-504.30.3.el6.x86_64
I have bond0 interface with following configuration.
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 7
Number of ports: 2
Actor Key: 17
Partner Key: 3
Partner Mac Address: a4:56:30:c6:0d:00
Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f2
Aggregator ID: 7
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 9c:8e:99:0d:1a:f4
Aggregator ID: 7
Slave queue ID: 0
modprobe.conf
alias bond0 bonding
options bond0 mode=4 miimon=100
Cisco switch config:
interface Port-channel1
description Linux-bond0
switchport access vlan 10
interface GigabitEthernet0/7
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
interface GigabitEthernet0/8
switchport access vlan 10
spanning-tree portfast
spanning-tree guard root
channel-protocol lacp
channel-group 1 mode active
!
Issue
I am getting RX packet drop on bond0 and eth0 interface on Linux ( but not on eth1)
bond0 : RX packets:575214161 errors:0 dropped:6407 overruns:0 frame:0
eth0 : RX packets:573623915 errors:0 dropped:6410 overruns:0 frame:0
eth1 : RX packets:1590356 errors:0 dropped:0 overruns:0 frame:0
I have check MRTG to see traffic flow and its around ~30mbps
But interesting thing. I am seeing all 30mbps traffic on eth0, eth1 has few kbps traffic. That means my link is not doing load balancing right?
cacti reporting following data. LACP should share traffic on both link right?
eth0: 30mbps
eth1: 600kbps
linux ethernet bonding cisco
linux ethernet bonding cisco
asked Dec 23 '15 at 17:18
Satish
63311131
asked Dec 23 '15 at 17:18
Satish
63311131
asked Dec 23 '15 at 17:18
Satish
63311131
asked Dec 23 '15 at 17:18
Satish
63311131
asked Dec 23 '15 at 17:18
Satish
63311131
63311131
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
This server isfileserverjust like backup machine. multiple machine accessing this single Linux server
– Satish
Dec 23 '15 at 17:29
add a comment |
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
This server isfileserverjust like backup machine. multiple machine accessing this single Linux server
– Satish
Dec 23 '15 at 17:29
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
This server is
fileserver just like backup machine. multiple machine accessing this single Linux server– Satish
Dec 23 '15 at 17:29
This server is
fileserver just like backup machine. multiple machine accessing this single Linux server– Satish
Dec 23 '15 at 17:29
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
I wouldn't worry about the dropped packets. Doing the math, you're losing less than .001% of your total packets.
As far as the load balancing goes, you probably need to (re)configure the load distribution algorithm in use on your switch (since it's the switch to server traffic that's not balancing). Cisco has some pretty good documentation on that here but it's too much to copy into this answer. Also note that the distribution algorithm assigns traffic streams to a physical interface which means that if there is only one stream it will only use one physical interface.
answered Dec 23 '15 at 17:42
David King
2,813421
We are runningLACPat Layer2 switch, it should shared frames equally on both switch port. it has nothing to do withTCPat Layer 4, am i missing something here?
– Satish
Dec 23 '15 at 17:46
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f251186%2flinux-interface-rx-packet-dropped-with-lacp-and-not-doing-loadbalacing%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
I wouldn't worry about the dropped packets. Doing the math, you're losing less than .001% of your total packets.
As far as the load balancing goes, you probably need to (re)configure the load distribution algorithm in use on your switch (since it's the switch to server traffic that's not balancing). Cisco has some pretty good documentation on that here but it's too much to copy into this answer. Also note that the distribution algorithm assigns traffic streams to a physical interface which means that if there is only one stream it will only use one physical interface.
answered Dec 23 '15 at 17:42
David King
2,813421
We are runningLACPat Layer2 switch, it should shared frames equally on both switch port. it has nothing to do withTCPat Layer 4, am i missing something here?
– Satish
Dec 23 '15 at 17:46
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
add a comment |
up vote
0
down vote
I wouldn't worry about the dropped packets. Doing the math, you're losing less than .001% of your total packets.
As far as the load balancing goes, you probably need to (re)configure the load distribution algorithm in use on your switch (since it's the switch to server traffic that's not balancing). Cisco has some pretty good documentation on that here but it's too much to copy into this answer. Also note that the distribution algorithm assigns traffic streams to a physical interface which means that if there is only one stream it will only use one physical interface.
answered Dec 23 '15 at 17:42
David King
2,813421
We are runningLACPat Layer2 switch, it should shared frames equally on both switch port. it has nothing to do withTCPat Layer 4, am i missing something here?
– Satish
Dec 23 '15 at 17:46
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
add a comment |
up vote
0
down vote
up vote
0
down vote
I wouldn't worry about the dropped packets. Doing the math, you're losing less than .001% of your total packets.
As far as the load balancing goes, you probably need to (re)configure the load distribution algorithm in use on your switch (since it's the switch to server traffic that's not balancing). Cisco has some pretty good documentation on that here but it's too much to copy into this answer. Also note that the distribution algorithm assigns traffic streams to a physical interface which means that if there is only one stream it will only use one physical interface.
answered Dec 23 '15 at 17:42
David King
2,813421
I wouldn't worry about the dropped packets. Doing the math, you're losing less than .001% of your total packets.
As far as the load balancing goes, you probably need to (re)configure the load distribution algorithm in use on your switch (since it's the switch to server traffic that's not balancing). Cisco has some pretty good documentation on that here but it's too much to copy into this answer. Also note that the distribution algorithm assigns traffic streams to a physical interface which means that if there is only one stream it will only use one physical interface.
answered Dec 23 '15 at 17:42
David King
2,813421
edited Dec 23 '15 at 17:58
answered Dec 23 '15 at 17:42
David King
2,813421
answered Dec 23 '15 at 17:42
David King
2,813421
answered Dec 23 '15 at 17:42
David King
2,813421
2,813421
We are runningLACPat Layer2 switch, it should shared frames equally on both switch port. it has nothing to do withTCPat Layer 4, am i missing something here?
– Satish
Dec 23 '15 at 17:46
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
add a comment |
We are runningLACPat Layer2 switch, it should shared frames equally on both switch port. it has nothing to do withTCPat Layer 4, am i missing something here?
– Satish
Dec 23 '15 at 17:46
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
We are running
LACP at Layer2 switch, it should shared frames equally on both switch port. it has nothing to do with TCP at Layer 4, am i missing something here?– Satish
Dec 23 '15 at 17:46
We are running
LACP at Layer2 switch, it should shared frames equally on both switch port. it has nothing to do with TCP at Layer 4, am i missing something here?– Satish
Dec 23 '15 at 17:46
1
1
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
It's still using a load balancing algorithm. I misspoke about the streams being TCP, it applies to any kind of stream. Typically the physical interface is assigned based on a has of either the source and dest MAC or the source and dest IP. Regardless, a single stream will use a single physical interface.
– David King
Dec 23 '15 at 17:58
1
1
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
Regarding packet dropped, my monitoring system was flapping with SSH connection. and same time i tired SSH it just hung for min or longer and then back... does it normal??
– Satish
Dec 23 '15 at 17:58
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
If all of the drops happened at about the same time it would cause major problems but if they were spread out somewhat evenly throughout the life of the connection you should barely notice them.
– David King
Dec 23 '15 at 18:00
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f251186%2flinux-interface-rx-packet-dropped-with-lacp-and-not-doing-loadbalacing%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Is the traffic all coming from the same source or do you have multiple sources hitting this server?
– David King
Dec 23 '15 at 17:26
This server is
fileserverjust like backup machine. multiple machine accessing this single Linux server– Satish
Dec 23 '15 at 17:29