Routing problem with multiple PPPoE uplinks

[stas.vitkovsky@…]

When multiple PPPoE WAN links are set up, Zentyal is unable to set up proper ip rules (it adds commands like /sbin/ip rule add from table 102, which obviously fail).

012/05/15 15:44:18 INFO> GlobalImpl.pm:653 EBox::GlobalImpl::saveAllModules - Changes saved successfully
2012/05/15 15:44:22 INFO> Base.pm:250 EBox::Module::Base::__ANON__ - Saving config for module: network
2012/05/15 15:44:22 INFO> Base.pm:250 EBox::Module::Base::__ANON__ - Saving config for module: network
2012/05/15 15:44:23 ERROR> Sudo.pm:213 EBox::Sudo::_rootError - root command set -e
/usr/share/zentyal-network/flush-fwmarks
/sbin/ip route flush table 102
/sbin/ip rule add fwmark 2/0xFF table 102
/sbin/ip rule add from 10.67.15.6 table 102
/sbin/ip rule add from  table 102
/sbin/ip route add default dev ppp0 table 102
/sbin/ip route flush table 101
/sbin/ip rule add fwmark 1/0xFF table 101
/sbin/ip rule add from 10.67.15.7 table 101
/sbin/ip rule add from  table 101
/sbin/ip route add default dev ppp1 table 101
/sbin/ip rule add table main
/sbin/iptables -t mangle -A PREROUTING -m mark --mark 0/0xff -i ppp1 -j MARK --set-mark 1
/sbin/iptables -t mangle -A POSTROUTING -o ppp1 -j MARK --set-mark 1
/sbin/iptables -t mangle -A PREROUTING -m mark --mark 0/0xff -i ppp0 -j MARK --set-mark 2
/sbin/iptables -t mangle -A POSTROUTING -o ppp0 -j MARK --set-mark 2
/sbin/iptables -t mangle -A POSTROUTING -o ppp0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
/sbin/iptables -t mangle -A POSTROUTING -o ppp1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
/sbin/iptables -t mangle -A PREROUTING  -m mark --mark 0/0xff -j  MARK --set-mark 2
/sbin/iptables -t mangle -A OUTPUT -m mark --mark 0/0xff -j  MARK --set-mark 2 failed.
Error output: Error: an inet prefix is expected rather than "table".

Command output: .
Exit value: 1
2012/05/15 15:44:23 ERROR> ppp-set-iface.pl:40 main::__ANON__ - Call to setRealPPPIface for eth1 failed
2012/05/15 15:44:23 ERROR> Sudo.pm:213 EBox::Sudo::_rootError - root command set -e
/usr/share/zentyal-network/flush-fwmarks
/sbin/ip route flush table 102
/sbin/ip rule add fwmark 2/0xFF table 102
/sbin/ip rule add from 10.67.15.6 table 102
/sbin/ip rule add from  table 102
/sbin/ip route add default dev ppp0 table 102
/sbin/ip route flush table 101
/sbin/ip rule add fwmark 1/0xFF table 101
/sbin/ip rule add from 10.67.15.7 table 101
/sbin/ip rule add from  table 101
/sbin/ip route add default dev ppp1 table 101
/sbin/ip rule add table main
/sbin/iptables -t mangle -A PREROUTING -m mark --mark 0/0xff -i ppp1 -j MARK --set-mark 1
/sbin/iptables -t mangle -A POSTROUTING -o ppp1 -j MARK --set-mark 1
/sbin/iptables -t mangle -A PREROUTING -m mark --mark 0/0xff -i ppp0 -j MARK --set-mark 2
/sbin/iptables -t mangle -A POSTROUTING -o ppp0 -j MARK --set-mark 2
/sbin/iptables -t mangle -A POSTROUTING -o ppp0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
/sbin/iptables -t mangle -A POSTROUTING -o ppp1 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
/sbin/iptables -t mangle -A PREROUTING  -m mark --mark 0/0xff -j  MARK --set-mark 2
/sbin/iptables -t mangle -A OUTPUT -m mark --mark 0/0xff -j  MARK --set-mark 2 failed.
Error output: Error: an inet prefix is expected rather than "table".

Command output: .
Exit value: 1
2012/05/15 15:44:23 ERROR> ppp-set-iface.pl:40 main::__ANON__ - Call to setRealPPPIface for eth2 failed

as the result, we have the following rules and routing tables set up (eth0 is LAN, eth1/2 are WANs):

root@zentyal:~# ip rule ls
0:      from all lookup local
32764:  from 10.67.15.6 lookup 102
32765:  from all fwmark 0x2/0xff lookup 102
32766:  from all lookup main
32767:  from all lookup default
root@zentyal:~# ip route ls
10.0.0.1 dev ppp0  proto kernel  scope link  src 10.67.15.6
10.0.0.1 dev ppp1  proto kernel  scope link  src 10.67.15.7
10.99.99.0/24 dev eth0  proto kernel  scope link  src 10.99.99.170
root@zentyal:~# ip route ls table default
default
        nexthop dev ppp1 weight 1
        nexthop dev ppp0 weight 1
root@zentyal:~# ip route ls table 102
root@zentyal:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:5c:00:8c
          inet addr:10.99.99.170  Bcast:10.99.99.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe5c:8c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8927 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3445 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1114297 (1.1 MB)  TX bytes:979299 (979.2 KB)

eth1      Link encap:Ethernet  HWaddr 08:00:27:d4:72:76
          inet6 addr: fe80::a00:27ff:fed4:7276/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8936 errors:0 dropped:0 overruns:0 frame:0
          TX packets:222 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:730194 (730.1 KB)  TX bytes:10271 (10.2 KB)

eth2      Link encap:Ethernet  HWaddr 08:00:27:7e:eb:e3
          inet6 addr: fe80::a00:27ff:fe7e:ebe3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7731 errors:0 dropped:0 overruns:0 frame:0
          TX packets:190 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:564423 (564.4 KB)  TX bytes:8224 (8.2 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:31041 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31041 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3007644 (3.0 MB)  TX bytes:3007644 (3.0 MB)

ppp0      Link encap:Point-to-Point Protocol
          inet addr:10.67.15.6  P-t-P:10.0.0.1  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1492  Metric:1
          RX packets:19 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:8686 (8.6 KB)  TX bytes:2548 (2.5 KB)

ppp1      Link encap:Point-to-Point Protocol
          inet addr:10.67.15.7  P-t-P:10.0.0.1  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1492  Metric:1
          RX packets:29 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:5518 (5.5 KB)  TX bytes:2434 (2.4 KB)

As it is pointed out  here (Hungarian), the problem could be "fixed" by changing the line 2695 at file /usr/share/perl5/EBox/Network.pm from

if ( scalar keys %interfaces > 1 ) { 

to something like

if ((scalar keys% interfaces> 1) and ($ method ne 'ppp')) {

and that fixes the problem.

[jamor@…]

Hello Stas Vitkovsky,

Thanks a lot for taking time to write this detailed report.

I have fixed it in [ef29ac6] but instead of skipping ppp ifaces, I skip intrfaces without assigned address.

You can see the diff there ->  http://git.zentyal.org/zentyal.git/blobdiff/b282cbe8ac83503428f9f9fe56be8170b3bd6949..ef29ac6e94944001765a9409f395a46b0d522246:/main/network/src/EBox/Network.pm

Regards,

Javier