Julien Fortin [Thu, 17 Nov 2016 20:56:37 +0000 (21:56 +0100)]
ifupdownmain: warn user if an interface matches a regex but doesn't exists in the config anymore
Ticket: CM-13316
Reviewed By: Roopa, Nikhil G
Testing Done:
auto v0
iface v0
vlan-id 100
vlan-raw-device tap1
auto v1
iface v1
vlan-id 100
vlan-raw-device tap2
auto v2
iface v2
vlan-id 100
vlan-raw-device tap3
auto br0
iface br0
bridge-ports tap9 regex v.*
In this specific case, if you remove v0 then execute ifreload ifupdown2 will
barf and say that v0 still exists as a dependency of br0.
From roopa:
The problem here is that the dependency gathering should have been done after
the ifdown's. However the ifdown's need the dependency tree for built in
interfaces which could be part of globs. And this dependency tree needs to be
the current state of things so can't do the selective picking of interfaces
either. So, its a chicken and egg problem.
Ideally when one uses regex, if he is removing an interface that was matched by
a regex, he has to do a 'ifdown' manually today. 'ifdown v0' in this case.
So today we decided to change the warning message to let the user know that he
should manually down the interface that matches a regex.
This is not a permanent fix but a work-around. We don't want to introduce new
changes that may introduce false positive case for existing and working config
post 3.2:
-----------
we should consider doing the down's in two phases during ifreload
a) down everything that we are very certain was removed from the file
(to make sure regex's don't catch it).
b) build dependency tree and run through the down's once more, to
catch all glob matches for built-in interfaces
Julien Fortin [Tue, 15 Nov 2016 17:33:07 +0000 (18:33 +0100)]
dhcp: add support for inet + inet6 dhcp on same interface
Ticket: CM-12370
Reviewed By: Roopa, Kanna, Scott E
Testing Done:
This patch also fixes a problem where dhcp6 used to create lease file with
a trailing whitespace. dhcp6 operation were also sometimes using the wrong
pid file. I added some code in the debian.postinst script to correctly
rename these files if they exists when we install/update ifupdown2.
(cumulus-qa-infra/cl-tests/tests/smoke/testdhcp.py:Testdhcp_relay)
auto swp1
iface swp1 inet dhcp
link-speed 10000
link-duplex full
link-autoneg off
Julien Fortin [Thu, 10 Nov 2016 18:14:43 +0000 (19:14 +0100)]
netlink: force python to load ifupdown2's nlmanager cpy instead of python-nlmanager
Ticket: CM-13453
Reviewed By: Roopa, Nikhil G, Daniel W
Testing Done: ifupdown2-tests
We had an issue where python-nlmanager and ifupdown2's nlmanager differed
When netd starts it loads ifupdown2's addons, so it also tries to do the
imports. Since the current working directy is not /usr/share/ifupdown2 but
netd directory it won't load ifupdown2's nlmanager first but python-nlmanager
if it exists.
Here we insert/hardcode /usr/share/ifupdown2/ first in the path to make sure
we load the right nlmanager
Roopa Prabhu [Mon, 7 Nov 2016 18:48:24 +0000 (10:48 -0800)]
addons: address: various fixes for mtu handling
Ticket: CM-6908, CM-6110, CM-13221
Reviewed By: julien, nikhil
Testing Done: added a new test in ifupdown2-tests which covers all cases
- move all mtu handling to a single function in addons/address.py
- Have an ifupdown2 default of 1500 mtu
- add a policy manager max_mtu check (we want to default cumulus max mtu
to 9216)
- special handling for bond and bridges
- print an info log abt setting mtu on bridge
- this can be enhanced in the future to look
at individual port mtu and rejecting the bridge mtu.
this operation can be expensive right now. Hence
just an info log.
- bond and vxlan dev mtu follow the rules of physical device mtu
Scott Emery [Mon, 29 Aug 2016 18:28:52 +0000 (11:28 -0700)]
nlmanager: Provide option to prevent color output
Ticket: CM-7361
Reviewed By: CCR-
Testing Done: Tested with clag
Certain loggers, most notably the syslog, don't allow non-printable characters
in their output, and so they convert them to a string of printable characters.
For example, the newline character is converted to #012. This results in output
that looks like this:
which is rather hard to interpret. So this patch modifes the nlpacket debug
output so that the user can specify whether or not color output should be used
by including an extra, optional parameter when instantiating a NetlinkPacket
object. The default is for color output, just like before this patch. But if
color output is not desired, then no VT100 control sequences will be output. Nor
will there be any newline characters embedded in the output.
The NetlinkManagerWithListener and NetlinkManager classes were also modified to
add the same optional use_color attribute, which defaults to True. Thus when
class members which instantiate NetlinkPacket objects are created the
appropriate value for the use_color attribute will be applied.
I also noticed that the default pid_offset of the NetlinkListener class was 2,
when it should have been 1. So I fixed that too.
Scott Emery [Mon, 22 Aug 2016 19:51:40 +0000 (12:51 -0700)]
nlmanager: Support new attribute types
Ticket: CM-7361
Reviewed By: CCR-5124
Testing Done: Tested with clag
CLAG requires some additional nlmanager functionality. Firstly, different
threads in CLAG with the same PID may instantiate NetlinkManager objects. This
currently assigned each the same pid value, which causes issues. So an optional
pid_offset parameter is added which is added to the PID when instantiating a
NetlinkManager object. This allows multiple threads in the same PID to be able
to have distinct identifiers. If a pid_offset is supplied, it should be 2 or
greater, since 0 and 1 are used by the NetlinkManagerWithListener.
Added two new attribute classes: FourByteList and TwoByteValue. Added an encode
function for the IPAddress attribute, and corrected some length errors in the
MACAddress attribute.
The Neighbor class got actual values for the CacheInfo, VLAN, VNI, IfIndex, and
Master attributes. The "Self" and "Master" flag values were added.
Also added the "family" parameter to the constructor for all attribute classes.
This eliminates the special-casing required for certain attribute's
instantiation.
Nikhil [Thu, 6 Oct 2016 00:26:09 +0000 (17:26 -0700)]
addons: mstpctl: ifquery -with-defaults update for vlan-aware bridge ports
Ticket: CM-13086
Reviewed By: roopa, julien
Testing Done: smoke testifupdown2:TestWithdefaults is run
'ifquery -with-defaults' should display default values of
mstpctl-portbpdufilter and mstpctl-bpduguard under bridge port,
but not under the bridge itself
added PORT_PROCESSED flag check to prevent processing
bridge ports again
Example:
auto vxlan1wd
iface vxlan1wd
vxlan-id 1001
auto vxlan2wd
iface vxlan2wd
vxlan-id 1002
auto brwithdef2
iface brwithdef2
bridge_ports vxlan1wd vxlan2wd
bridge-vlan-aware yes
Sample 'ifquery --with-defaults vxlan1wd' output before fix:
auto vxlan1wd
iface vxlan1wd
vxlan-id 1001
Sample 'ifquery --with-defaults vxlan1wd' output after fix:
auto vxlan1wd
iface vxlan1wd
vxlan-id 1001
mstpctl-portbpdufilter yes
mstpctl-bpduguard yes
Roopa Prabhu [Wed, 19 Oct 2016 17:43:45 +0000 (10:43 -0700)]
[PATCH ifupdown2] addons: addressvirtual: fixup macvlan device enslavements for vrfs
Ticket: CM-12988
Reviewed By: julien, nikhil, dsa
Testing Done: tested ifup and ifdown of vrf devices with address virtual
slaves
This patch fixes up macvlan device enslavements when vrf device
or vrf slave is brought down and up. address virtual macvlan
devices on vrf slaves need to enslaved to the vrf. This
patch checks and fixes up those vrf enslavements for the following
cases:
ifdown <vrf_device> && ifup <vrf_device>
ifdown <vrf_slave> && ifup <vrf_slave>
starting state:
------------
$ip -br link show
myvrf UP 46:c6:44:db:37:60 <NOARP,MASTER,UP,LOWER_UP>
bridge.901@bridge UP 44:38:39:00:77:88 <UP,BROADCAST,MULTICAST,UP>
bridge-901-v0@bridge.901 UP 00:00:5e:00:01:81 <UP,BROADCAST,MULTICAST,UP>
$ifdown myvrf
$ip -br link show
bridge.901@bridge DOWN 44:38:39:00:77:88 <BROADCAST,MULTICAST>
bridge-901-v0@bridge.901 DOWN 00:00:5e:00:01:81 <BROADCAST,MULTICAST,M-DOWN>
before patch (macvlan device bridge-901-v0 did not come up:
----------------------------------------
$ifup myvrf
$ip -br link show
bridge.901@bridge UP 44:38:39:00:77:88 <UP,BROADCAST,MULTICAST,UP>
bridge-901-v0@bridge.901 DOWN 00:00:5e:00:01:81 <BROADCAST,MULTICAST>
myvrf UP ce:a6:e1:85:75:73 <NOARP,MASTER,UP,LOWER_UP>
after patch:
------------
$ifup myvrf
$ip -br link show
bridge.901@bridge UP 44:38:39:00:77:88 <UP,BROADCAST,MULTICAST,UP>
bridge-901-v0@bridge.901 UP 00:00:5e:00:01:81 <UP,BROADCAST,MULTICAST,UP>
myvrf UP ce:a6:e1:85:75:73 <NOARP,MASTER,UP,LOWER_UP>
Ticket:
Reviewed By: Roopa, Nikhil G
Testing Done:
When using ifupdown2 on regular distribution (debian/ubuntu etc...) users may
not have the latest iproute2/bridge-utils version. Thus the output parsed from
the various subcommands executed by ifupdown2 might defer and cause exception
to be throwned.
We identified several cases:
/sbin/bridge -c -json vlan show
/sbin/brctl showmcqv4src bridge0
/sbin/brctl showstp bridge0
Julien Fortin [Tue, 4 Oct 2016 08:13:23 +0000 (10:13 +0200)]
addons: vrf: removing ifaceobj.name from exception message
Ticket: None
Reviewed By:
Testing Done:
The function _create_vrf_dev is surrounded by a try/catch as follow:
try:
vrf_table = self._create_vrf_dev(ifaceobj, vrf_table)
except Exception, e:
self.log_error('%s: %s' %(ifaceobj.name, str(e)), ifaceobj)
Thus we shouldn't include the ifaceobj.name in any error message passed to
log_error(). Since this function will raise an exception by default, it will
result in something like this:
error:br0.4002: red: red: create failed (cmd 'ip link add name red type vrf table 1001' failed: returned 1 ...[snip]
error:red: red: create failed (cmd 'ip link add name red type vrf table 1002' failed: returned 1 ...[snip]
after this patch:
error: red: create failed (cmd 'ip link add name red type vrf table 1002' failed: returned 1 ... [snip]
addons may provide a list of ifupdown scripts to ignore
Ticket:
Reviewed By: Roopa, Nikhil G
Testing Done:
ifupdown2's python addons are replacing some/most of the old ifupdown shell
scripts. When addon_scripts_support is set to 1, ifupdown2 will execute every
scripts present in /etc/network/if-$ACTION.d/ if a script exists with a name
identical to an ifupdown2 addon, this script won't be executed.
Sometimes an ifupdown2 addons doesn't have the same name as the ifupdown script
it's supposed to replace. Or maybe one addon is replacing several scripts.
For example: bond.py is replacing /etc/network/if-*.d/ifenslave
Now each addon is able to provide a list of script to ignore.
addons: mstpctl: warning user when using portadminedge attr on bridge vlan aware
Ticket: CM-12478
Reviewed By: Roopa, Nikhil G
Roopa: "For vlan aware bridge, we don't support the port attributes under the
bridge. It was deliberately done when vlan aware bridge config was introduced
because people wanted to get rid of the legacy under the bridge way of decla-
-ring port attributes. It was supported for a bit for evorack...in limited
form. we should probably just warn the user about this."
upperiface bringup is best effort, so ignore errors.
$ifquery -c -a
..[snip]...
auto br0
iface br0
bridge-vlan-aware yes [pass]
bridge-ports vx-14 vx-11 vx-10 vx-13 vx-12 [pass]
$ifdown vx-10
$ifdown vx-11
before patch:
$ifup vx-10
..[snip]...
info: running upperifaces (parent interfaces) if available ..
info: br0: running ops ...
error: br0: bridge port vx-11 does not exist
....
After patch:
$ifup vx-10
..[snip]...
info: running upperifaces (parent interfaces) if available ..
info: br0: running ops ...
....
ifupdownmain: dont catch link_up and link_down errors in ifupdownmain
Ticket: CM-12843
Reviewed By: julien, nikhil
Testing Done: Tested the failing test case in the bug
The scheduler already has some checking for errors from
link_up and link_down introduced by
commit 61c4d7244735 ("Suppress 'Network down' warnings when
link_master_slave feature is on").
The above fix was removed by a recent commit 707aeb737882 ("netlink ip link set up/down may silently fail, adding
try/except statements"). The scheduler does catch these errors.
so we will need to figure out why the scheduler is not catching it.
Right now this is failing an automation test. So the best fix
is to make sure the previous behaviour is restored.
addons: vrf: close sockets when vrf interface goes down
Ticket: CM-11393
Reviewed By: dsa, julien, nikhil
Testing Done: tested up and down of a vrf interface
$ifdown -v blue
..snip ..
info: executing /usr/lib/vrf/vrf-helper delete blue 1030
info: executing ip link del blue
info: executing /bin/ss -aK "dev == 54"
info: vrf: syncing table map to
/etc/iproute2/rt_tables.d/ifupdown2_vrf_map.conf
$ cat /etc/network/interfaces
auto tap0
iface tap0
#mtu 9000
mtu 1500
auto bridge
iface bridge
bridge-ports tap0
auto bridge.20
iface bridge.20
address 10.7.192.194/27
address-virtual 44:38:39:ff:00:20 10.7.192.193
$ ifreload -a
$ ip link show
bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
bridge.20@bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
bridge-20-v0@bridge.20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
$ #change mtu to 9000
$ ifreload -a
$ ip link show
bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default
bridge.20@bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default
bridge-20-v0@bridge.20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default
$
normalizing ip address(s) to IPNetwork format for string comparaison
Ticket: CM-12798
Reviewed By: Roopa, Nikhil G
Testing Done:
We are seeing some issue when using IP addresses with inner values padded with
zeros. Such as: 2a01:75e0:0000:09b0::1/64
The kernel will process the ip properly but when we query the kernel again (with
iproute2 or netlink) it returns 2a01:75e0:0:09b0::1/64
Since we are doing string comparaison we are seeing failures. We are now
converting all ip address to a standard format using IPNetwork or IPAddress obj
A customer discovered a corner case: the kernel is shrinking/reducing the ip
address fields containing 0s. For example if we configure and address such as
2a01:75e0:0000:09b0::1/64
Then if we query the kernel (using iproute2 or netlink) it will come back as:
2a01:75e0:0:09b0::1/64
Because of this issue we were seeing
root@r4u28:~# ifreload -a
error: bridge.200: cmd 'ip -force -batch - [addr add 10.50.103.193/26 dev bridge.200
addr add 2a01:75e0:0000:09b1::1/64 dev bridge.200
]' failed: returned 1 (RTNETLINK answers: File exists
Command failed -:1
)
Before adding an address to an iface we query it to know if the change is
necessary, since we only do a string comparaison between:
2a01:75e0:0000:09b1::1/64 and 2a01:75e0:0:09b1::1/64
it fails, ifupdown2 thinks that a change is needed. So we try to add the new
address via iproute2. iproute2 will fail because this address in a "shrinked"
format already exists.
This patch hot-fixes this problem by ignoring the error if it's an "exists"
error, we don't want to confuse the user.
A later fix will the real issue by normalizing all ip addr to the IPNetwork
format.
addons: vrf: be more friendly to older iproute2 versions
Ticket: CM-12049
Reviewed By: dsa, julien, nikhil
Testing Done: tested with and without /etc/iproute2/rt_tables.d/
OS does not have /etc/iproute2/rt_tables.d, on errors just info:
$ifreload -a -v
info: unable to save iproute2 vrf to table map ([Errno 2] No such file
or directory: '/etc/iproute2/rt_tables.d/ifupdown2_vrf_map.conf')
info: cannot find /etc/iproute2/rt_tables.d. pls check if your iproute2
version supports rt_tables.d
OS has /etc/iproute2/rt_tables.d, on errors warn:
$ifreload -a
warn: unable to save iproute2 vrf to table map ([Errno 2] No such file
Julien Fortin [Thu, 25 Aug 2016 21:57:21 +0000 (14:57 -0700)]
nlmanager: fixing error messages when receiving NLMSG_ERROR
Ticket: CM-12596
Reviewed By: Roopa, Daniel W
Testing Done:
before this patch:
$ cat /etc/network/interfaces
auto tapppppppppppp0.42
iface tapppppppppppp0.42
$
$ ifreload -a
warning: /etc/network/interfaces: line8: tapppppppppppp0.42: interface name too long
error: netlink: tapppppppppppp0: cannot create vlan 42: Operation failed with 'None' (RXed NLMSG_ERROR, pid 3346, seq 6, 108 bytes)
$
after:
$ ifreload -a
warning: /etc/network/interfaces: line8: tapppppppppppp0.42: interface name too long
error: netlink: tapppppppppppp0: cannot create vlan 42: Operation failed with 'Numerical result out of range'
$
Roopa Prabhu [Fri, 19 Aug 2016 18:09:57 +0000 (11:09 -0700)]
addons: bridge: fix running_vids value when cache is stale
Ticket: CM-12552
Reviewed By: julien, nikhil
Testing Done: tested with failing config with bridge-access 1
This is similar to the fix done in the below commit for pvid:
"5061730ea5bf ("addons: bridge: fix default pvid handling in cases where
cache is stale")"
easier steps to reproduce:
- have a vlan aware bridge with more than one ports
- add 'bridge-access 1' to one of the ports
- boot the box with the config
- check that the vlans are fine
- ifdown <interface_with_bridge_access_1>
- ifreload -a
- the interface with bridge_access 1 does
not have the pvid flag on vlan 1
This patch makes sure we assume the right running
vid and pvid value ie [1] and 1 if the
cache returns no values. vid = [1] and pvid = 1
are the kernel default/initial values for a port.
Roopa Prabhu [Sat, 13 Aug 2016 06:49:29 +0000 (23:49 -0700)]
addons: vrf: don't delete vrf map file at init if mgmt vrf already exists
Ticket: CM-12084
Reviewed By: julien, nikhil
Testing Done: tested with --allow=mgmt class
This fixes a problem where vrf map file was getting deleted
at boot during the second invocation of ifup. This is because
the code uses PERFMODE to check it is boot..but does
not realize that this may not be the first invocation of
ifup during the boot.
This patch adds additional checks for mgmt netdev to
make sure we delete the vrf map file only the first time.
My previous fix for the same bug did not fix the problem completly.
In this patch all running pvid calculation
is done before the pvid_to_del is calculated.
This makes sure the old pvid is deleted if it is not
the same as current pvid in all cases
Julien Fortin [Thu, 11 Aug 2016 01:05:04 +0000 (03:05 +0200)]
nlpacket: add new attribute: AttributeStringInterfaceName with length check
Ticket: CM-12302
Reviewed By: Daniel, Roopa, Nikhil G
Testing Done: ifupdown2 smoke test
With this pretty straight forward, we introduce a new Attribute in nlmanager,
so that we have single check and no code redundancy for ifname length. This
also prevent loosing time sending a netlink packet for an known error.
This can happen if the bridge vlans were
cached before the interface become a bridge port.
and when the interface becomes a bridge port
kernel adds a default pvid of 1. so this patch
assumes a default pvid is 1 if we did not find
anything in the cache. This will delete
the default pvid of 1 if the user has configured a
different pvid.
The message logged and the exception raised by nlmanager, when receiving
an NLMSG_ERROR packet from the kernel, wasn't super explicit plus if the
error wasn't a netlink related failure we didn't provide any information
Now we use strerror with the provided error code to generate a nice msg.
ifupdownaddons: iproute2: Cache vxlan remote ip addresses only if
statically configured
Ticket: CM-12277
Reviewed By: CCR-5059
Testing Done: ifreload with a VxlanClagClosConfig.py config
get_vxlan_peers gets remote ip addresses if they are statically configured
through ifupdown2 (vxlan-remoteip <address>). These are cached in
ifupdown2's link object so that they can be used in 'ip link set' (if link
exists already), 'ifquery -r' and 'ifquery -c'. ifupdown2 should ignore and
not cache these addresses if they are provisioned by vxrd. Check if vxrd is
running before caching remote ip addresses. This helps in optimizing time
taken to ifreload with 1k bridge/vxlan interfaces (vlan-aware bridge) in a
vxrd/vxsnd setup.
Roopa Prabhu [Mon, 8 Aug 2016 22:46:13 +0000 (15:46 -0700)]
addons: address: never reset mtu on lo implicity
we should leave the mtu on lo to the default mtu
if user has not requested to change it. lo has
a larger mtu because it does not really depend on a
physical mtu.
vrf slave brings up the master if master is not up yet.
Today this is done only when ALL (auto) option is set
just as an optimization. because you dont want to bring
up the master in cases where user just wants to
bring up the vrf slave. eg ifup -v eth0.
This does not work so well, when user uses
--allow classes to bring up vrf master and slaves
together (eg mgmt vrf).
This patch removes the ALL check when bringing
up master and replaces it with an ALL or
CLASS check. ie make sure vrf master belongs to the
same class as you when CLASS is specified.
Roopa Prabhu [Fri, 5 Aug 2016 18:28:28 +0000 (11:28 -0700)]
ifupdown: move IFACE_CLASS ifupdownmain flag to ifupdownflags.CLASS
This patch moves ifupdownmain flag IFACE_CLASS to
ifupdownflags.CLASS. ifupdownflags.CLASS is set to true
if the user is asking for a class of interfaces.
example: ifreload --allow=mgmt
By moving it to ifupdownflags, we make it visible
to add modules.
Roopa Prabhu [Fri, 19 Aug 2016 18:09:57 +0000 (11:09 -0700)]
addons: bridge: fix running_vids value when cache is stale
Ticket: CM-12552
Reviewed By: julien, nikhil
Testing Done: tested with failing config with bridge-access 1
This is similar to the fix done in the below commit for pvid:
"5061730ea5bf ("addons: bridge: fix default pvid handling in cases where
cache is stale")"
easier steps to reproduce:
- have a vlan aware bridge with more than one ports
- add 'bridge-access 1' to one of the ports
- boot the box with the config
- check that the vlans are fine
- ifdown <interface_with_bridge_access_1>
- ifreload -a
- the interface with bridge_access 1 does
not have the pvid flag on vlan 1
This patch makes sure we assume the right running
vid and pvid value ie [1] and 1 if the
cache returns no values. vid = [1] and pvid = 1
are the kernel default/initial values for a port.
Roopa Prabhu [Sat, 13 Aug 2016 06:49:29 +0000 (23:49 -0700)]
addons: vrf: don't delete vrf map file at init if mgmt vrf already exists
Ticket: CM-12084
Reviewed By: julien, nikhil
Testing Done: tested with --allow=mgmt class
This fixes a problem where vrf map file was getting deleted
at boot during the second invocation of ifup. This is because
the code uses PERFMODE to check it is boot..but does
not realize that this may not be the first invocation of
ifup during the boot.
This patch adds additional checks for mgmt netdev to
make sure we delete the vrf map file only the first time.
projects / mirror_ifupdown2.git / log