NDB: lost sync in apply() (due 'File exists'?) when trying to bring up an interface inside a network namespace

[rytilahti]

Hi and thanks for the great library!

I have been experimenting with the NDB interface and network namespaces, and I'm currently struggling to make the network interface inside a netns to go up.

As similar procedure works fine with the "IPDB-way", I think it is a bug, but I couldn't figure out where should I start looking to fix it.

P.S. A quick side-question, is it safe to use multiple context managers, ala with ndb.interfaces[peer_dict] as peer_veth, nbd.interfaces[root_ns_if] as root_veth:?

NDB

interface on main namespace

1272: prcf30af0d-r@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a6:6f:4e:64:dd:36 brd ff:ff:ff:ff:ff:ff link-netns prcf30af0d
    inet 192.168.233.0/31 scope global prcf30af0d-r
       valid_lft forever preferred_lft forever
    inet6 fe80::a46f:4eff:fe64:dd36/64 scope link 
       valid_lft forever preferred_lft forever

Interface inside the NS

• Note the smallish index number, potentially related as IPDB assigns a sequantial index?
• On the main namespace that index is reserved for the ethernet adapter.

2: prcf30af0d-p@if1272: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 36:65:27:22:ae:76 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.233.1/31 scope global prcf30af0d-p
       valid_lft forever preferred_lft forever

Debug log

DEBUG:pyroute2.ndb.139946316556880.view.interfaces:cache del (('target', 'localhost'), ('tflags', 0), ('index', 1272))
DEBUG:pyroute2.ndb.139946316556880.view.interfaces:cache add (('target', 'prcf30af0d'), ('tflags', 0), ('index', 2))
INFO:__main__:setting peer iface prcf30af0d-p up with 192.168.233.1/31
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:commit: [(1565646783.800148, 'invalid'), (1565646783.8018253, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:init
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:complete key OrderedDict([('target', 'prcf30af0d'), ('tflags', 0), ('index', 2)]) from table interfaces_139946275317232
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:got OrderedDict([('target', 'prcf30af0d'), ('tflags', 0), ('index', 2)])
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 4098, 0, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 1, 0, None, 0, 65535, 65536, 0, 1, 'down', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot interfaces_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot addresses_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot neighbours_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot routes_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot nh_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot rules_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot netns_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot p2p_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_bridge_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_bond_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_vlan_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_vxlan_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_gre_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_vrf_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_vti_139946275317232
DEBUG:pyroute2.ndb.139946316556880.schema:create snapshot ifinfo_vti6_139946275317232
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply: [(1565646783.800148, 'invalid'), (1565646783.8018253, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 4098, 0, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 1, 0, None, 0, 65535, 65536, 0, 1, 'down', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply req: {'target': 'prcf30af0d', 'tflags': 0, 'index': 2, 'state': 'up', 'master': None}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply idx_req: {'target': 'prcf30af0d', 'tflags': 0, 'index': 2}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply state: system
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run set ({'target': 'prcf30af0d', 'tflags': 0, 'index': 2, 'state': 'up', 'master': None})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: apply set {objid 139946275317232, wtime 0.4, mqsize 4, nqsize 0}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_rtnl: {'length': 1360, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=0, delay=0)}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 69635, 1, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 1, 2, 0, None, 0, 65535, 65536, 1, 1, 'up', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 69635, 1, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 1, 2, 0, None, 0, 65535, 65536, 1, 1, 'up', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: [(1565646783.800148, 'invalid'), (1565646783.844721, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: True
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:checked
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: 139946275317232 pass
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:init
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:complete key {'target': 'localhost', 'address': '192.168.233.1', 'prefixlen': 31} from table addresses
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_rtnl: {'length': 1360, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=0, delay=0)}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('localhost', 0, 0, 1, 1272, 69699, 0, 'a6:6f:4e:64:dd:36', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-r', 1500, 2, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 1, 2, 0, None, 0, 65535, 65536, 1, 1, 'up', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_rtnl: {'length': 1360, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=0, delay=0)}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 69699, 0, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 1, 2, 0, None, 0, 65535, 65536, 1, 1, 'up', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:got none
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply: [(1565646783.8450475, 'invalid')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: None
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply req: {'target': 'prcf30af0d', 'prefixlen': 31, 'IFA_ADDRESS': '192.168.233.1', 'address': '192.168.233.1'}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply idx_req: {'target': 'prcf30af0d', 'prefixlen': 31, 'IFA_ADDRESS': '192.168.233.1'}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:apply state: invalid
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run add ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: apply add {objid 139946275357616, wtime 0.0, mqsize 0, nqsize 0}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 2, 31, 128, 0, 2, '192.168.233.1', '192.168.233.1', 'prcf30af0d-p', None, None, None, 128)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: [(1565646783.8450475, 'invalid'), (1565646783.854934, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check changed: {'target'}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check failed
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run add ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:error: (17, 'File exists')
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:ignore error 17 for {'target': 'localhost', 'address': '192.168.233.1', 'prefixlen': 31, 'tflags': 0, 'family': 2, 'flags': 128, 'scope': 0, 'index': 2, 'local': '192.168.233.1', 'label': 'prcf30af0d-p', 'broadcast': None, 'anycast': None, 'multicast': None}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run fallback set ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: apply add {objid 139946275357616, wtime 0.1, mqsize 1, nqsize 0}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 2, 31, 128, 0, 2, '192.168.233.1', '192.168.233.1', 'prcf30af0d-p', None, None, None, 128)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: [(1565646783.8450475, 'invalid'), (1565646783.8588128, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check changed: {'target'}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check failed
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run add ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:error: (17, 'File exists')
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:ignore error 17 for {'target': 'localhost', 'address': '192.168.233.1', 'prefixlen': 31, 'tflags': 0, 'family': 2, 'flags': 128, 'scope': 0, 'index': 2, 'local': '192.168.233.1', 'label': 'prcf30af0d-p', 'broadcast': None, 'anycast': None, 'multicast': None}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run fallback set ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: apply add {objid 139946275357616, wtime 0.2, mqsize 0, nqsize 0}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 2, 31, 128, 0, 2, '192.168.233.1', '192.168.233.1', 'prcf30af0d-p', None, None, None, 128)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: [(1565646783.8450475, 'invalid'), (1565646783.9650478, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check changed: {'target'}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check failed
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run add ({'target': 'prcf30af0d', 'address': '192.168.233.1', 'prefixlen': 31, 'index': 2})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:error: (17, 'File exists')

<I snipped off some more tries from here>

DEBUG:pyroute2.ndb.139946316556880.rtnl_object:run set ({'target': 'prcf30af0d', 'tflags': 0, 'index': 2, 'state': 'down', 'master': None})
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: apply set {objid 139946275002032, wtime 0.4, mqsize 4, nqsize 0}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_rtnl: {'length': 1360, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=0, delay=0)}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 4098, 1, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, 0, None, 0, 65535, 65536, 1, 2, 'down', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('prcf30af0d', 0, 0, 1, 2, 4098, 0, '36:65:27:22:ae:76', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-p', 1500, 1272, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 1, 0, None, 0, 65535, 65536, 0, 1, 'down', 'veth', None)
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: [(1565646783.808394, 'invalid'), (1565646798.4870124, 'system')]
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:check: True
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:checked
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:stats: 139946275002032 pass
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_rtnl: {'length': 1360, 'type': 16, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=-1, delay=0)}
DEBUG:pyroute2.ndb.139946316556880.rtnl_object:load_sql: ('localhost', 0, 0, 1, 1272, 4099, 0, 'a6:6f:4e:64:dd:36', 'ff:ff:ff:ff:ff:ff', 'prcf30af0d-r', 1500, 2, 'noqueue', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, 0, None, 0, 65535, 65536, 1, 2, 'up', 'veth', None)
Traceback (most recent call last):
  File "test_ndb.py", line 65, in <module>
    create_namespace_nbd()
  File "test_ndb.py", line 39, in create_namespace_nbd
    _LOGGER.info(f"setting peer iface {peer_ns_if} up with {peer_ip_w_cidr}")
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/__init__.py", line 261, in __exit__
    self.commit()
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/__init__.py", line 496, in commit
    self.apply()
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/interface.py", line 315, in apply
    super(Interface, self).apply(rollback)
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/__init__.py", line 715, in apply
    raise ret
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/interface.py", line 211, in do_add_ip
    self.ipaddr.create(spec).apply()
  File "/code/path/lib/pyroute2/pyroute2/ndb/objects/__init__.py", line 672, in apply
    raise Exception('lost sync in apply()')
Exception: lost sync in apply()

IPDB variant

• Running the IPDB version works as expected.
• The interface numbers are sequential here.

Main namespace

1274: pr5ddf7a81-r@if1273: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 62:f4:1a:e1:10:23 brd ff:ff:ff:ff:ff:ff link-netns pr5ddf7a81
    inet 192.168.233.0/31 scope global pr5ddf7a81-r
       valid_lft forever preferred_lft forever
    inet6 fe80::60f4:1aff:fee1:1023/64 scope link 
       valid_lft forever preferred_lft forever

Inside the namespace

1273: pr5ddf7a81-p@if1274: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether d6:80:b2:5a:a0:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.233.1/31 scope global pr5ddf7a81-p
       valid_lft forever preferred_lft forever
    inet6 fe80::d480:b2ff:fe5a:a004/64 scope link 
       valid_lft forever preferred_lft forever

Test script

• Note, this does not clean the interfaces nor the namespace afterwards, for testing I simple used ip -all netns delete.

from pyroute2 import IPRoute, netns, NetNS, NDB, IPDB
from pyroute2.common import uifname
import logging

logging.basicConfig(level=logging.DEBUG)

_LOGGER = logging.getLogger(__name__)

root_ip, peer_ip = "192.168.233.0", "192.168.233.1"
root_ip_w_cidr = f"{root_ip}/31"
peer_ip_w_cidr = f"{peer_ip}/31"

nsname = uifname()

root_ns_if = f"{nsname}-r"
peer_ns_if = f"{nsname}-p"


def create_namespace_nbd():
    ndb = NDB(debug=True)

    ndb.sources.add(netns=nsname)

    ndb.interfaces.create(
        ifname=root_ns_if, kind="veth", peer={"ifname": peer_ns_if, "net_ns_fd": nsname}
    ).commit()

    with ndb.interfaces[root_ns_if] as root_veth:
        root_veth.add_ip(root_ip_w_cidr)
        root_veth["state"] = "up"
        _LOGGER.info(f"setting root iface {root_ns_if} up with {root_ip_w_cidr}")

    # need to use net_ns_fd to locate the peer interface
    peer_dict = {"net_ns_fd": nsname, "ifname": peer_ns_if}
    with ndb.interfaces[peer_dict] as peer_veth:
        peer_veth.add_ip(peer_ip_w_cidr)
        peer_veth["state"] = "up"
        _LOGGER.info(f"setting peer iface {peer_ns_if} up with {peer_ip_w_cidr}")


def create_namespace_ipr():
    ipdb = IPDB()

    ns = NetNS(nsname)

    ipdb.create(ifname=root_ns_if, kind="veth", peer=peer_ns_if).commit()
    with ipdb.interfaces[peer_ns_if] as i:
        i.net_ns_fd = nsname

    with ipdb.interfaces[root_ns_if] as root_veth:
        root_veth.add_ip(root_ip_w_cidr)
        root_veth.up()

    ipdb_ns = IPDB(nl=ns)

    with ipdb_ns.interfaces[peer_ns_if] as peer_veth:
        peer_veth.add_ip(peer_ip_w_cidr)
        peer_veth.up()

    _LOGGER.info(f"created successfully.")
    ipdb_ns.release()
    ns.close()


# create_namespace_nbd()
create_namespace_ipr()

[svinota]

Sorry for the delay.

Please try

# notice "target" instead of "net_ns_fd" here:
peer_dict = {"target": nsname, "ifname": peer_ns_if}

Regarding the question:

is it safe to use multiple context managers

must be OK, if not — pls open a ticket

[afics]

@svinota for me this still happens with your suggested change (net_ns_fd -> target).

[mawiegand]

Any news here?

I run into the same error if I try to execute the following:

#!/usr/bin/python3
 
from pyroute2 import NDB
 
ndb = NDB(debug=True,log="on")
 
ndb.sources.add(netns="ns1")
 
# move interface to namespace "ns1"
dev = ndb.interfaces[{"ifname": "enp0s3"}]
dev = dev.set("target", "ns1").commit()
 
# add ip address
dev = dev.add_ip("192.168.42.21/24")
dev = dev.commit()

[svinota]

(/me back from long absence)

Found the issue, will be fixed on Thursday.

[svinota]

Workaround is merged, pls check if it helps. But the proper fix is on the way.

[mawiegand]

Workaround is merged, pls check if it helps. But the proper fix is on the way.

Seems still to be broken:

Traceback (most recent call last):
  File "./pyroute2_test.py", line 10, in <module>
    dev = ndb.interfaces[{"ifname": "enp0s3"}]
  File "/home/mawiegand/pyroute2/pyroute2/ndb/main.py", line 299, in __getitem__
    match_pairs=match_pairs)
  File "/home/mawiegand/pyroute2/pyroute2/ndb/objects/interface.py", line 150, in __init__
    super(Interface, self).__init__(*argv, **kwarg)
  File "/home/mawiegand/pyroute2/pyroute2/ndb/objects/__init__.py", line 231, in __init__
    raise KeyError('object does not exists')
KeyError: 'object does not exists'

Log:

2020-01-16 06:24:07,482    DEBUG pyroute2.ndb.140602995765824.sources.ns1: init                                                                                                                                                                                       [22/1939]
2020-01-16 06:24:07,482    DEBUG pyroute2.ndb.140602995765824.sources.ns1: starting the source
2020-01-16 06:24:07,483    DEBUG pyroute2.ndb.140602995765824.sources.ns1: connecting                                                                                                                                                                                         
2020-01-16 06:24:07,489    DEBUG pyroute2.ndb.140602995765824.sources.ns1: loading                                                                                                                                                                                            
2020-01-16 06:24:07,500    DEBUG pyroute2.ndb.140602995765824.sources.ns1: running                                                                                                                                                                                            
2020-01-16 06:24:07,500    DEBUG pyroute2.ndb.140602995765824.rtnl_object: init                                                                                                                                                                                               
2020-01-16 06:24:07,500    DEBUG pyroute2.ndb.140602995765824.rtnl_object: complete key {'ifname': 'enp0s3'} from table interfaces                                                                                                                                            
2020-01-16 06:24:07,500    DEBUG pyroute2.ndb.140602995765824.rtnl_object: got {'ifname': 'enp0s3', 'target': 'localhost', 'tflags': 0, 'index': 2}                                                                                                                           
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: ('localhost', 0, 0, 1, 2, 4098, 0, '08:00:27:61:87:28', 'ff:ff:ff:ff:ff:ff', 'enp0s3', 1500, None, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, None, None, 0, 65535,
65536, 1, 2, 'down', None, None)
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.view.interfaces: cache add (('target', 'localhost'), ('tflags', 0), ('index', 2))                                                                                                                               
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: commit: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns')]                                                                                           
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: init
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: complete key OrderedDict([('target', 'localhost'), ('tflags', 0), ('index', 2)]) from table interfaces_140602968230464                                                                             
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: got OrderedDict([('target', 'localhost'), ('tflags', 0), ('index', 2)])                                                                                                                            
2020-01-16 06:24:07,501    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: ('localhost', 0, 0, 1, 2, 4098, 0, '08:00:27:61:87:28', 'ff:ff:ff:ff:ff:ff', 'enp0s3', 1500, None, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, None, None, 0, 65535,
65536, 1, 2, 'down', None, None)
2020-01-16 06:24:07,502    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot interfaces_140602968230464
2020-01-16 06:24:07,503    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot addresses_140602968230464
2020-01-16 06:24:07,503    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot neighbours_140602968230464
2020-01-16 06:24:07,503    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot routes_140602968230464
2020-01-16 06:24:07,503    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot nh_140602968230464
2020-01-16 06:24:07,504    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot rules_140602968230464
2020-01-16 06:24:07,504    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot netns_140602968230464
2020-01-16 06:24:07,504    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot p2p_140602968230464
2020-01-16 06:24:07,504    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_bridge_140602968230464
2020-01-16 06:24:07,505    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_bond_140602968230464
2020-01-16 06:24:07,505    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_vlan_140602968230464
2020-01-16 06:24:07,506    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_vxlan_140602968230464
2020-01-16 06:24:07,506    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_gre_140602968230464
2020-01-16 06:24:07,506    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_vrf_140602968230464
2020-01-16 06:24:07,507    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_vti_140602968230464
2020-01-16 06:24:07,507    DEBUG pyroute2.ndb.140602995765824.schema: create snapshot ifinfo_vti6_140602968230464
2020-01-16 06:24:07,509    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns')]
2020-01-16 06:24:07,510    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: ('localhost', 0, 0, 1, 2, 4098, 0, '08:00:27:61:87:28', 'ff:ff:ff:ff:ff:ff', 'enp0s3', 1500, None, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, None, None, 0, 65535,
65536, 1, 2, 'down', None, None)
2020-01-16 06:24:07,510    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply req: {'target': 'localhost', 'tflags': 0, 'index': 2, 'net_ns_fd': 'ns1'}
2020-01-16 06:24:07,510    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply idx_req: {'target': 'localhost', 'tflags': 0, 'index': 2}
2020-01-16 06:24:07,510    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply state: setns
2020-01-16 06:24:07,510    DEBUG pyroute2.ndb.140602995765824.rtnl_object: run set ({'target': 'localhost', 'tflags': 0, 'index': 2, 'net_ns_fd': 'ns1'})
2020-01-16 06:24:07,521    DEBUG pyroute2.ndb.140602995765824.rtnl_object: stats: apply set {objid 140602968230464, wtime 0.0, mqsize 0, nqsize 0}
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: ('localhost', 0, 0, 1, 2, 4098, 0, '08:00:27:61:87:28', 'ff:ff:ff:ff:ff:ff', 'enp0s3', 1500, None, 'noop', None, 1000, 0, None, None, None, 0, None, 0, 1, 1, 0, 3, None, None, 0, 65535,
65536, 1, 2, 'down', None, None)
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: check: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns')]
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: check state: False
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: check failed
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: run set ({'target': 'localhost', 'tflags': 0, 'index': 2, 'net_ns_fd': 'ns1'})
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: error: (19, 'No such device')
2020-01-16 06:24:07,526    DEBUG pyroute2.ndb.140602995765824.rtnl_object: ignore error 19 for {'flags': 4098, 'state': 'down', 'ifname': 'enp0s3', 'target': 'localhost', 'tflags': 0, 'index': 2, 'family': 0, 'ifi_type': 1, 'change': 0, 'address': '08:00:27:61:87:28', 'b
roadcast': 'ff:ff:ff:ff:ff:ff', 'mtu': 1500, 'link': None, 'qdisc': 'noop', 'master': None, 'txqlen': 1000, 'linkmode': 0, 'net_ns_pid': None, 'ifalias': None, 'num_vf': None, 'group': 0, 'ext_mask': None, 'promiscuity': 0, 'num_tx_queues': 1, 'num_rx_queues': 1, 'carrie
r': 0, 'carrier_changes': 3, 'link_netnsid': None, 'phys_port_name': None, 'proto_down': 0, 'gso_max_segs': 65535, 'gso_max_size': 65536, 'carrier_up_count': 1, 'carrier_down_count': 2, 'kind': None, 'slave_kind': None, 'net_ns_fd': 'ns1'}
2020-01-16 06:24:07,527    DEBUG pyroute2.ndb.140602995765824.rtnl_object: stats: apply set {objid 140602968230464, wtime 0.2, mqsize 2, nqsize 0}
2020-01-16 06:24:07,528    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_rtnl: {'length': 580, 'type': 17, 'flags': 0, 'sequence_number': 0, 'pid': 0, 'error': None, 'stats': Stats(qsize=0, delta=0, delay=0)}
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: None
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: check: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns'), (1579177447.5283737, 'invalid')]
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: check: True
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: checked
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: stats: 140602968230464 pass
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: commit: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns'), (1579177447.5283737, 'invalid')]
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply: [(1579177447.5005624, 'invalid'), (1579177447.5011153, 'system'), (1579177447.5012302, 'setns'), (1579177447.5283737, 'invalid')]
2020-01-16 06:24:07,530    DEBUG pyroute2.ndb.140602995765824.rtnl_object: load_sql: None
2020-01-16 06:24:07,531    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply req: {'target': 'localhost', 'tflags': 0, 'index': 2}
2020-01-16 06:24:07,531    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply idx_req: {'target': 'localhost', 'tflags': 0, 'index': 2}
2020-01-16 06:24:07,531    DEBUG pyroute2.ndb.140602995765824.rtnl_object: apply state: invalid
2020-01-16 06:24:07,531    DEBUG pyroute2.ndb.140602995765824.rtnl_object: run add ({'flags': 4098, 'state': 'down', 'ifname': 'enp0s3', 'target': 'localhost', 'tflags': 0, 'index': 2, 'family': 0, 'ifi_type': 1, 'change': 0, 'address': '08:00:27:61:87:28', 'broadcast':
'ff:ff:ff:ff:ff:ff', 'mtu': 1500, 'qdisc': 'noop', 'txqlen': 1000, 'linkmode': 0, 'group': 0, 'promiscuity': 0, 'num_tx_queues': 1, 'num_rx_queues': 1, 'carrier': 0, 'carrier_changes': 3, 'proto_down': 0, 'gso_max_segs': 65535, 'gso_max_size': 65536, 'carrier_up_count':
1, 'carrier_down_count': 2, 'net_ns_fd': 'ns1'})
2020-01-16 06:24:07,532    DEBUG pyroute2.ndb.140602995765824.rtnl_object: error: (95, 'Operation not supported')
2020-01-16 06:24:07,532    DEBUG pyroute2.ndb.140602995765824.rtnl_object: init
2020-01-16 06:24:07,532    DEBUG pyroute2.ndb.140602995765824.rtnl_object: complete key {'flags': 4098, 'state': 'down', 'ifname': 'enp0s3', 'target': 'localhost', 'tflags': 0, 'index': 2, 'family': 0, 'ifi_type': 1, 'change': 0, 'address': '08:00:27:61:87:28', 'broadcas
t': 'ff:ff:ff:ff:ff:ff', 'mtu': 1500, 'link': None, 'qdisc': 'noop', 'txqlen': 1000, 'linkmode': 0, 'net_ns_pid': None, 'ifalias': None, 'num_vf': None, 'group': 0, 'ext_mask': None, 'promiscuity': 0, 'num_tx_queues': 1, 'num_rx_queues': 1, 'carrier': 0, 'carrier_changes
': 3, 'link_netnsid': None, 'phys_port_name': None, 'proto_down': 0, 'gso_max_segs': 65535, 'gso_max_size': 65536, 'carrier_up_count': 1, 'carrier_down_count': 2, 'kind': None, 'slave_kind': None, 'net_ns_fd': 'ns1'} from table interfaces
2020-01-16 06:24:07,532    DEBUG pyroute2.ndb.140602995765824.rtnl_object: got {'flags': 4098, 'state': 'down', 'ifname': 'enp0s3', 'target': 'localhost', 'tflags': 0, 'index': 2, 'family': 0, 'ifi_type': 1, 'change': 0, 'address': '08:00:27:61:87:28', 'broadcast': 'ff:f
f:ff:ff:ff:ff', 'mtu': 1500, 'link': None, 'qdisc': 'noop', 'txqlen': 1000, 'linkmode': 0, 'net_ns_pid': None, 'ifalias': None, 'num_vf': None, 'group': 0, 'ext_mask': None, 'promiscuity': 0, 'num_tx_queues': 1, 'num_rx_queues': 1, 'carrier': 0, 'carrier_changes': 3, 'li
nk_netnsid': None, 'phys_port_name': None, 'proto_down': 0, 'gso_max_segs': 65535, 'gso_max_size': 65536, 'carrier_up_count': 1, 'carrier_down_count': 2, 'kind': None, 'slave_kind': None, 'net_ns_fd': 'ns1'}
2020-01-16 06:24:17,377    DEBUG pyroute2.ndb.140602995765824.sources.ns1: sync
2020-01-16 06:24:17,377    DEBUG pyroute2.ndb.140602995765824.sources.ns1: flush DB for the target
2020-01-16 06:24:17,380    DEBUG pyroute2.ndb.140602995765824.sources.ns1: stopped
2020-01-16 06:24:17,382     INFO pyroute2.ndb.140602995765824.sources.localhost: source shutdown
2020-01-16 06:24:17,383    DEBUG pyroute2.ndb.140602995765824.sources.localhost: sync
2020-01-16 06:24:17,383    DEBUG pyroute2.ndb.140602995765824.sources.localhost: shutdown handled by the main thread
2020-01-16 06:24:17,383    DEBUG pyroute2.ndb.140602995765824.sources.localhost: stopped
2020-01-16 06:24:17,384    DEBUG pyroute2.ndb.140602995765824.main: flush DB for the target localhost
2020-01-16 06:24:17,386     INFO pyroute2.ndb.140602995765824.sources.localhost/netns: source shutdown
2020-01-16 06:24:17,386    DEBUG pyroute2.ndb.140602995765824.sources.localhost/netns: sync
2020-01-16 06:24:17,387    DEBUG pyroute2.ndb.140602995765824.sources.localhost/netns: shutdown handled by the main thread
2020-01-16 06:24:17,387    DEBUG pyroute2.ndb.140602995765824.sources.localhost/netns: stopped
2020-01-16 06:24:17,387    DEBUG pyroute2.ndb.140602995765824.main: flush DB for the target localhost/netns
2020-01-16 06:24:17,388     INFO pyroute2.ndb.140602995765824.sources.ns1: source shutdown
2020-01-16 06:24:17,388    DEBUG pyroute2.ndb.140602995765824.main: flush DB for the target ns1

[svinota]

Thanks, analyzing the log

[svinota]

@mawiegand may I look at the code you run? I believe you tried to move a real interace between netns, but for me it works. Probably it's not a bug but a poor documentation that should be updated.

[mawiegand]

@mawiegand may I look at the code you run? I believe you tried to move a real interace between netns, but for me it works. Probably it's not a bug but a poor documentation that should be updated.

I used this code snippet:

Any news here?

I run into the same error if I try to execute the following:

#!/usr/bin/python3
 
from pyroute2 import NDB
 
ndb = NDB(debug=True,log="on")
 
ndb.sources.add(netns="ns1")
 
# move interface to namespace "ns1"
dev = ndb.interfaces[{"ifname": "enp0s3"}]
dev = dev.set("target", "ns1").commit()
 
# add ip address
dev = dev.add_ip("192.168.42.21/24")
dev = dev.commit()

And yes it is a real interface. Do I need to move real interfaces differently?

[svinota]

Pls try that.

# dev = dev.set("target", "ns1").commit()
dev = dev.set("net_ns_fd", "ns1").commit()