In this situation, Linux and VPP really diverge. In VPP, any
sub-interface can carry arbitrary configuration, they can be dot1q,
dot1ad, with or without an inner dot1q. So the following is valid in
VPP:
vppctl create sub TenGigabitEthernet3/0/0 10 dot1ad 100 inner-dot1q 200 exact-match
In Linux however, double tagged interfaces have to be created as a chain
of two interfaces, first with the outer and then with the inner tag. So
there is no equivalent of the above command in Linux, where we must:
ip link add link e0 name e0.100 type vlan id 100 proto 802.1ad
ip link add link e0.100 name e0.100.200 type vlan id 200 proto 802.1q
So in order to create Q-in-Q sub-interfaces, for Linux their intermediary
parent must exist, while in VPP this is not true.
I have to make a compromise, so I'll be a bit more explicit and allow
this type of LCP to be created under these conditions:
* A sub-int exists with the intermediary (in this case, `dot1ad 100
exact-match`)
* That sub-int itself has an LCP, with a Linux interface device that
we can spawn the inner-dot1q 200 interface off of
Creation of qinq and qinad interfaces becomes thus:
vppctl create sub TenGigabitEthernet3/0/0 10 dot1ad 100 exact-match
vppctl create sub TenGigabitEthernet3/0/0 11 dot1ad 100 inner-dot1q 200 exact-match
vppctl lcpng create TenGigabitEthernet3/0/0 host-if e0
vppctl lcpng create TenGigabitEthernet3/0/0.10 host-if e0.10
vppctl lcpng create TenGigabitEthernet3/0/0.11 host-if e0.11
And the resulting situation in Linux:
pim@hippo:~/src/lcpng$ ip link | grep e0
397: e0: <BROADCAST,MULTICAST> mtu 9000 qdisc mq state DOWN mode DEFAULT group default qlen 1000
398: e0.10@e0: <BROADCAST,MULTICAST,M-DOWN> mtu 9000 qdisc noop state DOWN mode DEFAULT group default qlen 1000
399: e0.11@e0.10: <BROADCAST,MULTICAST,M-DOWN> mtu 9000 qdisc noop state DOWN mode DEFAULT group default qlen 1000
If there are no LCPs defined yet, and I try to create an LCP for a sub-int,
there is a NULL deref in lcp_itf_pair_get() because the pool hasn't been
initiated yet. Fix this by returning NULL if the pool isn't initialized,
and catching the invalid `lip` returning an error.
While I was here, I took a look at a few possible crashes:
- if the pair_create() is called with an invalid phy_sw_if_index, catch
this and return an error.
- if a pair_add_sub() is called but the parent cannot be found, catch
this and return an error.
DBGvpp# set interface state TenGigabitEthernet3/0/0 down
DBGvpp# lcpng create TenGigabitEthernet3/0/0 host-if e0
DBGvpp# set interface state TenGigabitEthernet3/0/1 up
DBGvpp# lcpng create TenGigabitEthernet3/0/1 host-if e1
Yields:
pim@hippo:~/src/lcpng$ ip link show e0
304: e0: <BROADCAST,MULTICAST> mtu 9000 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 68:05:ca:32:46:14 brd ff:ff:ff:ff:ff:ff
pim@hippo:~/src/lcpng$ ip link show e1
305: e1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 68:05:ca:32:46:15 brd ff:ff:ff:ff:ff:ff
This is to debug why creating QinQ subinterfaces is failing. Consider
the following:
vppctl create sub TenGigabitEthernet3/0/0 1234
vppctl create sub TenGigabitEthernet3/0/0 1235 dot1q 1234 inner-dot1q 1000
vppctl lcpng create TenGigabitEthernet3/0/0 host-if e0
vppctl lcpng create TenGigabitEthernet3/0/0.1234 host-if e0.1234
vppctl lcpng create TenGigabitEthernet3/0/0.1235 host-if e0.1235
Creating the sub-interfaces works, the first is a normal .1q with
tag 1234. Creating its LCP works well, the parent is looked up,
a netlink VLAN link is created as a child, and it gets tag 1234.
Now the second one: it's also operating on parent Te3/0/0 which is
looked up, but now a netlink VLAN link is created as a child, again
with dot1q 1234: this interface already exists, so that's a no-go
and an error is thrown.
-- Thoughts on a fix for this:
I think the fix is probably retrieving the correct lip, ie not
the lip of the phy parent interface (e0) but the lip of the pair that
has the outer vlan 1234 already (e0.1234), and then asking netlink to
create a child interface with vlan 1235 on that e0.1234, rather than
the phy e0.
Although creating a dot1ad.dot1q or a dot1q.dot1q interface in VPP
is strictly valid, we will not be able to succeed without the
intermediate interface in the Linux model, so we return an error
in that case.
Remove the functionality that allows for a configuration of pairs in the
startup.conf -- I am intending on creating interfaces for each/any phy
and sub and tunnel interfaceb that is created.
Instead of injecting events and having a process listening for them,
simply use the callback to create LCP interfaces. This change removes
the old code and sets a helpful logging entry on line 870, in which
future automatic creations and removals of LCP taps will occur.
This way, three desirable usability properties are obtained:
1) on startup, all physical interfaces will be copied into LCP
2) on sub-interface or tunnel creation (or phy insertion), new
interfaces will be created.
3) Deletions and removals will allow for auto-cleanup of the LCP.
Still use a sensible default of 9216, but if the L3 packet size is set
on the VPP interface, copy it forward (just as we do in the 'host'
interface of the TAP itself, ie the interface created in the linux
namespace). Now they will all line up initially.
The TAP interface does copy forward the host MTU based on the VPP
interface's L3 MTU, but it should also ensure that the VPP tap
interface has an MTU that is greater-or-equal to those. Considering
users can set the interfaces at runtime (set interface mtu packet ...)
ensure that the tap MTU is large enough.
Ideally, a callback updates the MTU to the same value as the L3 MTU of
the VPP interface, or the L3 MTU of the host interface, if either of
those change. For now, it's a safe bet to take jumbo 9216.
This gives a lot of operational problems later. It's definitely reasonable
to be able to create tap interfaces in other namespaces, and this is
still possible (see below for syntax).
However, changing the runtime netns makes the netlink listener much more
complicated because it will have to listen on not just one netns, but all
of them, for netlink updates.
So, for now, let's remove the ability to set the namespace in the API.
Still possible:
- set at startup.conf in lcpng { netns <x> }
- force creating in 'lcpng create ... netns <x>'
This will nudge folks to create one singular namespace (say,
'dataplane', in the startup.conf), and then handle all netlink messages
in that namespace only.
