ipng/vppcfg
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feature: stateless planning

Browse Source 
Add a  feature to plan a configuration without reading from the VPP Dataplane.

In this mode, the configuration file is read and validated in the same way as `check` or `plan`,
but then instead of retrieving the running state from the VPP API, a state is re-created using
the physical interfaces specified in the YAML config.

Implement this by creating vppapi:mockconfig() which reads the 'interfaces' scope from the YAML
config file, and creates a VPPMessage() of type sw_interface_details for each interface that is a
PHY (for now, only supporting device-type 'dpdk').

If the flag --novpp is specified in the planner, call mockconfig() instead of readconfig().

Some further details:
- if the MAC is not set in the YAML config, it won't be set in the output exec file.
- for bondethernets, no MAC can be generated unless it's set in the first member.
- the MTU is always set, because it's mocked to 64b and the YAML file will always be higher.

TESTED:
- the unit tests and YAML tests all pass
- the integration tests all pass, but they do not call this new codepath

- Based on an empty VPP on Hippo, I compared the output of these two, side by side:
for i in intest/*yaml; do ./vppcfg.py plan -c $i -o /tmp/$i-vpp.exec; done
for i in intest/*yaml; do ./vppcfg.py plan --novpp -c $i -o /tmp/$i-novpp.exec; done

==> The only changes here are:
* if I cannot determine the bondether MAC in the --novpp case, it is not emitted
* if the MAC address is set in the YAML file, the --novpp case will always emit it
* if VPP has mtu 9000, the --novpp case will end up still emitting interface and packet MTU,
  because it mocks the interface MTU at 64.

In all cases, --novpp emits more configuration statements, and the statements that it emits are
redundant.
This commit is contained in:
Pim van Pelt
2022-12-03 15:57:00 +00:00
parent 490c294014
commit 305a30b1a1
4 changed files with 149 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
docs/user-guide.md
View File

@ -218,6 +218,9 @@ a set of CLI commands that could be pasted into a `vppctl` shell in the order th
Alternatively, the output file can be consumed by VPP by issuing `vppctl exec <filename>`, noting
that the filename has to be an absolute path.
For an in-depth discussion on path-planning and how `vppcfg` operates, see
[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
Users are not encouraged to program VPP this way (see the **apply** module for that), however
for the sake of completeness:
@ -241,8 +244,32 @@ $ vppcfg plan -c example.yaml
[INFO ] root.main: Planning succeeded
```
For an in-depth discussion on path-planning and how `vppcfg` operates, see
[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
#### Stateless planning
A special feature of `vppcfg` is to plan a configuration without reading from the VPP Dataplane.
In this mode, the configuration file is read and validated in the same way as `check` or `plan`,
but then instead of retrieving the running state from the VPP API, a state is re-created using
the physical interfaces specified in the YAML config. This is useful for operators who wish to
pre-compute a configuration snippet and include it in VPP's `startup.conf`, like so:
```
$ mkdir /etc/vpp/config/
$ cat << EOF > /etc/vpp/config/bootstrap.vpp
exec /etc/vpp/config/head.vpp
exec /etc/vpp/config/vppcfg.vpp
exec /etc/vpp/config/tail.vpp
EOF
$ touch /etc/vpp/config/head.vpp /etc/vpp/config/tail.vpp
$ vppcfg plan --novpp -c /etc/vpp/vppcfg.yaml -o /etc/vpp/config/vppcfg.vpp
```
After adding `unix { startup /etc/vpp/config/bootstrap.vpp }`, the VPP dataplane will execute
all of the commands it finds, so in turn executing `head.vpp`, then the generated `vppcfg.vpp`
and finally `tail.vpp`. This pattern is useful to be able to pre-flight set up the dataplane
with `head.vpp` (think of things like custom logging, plugin defaults, DPDK affinity, and so on),
then letting `vppcfg` do its part, and finally leaving the ability to also program the dataplane
with things that `vppcfg` does not (yet) support in `tail.vpp`.
### vppcfg apply

5
vppcfg/vpp/reconciler.py
View File

@ -1015,7 +1015,10 @@ class Reconciler:
)
member_iface = self.vpp.get_interface_by_name(member_ifname)
if not member_iface or member_ifname not in vpp_members:
if len(vpp_members) == 0:
if (
len(vpp_members) == 0
and member_iface.l2_address != "00:00:00:00:00:00"
):
bondmac = member_iface.l2_address
cli = f"bond add {config_bond_ifname} {member_iface.interface_name}"
self.cli["sync"].append(cli)

101
vppcfg/vpp/vppapi.py
View File

@ -19,11 +19,9 @@ derived classes VPPApiDumper() and VPPApiApplier()
"""
import os
import fnmatch
import logging
import socket
import time
from vpp_papi import VPPApiClient, VPPApiJSONFiles
from vpp_papi import VPPApiClient, VPPApiJSONFiles, MACAddress
class VPPApi:
@ -41,23 +39,34 @@ class VPPApi:
self.vpp_api_socket = vpp_api_socket
self.vpp_json_dir = vpp_json_dir
self.vpp_jsonfiles = []
self.vpp_messages = {}
self.connected = False
self.clientname = clientname
self.vpp = None
self.cache_read = False
self.cache_clear()
self.lcp_enabled = False
if self.vpp_json_dir is None:
self.vpp_json_dir = VPPApiJSONFiles.find_api_dir([])
elif not os.path.isdir(self.vpp_json_dir):
self.logger.error(f"VPP api json directory not found: {self.vpp_json_dir}")
return False
self.logger.error(f"VPP API JSON directory not found: {self.vpp_json_dir}")
# construct a list of all the json api files
# Construct a list of all the JSON API files
self.vpp_jsonfiles = VPPApiJSONFiles.find_api_files(api_dir=self.vpp_json_dir)
if not self.vpp_jsonfiles:
self.logger.error("no json api files found")
return False
self.logger.error("No JSON API files found")
# Enumerate all VPPMessage signatures from the JSON API files, and give their
# API namedtuple defaults so creating instances can set only those fields which
# are relevant.
for json_filename in self.vpp_jsonfiles:
with open(json_filename, "r", encoding="utf-8") as file_handle:
for name, msg in VPPApiJSONFiles.process_json_file(file_handle)[
0
].items():
msg.tuple.__new__.__defaults__ = (None,) * len(msg.tuple._fields)
self.vpp_messages[name] = msg
def connect(self, retries=30):
"""Connect to the VPP Dataplane, if we're not already connected"""
@ -201,6 +210,82 @@ class VPPApi:
self.cache["taps"].pop(iface.sw_if_index, None)
return True
def mockconfig(self, yaml_config):
"""Mock a minimal configuration cache without talking to a running VPP Dataplane, by
enumerating the 'interfaces' scope from yaml_config"""
if not "interfaces" in yaml_config:
self.logger.error(f"YAML config does not contain any interfaces")
return False
self.logger.debug(f"config: {yaml_config['interfaces']}")
self.cache_clear()
## Add mock local0
idx = 0
self.cache["interfaces"][idx] = self.vpp_messages["sw_interface_details"].tuple(
sw_if_index=idx,
sup_sw_if_index=idx,
l2_address=MACAddress("00:00:00:00:00:00"),
flags=0,
type=0,
link_duplex=0,
link_speed=0,
sub_id=0,
sub_number_of_tags=0,
sub_outer_vlan_id=0,
sub_inner_vlan_id=0,
sub_if_flags=0,
vtr_op=0,
vtr_push_dot1q=0,
vtr_tag1=0,
vtr_tag2=0,
outer_tag=0,
link_mtu=0,
mtu=[0, 0, 0, 0],
interface_name="local0",
interface_dev_type="local",
tag="mock",
)
## Add mock PHYs
for ifname, iface in yaml_config["interfaces"].items():
if not "device-type" in iface or iface["device-type"] not in ["dpdk"]:
continue
idx += 1
self.cache["interfaces"][idx] = self.vpp_messages[
"sw_interface_details"
].tuple(
sw_if_index=idx,
sup_sw_if_index=idx,
l2_address=MACAddress("00:00:00:00:00:00"),
flags=0,
type=0,
link_duplex=0,
link_speed=0,
sub_id=0,
sub_number_of_tags=0,
sub_outer_vlan_id=0,
sub_inner_vlan_id=0,
sub_if_flags=0,
vtr_op=0,
vtr_push_dot1q=0,
vtr_tag1=0,
vtr_tag2=0,
outer_tag=0,
link_mtu=64,
mtu=[64, 0, 0, 0],
interface_name=ifname,
interface_dev_type=iface["device-type"],
tag="mock",
)
## Create interface_names and interface_address indexes
for idx, iface in self.cache["interfaces"].items():
self.cache["interface_names"][iface.interface_name] = idx
self.cache["interface_addresses"][idx] = []
self.logger.debug(f"cache(mock): {self.cache}")
return True
def readconfig(self):
"""Read the configuration out of a running VPP Dataplane and put it into a
VPP config cache"""

36
vppcfg/vppcfg.py
View File

@ -130,6 +130,12 @@ def main():
type=str,
help="""YAML configuration file for vppcfg""",
)
plan_p.add_argument(
"--novpp",
dest="novpp",
action="store_true",
help="""Don't query VPP API, assume 'empty' dataplane config""",
)
plan_p.add_argument(
"-o",
"--output",
@ -238,22 +244,26 @@ def main():
sys.exit(0)
reconciler = Reconciler(cfg, **opt_kwargs)
if not reconciler.vpp.readconfig():
sys.exit(-3)
if args.novpp:
if not reconciler.vpp.mockconfig(cfg):
sys.exit(-7)
else:
if not reconciler.vpp.readconfig():
sys.exit(-3)
if not reconciler.phys_exist_in_vpp():
logging.error("Not all PHYs in the config exist in VPP")
sys.exit(-4)
if not reconciler.phys_exist_in_vpp():
logging.error("Not all PHYs in the config exist in VPP")
sys.exit(-4)
if not reconciler.phys_exist_in_config():
logging.error("Not all PHYs in VPP exist in the config")
sys.exit(-5)
if not reconciler.phys_exist_in_config():
logging.error("Not all PHYs in VPP exist in the config")
sys.exit(-5)
if not reconciler.lcps_exist_with_lcp_enabled():
logging.error(
"Linux Control Plane is needed, but linux-cp API is not available"
)
sys.exit(-6)
if not reconciler.lcps_exist_with_lcp_enabled():
logging.error(
"Linux Control Plane is needed, but linux-cp API is not available"
)
sys.exit(-6)
failed = False
if not reconciler.prune():