feature: stateless planning

Add a feature to plan a configuration without reading from the VPP Dataplane. In this mode, the configuration file is read and validated in the same way as `check` or `plan`, but then instead of retrieving the running state from the VPP API, a state is re-created using the physical interfaces specified in the YAML config. Implement this by creating vppapi:mockconfig() which reads the 'interfaces' scope from the YAML config file, and creates a VPPMessage() of type sw_interface_details for each interface that is a PHY (for now, only supporting device-type 'dpdk'). If the flag --novpp is specified in the planner, call mockconfig() instead of readconfig(). Some further details: - if the MAC is not set in the YAML config, it won't be set in the output exec file. - for bondethernets, no MAC can be generated unless it's set in the first member. - the MTU is always set, because it's mocked to 64b and the YAML file will always be higher. TESTED: - the unit tests and YAML tests all pass - the integration tests all pass, but they do not call this new codepath - Based on an empty VPP on Hippo, I compared the output of these two, side by side: for i in intest/*yaml; do ./vppcfg.py plan -c $i -o /tmp/$i-vpp.exec; done for i in intest/*yaml; do ./vppcfg.py plan --novpp -c $i -o /tmp/$i-novpp.exec; done ==> The only changes here are: * if I cannot determine the bondether MAC in the --novpp case, it is not emitted * if the MAC address is set in the YAML file, the --novpp case will always emit it * if VPP has mtu 9000, the --novpp case will end up still emitting interface and packet MTU, because it mocks the interface MTU at 64. In all cases, --novpp emits more configuration statements, and the statements that it emits are redundant.
2022-12-03 15:57:00 +00:00
parent 490c294014
commit 305a30b1a1
4 changed files with 149 additions and 24 deletions
--- a/docs/user-guide.md
+++ b/docs/user-guide.md
@ -218,6 +218,9 @@ a set of CLI commands that could be pasted into a `vppctl` shell in the order th
 Alternatively, the output file can be consumed by VPP by issuing `vppctl exec <filename>`, noting
 that the filename has to be an absolute path.

+For an in-depth discussion on path-planning and how `vppcfg` operates, see
+[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
+
 Users are not encouraged to program VPP this way (see the **apply** module for that), however
 for the sake of completeness:

@ -241,8 +244,32 @@ $ vppcfg plan -c example.yaml
 [INFO    ] root.main: Planning succeeded
 ```

-For an in-depth discussion on path-planning and how `vppcfg` operates, see
-[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
+#### Stateless planning
+
+A special feature of `vppcfg` is to plan a configuration without reading from the VPP Dataplane.
+In this mode, the configuration file is read and validated in the same way as `check` or `plan`,
+but then instead of retrieving the running state from the VPP API, a state is re-created using
+the physical interfaces specified in the YAML config. This is useful for operators who wish to
+pre-compute a configuration snippet and include it in VPP's `startup.conf`, like so:
+
+```
+$ mkdir /etc/vpp/config/
+$ cat << EOF > /etc/vpp/config/bootstrap.vpp
+exec /etc/vpp/config/head.vpp
+exec /etc/vpp/config/vppcfg.vpp
+exec /etc/vpp/config/tail.vpp
+EOF
+
+$ touch /etc/vpp/config/head.vpp /etc/vpp/config/tail.vpp
+$ vppcfg plan --novpp -c /etc/vpp/vppcfg.yaml -o /etc/vpp/config/vppcfg.vpp
+```
+
+After adding `unix { startup /etc/vpp/config/bootstrap.vpp }`, the VPP dataplane will execute
+all of the commands it finds, so in turn executing `head.vpp`, then the generated `vppcfg.vpp`
+and finally `tail.vpp`. This pattern is useful to be able to pre-flight set up the dataplane
+with `head.vpp` (think of things like custom logging, plugin defaults, DPDK affinity, and so on),
+then letting `vppcfg` do its part, and finally leaving the ability to also program the dataplane
+with things that `vppcfg` does not (yet) support in `tail.vpp`.

 ### vppcfg apply

--- a/vppcfg/vpp/reconciler.py
+++ b/vppcfg/vpp/reconciler.py
@ -1015,7 +1015,10 @@ class Reconciler:
                )
                member_iface = self.vpp.get_interface_by_name(member_ifname)
                if not member_iface or member_ifname not in vpp_members:
-                    if len(vpp_members) == 0:
+                    if (
+                        len(vpp_members) == 0
+                        and member_iface.l2_address != "00:00:00:00:00:00"
+                    ):
                        bondmac = member_iface.l2_address
                    cli = f"bond add {config_bond_ifname} {member_iface.interface_name}"
                    self.cli["sync"].append(cli)
--- a/vppcfg/vpp/vppapi.py
+++ b/vppcfg/vpp/vppapi.py
@ -19,11 +19,9 @@ derived classes VPPApiDumper() and VPPApiApplier()
 """

 import os
-import fnmatch
 import logging
-import socket
 import time
-from vpp_papi import VPPApiClient, VPPApiJSONFiles
+from vpp_papi import VPPApiClient, VPPApiJSONFiles, MACAddress


 class VPPApi:
@ -41,23 +39,34 @@ class VPPApi:
        self.vpp_api_socket = vpp_api_socket
        self.vpp_json_dir = vpp_json_dir
        self.vpp_jsonfiles = []
+        self.vpp_messages = {}
        self.connected = False
        self.clientname = clientname
        self.vpp = None
+        self.cache_read = False
        self.cache_clear()
        self.lcp_enabled = False

        if self.vpp_json_dir is None:
            self.vpp_json_dir = VPPApiJSONFiles.find_api_dir([])
        elif not os.path.isdir(self.vpp_json_dir):
-            self.logger.error(f"VPP api json directory not found: {self.vpp_json_dir}")
-            return False
+            self.logger.error(f"VPP API JSON directory not found: {self.vpp_json_dir}")

-        # construct a list of all the json api files
+        # Construct a list of all the JSON API files
        self.vpp_jsonfiles = VPPApiJSONFiles.find_api_files(api_dir=self.vpp_json_dir)
        if not self.vpp_jsonfiles:
-            self.logger.error("no json api files found")
-            return False
+            self.logger.error("No JSON API files found")
+
+        # Enumerate all VPPMessage signatures from the JSON API files, and give their
+        # API namedtuple defaults so creating instances can set only those fields which
+        # are relevant.
+        for json_filename in self.vpp_jsonfiles:
+            with open(json_filename, "r", encoding="utf-8") as file_handle:
+                for name, msg in VPPApiJSONFiles.process_json_file(file_handle)[
+                    0
+                ].items():
+                    msg.tuple.__new__.__defaults__ = (None,) * len(msg.tuple._fields)
+                    self.vpp_messages[name] = msg

    def connect(self, retries=30):
        """Connect to the VPP Dataplane, if we're not already connected"""
@ -201,6 +210,82 @@ class VPPApi:
        self.cache["taps"].pop(iface.sw_if_index, None)
        return True

+    def mockconfig(self, yaml_config):
+        """Mock a minimal configuration cache without talking to a running VPP Dataplane, by
+        enumerating the 'interfaces' scope from yaml_config"""
+
+        if not "interfaces" in yaml_config:
+            self.logger.error(f"YAML config does not contain any interfaces")
+            return False
+        self.logger.debug(f"config: {yaml_config['interfaces']}")
+
+        self.cache_clear()
+        ## Add mock local0
+        idx = 0
+        self.cache["interfaces"][idx] = self.vpp_messages["sw_interface_details"].tuple(
+            sw_if_index=idx,
+            sup_sw_if_index=idx,
+            l2_address=MACAddress("00:00:00:00:00:00"),
+            flags=0,
+            type=0,
+            link_duplex=0,
+            link_speed=0,
+            sub_id=0,
+            sub_number_of_tags=0,
+            sub_outer_vlan_id=0,
+            sub_inner_vlan_id=0,
+            sub_if_flags=0,
+            vtr_op=0,
+            vtr_push_dot1q=0,
+            vtr_tag1=0,
+            vtr_tag2=0,
+            outer_tag=0,
+            link_mtu=0,
+            mtu=[0, 0, 0, 0],
+            interface_name="local0",
+            interface_dev_type="local",
+            tag="mock",
+        )
+        ## Add mock PHYs
+        for ifname, iface in yaml_config["interfaces"].items():
+            if not "device-type" in iface or iface["device-type"] not in ["dpdk"]:
+                continue
+            idx += 1
+            self.cache["interfaces"][idx] = self.vpp_messages[
+                "sw_interface_details"
+            ].tuple(
+                sw_if_index=idx,
+                sup_sw_if_index=idx,
+                l2_address=MACAddress("00:00:00:00:00:00"),
+                flags=0,
+                type=0,
+                link_duplex=0,
+                link_speed=0,
+                sub_id=0,
+                sub_number_of_tags=0,
+                sub_outer_vlan_id=0,
+                sub_inner_vlan_id=0,
+                sub_if_flags=0,
+                vtr_op=0,
+                vtr_push_dot1q=0,
+                vtr_tag1=0,
+                vtr_tag2=0,
+                outer_tag=0,
+                link_mtu=64,
+                mtu=[64, 0, 0, 0],
+                interface_name=ifname,
+                interface_dev_type=iface["device-type"],
+                tag="mock",
+            )
+
+        ## Create interface_names and interface_address indexes
+        for idx, iface in self.cache["interfaces"].items():
+            self.cache["interface_names"][iface.interface_name] = idx
+            self.cache["interface_addresses"][idx] = []
+
+        self.logger.debug(f"cache(mock): {self.cache}")
+        return True
+
    def readconfig(self):
        """Read the configuration out of a running VPP Dataplane and put it into a
        VPP config cache"""
--- a/vppcfg/vppcfg.py
+++ b/vppcfg/vppcfg.py
@ -130,6 +130,12 @@ def main():
        type=str,
        help="""YAML configuration file for vppcfg""",
    )
+    plan_p.add_argument(
+        "--novpp",
+        dest="novpp",
+        action="store_true",
+        help="""Don't query VPP API, assume 'empty' dataplane config""",
+    )
    plan_p.add_argument(
        "-o",
        "--output",
@ -238,6 +244,10 @@ def main():
        sys.exit(0)

    reconciler = Reconciler(cfg, **opt_kwargs)
+    if args.novpp:
+        if not reconciler.vpp.mockconfig(cfg):
+            sys.exit(-7)
+    else:
        if not reconciler.vpp.readconfig():
            sys.exit(-3)