feature: stateless planning

Add a  feature to plan a configuration without reading from the VPP Dataplane.

In this mode, the configuration file is read and validated in the same way as `check` or `plan`,
but then instead of retrieving the running state from the VPP API, a state is re-created using
the physical interfaces specified in the YAML config.

Implement this by creating vppapi:mockconfig() which reads the 'interfaces' scope from the YAML
config file, and creates a VPPMessage() of type sw_interface_details for each interface that is a
PHY (for now, only supporting device-type 'dpdk').

If the flag --novpp is specified in the planner, call mockconfig() instead of readconfig().

Some further details:
- if the MAC is not set in the YAML config, it won't be set in the output exec file.
- for bondethernets, no MAC can be generated unless it's set in the first member.
- the MTU is always set, because it's mocked to 64b and the YAML file will always be higher.

TESTED:
- the unit tests and YAML tests all pass
- the integration tests all pass, but they do not call this new codepath

- Based on an empty VPP on Hippo, I compared the output of these two, side by side:
for i in intest/*yaml; do ./vppcfg.py plan -c $i -o /tmp/$i-vpp.exec; done
for i in intest/*yaml; do ./vppcfg.py plan --novpp -c $i -o /tmp/$i-novpp.exec; done

==> The only changes here are:
* if I cannot determine the bondether MAC in the --novpp case, it is not emitted
* if the MAC address is set in the YAML file, the --novpp case will always emit it
* if VPP has mtu 9000, the --novpp case will end up still emitting interface and packet MTU,
  because it mocks the interface MTU at 64.

In all cases, --novpp emits more configuration statements, and the statements that it emits are
redundant.

docs/user-guide.md

@@ -218,6 +218,9 @@ a set of CLI commands that could be pasted into a `vppctl` shell in the order th
 Alternatively, the output file can be consumed by VPP by issuing `vppctl exec <filename>`, noting
 that the filename has to be an absolute path.
 
+For an in-depth discussion on path-planning and how `vppcfg` operates, see
+[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
+
 Users are not encouraged to program VPP this way (see the **apply** module for that), however
 for the sake of completeness:
 
@@ -241,8 +244,32 @@ $ vppcfg plan -c example.yaml
 [INFO    ] root.main: Planning succeeded
 ```
 
-For an in-depth discussion on path-planning and how `vppcfg` operates, see
+#### Stateless planning
-[this post](https://ipng.ch/s/articles/2022/04/02/vppcfg-2.html).
+
+A special feature of `vppcfg` is to plan a configuration without reading from the VPP Dataplane.
+In this mode, the configuration file is read and validated in the same way as `check` or `plan`,
+but then instead of retrieving the running state from the VPP API, a state is re-created using
+the physical interfaces specified in the YAML config. This is useful for operators who wish to
+pre-compute a configuration snippet and include it in VPP's `startup.conf`, like so:
+
+```
+$ mkdir /etc/vpp/config/
+$ cat << EOF > /etc/vpp/config/bootstrap.vpp
+exec /etc/vpp/config/head.vpp
+exec /etc/vpp/config/vppcfg.vpp
+exec /etc/vpp/config/tail.vpp
+EOF
+
+$ touch /etc/vpp/config/head.vpp /etc/vpp/config/tail.vpp
+$ vppcfg plan --novpp -c /etc/vpp/vppcfg.yaml -o /etc/vpp/config/vppcfg.vpp
+```
+
+After adding `unix { startup /etc/vpp/config/bootstrap.vpp }`, the VPP dataplane will execute
+all of the commands it finds, so in turn executing `head.vpp`, then the generated `vppcfg.vpp`
+and finally `tail.vpp`. This pattern is useful to be able to pre-flight set up the dataplane
+with `head.vpp` (think of things like custom logging, plugin defaults, DPDK affinity, and so on),
+then letting `vppcfg` do its part, and finally leaving the ability to also program the dataplane
+with things that `vppcfg` does not (yet) support in `tail.vpp`.
 
 ### vppcfg apply
 

vppcfg/vpp/reconciler.py

@@ -1015,7 +1015,10 @@ class Reconciler:
                 )
                 member_iface = self.vpp.get_interface_by_name(member_ifname)
                 if not member_iface or member_ifname not in vpp_members:
-                    if len(vpp_members) == 0:
+                    if (
+                        len(vpp_members) == 0
+                        and member_iface.l2_address != "00:00:00:00:00:00"
+                    ):
                         bondmac = member_iface.l2_address
                     cli = f"bond add {config_bond_ifname} {member_iface.interface_name}"
                     self.cli["sync"].append(cli)

vppcfg/vpp/vppapi.py

@@ -19,11 +19,9 @@ derived classes VPPApiDumper() and VPPApiApplier()
 """
 
 import os
-import fnmatch
 import logging
-import socket
 import time
-from vpp_papi import VPPApiClient, VPPApiJSONFiles
+from vpp_papi import VPPApiClient, VPPApiJSONFiles, MACAddress
 
 
 class VPPApi:
@@ -41,23 +39,34 @@ class VPPApi:
         self.vpp_api_socket = vpp_api_socket
         self.vpp_json_dir = vpp_json_dir
         self.vpp_jsonfiles = []
+        self.vpp_messages = {}
         self.connected = False
         self.clientname = clientname
         self.vpp = None
+        self.cache_read = False
         self.cache_clear()
         self.lcp_enabled = False
 
         if self.vpp_json_dir is None:
             self.vpp_json_dir = VPPApiJSONFiles.find_api_dir([])
         elif not os.path.isdir(self.vpp_json_dir):
-            self.logger.error(f"VPP api json directory not found: {self.vpp_json_dir}")
+            self.logger.error(f"VPP API JSON directory not found: {self.vpp_json_dir}")
-            return False
 
-        # construct a list of all the json api files
+        # Construct a list of all the JSON API files
         self.vpp_jsonfiles = VPPApiJSONFiles.find_api_files(api_dir=self.vpp_json_dir)
         if not self.vpp_jsonfiles:
-            self.logger.error("no json api files found")
+            self.logger.error("No JSON API files found")
-            return False
+
+        # Enumerate all VPPMessage signatures from the JSON API files, and give their
+        # API namedtuple defaults so creating instances can set only those fields which
+        # are relevant.
+        for json_filename in self.vpp_jsonfiles:
+            with open(json_filename, "r", encoding="utf-8") as file_handle:
+                for name, msg in VPPApiJSONFiles.process_json_file(file_handle)[
+                    0
+                ].items():
+                    msg.tuple.__new__.__defaults__ = (None,) * len(msg.tuple._fields)
+                    self.vpp_messages[name] = msg
 
     def connect(self, retries=30):
         """Connect to the VPP Dataplane, if we're not already connected"""
@@ -201,6 +210,82 @@ class VPPApi:
         self.cache["taps"].pop(iface.sw_if_index, None)
         return True
 
+    def mockconfig(self, yaml_config):
+        """Mock a minimal configuration cache without talking to a running VPP Dataplane, by
+        enumerating the 'interfaces' scope from yaml_config"""
+
+        if not "interfaces" in yaml_config:
+            self.logger.error(f"YAML config does not contain any interfaces")
+            return False
+        self.logger.debug(f"config: {yaml_config['interfaces']}")
+
+        self.cache_clear()
+        ## Add mock local0
+        idx = 0
+        self.cache["interfaces"][idx] = self.vpp_messages["sw_interface_details"].tuple(
+            sw_if_index=idx,
+            sup_sw_if_index=idx,
+            l2_address=MACAddress("00:00:00:00:00:00"),
+            flags=0,
+            type=0,
+            link_duplex=0,
+            link_speed=0,
+            sub_id=0,
+            sub_number_of_tags=0,
+            sub_outer_vlan_id=0,
+            sub_inner_vlan_id=0,
+            sub_if_flags=0,
+            vtr_op=0,
+            vtr_push_dot1q=0,
+            vtr_tag1=0,
+            vtr_tag2=0,
+            outer_tag=0,
+            link_mtu=0,
+            mtu=[0, 0, 0, 0],
+            interface_name="local0",
+            interface_dev_type="local",
+            tag="mock",
+        )
+        ## Add mock PHYs
+        for ifname, iface in yaml_config["interfaces"].items():
+            if not "device-type" in iface or iface["device-type"] not in ["dpdk"]:
+                continue
+            idx += 1
+            self.cache["interfaces"][idx] = self.vpp_messages[
+                "sw_interface_details"
+            ].tuple(
+                sw_if_index=idx,
+                sup_sw_if_index=idx,
+                l2_address=MACAddress("00:00:00:00:00:00"),
+                flags=0,
+                type=0,
+                link_duplex=0,
+                link_speed=0,
+                sub_id=0,
+                sub_number_of_tags=0,
+                sub_outer_vlan_id=0,
+                sub_inner_vlan_id=0,
+                sub_if_flags=0,
+                vtr_op=0,
+                vtr_push_dot1q=0,
+                vtr_tag1=0,
+                vtr_tag2=0,
+                outer_tag=0,
+                link_mtu=64,
+                mtu=[64, 0, 0, 0],
+                interface_name=ifname,
+                interface_dev_type=iface["device-type"],
+                tag="mock",
+            )
+
+        ## Create interface_names and interface_address indexes
+        for idx, iface in self.cache["interfaces"].items():
+            self.cache["interface_names"][iface.interface_name] = idx
+            self.cache["interface_addresses"][idx] = []
+
+        self.logger.debug(f"cache(mock): {self.cache}")
+        return True
+
     def readconfig(self):
         """Read the configuration out of a running VPP Dataplane and put it into a
         VPP config cache"""

vppcfg/vppcfg.py

@@ -130,6 +130,12 @@ def main():
         type=str,
         help="""YAML configuration file for vppcfg""",
     )
+    plan_p.add_argument(
+        "--novpp",
+        dest="novpp",
+        action="store_true",
+        help="""Don't query VPP API, assume 'empty' dataplane config""",
+    )
     plan_p.add_argument(
         "-o",
         "--output",
@@ -238,22 +244,26 @@ def main():
         sys.exit(0)
 
     reconciler = Reconciler(cfg, **opt_kwargs)
-    if not reconciler.vpp.readconfig():
+    if args.novpp:
-        sys.exit(-3)
+        if not reconciler.vpp.mockconfig(cfg):
+            sys.exit(-7)
+    else:
+        if not reconciler.vpp.readconfig():
+            sys.exit(-3)
 
-    if not reconciler.phys_exist_in_vpp():
+        if not reconciler.phys_exist_in_vpp():
-        logging.error("Not all PHYs in the config exist in VPP")
+            logging.error("Not all PHYs in the config exist in VPP")
-        sys.exit(-4)
+            sys.exit(-4)
 
-    if not reconciler.phys_exist_in_config():
+        if not reconciler.phys_exist_in_config():
-        logging.error("Not all PHYs in VPP exist in the config")
+            logging.error("Not all PHYs in VPP exist in the config")
-        sys.exit(-5)
+            sys.exit(-5)
 
-    if not reconciler.lcps_exist_with_lcp_enabled():
+        if not reconciler.lcps_exist_with_lcp_enabled():
-        logging.error(
+            logging.error(
-            "Linux Control Plane is needed, but linux-cp API is not available"
+                "Linux Control Plane is needed, but linux-cp API is not available"
-        )
+            )
-        sys.exit(-6)
+            sys.exit(-6)
 
     failed = False
     if not reconciler.prune():