diff --git a/docs/caveats.md b/docs/caveats.md index f1359c2724..f3b1d9afae 100644 --- a/docs/caveats.md +++ b/docs/caveats.md @@ -334,6 +334,12 @@ Netlab enables VRRPv3 by default on Dell OS10, overriding any platform defaults. * You have to build the *dnsmasq* container image with the **netlab clab build dnsmasq** command. +(caveats-vpp)= +## VPP (Vector Packet Processor) + +* You must [build the VPP container image](build-vpp) with the **netlab clab build vpp** command. +* VPP can use FRR or BIRD control plane. [More details](vpp-control-plane). + (caveats-exos)= ## Extreme Networks EXOS diff --git a/docs/labs/clab.md b/docs/labs/clab.md index b9f5ae56b6..8cf32d36a8 100644 --- a/docs/labs/clab.md +++ b/docs/labs/clab.md @@ -60,10 +60,11 @@ Lab topology file created by **[netlab up](../netlab/up.md)** or **[netlab creat | Mikrotik RouterOS 7 | vrnetlab/vr-routeros:7.6 | | Nokia SR Linux | ghcr.io/nokia/srlinux:24.10.1 | | Nokia SR OS | vrnetlab/vr-sros:latest | +| VPP [❗](build-vpp) | netlab/vpp:latest | | VyOS | ghcr.io/sysoleg/vyos-container | -* Cumulus VX, FRR, Linux, Nokia SR Linux, and VyOS images are automatically downloaded from public container registries. -* Build the BIRD, dnsmasq, and Netscaler images with the **netlab clab build** command. BIRD supports several build targets and configurable source releases — see [](build-bird) for details. +* FRR, Linux, Nokia SR Linux, and VyOS images are automatically downloaded from public container registries. +* Build the [BIRD](build-bird), dnsmasq, [VPP](build-vpp), and [Netscaler](build-netscaler) images with the **netlab clab build** command. BIRD and VPP FD.io build process supports configurable software releases. * The Arista cEOS image has to be [downloaded and installed manually](ceos.md). * Nokia SR OS and SR-SIM container images require a license; see also [vrnetlab instructions](https://containerlab.srlinux.dev/manual/vrnetlab/). * Follow Cisco's documentation to install the IOS XRd container, making sure the container image name matches the one _netlab_ uses (alternatively, [change the default image name](default-device-image) for the IOS XRd container). @@ -434,7 +435,7 @@ server. ```{eval-rst} .. toctree:: - :caption: Installing Container Images + :caption: Building and Installing Container Images :maxdepth: 1 :hidden: @@ -442,5 +443,6 @@ server. bird.md linux.md netscaler.md + vpp.md .. ``` diff --git a/docs/labs/vpp.md b/docs/labs/vpp.md new file mode 100644 index 0000000000..042136457f --- /dev/null +++ b/docs/labs/vpp.md @@ -0,0 +1,38 @@ +(build-vpp)= +# Using a VPP Container + +VPP is a [containerlab](lab-clab) device using a locally built **netlab/vpp:latest** image. + +* Build the container with `netlab clab build vpp` +* Pin the FD.io release with **defaults.devices.vpp.clab.sw_version** (optional; latest release is used by default), then run `netlab clab build vpp` +* Use `device: vpp` in the lab topology + +VPP nodes default to the **router** role and also support the **bridge** role. They do not support the **host** role. + +The image installs VPP packages from the FD.io Packagecloud repository selected with **defaults.devices.vpp.clab.repo** (default: `release`). + +(vpp-control-plane)= +## Control Plane + +VPP runs a Linux control-plane daemon in the dataplane network namespace. The default is **bird**; set **control_plane** to **frr** when you need FRRouting (for example IS-IS). + +Lab-wide default: + +``` +defaults.devices.vpp.control_plane: frr +``` + +Per-node override: + +``` +nodes: + r1: + device: vpp + control_plane: frr +``` + +Version pin example: + +``` +defaults.devices.vpp.clab.sw_version: 25.06-release +``` diff --git a/docs/netlab/clab.md b/docs/netlab/clab.md index 7328ac0140..064fa412bb 100644 --- a/docs/netlab/clab.md +++ b/docs/netlab/clab.md @@ -95,6 +95,8 @@ $ netlab clab build bird --version 2.17.4 --tag netlab/bird:latest To use a custom container image, specify it with the **image** node parameter or the **defaults.daemons._daemon_.clab.image** [default setting](topo-defaults). +See [](build-bird) for the **--version** parameter and [](build-vpp) for pinning the FD.io release with **defaults.devices.vpp.clab.sw_version**. + (netlab-clab-cleanup)= ## Docker Cleanup diff --git a/docs/netlab/defaults.md b/docs/netlab/defaults.md index aaf8867874..d09b8cfa1a 100644 --- a/docs/netlab/defaults.md +++ b/docs/netlab/defaults.md @@ -98,6 +98,7 @@ You can also use glob expressions to select the parameters you want to display. $ netlab defaults '*clab.image' daemons.bird.clab.image = netlab/bird:latest daemons.dnsmasq.clab.image = netlab/dnsmasq:latest +devices.vpp.clab.image = netlab/vpp:latest devices.arubacx.clab.image = vrnetlab/vr-aoscx:20240129204649 devices.cat8000v.clab.image = vrnetlab/vr-c8000v:17.13.01a devices.csr.clab.image = vrnetlab/cisco_csr1000v:17.03.08 diff --git a/docs/platforms.md b/docs/platforms.md index 53bf721a2a..17487b312c 100644 --- a/docs/platforms.md +++ b/docs/platforms.md @@ -48,6 +48,7 @@ | Nokia SR-SIM [❗](caveats-srsim) | srsim | full | | OpenBSD [❗](caveats-openbsd) | openbsd | best effort | | SONiC [❗](caveats-sonic-vm) | sonic | minimal | +| VPP (fd.io) [❗](caveats-vpp) | vpp | minimal | | VyOS 1.4 [❗](caveats-vyos) | vyos | full | [^SROSBE]: With the launch of the Nokia SR SIM, we stopped running integration tests for the SR-OS VM, assuming the behavior of the two products would be nearly identical. @@ -89,6 +90,7 @@ Most devices behave as routers (or layer-3 switches); the following devices can | Generic Linux | ❌ | ✅ | ✅ | | Kubernetes in Docker | ❌ | ✅ | ❌ | | Open BSD | ✅ | ✅ | ❌ | +| VPP | ✅ | ❌ | ✅ | **Notes:** @@ -145,6 +147,7 @@ You cannot use all supported network devices with all virtualization providers. | Nokia SR-SIM | ❌ | ✅ | | OpenBSD | [✅](build-openbsd) | [✅](clab-vrnetlab) | | SONiC | [✅](build-sonic-box) | [✅](build-sonic-container) | +| VPP | ❌ | ✅ | | VyOS | ✅ | ✅[❗](caveats-vyos) | **Note:** @@ -211,7 +214,7 @@ Ansible playbooks included with **netlab** can deploy and collect device configu | Nokia SR OS[^SROS] | ✅ | ✅ | | OpenBSD | ✅ | ❌ | | Sonic | ✅ | ✅ | -| Sonic (containerlab) | ✅ | ✅ | +| VPP FD.io | ✅ | ❌ | | VyOS | ✅ | ✅ | **Note:** *netlab* can deploy daemon configurations, but cannot collect them. Use the **netlab initial -o** command to create daemon configuration files in a custom directory. @@ -229,6 +232,9 @@ Ansible playbooks included with **netlab** can deploy and collect device configu (platform-config-mode)= _netlab_ uses Ansible playbooks and device-specific task lists to deploy device configuration snippets onto most devices, with these notable exceptions: +:::{table} +:class: table-wrap + | Device | Provider | Configuration deployment method | |--------|----------|---------------------------------| | bird | clab | **bash** scripts or daemon configuration files[^BBS] | @@ -238,6 +244,8 @@ _netlab_ uses Ansible playbooks and device-specific task lists to deploy device | KinD | clab | **bash** scripts copied into and executed in containers | | linux | clab | host- or container-side **bash** scripts[^LBS] | | Sonic (containerlab) | clab | **bash** or **vtysh** scripts[^FRRBV] over **docker exec** | +| vpp | clab | **bash** scripts, node configuration files, and VPP CLI configuration files[^VPPC] | +::: [^FRRBV]: Configurations starting with a *shebang* are assumed to be Linux scripts; all other configurations are assumed to be **vtysh** scripts and get a `#!/usr/bin/vtysh -f` shebang prepended to them. @@ -247,6 +255,8 @@ _netlab_ uses Ansible playbooks and device-specific task lists to deploy device [^DBS]: Initial device configurations, VLANs, static routes, and link aggregation are configured with **bash** scripts. All other features are configured with the dnsmasq configuration files. +[^VPPC]: Initial device configuration is deployed with **bash** scripts executed within the container. VPP **startup.conf** is deployed as a node configuration file. The VPP startup configuration loads `/etc/vpp/config/setup.vpp`, a generated VPP CLI configuration file; the initial script creates `/etc/vpp/config/clab-interfaces.vpp`. The container waits for **netlab initial** to finish before starting VPP. + [^cRBS]: The configuration deployment uses a custom **bash** script that calls **cli** command to execute **load merge** followed by **commit**. The custom script is used as the *shebang* interpreter for the configuration snippets. Several other devices can use alternate (faster) configuration methods that are not enabled by default; you have to set the **netlab_config_mode** group variable[^NCMGV] or node parameter to use them: @@ -362,7 +372,7 @@ The following interface addresses are supported on various platforms; most daemo | Nokia SR OS[^SROS] | ✅ | ✅ | ✅ | ❌ | | OpenBSD | ✅ | ✅ | ❌ | ❌ | | Sonic | ✅ | ✅ | ✅ | ❌ | -| Sonic (containerlab) | ✅ | ✅ | ✅ | ❌ | +| VPP | ✅ | ✅ | ✅ | ✅ | | VyOS | ✅ | ✅ | ✅ | ❌ | ```{tip} diff --git a/netsim/ansible/templates/initial/vpp.j2 b/netsim/ansible/templates/initial/vpp.j2 new file mode 100644 index 0000000000..d30123b783 --- /dev/null +++ b/netsim/ansible/templates/initial/vpp.j2 @@ -0,0 +1,34 @@ +{% from '_extra_initial.j2' import extra_initial with context %} +#!/usr/bin/env bash +# + +NETNS=${NETNS:="dataplane"} +CLAB_VPP_FILE="/etc/vpp/config/clab-interfaces.vpp" + +echo "Creating dataplane namespace" +/usr/bin/mkdir -p /etc/netns/$NETNS +/usr/bin/touch /etc/netns/$NETNS/resolv.conf +/usr/sbin/ip netns add $NETNS +/usr/bin/nsenter --net=/run/netns/$NETNS /usr/sbin/ip link set lo up + +echo "Generating $CLAB_VPP_FILE" +mkdir -p $(dirname $CLAB_VPP_FILE) +: > $CLAB_VPP_FILE +MTU=9216 +for IFNAME in $(ip -br link | awk '{ print $1 }' | cut -f1 -d@ | grep -E '^eth[1-9][0-9]*$' | sort); do + MAC=$(ip -br link show dev $IFNAME | awk '{ print $3 }') + echo " * $IFNAME hw-addr $MAC mtu $MTU" + ip link set $IFNAME up mtu $MTU + cat << EOF >> $CLAB_VPP_FILE +create host-interface name $IFNAME hw-addr $MAC +set interface mtu $MTU host-$IFNAME +set interface state host-$IFNAME up + +EOF +done + +{{ extra_initial() }} + +touch /var/run/initial.done + +exit 0 diff --git a/netsim/ansible/templates/initial/vpp.vlan.j2 b/netsim/ansible/templates/initial/vpp.vlan.j2 new file mode 100644 index 0000000000..58abc78a0d --- /dev/null +++ b/netsim/ansible/templates/initial/vpp.vlan.j2 @@ -0,0 +1,63 @@ +{# + Create VPP VLAN objects (bridge-domains, trunk subifs, BVIs). Access-port + bridge assignment lives in vlan/vpp.j2. Routed subif LCP peers are created + by lcp-auto-subint after netlab-start aligns parent TAP MACs with host-*. + BVI must be attached to its BD immediately after create; BVI still needs + an explicit lcp create. +#} +{% for vname,vdata in vlans|default({})|dictsort + if vdata.mode|default('irb') != 'route' %} +create bridge-domain {{ vdata.id }} +{% endfor %} +{% for ifdata in interfaces if ifdata.type|default('') == 'vlan_member' + and ifdata.vlan.access_id is defined + and ifdata.parent_ifname is defined %} +{% set vpp_if = 'host-' ~ ifdata.ifname %} +create sub-interfaces host-{{ ifdata.parent_ifname }} {{ ifdata.vlan.access_id }} +{% if ifdata.mtu is defined %} +set interface mtu {{ ifdata.mtu }} {{ vpp_if }} +{% endif %} +set interface state {{ vpp_if }} up +{% if ifdata.vlan.mode|default('irb') == 'route' %} +{% if ifdata.ipv4 is defined and ifdata.ipv4 is string %} +set interface ip address {{ vpp_if }} {{ ifdata.ipv4 }} +{% endif %} +{% if ifdata.ipv6 is defined and ifdata.ipv6 is string and ifdata.ipv6|ansible.utils.ipv6 %} +set interface ip address {{ vpp_if }} {{ ifdata.ipv6 }} +{% if role == 'router' %} +{% if ifdata.ra.disable|default(false) is true %} +ip6 nd {{ vpp_if }} ra-suppress +{% else %} +ip6 nd {{ vpp_if }} ra-interval 5 +{% set nd_prefix = ifdata.ipv6|ansible.utils.ipaddr(0) %} +ip6 nd {{ vpp_if }} prefix {{ nd_prefix }} default +{% endif %} +{% endif %} +{% endif %} +{# LCP for VLAN subifs comes from lcp-auto-subint on the parent #} +{% endif %} +{% endfor %} +{% for ifdata in interfaces if ifdata.type|default('') == 'svi' + and ((ifdata.ipv4 is defined and ifdata.ipv4 is string) + or (ifdata.ipv6 is defined and ifdata.ipv6 is string)) %} +{% set bd = vlans[ifdata.vlan.name].id %} +bvi create instance {{ bd }} +set interface l2 bridge bvi{{ bd }} {{ bd }} bvi +set interface state bvi{{ bd }} up +{% if ifdata.ipv4 is defined and ifdata.ipv4 is string %} +set interface ip address bvi{{ bd }} {{ ifdata.ipv4 }} +{% endif %} +{% if ifdata.ipv6 is defined and ifdata.ipv6 is string and ifdata.ipv6|ansible.utils.ipv6 %} +set interface ip address bvi{{ bd }} {{ ifdata.ipv6 }} +{% if role == 'router' %} +{% if ifdata.ra.disable|default(false) is true %} +ip6 nd bvi{{ bd }} ra-suppress +{% else %} +ip6 nd bvi{{ bd }} ra-interval 5 +{% set nd_prefix = ifdata.ipv6|ansible.utils.ipaddr(0) %} +ip6 nd bvi{{ bd }} prefix {{ nd_prefix }} default +{% endif %} +{% endif %} +{% endif %} +lcp create bvi{{ bd }} host-if {{ ifdata.ifname }} +{% endfor %} diff --git a/netsim/ansible/templates/mpls/vpp.initial.j2 b/netsim/ansible/templates/mpls/vpp.initial.j2 new file mode 100644 index 0000000000..48bf7ede03 --- /dev/null +++ b/netsim/ansible/templates/mpls/vpp.initial.j2 @@ -0,0 +1 @@ +sysctl -w net.mpls.platform_labels=1048575 diff --git a/netsim/ansible/templates/mpls/vpp.j2 b/netsim/ansible/templates/mpls/vpp.j2 new file mode 100644 index 0000000000..6e00e79309 --- /dev/null +++ b/netsim/ansible/templates/mpls/vpp.j2 @@ -0,0 +1 @@ +mpls table add 0 \ No newline at end of file diff --git a/netsim/ansible/templates/vlan/vpp.j2 b/netsim/ansible/templates/vlan/vpp.j2 new file mode 100644 index 0000000000..6d6eed3d09 --- /dev/null +++ b/netsim/ansible/templates/vlan/vpp.j2 @@ -0,0 +1,16 @@ +{# + VPP interface VLAN parameters (access/trunk membership). Bridge-domains, + subinterfaces, and BVIs (including BVI-to-BD attach) are created in + initial/vpp.vlan.j2. +#} +{% for ifdata in interfaces if ifdata.type|default('') == 'vlan_member' + and ifdata.vlan.access_id is defined + and ifdata.vlan.mode|default('irb') != 'route' %} +set interface l2 tag-rewrite host-{{ ifdata.ifname }} pop 1 +{% endfor %} +{% for ifdata in interfaces if ifdata.vlan is defined and ifdata.type|default('') != 'svi' %} +{% set vname = ifdata.vlan.access|default(ifdata.vlan.native) %} +{% if vname is defined and vlans[vname].mode|default('irb') != 'route' %} +set interface l2 bridge host-{{ ifdata.ifname }} {{ vlans[vname].id }} +{% endif %} +{% endfor %} diff --git a/netsim/daemons/bird/bird.j2 b/netsim/daemons/bird/bird.j2 index 5ae1abe20f..43e3f282d0 100644 --- a/netsim/daemons/bird/bird.j2 +++ b/netsim/daemons/bird/bird.j2 @@ -7,6 +7,7 @@ log stderr all; {% include 'radv.j2' %} {% endif %} -{% for k,v in _daemon_config.items() if k != device|default(netlab_device_type) and not v.endswith('.sh') %} +{% for k,v in _daemon_config.items() + if k not in [device|default(netlab_device_type), 'bird'] and v.endswith('.conf') %} include "{{ v }}"; {% endfor %} diff --git a/netsim/daemons/vpp/Dockerfile.j2 b/netsim/daemons/vpp/Dockerfile.j2 new file mode 100644 index 0000000000..c19cae14b2 --- /dev/null +++ b/netsim/daemons/vpp/Dockerfile.j2 @@ -0,0 +1,143 @@ +{% set _clab = defaults.devices.vpp.clab %} +{% set _sw_version = sw_version if sw_version is defined else _clab.get('sw_version',none) %} +FROM debian:bookworm-slim +ENV DEBIAN_FRONTEND=noninteractive + +LABEL maintainer="Netlab project " +LABEL description="FD.io Vector Packet Processor (VPP){% if _sw_version %} ({{ _sw_version }}){% endif %}" + +RUN apt-get update && apt-get install -y --no-install-recommends \ + apt-transport-https \ + ca-certificates \ + curl \ + gnupg \ + iproute2 \ + procps \ + && rm -rf /var/lib/apt/lists/* + +ARG REPO +ARG VPP_VERSION={{ _sw_version or '' }} + +WORKDIR /vpp + +RUN set -eux; \ + export VPP_VERSION="{{ _sw_version or '' }}"; \ + export REPO="{{ _clab.get('repo','release') }}"; \ + bash <<'GETVPP' +#!/bin/bash + +[ -z "$REPO_URL" ] && REPO_URL="https://packagecloud.io/install/repositories/fdio/${REPO:=release}" + +function get_vpp () { + ls *.deb 2>/dev/null && { die "remove existing *.deb files"; } + + set -exuo pipefail + trap '' PIPE + + curl -sS "${REPO_URL}"/script.deb.sh | bash || { + die "Packagecloud FD.io repo fetch failed." + } + + artifacts=() + both_quotes='"'"'" + match="[^${both_quotes}]*" + qmatch="[${both_quotes}]\?" + sed_command="s#.*apt_source_path=${qmatch}\(${match}\)${qmatch}#\1#p" + apt_fdio_repo_file=$(curl -s "${REPO_URL}"/script.deb.sh | \ + sed -n ${sed_command}) || { + die "Local fdio repo file path fetch failed." + } + + if [ ! -f ${apt_fdio_repo_file} ]; then + die "${apt_fdio_repo_file} not found, \ + repository installation was not successful." + fi + + packages=$(apt-cache -o Dir::Etc::SourceList=${apt_fdio_repo_file} \ + -o Dir::Etc::SourceParts=${apt_fdio_repo_file} dumpavail \ + | grep Package: | cut -d " " -f 2) || { + die "Retrieval of available VPP packages failed." + } + if [ -z "${VPP_VERSION-}" ]; then + allVersions=$(apt-cache -o Dir::Etc::SourceList=${apt_fdio_repo_file} \ + -o Dir::Etc::SourceParts=${apt_fdio_repo_file} \ + show vpp | grep Version: | cut -d " " -f 2) || { + die "Retrieval of available VPP versions failed." + } + if [ "${REPO}" != "master" ]; then + nonRcVersions=$(echo "$allVersions" | grep -v "\-rc[0-9]") || true + [ -n "${nonRcVersions}" ] && allVersions=$nonRcVersions + fi + VPP_VERSION=$(echo "$allVersions" | head -n1) || true + fi + + set +x + echo "Finding packages with version: ${VPP_VERSION-}" + for package in ${packages}; do + pkg_info=$(apt-cache show -- ${package}) || { + die "apt-cache show on ${package} failed." + } + ver=$(echo ${pkg_info} | grep -o "Version: ${VPP_VERSION-}[^ ]*" | head -1) || true + if [ -n "${ver-}" ]; then + if [ "${package}" == "vom" ]; then + echo " x '${package}' skipped" + else + echo "+++'${package}' found" + ver=$(echo "$ver" | cut -d " " -f 2) + artifacts+=(${package[@]/%/=${ver-}}) + fi + else + echo " - '${package}'" + fi + done + set -x + + if [ "${INSTALL:-false}" = true ]; then + apt-get -y install "${artifacts[@]}" || { + die "Install VPP artifacts failed." + } + else + apt-get -y download "${artifacts[@]}" || { + die "Download VPP artifacts failed." + } + fi +} + +function die () { + set -x + set +eu + echo "${1:-Unspecified run-time error occurred!}" + exit "${2:-1}" +} + +get_vpp +GETVPP + +RUN set -eux; \ + sysctl_bin="$(command -v sysctl)"; \ + mv "$sysctl_bin" "${sysctl_bin}.real"; \ + printf '#!/bin/sh\nexit 0\n' > "$sysctl_bin"; \ + chmod +x "$sysctl_bin"; \ + apt-get update; \ + apt-get install -y -V ./*.deb; \ + mv "${sysctl_bin}.real" "$sysctl_bin"; \ + dpkg-query -f '${Version}\n' -W vpp > /vpp/version; \ + rm -rf vom*.deb vpp-dbg*.deb; \ + rm -rf /var/lib/apt/lists/*; + +RUN set -eux; \ + apt-get update; \ + apt-get install -y --no-install-recommends \ + bird2 \ + curl \ + gnupg; \ + curl -fsSL https://deb.frrouting.org/frr/keys.asc | gpg --dearmor -o /etc/apt/trusted.gpg.d/frr.gpg; \ + echo "deb https://deb.frrouting.org/frr bookworm frr-stable" > /etc/apt/sources.list.d/frr.list; \ + apt-get update; \ + apt-get install -y --no-install-recommends frr frr-pythontools; \ + mkdir -p /etc/bird /run/bird /var/log/frr /var/run/frr; \ + rm -rf /var/lib/apt/lists/*; + +RUN mkdir -p /var/log/vpp + +CMD ["/usr/bin/vpp", "-c", "/etc/vpp/startup.conf"] diff --git a/netsim/devices/vpp.py b/netsim/devices/vpp.py new file mode 100644 index 0000000000..86d59b24b4 --- /dev/null +++ b/netsim/devices/vpp.py @@ -0,0 +1,162 @@ +# +# VPP quirks +# +from box import Box + +from ..augment import devices as a_devices +from ..utils import log +from . import _Quirks, report_quirk +from .bird import bird_vlan_evpn_rt, bird_vrf_rt + +_VPP_OWNED_GVARS = frozenset({"netlab_start_daemon", "netlab_dp_module"}) + + +def control_plane(node: Box, defaults: Box) -> str: + vpp_defaults = defaults.devices.vpp + return node.get("control_plane", vpp_defaults.get("control_plane", "bird")) + + +def _active_cp_modules(node: Box, cp: str) -> set[str]: + active = {cp, "routing"} + active.update(node.get("module", [])) + return active + + +def _merge_cp_daemon_config(node: Box, topology: Box, cp: str) -> None: + """ + Copy control-plane daemon_config entries that would survive cleanup_non_ansible_config. + + Bird ships optional plugin maps (e.g. ospf@areas). Merging those after node + cleanup would force a config template lookup for plugins the lab does not use. + """ + cp_dev = topology.defaults.devices.get(cp) + if not cp_dev or "daemon_config" not in cp_dev: + return + + if not node.get("_daemon_config"): + node._daemon_config = {} + + modules = set(node.get("module", [])) + configs = set(node.get("config", [])) + for k, v in cp_dev.daemon_config.items(): + kn = k.replace("@", ".") + if kn != cp and kn not in modules and kn not in configs and kn != "initial": + continue + node._daemon_config[k] = v + + +def _merge_cp_group_vars(node: Box, topology: Box, cp: str) -> None: + cp_dev = topology.defaults.devices.get(cp, {}) + cp_clab_gvars = cp_dev.get("clab", {}).get("group_vars") + cp_gvars = cp_dev.get("group_vars", {}) + vpp_clab_gvars = topology.defaults.devices.vpp.get("clab", {}).get("group_vars", {}) + + if cp_clab_gvars: + node.clab.group_vars = cp_clab_gvars + vpp_clab_gvars + node.clab.get("group_vars", {}) + + for gvars in (cp_clab_gvars, cp_gvars): + if not gvars: + continue + for k, v in gvars.items(): + if k in _VPP_OWNED_GVARS: + continue + if k not in node and k not in vpp_clab_gvars: + node[k] = v + + +def _merge_cp_config_templates(node: Box, topology: Box, cp: str) -> None: + cp_templates = topology.defaults.devices.get(cp, {}).get("clab", {}).get("node", {}).get("config_templates") + if cp_templates: + node.clab.config_templates = cp_templates + node.clab.get("config_templates", {}) + + +def _bird_rt_transforms(node: Box) -> None: + bird_vrf_rt(node) + bird_vlan_evpn_rt(node) + + +def _configure_bird_cp(node: Box, topology: Box) -> None: + _merge_cp_daemon_config(node, topology, "bird") + _bird_rt_transforms(node) + _merge_cp_group_vars(node, topology, "bird") + + +def _register_frr_cp_scripts(node: Box, topology: Box) -> None: + """ + Bind FRR control-plane module scripts for startup-time deployment. + + netlab initial must not docker-exec these before FRR starts in the dataplane netns. + """ + features = a_devices.get_device_features(node, topology.defaults) + mod_list: list[str] = ["normalize"] if features.initial.get("normalize", False) else [] + mod_list += ["initial"] + node.get("module", []) + node.get("config", []) + cp_modules = _active_cp_modules(node, "frr") - {"frr", "routing", "initial"} + if not cp_modules: + return + + if not node.get("_daemon_config"): + node._daemon_config = {} + + for idx, m in enumerate(mod_list, start=1): + if m not in cp_modules: + continue + key = m.replace(".", "@") + # Keep VPP dataplane snippets (*.vpp) for setup.vpp exec; do not feed them to vtysh + existing = node._daemon_config.get(key) + if isinstance(existing, str) and existing.endswith(".vpp"): + continue + node._daemon_config[key] = f"/etc/config/{idx:02d}-{m}.sh" + + +def _configure_frr_cp(node: Box, topology: Box) -> None: + _merge_cp_group_vars(node, topology, "frr") + _merge_cp_config_templates(node, topology, "frr") + _register_frr_cp_scripts(node, topology) + + +def _seed_daemon_config(node: Box) -> None: + """ + Promote device node_config into _daemon_config for setup.vpp.j2 and CP merges. + + Regular devices only auto-copy node_config → _node_config; VPP templates and + BIRD/FRR control-plane merging still expect _daemon_config. + + Entries with a mode suffix (path:sh) stay in _node_config for binds/exec and + must not become BIRD include paths. + """ + if not node.get("_daemon_config"): + node._daemon_config = {} + + for k, v in node.get("_node_config", {}).items(): + if k not in node._daemon_config and isinstance(v, str) and ':' not in v: + node._daemon_config[k] = v + + +def configure_control_plane(node: Box, topology: Box) -> None: + cp = control_plane(node, topology.defaults) + node.control_plane = cp + _seed_daemon_config(node) + + if cp == "bird": + if "isis" in node.get("module", []): + report_quirk( + f"IS-IS is not supported on VPP node {node.name} with bird control plane", + more_hints=["Use control_plane: frr to enable IS-IS"], + quirk="bird_isis", + category=log.IncorrectValue, + node=node, + ) + return + # Enable template-search parent fallback without device.daemon (see templates.py) + node._daemon = True + node._daemon_parent = "bird" + _configure_bird_cp(node, topology) + else: + _configure_frr_cp(node, topology) + + +class VPP(_Quirks): + @classmethod + def device_quirks(self, node: Box, topology: Box) -> None: + configure_control_plane(node, topology) + diff --git a/netsim/devices/vpp.yml b/netsim/devices/vpp.yml new file mode 100644 index 0000000000..8588be4a62 --- /dev/null +++ b/netsim/devices/vpp.yml @@ -0,0 +1,96 @@ +--- +description: VPP fd.io container +parent: linux +role: router +interface_name: eth{ifindex} +mgmt_if: eth0 +loopback_interface_name: loop{ifindex} +control_plane: bird +support: + level: minimal +routing: + _rm_per_af: True +group_vars: + ansible_network_os: frr + ansible_python_interpreter: auto_silent + netlab_device_type: vpp +node_config: + vpp: /etc/vpp/startup.conf + mpls: /etc/vpp/config/mpls.vpp + vlan: /etc/vpp/config/vlan.vpp +clab: + node.cmd: /etc/dataplane-wait.sh + node.config_templates: + dataplane-wait: /etc/dataplane-wait.sh:sh + setup@vpp: /etc/vpp/config/setup.vpp + vlan-init@vpp: /etc/vpp/config/vlan-init.vpp + netlab-start@sh: /usr/local/bin/netlab-start.sh:sh + hosts: /etc/hosts:shared + node_config: + initial: /etc/config/01-initial.sh:sh + # Override linux parent :ns — VLAN dataplane via vlan-init + vlan.vpp + vlan: /etc/vpp/config/vlan.vpp + group_vars: + ansible_connection: docker + ansible_user: root + docker_shell: bash -il + netlab_config_mode: sh + netlab_dp_module: [ initial ] + netlab_start_daemon: "/usr/local/bin/netlab-start.sh" + image: netlab/vpp:latest + # image: git.ipng.ch/ipng/vpp-containerlab:stable + build: True + # node: + # kind: fdio_vpp + sw_version: # Marks versioning support, latest release is used by default + repo: release + mtu: 1500 + features: + initial: + config_mode: [ sh ] + roles: [ router, bridge ] + + # Features that work in Linux containers, but not on VPP + # + services: False +libvirt: + image: +virtualbox: + image: +features: + initial: + ipv4: + unnumbered: peer + ipv6: + lla: true + use_ra: true + ra: true + reload: false + roles: [ router, bridge ] + vlan: + model: router + svi_interface_name: vlan{vlan} + subif_name: '{ifname}.{vlan.access_id}' + ospf: + # bird and frr + unnumbered: true + password: true + priority: true + timers: true + areas: true + import: [ bgp, connected, static, vrf, ripv2, isis ] # bird: no ripv2/isis +# default: true # frr only +# gr: [ ipv4, ipv6 ] # frr only + +# Features inherited from Linux that do not work on VPP +# + lag: False + routing: False + services: False +external: + image: none +graphite.icon: router + +attributes: + node: + control_plane: { type: str, valid_values: [ bird, frr ] } diff --git a/netsim/templates/provider/clab/vpp/bird.j2 b/netsim/templates/provider/clab/vpp/bird.j2 new file mode 100644 index 0000000000..8c77c21b6f --- /dev/null +++ b/netsim/templates/provider/clab/vpp/bird.j2 @@ -0,0 +1 @@ +{% include 'bird/bird.j2' %} diff --git a/netsim/templates/provider/clab/vpp/daemons.j2 b/netsim/templates/provider/clab/vpp/daemons.j2 new file mode 100644 index 0000000000..6ee72b7965 --- /dev/null +++ b/netsim/templates/provider/clab/vpp/daemons.j2 @@ -0,0 +1 @@ +{% include 'frr/daemons.j2' %} diff --git a/netsim/templates/provider/clab/vpp/hosts.j2 b/netsim/templates/provider/clab/vpp/hosts.j2 new file mode 100644 index 0000000000..42d1a3c4bd --- /dev/null +++ b/netsim/templates/provider/clab/vpp/hosts.j2 @@ -0,0 +1 @@ +{% include 'linux/hosts.j2' %} diff --git a/netsim/templates/provider/clab/vpp/netlab-start.sh.j2 b/netsim/templates/provider/clab/vpp/netlab-start.sh.j2 new file mode 100644 index 0000000000..87bc32f95e --- /dev/null +++ b/netsim/templates/provider/clab/vpp/netlab-start.sh.j2 @@ -0,0 +1,123 @@ +#!/bin/bash +# +# Start VPP, attach Linux peer addresses for IPv4 unnumbered LCP interfaces, +# then start the selected control-plane daemon in the dataplane namespace. +# +set -e + +NETNS=dataplane +VPP_CONTROL_PLANE={{ control_plane|default('bird') }} +WAIT_TIMEOUT=30 + +/usr/bin/vpp -c /etc/vpp/startup.conf & +VPP_PID=$! +timeout "$WAIT_TIMEOUT" sh -c 'until [ -S /run/vpp/cli.sock ]; do sleep 0.1; done' \ + || { echo "VPP CLI socket did not appear within ${WAIT_TIMEOUT}s" >&2; exit 1; } +vppctl show version >/dev/null + +# +# The CLI socket appears before unix.startup-config (setup.vpp) finishes. +# Wait for LCP peers so unnumbered addressing and FRR/BIRD see dataplane ifaces. +# +{% for l in interfaces + if not l.virtual_interface|default(false) + and l.type|default('') not in ['svi', 'vlan_member'] + and (l.type|default('') != 'loopback' + or (('ipv4' in l and l.ipv4 is not false) or ('ipv6' in l and l.ipv6 is not false))) %} +timeout "$WAIT_TIMEOUT" sh -c "until ip -n \"$NETNS\" link show {{ l.ifname }} >/dev/null 2>&1; do sleep 0.1; done" \ + || { echo "LCP interface {{ l.ifname }} did not appear in $NETNS within ${WAIT_TIMEOUT}s" >&2; exit 1; } +{% endfor %} + +# +# Align each LCP TAP MAC with its host-* / loopback phy MAC. initial/vpp.j2 pins +# host-interface hw-addr to the container NIC MAC; TAP defaults to 02:fe:… and +# VLAN subif LCP then tries to add that as a secondary MAC on af_packet (unsupported). +# +while read -r phy tap _rest; do + [ -n "$phy" ] && [ -n "$tap" ] || continue + case "$phy" in + host-*|loop*|bvi*) ;; + *) continue ;; + esac + phy_mac=$(vppctl show hardware-interfaces "$phy" 2>/dev/null \ + | awk '/Ethernet address/{print $3; exit}') + [ -n "$phy_mac" ] || continue + vppctl set interface mac address "$tap" "$phy_mac" >/dev/null 2>&1 || true +done < <(vppctl show lcp 2>/dev/null | awk '/^itf-pair:/{print $3,$4}') + +# +# VLAN create + L2 membership after MAC alignment (see vlan-init.vpp / vlan.vpp). +# +{% if vlans is defined %} +vppctl exec /etc/vpp/config/vlan-init.vpp +{% endif %} +{% if 'vlan' in module|default([]) %} +vppctl exec /etc/vpp/config/vlan.vpp +{% endif %} +{% for m in module|default([]) + if m != 'vlan' + and m in _daemon_config|default({}) + and _daemon_config[m].endswith('.vpp') %} +vppctl exec {{ _daemon_config[m] }} +{% endfor %} + +{% for l in interfaces + if l.type|default('') in ['vlan_member', 'svi'] + and ((l.ipv4 is defined and l.ipv4 is string) + or (l.ipv6 is defined and l.ipv6 is string) + or l.type|default('') == 'vlan_member') %} +{% if l.type|default('') == 'vlan_member' and l.vlan.mode|default('irb') == 'route' %} +timeout "$WAIT_TIMEOUT" sh -c "until ip -n \"$NETNS\" link show {{ l.ifname }} >/dev/null 2>&1; do sleep 0.1; done" \ + || { echo "LCP VLAN interface {{ l.ifname }} did not appear in $NETNS within ${WAIT_TIMEOUT}s" >&2; exit 1; } +{% elif l.type|default('') == 'svi' %} +timeout "$WAIT_TIMEOUT" sh -c "until ip -n \"$NETNS\" link show {{ l.ifname }} >/dev/null 2>&1; do sleep 0.1; done" \ + || { echo "LCP SVI {{ l.ifname }} did not appear in $NETNS within ${WAIT_TIMEOUT}s" >&2; exit 1; } +{% endif %} +{% endfor %} + +# +# Attach Linux peer addresses for IPv4 unnumbered LCP interfaces +# +{% for l in interfaces if l._unnumbered_peer is defined %} +ip -n "$NETNS" addr flush dev {{ l.ifname }} scope global 2>/dev/null || true +ip -n "$NETNS" addr add {{ l._parent_ipv4 }} peer {{ l._unnumbered_peer }} dev {{ l.ifname }} +{% endfor %} + +case "$VPP_CONTROL_PLANE" in + bird) + mkdir -p /run/bird + # BIRD 2.x: omit -d so bird daemonizes; VPP stays foreground via wait + ip netns exec "$NETNS" /usr/sbin/bird -c /etc/bird/bird.conf + wait $VPP_PID + ;; + frr) + ip netns exec "$NETNS" bash -c ' + mkdir -p /run/frr /var/run/frr /var/log/frr + chown -R frr:frrvty /run/frr /var/run/frr /var/log/frr 2>/dev/null \ + || chown -R frr:frr /run/frr /var/run/frr /var/log/frr + /usr/lib/frr/frrinit.sh start + # Match ansible/tasks/deploy-config/frr.yml: bash shebang → shell, + # otherwise treat the file as vtysh configuration (e.g. OSPF/BGP). + for script in /etc/config/[0-9][0-9]-*.sh; do + case "$script" in + /etc/config/01-initial.sh) continue ;; + esac + [ -f "$script" ] || continue + shebang=$(head -n1 "$script") + case "$shebang" in + "#!/bin/bash"|"#!/bin/sh"|"#!/usr/bin/env bash"|"#!/usr/bin/env sh") + bash "$script" || exit 1 + ;; + *) + /usr/bin/vtysh -f "$script" || exit 1 + ;; + esac + done + ' + wait $VPP_PID + ;; + *) + echo "Unknown control_plane: $VPP_CONTROL_PLANE" >&2 + exit 1 + ;; +esac diff --git a/netsim/templates/provider/clab/vpp/resolv.j2 b/netsim/templates/provider/clab/vpp/resolv.j2 new file mode 100644 index 0000000000..af99c30565 --- /dev/null +++ b/netsim/templates/provider/clab/vpp/resolv.j2 @@ -0,0 +1 @@ +{% include 'linux/resolv.j2' %} diff --git a/netsim/templates/provider/clab/vpp/setup.vpp.j2 b/netsim/templates/provider/clab/vpp/setup.vpp.j2 new file mode 100644 index 0000000000..d4965595d8 --- /dev/null +++ b/netsim/templates/provider/clab/vpp/setup.vpp.j2 @@ -0,0 +1,88 @@ +# +# bound to /etc/vpp/config/setup.vpp +# +set logging class linux-cp rate-limit 1000 level warn syslog-level notice + +lcp default netns dataplane +lcp lcp-sync on +lcp lcp-sync-unnumbered off +lcp param del-static-on-link-down on +lcp param del-dynamic-on-link-down on +{# + Auto-create Linux VLAN devices on the parent LCP TAP when VPP subifs appear. + Parent TAP MAC must match host-interface MAC first (see netlab-start.sh). +#} +lcp lcp-auto-subint on + +exec /etc/vpp/config/clab-interfaces.vpp + +{% for af_name in ['ipv4','ipv6'] if af_name in af %} +set ip neighbor-config {{ af_name|replace('ipv','ip') }} age 900 +{% endfor %} + +{# + Skip SVIs / vlan_member (vlan-init.vpp via netlab-start.sh). Keep loopbacks + even though they are virtual_interface — they still need a VPP loopback + address. +#} +{% for l in netlab_interfaces + if l.type|default('') == 'loopback' + or (not l.virtual_interface|default(false) + and l.type|default('') not in ['svi', 'vlan_member']) %} +{% set is_loopback = l.type|default('') == 'loopback' %} +{% if is_loopback %} +{% set lb_instance = l.ifname[4:] %} +create loopback interface instance {{ lb_instance }} +{% set vpp_if = l.ifname %} +{% else %} +{% set vpp_if = 'host-' ~ l.ifname %} +{% endif %} +{% set has_ipv4 = 'ipv4' in l and l.ipv4 is not false %} +{% set has_ipv6 = 'ipv6' in l and l.ipv6 is not false %} +{% if l.mtu is defined %} +set interface mtu {{ l.mtu }} {{ vpp_if }} +{% endif %} +{% if not is_loopback or has_ipv4 or has_ipv6 %} +lcp create {{ vpp_if }} host-if {{ l.ifname }} +{% endif %} +set interface state {{ vpp_if }} {{ 'down' if l.shutdown|default(False) else 'up' }} +{% if l.ipv4 is defined and l.ipv4 is string %} +set interface ip address {{ vpp_if }} {{ l.ipv4 }} +{% elif l._parent_intf is defined %} +set interface unnumbered {{ vpp_if }} use {{ l._parent_intf }} +{% endif %} +{% if 'ipv6' in l and l.ipv6 is string and l.ipv6|ansible.utils.ipv6 %} +set interface ip address {{ vpp_if }} {{ l.ipv6 }} +{% if not is_loopback %} +{% if role == 'router' %} +{% if l.ra.disable|default(false) is true %} +ip6 nd {{ vpp_if }} ra-suppress +{% else %} +ip6 nd {{ vpp_if }} ra-interval 5 +{% if 'ra' in l and l.ipv6 is string %} +{% set nd_prefix = l.ipv6|ansible.utils.ipaddr(0) %} +{% set nd_no_autoconfig = 'no-autoconfig' if l.ra.slaac|default(true) is false else '' %} +{% set nd_off_link = 'off-link' if l.ra.onlink|default(true) is false else '' %} +{% if nd_no_autoconfig or nd_off_link %} +ip6 nd {{ vpp_if }} prefix {{ nd_prefix }} infinite {{ nd_no_autoconfig }} {{ nd_off_link }} +{% else %} +ip6 nd {{ vpp_if }} prefix {{ nd_prefix }} default +{% endif %} +{% endif %} +{% if l.ra.dhcp|default(false) == 'all' %} +ip6 nd {{ vpp_if }} ra-managed-config-flag +{% endif %} +{% if l.ra.dhcp|default(false) == 'other' %} +ip6 nd {{ vpp_if }} ra-other-config-flag +{% endif %} +{% endif %} +{% elif role == 'host' %} +ip6 nd address autoconfig {{ vpp_if }} default-route +{% endif %} +{% endif %} +{% endif %} +{% endfor %} + +{# + VLAN and other module .vpp snippets run from netlab-start.sh after LCP TAP + MACs are copied from host-* interfaces (af_packet rejects secondary MACs). +#} diff --git a/netsim/templates/provider/clab/vpp/vlan-init.vpp.j2 b/netsim/templates/provider/clab/vpp/vlan-init.vpp.j2 new file mode 100644 index 0000000000..84e2e99994 --- /dev/null +++ b/netsim/templates/provider/clab/vpp/vlan-init.vpp.j2 @@ -0,0 +1,5 @@ +{# + Deferred VLAN object creation (executed from netlab-start.sh after LCP TAP + MACs are aligned with host-interface MACs from clab-interfaces.vpp). +#} +{% include 'initial/vpp.vlan.j2' %} diff --git a/netsim/templates/provider/clab/vpp/vpp.j2 b/netsim/templates/provider/clab/vpp/vpp.j2 new file mode 100644 index 0000000000..b42d6bbb62 --- /dev/null +++ b/netsim/templates/provider/clab/vpp/vpp.j2 @@ -0,0 +1,40 @@ +unix { + interactive + nosyslog + + full-coredump + cli-listen /run/vpp/cli.sock + cli-prompt {{ inventory_hostname }}-vpp$ + cli-no-pager + poll-sleep-usec 100 + startup-config /etc/vpp/config/setup.vpp +} + +api-trace { + on +} + +memory { + main-heap-size 512M + main-heap-page-size 4k +} + +buffers { + buffers-per-numa 16000 + default data-size 2048 + page-size 4k +} + +statseg { + size 64M + page-size 4k + per-node-counters on +} + +plugins { + plugin default { enable } + plugin dpdk_plugin.so { disable } + plugin linux_cp_plugin.so { enable } + plugin linux_nl_plugin.so { enable } + plugin sflow_plugin.so { enable } +} diff --git a/tests/integration/initial/04-mtu.yml b/tests/integration/initial/04-mtu.yml index db24661f07..e4262f5460 100644 --- a/tests/integration/initial/04-mtu.yml +++ b/tests/integration/initial/04-mtu.yml @@ -89,13 +89,6 @@ validate: eos: ping t1400 df-bit size 1400 valid: | 'bytes from' in stdout - flush_v4_1400: - description: Flush IPv4 PMTU cache on p1 - nodes: [ p1 ] - exec: - eos: bash sudo ip route flush cache - valid: | - True v4_1400_ptb: description: IPv4 ping p1 => target (expecting Frag Needed) nodes: [ p1 ] @@ -113,19 +106,11 @@ validate: 'bytes from' not in stdout v6_1400_low: description: IPv6 ping p1 => target (under MTU) - wait: ping_long nodes: [ p1 ] exec: eos: ping ipv6 t1400 df-bit size 1400 valid: | 'bytes from' in stdout - flush_v6_1400: - description: Flush IPv6 PMTU cache on p1 - nodes: [ p1 ] - exec: - eos: bash sudo ip -6 route flush cache - valid: | - True v6_1400_ptb: description: IPv6 ping p1 => target (expecting PTB) nodes: [ p1 ]