Tripleo Composable Roles - How to have different parameter values per compute role
The Problem
With the release of Composable Roles, there are a plethora of things you can do with the broken down individual services. However there are equal no of things which is stil WIP. Some of it needs a little bit of workaround, and some of it is plain simple and intutive. We will discuss here one of the lesser known features of having 2 different compute-roles to have 2 different parameter values for the same services. While this sounds simple, in the composable paradigm, there needs to be a little workaround to get it working
Available Documentation
We will talk of some of the awesome documentation already existing, for the un-intitated into the composable roles. Here are they -
Graeme Gillies's, excellent blog post on getting you started Using TripleO Composable roles in Openstack Newton to perform extra Overcloud configuration
Tripleo Composable roles walkthrough.
Steven Hardy's blogpost on tripleo composable roles
The configuration
Recently with the OSPD - 10 Release, we added, OVS-DPDK. We will try in this blog to configure one compute node to have ovs bridge and another compute as a ovs-dpdk bridge, for network isolation. We will also further have a service parameter for ovs to have a different value on the ovs-dpdk compute node. so lets start!
Refer these document for initial understanding and setting up of Director with ovs-dpdk as the intigration bridge.
Network Functions Virtualization Configuration Guide - Red Hat Customer Portal
Network Functions Virtualization Configuration Guide - Red Hat Customer Portal
Further to this, now we want one compute node to be a normal compute node with openvswitch bridge, and the other compute node to have ovs-dpdk bridge
Our network-environment.yaml
resource_registry: #Our network Isolation OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/templates/nic-configs/compute.yaml OS::TripleO::ComputeOvsDpdk::Net::SoftwareConfig: /home/stack/templates/nic-configs/computeovsdpdk.yaml OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/nic-configs/controller.yaml #First boot OS::TripleO::NodeUserData: /home/stack/templates/first-boot.yaml OS::TripleO::NodeExtraConfigPost: /home/stack/templates/post-install.yaml OS::TripleO::AllNodes::Validation: /home/stack/templates/dummy_all_nodes-validation.yaml #Our ovs-dpdk composable service OS::TripleO::Services::ComputeNeutronOvsDpdkAgent: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-ovs-dpdk-agent.yaml #ports configuration (please note this) OS::TripleO::ComputeOvsDpdk::Ports::ExternalPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml OS::TripleO::ComputeOvsDpdk::Ports::InternalApiPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml OS::TripleO::ComputeOvsDpdk::Ports::StorageMgmtPort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml OS::TripleO::ComputeOvsDpdk::Ports::StoragePort: /usr/share/openstack-tripleo-heat-templates/network/ports/noop.yaml OS::TripleO::ComputeOvsDpdk::Ports::TenantPort: /usr/share/openstack-tripleo-heat-templates/network/ports/tenant.yaml #Parameters (these are shared across the roles for all the services or networks) PingTestIps: '' ControlPlaneDefaultRoute: "172.18.0.1" ControlPlaneSubnetCidr: "24" DnsServers: ['8.8.8.8','8.8.4.4'] EC2MetadataIp: "172.18.0.1" TenantNetCidr: 10.10.103.0/24 TenantAllocationPools: [{'start': '10.10.103.10', 'end': '10.10.103.200'}] #Flavors (Note, as this is the preferred way to state no of compute nodes or controller nodes) OvercloudControlFlavor: baremetal OvercloudComputeFlavor: baremetal OvercloudComputeOvsDpdkFlavor: baremetal # Number of nodes to deploy. ControllerCount: 1 ComputeCount: 1 ComputeOvsDpdkCount: 1 NeutronBridgeMappings: "datacentre:br-ex" NeutronFlatNetworks: "datacentre" NeutronNetworkType: "gre" NeutronTunnelTypes: "gre" # Netron DPDK specific parameters NeutronDpdkCoreList: "'2,4,20,23'" NeutronDpdkMemoryChannels: "2" NeutronDatapathType: "netdev" NeutronVhostuserSocketDir: "/var/run/openvswitch" NeutronDpdkSocketMemory: "'2048,2048'" NeutronDpdkDriverType: "vfio-pci" NovaReservedHostMemory: 4096 NovaVcpuPinSet: ['12-21','24-87'] NovaSchedulerDefaultFilters: "RamFilter,ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter" ComputeKernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=64 intel_iommu=on isolcpus=2,4,20,23,12-87" #Very specific a compute Role specific parameter is sent via ExtraConfigs and needs to be a hiera data format ComputeOvsDpdkExtraConfig: vswitch::dpdk::memory_channels: "'0,2048'"
The nic configurations for network isolation is attached, which needs to be changed as per requirement. For our lab, nic2 was the provisioning network, which we also used for br-ex as there was no other external network. nic4 is our ovs-dpdk interface which is being used for tenant network. Most of the network isolation documentation is covered in here.
We will see specifically the changes in roles_data.yaml - as thats where all the magic starts (the whole of roles_data.yaml is attached, we address the relevant section only here)
- name: ComputeOvsDpdk CountDefault: 1 HostnameFormatDefault: '%stackname%-computeovsdpdk-%index%' ServicesDefault: - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephClient - OS::TripleO::Services::CephExternal - OS::TripleO::Services::Timezone - OS::TripleO::Services::Ntp - OS::TripleO::Services::Snmp - OS::TripleO::Services::NovaCompute - OS::TripleO::Services::NovaLibvirt - OS::TripleO::Services::Kernel - OS::TripleO::Services::ComputeNeutronCorePlugin - OS::TripleO::Services::ComputeNeutronOvsDpdkAgent - OS::TripleO::Services::ComputeCeilometerAgent - OS::TripleO::Services::ComputeNeutronL3Agent - OS::TripleO::Services::ComputeNeutronMetadataAgent - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::NeutronSriovAgent - OS::TripleO::Services::OpenDaylightOvs - OS::TripleO::Services::SensuClient - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::VipHosts
This role is named ComputeOvsDpdk, which has extra composable service OS::TripleO::Services::ComputeNeutronOvsDpdkAgent as the thing which adds ovs-dpdk to this compute node
ComputeNeutronOvsDpdkAgent is mapped to /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-ovs-dpdk-agent.yaml (heck the network-environment.yaml, under ResourceRegistry section).
Finally Deploy!
openstack overcloud deploy --templates \ --timeout 180 \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/templates/network-environment.yaml \ -e roles_data.yaml
Remember, ExtraConfigs Parameters are always send as a hiera data