Integration Testing in Infrahub: How to Validate Your Automation in Real Environments
Learn how to set up integration testing Infrahub using Docker and Testcontainers to confidently validate schema, data, and Git workflows.
Getting Started with the Pytest Plugin for Infrahub
Learn how to use the Pytest plugin for Infrahub to test GraphQL, Jinja2, and data transformations using simple YAML—no Python needed.
How to Manage Infrastructure Data With the Infrahub Python SDK
Learn how to create, update, delete, and upsert infrastructure data using the Infrahub Python SDK. Includes hands-on code examples
Querying Data in Infrahub via the Python SDK
Infrahub provides multiple ways to interact with your infrastructure data, including the Web GUI, GraphQL queries, and the Python SDK. These can be used to query, modify, create, or delete data in Infrahub. In this post, we’ll focus on using the Python SDK to query data from Infrahub.
This post assumes you are familiar with basic Python and Infrahub. If you’re new to these topics, don’t worry, you can still follow along.
Throughout this post, we’ll be using the Always-On Infrahub demo instance, which is available for anyone to access via this link. The demo instance already has some data in it, so if you’d like to follow along or try this yourself, you can use it without needing to set up anything.
Introduction
The Python SDK supports both synchronous and asynchronous Python. However, in this post, we’ll focus on using synchronous Python, which I hope most of us are comfortable with. We’ll cover async in a future blog post.
Interacting with Infrahub through the Python SDK is done using a client object, which defines the Infrahub instance you’ll be working with. This client acts as the connection point, allowing you to query, create, modify and delete resources within Infrahub.
You can install the Infrahub Python SDK with the pip package installer. It’s always a good idea to use a virtual environment to keep dependencies isolated.
python3 -m venv venv
source venv/bin/activate
pip install infrahub-sdkThe InfrahubClientSync class provides the synchronous version of the Infrahub client. This is what we’ll use to interact with Infrahub in this post.
from infrahub_sdk import InfrahubClientSync
client = InfrahubClientSync(address="https://demo.infrahub.app")To instantiate a client object, we pass the Infrahub server address. For authentication, you can use the environment variable INFRAHUB_API_TOKEN to pass the API key. The API key can be created in the Infrahub GUI by navigating to Account Settings > Tokens. Once generated, you can set the environment variable as shown below.
export INFRAHUB_API_TOKEN="TOKEN_HERE"Please note that if you’re only querying data from the Infrahub demo instance, you don’t need an API token. However, if you’re modifying, creating, or deleting resources, authentication is required, and you’ll need to provide a valid API token.
We can also instantiate a client using the Config object. Instead of passing the server address directly to InfrahubClientSync, we create a Config object and pass it to the client object. The API token can also be provided as part of the Config object using the api_token parameter.
from infrahub_sdk import InfrahubClientSync, Config
config = Config(
address="https://demo.infrahub.app",
api_token="182687c6-9445-8f39-dcb7-10658e5cfa49",
)
client = InfrahubClientSync(config=config)Similarly, authentication can also be done using a username and password, as shown below.
config = Config(username="admin", password="infrahub")Please note that for the purpose of this post, we are adding the credentials directly in the script, but in production, you should never hardcode API tokens or passwords in plain text.
Querying Data
Now that we know how to create a client object and authenticate, let’s move on to querying data from Infrahub. On the demo instance, you will find devices, interfaces, VLANs, BGP peers, and more. Since most of us are familiar with VLANs, let’s start by querying the list of VLANs in Infrahub.
We can query data in 3 ways using the SDK.
- Querying all the nodes of a given kind, using the
allmethod - Querying a single node of a given kind, based on some filters, using the
getmethod - Querying multiple nodes of a given kind, based on some filters, using the
filtersmethod
You can view all VLANs in the Infrahub by navigating to Network Configuration > VLAN. As shown in the screenshot, each site has two VLANs, and there are a total of five sites. Now, let’s see how we can retrieve the list of VLANs using the Python SDK.
Please note that the demo instance may change over time, so VLANs may appear under a different menu with a different schema. Please keep this in mind if you don’t see them in the exact location mentioned in this post.
Querying All Nodes
Let’s start by querying all the VLANs. This can be done by calling the all() method on the client object and passing the kind of the resource as an argument. You can find the ‘kind’ of a resource by inspecting the schema.
To view the schema, you can click on the ? icon at the top right of the page and then select “Schema” from the dropdown menu and this will open the schema explorer. In the schema, we can also see other Properties, Attributes, and Relationships, which we’ll cover later in this post.
To query all VLANs, we can use the following script.
from infrahub_sdk import InfrahubClientSync, Config
config = Config(
address="https://demo.infrahub.app",
api_token="182687c6-9445-8f39-dcb7-10658e5cfa49",
)
client = InfrahubClientSync(config=config)
all_vlans = client.all("InfraVLAN")
for vlan in all_vlans:
print(f"VLAN ID: {vlan.vlan_id.value}, Name: {vlan.name.value}")The script initializes a connection to the Infrahub demo instance using the InfrahubClientSync class as usual. We then use the all() method to retrieve all VLANs by specifying “InfraVLAN”, which is the kind associated with VLANs in the schema. The script then loops through the returned list of VLAN objects and prints their vlan_id and name attribute values.
If we check type(all_vlans[0]), we get infrahub_sdk.node.InfraVLANInfrahubNodeSync. This means the returned objects are Python objects constructed from the schema. In this case, each VLAN returned from the query is an instance of the InfraVLANInfrahubNodeSync class.
Querying a Single Node
In the previous example, we used the all() method to query all nodes of the same kind. Now, let’s see how we can query a single node of a kind. Using the same VLAN example, let’s try to query the VLAN named atl1_server.
To retrieve a single node, we use the get() method. The first argument is still the kind, but we also need to pass one or more filters to specify which node we want.
What Are Filters?
Filters allow us to search for specific attributes of a node as well as its relationships. For every attribute and relationship in a schema, a set of filters is automatically generated, allowing us to refine queries based on both attributes and related objects.
For every attribute in a schema, the following filters are automatically generated.
- ids: (list) Filters for a list of node ids
- hfid: Human-friendly Identifier of the specific node
- attribute__value: Filters for a single attribute value.
- attribute__values: (list) Filters for multiple attribute values.
- attribute__is_visible: (boolean) Filters for whether an attribute is visible.
- attribute__is_protected: (boolean) Filters for whether an attribute is protected.
- attribute__source__id: Filters for the source property of an attribute.
- attribute__owner__id: Filters for the owner property of an attribute.
- attribute__isnull: (boolean) Filters for attributes that have a null (empty) value.
For relationships, the following filters are generated.
- relationship__attribute__value
- relationship__attribute__values
- relationship__attribute__is_visible
- relationship__attribute__is_protected
- relationship__attribute__source__id
- relationship__attribute__owner__id
- relationship__ids
- relationship__isnull
To explain this, if a node has an attribute such as name, the corresponding filter would be name__value, where the word “attribute” in the filter format is replaced with the actual attribute name. Similarly, if you want to find nodes that do not have a description, you can use the filter description__isnull to retrieve only those with an empty value. (Assuming the node has an attribute called description)
Similarly, an InfraDevice has a relationship to platform, so if you want to filter devices based on the platform name, the filter would be platform__name__value. This follows the same pattern, where the relationship name is used in place of “relationship”, allowing you to query nodes based on their related objects.
You can also find the available filters for any given kind in the GraphQL Sandbox by opening the explorer, navigating to the kind, and expanding it.
For the InfraVLAN kind, we can inspect the schema to see the available attributes.
As shown in the schema, InfraVLAN has attributes such as name, VLAN ID, description, status, and role. If we want to query a VLAN by name, we use the name__value filter.
Here’s how we can retrieve the VLAN named atl1_server.
vlan = client.get("InfraVLAN", name__value='atl1_server')
vlan.vlan_id.value # Output: 200This retrieves the VLAN and allows us to access its attributes, such as the VLAN ID.
We can also use multiple filters by specifying multiple attributes in a query. For example, we can refine our VLAN search by filtering based on more than one attribute. This might not be necessary in this specific case since we can already query the VLAN using its name, but it’s useful to understand how filtering works with multiple conditions. Here’s an example where we query a VLAN using name, VLAN ID, and role.
vlan = client.get(
"InfraVLAN", name__value="atl1_server", vlan_id__value=200, role__value="server"
)You might have noticed that we use the term “query a single node of a kind.” But what does that mean? What happens if the query returns multiple nodes instead of just one?
We can easily test this by using the vlan_id attribute as a filter. Since we know there are multiple VLANs with VLAN ID 200 (even though they belong to different sites), let’s try to query it.
vlan = client.get("InfraVLAN", vlan_id__value=200)Here, we are trying to retrieve a VLAN with vlan_id = 200, but when we run the script, we get the following error.
IndexError: More than 1 node returnedThis confirms that the get() method expects to return only one node, but since multiple nodes match the query, it fails.
Querying Multiple Nodes
You can query Infrahub for multiple nodes of a particular kind by using the filters() method and using 1 or more filters. Previously, we saw that the get() method is limited to querying a single node and returns an error if multiple nodes match the filter. To query multiple nodes, we can use the filters() method.
vlans = client.filters("InfraVLAN", role__value="server")
for vlan in vlans:
print(vlan.name.value, vlan.vlan_id.value)atl1_server 200
den1_server 200
dfw1_server 200
jfk1_server 200
ord1_server 200In this example, we query all VLANs where the role is “server”. Unlike get(), which returns a single object, filters() returns a list of matching nodes. We then loop through the results and print the name and VLAN ID of each VLAN.
Similarly, we can use a relationship filter to find VLANs based on their associated Site. Since Site is a relationship of InfraVLAN, as seen in the schema, we can apply a filter to retrieve all VLANs within a specific site.
For example, if we want to find all VLANs in atl1, we can use the following query.
vlans = client.filters("InfraVLAN", site__name__value="atl1")
for vlan in vlans:
print(vlan.name.value, vlan.vlan_id.value)#output
atl1_management 400
atl1_server 200Attributes and Relationships
In this final section, let’s look at how the Infrahub SDK fetches the attributes and relationships associated with a node. So far, we have focused on querying VLANs, but now let’s switch to querying devices as an example.
devices = client.filters(
"InfraDevice", type__value="7280R3", status__value="provisioning"
)
for device in devices:
print(device.name.value)#output
den1-edge1In the script, we are querying for nodes of kind “InfraDevice” that match specific filters:
- type = “7280R3”
- status = “provisioning”
From the Web GUI, we can see that only one device matches these criteria – den1-edge1. Running the script confirms this, as the output returns only that device.
We can also print the name attribute of the device. Similarly, we can access other attributes, such as status and role. These attributes are defined in the schema for this specific kind, and you can view them in the Web GUI under the Schema section.
If we open this device in the Web GUI, we can also see other information such as platform, primary IP address, and interfaces. The question now is, can we access this information when querying the device through the SDK?
By default, the result of a query will include:
- Attributes
- Relationships of cardinality one
- Relationships of kind Attribute or Parent
Relationships that are included in a query will be automatically initialized with some information such as id, hfid or display_label. But the related node itself will not be included. So, let’s explore this in detail with a few examples.
If we test this by running the following query, it will return the display_label of the platform, so the output will be “Arista EOS”.
device = client.get("InfraDevice", name__value="den1-edge1")
device.platform.display_label#output
Arista EOSUsing the fetch() method
However, if you want to access the platform’s attributes, which were not fetched as part of the query, you need to fetch them explicitly.
device = client.get("InfraDevice", name__value="den1-edge1")
device.platform.fetch()
device.platform.peer.napalm_driver.value#output
eos
One important thing to note here is that device.platform does not refer to the platform itself but rather represents the relationship between the device and its platform. If you want to access the actual platform node and its attributes, you need to use the peer property.
device = client.get("InfraDevice", name__value="den1-edge1")
device.primary_address.fetch()
device.primary_address.peer.address.valueFetching More Relationships
In the previous section, we looked into fetching relationships of cardinality one and relationships of kind Attribute or Parent, but what about other relationships like interfaces, which have a cardinality of many and are of kind components?
Interfaces, for example, are not included in the query by default, but we can use the include argument to fetch the interfaces relationship.
To retrieve these interfaces, we need to add the interfaces relationship to the include argument when querying the device.
device = client.get("InfraDevice", name__value="den1-edge1", include=['interfaces'])
device.interfaces.fetch()
for interface in device.interfaces.peers:
print(f"{interface.peer.name.value} - {interface.peer.role.value}")#output
Ethernet1 - peer
Ethernet10 - spare
Ethernet11 - server
Ethernet12 - server
Ethernet2 - peer
Ethernet3 - backbone
Ethernet4 - backbone
Ethernet5 - upstream
Ethernet6 - upstream
Ethernet7 - spare
Ethernet8 - spare
Ethernet9 - peering
Loopback0 - loopback
Management0 - management
port-channel1 - serverprefetch_relationships
You can also use prefetch_relationships to fetch related nodes automatically when querying a device. This eliminates the need to use the fetch() method later, as the relationships are retrieved upfront. However, keep in mind that depending on what you are querying, this can result in a large amount of data being returned. When using fetch(), you have full control over which relationships are retrieved and when, allowing for more efficient queries when dealing with large datasets.
device = client.get(
"InfraDevice",
name__value="den1-edge1",
prefetch_relationships=True,
populate_store=True,
)
print(device.platform.peer.name.value)
print(device.primary_address.peer.address.value)#output
Arista EOS
172.16.0.19/16Closing Up
In this post, we explored how to query data from Infrahub using the Python SDK, covering single and multiple node queries, filtering, and retrieving related objects using fetch() and prefetch_relationships. We also looked at how relationships of different cardinalities affect query results. If you’re following along with the Infrahub sandbox, try running some queries yourself and feel free to reach out if you have any questions. You can find us on the OpsMill Discord server.
Simplifying Network Automation Workflows with Infrahub, Nornir, and Jinja2
In this blog post, we will explore how Infrahub integrates with Jinja2 and Nornir to simplify network automation workflows. To demonstrate, we’ll add two Arista devices to Infrahub, treating them as basic access switches. We’ll then input the necessary details for these devices to generate configurations. We’ll focus on creating VLAN and some interface configurations to keep it simple.
For each device, we’ll assign a primary IP (used for SSH), configure a few interfaces with descriptions, and specify an untagged VLAN for each interface. Additionally, we’ll define these VLANs globally in Infrahub (not tied to any specific device). A Jinja2 template will then use this information to generate configurations for each device. Finally, we’ll use the nornir-infrahub plugin as the inventory source and Napalm to push the generated configurations to each device.
Prerequisites
This blog post assumes you are somewhat familiar with Git and Docker. If you’re new to InfraHub, don’t worry, you should still be able to follow along. Make sure Git and Docker are installed on your local machine before getting started.
You’ll also need the following tools as we proceed.
- Infrahub – Of course, you need an Infrahub instance to follow along.
- Infrahubctl – This is the CLI tool used to interact with Infrahub.
- Infrahub Python SDK – This is required for programmatically creating and managing data in Infrahub.
Let’s install them using pip. To keep your environment clean, create a Python virtual environment to isolate the packages.
python -m venv venv
source venv/bin/activate
pip install 'infrahub-sdk[ctl]'Lastly, you’ll need to set an environment variable for the API key/token. Generate a token in Infrahub as shown in the screenshot and export it as an environment variable.
<bash]
export INFRAHUB_API_TOKEN="1803a5a3-8cf7-ec6b-35cb-c51a83c2a410"
[/bash]
This blog post is based on InfrahHub v1.1.0 and uses the following schemas from the Schema Library.
- schema-library/base/
- schema-library/extensions/vlan/
- schema-library/extensions/location_minimal/
To import the schemas into your Infrahub instance, first clone the schema-library GitHub repo.
git clone https://github.com/opsmill/schema-library.gitNext, import the schemas using infrahubctl CLI tool.
infrahubctl schema load /schema-library/base/
infrahubctl schema load /schema-library/extensions/vlan/
infrahubctl schema load /schema-library/extensions/location_minimal/The Components Needed
To get started with the example, let’s create some data in Infrahub. To keep it simple, we’ll focus on creating IP addresses, VLANs, devices, and interfaces. You can add this data in several ways, including the web GUI, GraphQL queries, or the Python SDK.
Let’s start by creating a location and three VLANs in Infrahub, each with an ID and a name using the web GUI.
First, create a location by navigating to the Location > Site and create a Site called ‘HQ’
Next, navigate to Layer 2 Domain and create a domain called ‘campus’.
Next, create the following three VLANs by navigating to Layer 2 Domain > VLAN in Infrahub. Here, you can input the VLAN ID, name, domain (select the domain you created earlier), and status.
If you prefer to create them using GraphQL queries, feel free to do so. Below is an example of the query for creating Layer 2 Domain and VLANs using GraphQL.
You can access the GraphQL sandbox by navigating to Admin > GraphQL Sandbox
mutation {
LocationSiteCreate(
data: {name: {value: "HQ"}, shortname:{value: "hq"}}
) {
ok
object {
id
}
}
}mutation {
IpamL2DomainCreate(
data: {name: {value: "campus"}}
) {
ok
object {
id
}
}
}mutation {
vlan10: IpamVLANCreate(
data: {
vlan_id: {value: 10},
status: {value: "active"},
name: {value: "finance"},
l2domain: {hfid: "campus"},
description: {value: "VLAN for Finance Users"},
role: {value: "user"}
}
) {
ok
object {
id
}
}
vlan20: IpamVLANCreate(
data: {
vlan_id: {value: 20},
status: {value: "active"},
name: {value: "sales"},
l2domain: {hfid: "campus"},
description: {value: "VLAN for Sales Users"},
role: {value: "user"}
}
) {
ok
object {
id
}
}
vlan30: IpamVLANCreate(
data: {
vlan_id: {value: 30},
status: {value: "active"},
name: {value: "admin"},
l2domain: {hfid: "campus"},
description: {value: "VLAN for Admin Users"},
role: {value: "user"}
}
) {
ok
object {
id
}
}
}Next, we’ll add two devices to Infrahub, named access-01 and access-02, and assign a primary IP to each device. Like any other device in Infrahub, these can be associated with a specific location, status, device type, platform, and more.
Before creating the devices, let’s first create the Manufacturer, Device Type, Platform and IP address. For the platform, we’ll specify eos as the Napalm driver. This will be used later in the blog post to demonstrate its significance and how it integrates with the workflow.
For creating IP addresses, first, navigate to IPAM > IP Prefixes and create a prefix with the 192.168.100.0/24 subnet. Once the prefix is created, go to IPAM > IP Addresses and add two IP addresses for our devices.
If you prefer to create them using GraphQL, here are the queries.
mutation {
OrganizationManufacturerCreate(
data: {name: {value: "Arista"}}
) {
ok
object {
id
}
}
}mutation {
DcimDeviceTypeCreate(
data: {name: {value: "Arista Switch"}, manufacturer: {hfid: "Arista"}}
) {
ok
object {
id
}
}
}mutation {
DcimPlatformCreate(
data: {name: {value: "eos"}, napalm_driver: {value: "eos"}}
) {
ok
object {
id
}
}
}mutation {
IpamPrefixCreate(
data: {status: {value: "active"}, prefix: {value: "192.168.100.0/24"}, member_type: {value: "address"}}
) {
ok
object {
id
}
}
}mutation {
ip_211: IpamIPAddressCreate(
data: {
description: {value: "access-01"},
address: {value: "192.168.100.211/32"}
}
) {
ok
object {
id
}
}
ip_212: IpamIPAddressCreate(
data: {
description: {value: "access-02"},
address: {value: "192.168.100.212/32"}
}
) {
ok
object {
id
}
}
}Finally, let’s create the two devices and add two interfaces for each device. Each device will be associated with the IP address we created earlier. This IP address will serve as the primary IP for the device and will be used for SSH to manage the device. When we use Nornir, this is the IP address it will rely on to connect to the device.
mutation {
access_01: DcimDeviceCreate(
data: {
name: {value: "access-01"},
platform: {hfid: "eos"},
location: {id: "18178eec-8379-21fd-311d-c51b6d37a6bf"},
device_type: {hfid: "Arista Switch"},
status: {value: "active"},
primary_address: {id: "181832cf-12e5-55de-311e-c516b3a8b16c"}
}
) {
ok
object {
id
}
}
access_02: DcimDeviceCreate(
data: {
name: {value: "access-02"},
platform: {hfid: "eos"},
location: {id: "18178eec-8379-21fd-311d-c51b6d37a6bf"},
device_type: {hfid: "Arista Switch"},
status: {value: "active"},
primary_address: {id: "181832cf-411c-4dae-3111-c515154e7409"}
}
) {
ok
object {
id
}
}
}Please note that the location and primary_address fields use the id for reference. You can retrieve the corresponding IDs from the web GUI and pass them as needed.
We will then add a couple of interfaces to each device, including details like descriptions, status, and associated VLANs.
mutation {
access_01_eth5: DcimInterfaceL2Create(
data: {
name: {value: "eth5"},
description: {value: "device-01"},
enabled: {value: true},
device: {hfid: "access-01"},
untagged_vlan: {hfid: "finance"},
speed: {value: 1000},
l2_mode: {value: "Access"},
status: {value: "active"}
}
) {
ok
object {
id
}
}
access_01_eth6: DcimInterfaceL2Create(
data: {
name: {value: "eth6"},
description: {value: "device-02"},
enabled: {value: true},
device: {hfid: "access-01"},
untagged_vlan: {hfid: "admin"},
speed: {value: 1000},
l2_mode: {value: "Access"},
status: {value: "active"}
}
) {
ok
object {
id
}
}
access_02_eth5: DcimInterfaceL2Create(
data: {
name: {value: "eth5"},
description: {value: "device-03"},
enabled: {value: true},
device: {hfid: "access-02"},
untagged_vlan: {hfid: "sales"},
speed: {value: 1000},
l2_mode: {value: "Access"},
status: {value: "active"}
}
) {
ok
object {
id
}
}
access_02_eth6: DcimInterfaceL2Create(
data: {
name: {value: "eth6"},
description: {value: "device-04"},
enabled: {value: true},
device: {hfid: "access-02"},
untagged_vlan: {hfid: "admin"},
speed: {value: 1000},
l2_mode: {value: "Access"},
status: {value: "active"}
}
) {
ok
object {
id
}
}
}Once we have all the data in place, the next step is to use it to generate device configurations. If you’re familiar with any form of network automation, you likely know that Jinja2 is one of the best tools for generating device configurations.
We now have all the data required to generate the configuration, such as VLANs, interfaces, descriptions, and more. The next step is to create a Jinja2 template that takes these values as inputs and generates the configuration. Additionally, we need to ensure the generated configurations are correctly associated with each device.
Infrahub provides a way to achieve this by using a Jinja2 template along with a GraphQL query to generate the configuration. The generated configuration is saved in Infrahub as an artifact, which can then be associated with the devices using an artifact definition. In the next sections, we’ll look at how to configure all of this.
Jinja2 Transformation and Artifact
So, how do we use Jinja2 with Infrahub? We use an Infrahub feature called ‘Transformation’. As the name suggests, this involves taking the data stored in Infrahub and converting it into a different format. In our case, we use a Jinja2 template to transform the data into a text file (rendered configuration).
As discussed previously, we also need a GraphQL query that fetches all the inputs required for the Jinja2 template. If you’re familiar with Jinja2, you might typically use a YAML or JSON file to store the data and then pass it to the template. In our case, this data is stored in Infrahub.
The final step involves defining an Artifact definition, which groups together a transformation with a target group, forming the artifact’s definition.
We can package all these components together (Jinja2 template, GraphQL query, and Artifact definition) alongside a .infrahub.yml file in a Git repository. This repository can then be added to Infrahub. The .infrahub.yml file enables Infrahub to identify the necessary imports and tie together the various components.
Please note that the Artifact definition, for example, can also be created via the web GUI or GraphQL query, but in this example, we use a Git repository.
Here are the contents of each file.
config.gql
query MyQuery($device: String!) {
IpamVLAN {
edges {
node {
vlan_id {
value
}
name {
value
}
}
}
}
DcimDevice(name__value: $device) {
edges {
node {
interfaces {
edges {
node {
name {
value
}
description {
value
}
... on DcimInterfaceL2 {
l2_mode {
value
}
untagged_vlan {
node {
name {
value
}
vlan_id {
value
}
}
}
}
}
}
}
}
}
}
}Here is the sample output from the query for the device access-01.
{
"data": {
"IpamVLAN": {
"edges": [
{
"node": {
"vlan_id": { "value": 30 },
"name": { "value": "admin" }
}
},
{
"node": {
"vlan_id": { "value": 10 },
"name": { "value": "finance" }
}
},
{
"node": {
"vlan_id": { "value": 20 },
"name": { "value": "sales" }
}
}
]
},
"DcimDevice": {
"edges": [
{
"node": {
"interfaces": {
"edges": [
{
"node": {
"name": { "value": "Eth5" },
"description": { "value": "new-description" },
"l2_mode": { "value": "Access" },
"untagged_vlan": {
"node": {
"name": { "value": "finance" },
"vlan_id": { "value": 10 }
}
}
}
},
{
"node": {
"name": { "value": "Eth6" },
"description": { "value": "device-02" },
"l2_mode": { "value": "Access" },
"untagged_vlan": {
"node": {
"name": { "value": "admin" },
"vlan_id": { "value": 30 }
}
}
}
}
]
}
}
}
]
}
}
}config.j2
!
{% for vlan in data['IpamVLAN']['edges'] %}
vlan {{ vlan['node']['vlan_id']['value'] }}
name {{ vlan['node']['name']['value'] }}
!
{% endfor %}
{% for edge in data['DcimDevice']['edges'][0]['node']['interfaces']['edges'] %}
interface {{ edge['node']['name']['value'] }}
description {{ edge['node']['description']['value'] }}
{% if edge['node']['l2_mode']['value'] == 'Access' %}
switchport mode access
switchport access vlan {{ edge['node']['untagged_vlan']['node']['vlan_id']['value'] }}
{% endif %}
!
{% endfor %}.infrahub.yml
- - -
jinja2_transforms:
- name: device_config
description: "VLAN and Interface configuration"
query: "config_query"
template_path: "config.j2"
queries:
- name: config_query
file_path: "config.gql"
artifact_definitions:
- name: "config_file"
artifact_name: "configuration file"
parameters:
device: "name__value"
content_type: "text/plain"
targets: "Transformation"
transformation: "device_config"Both jinja2_transforms and queries defined in this file are straightforward, so let’s focus on artifact_definitions. Each Artifact Definition in .infrahub.yml must include the following.
- name – the name of the Artifact Definition
- artifact_name – the name of the Artifact created by this Artifact Definition
- parameters – mapping of the input parameters required to render this Artifact
- content_type – the content-type of the created Artifact
- targets – the Infrahub Group to target when generating the Artifact
- transformation – the name of the Transformation to use when generating the Artifact
Here, we defined a group called ‘Transformation’ and added the two devices to this group. You can create the Group by navigating to Object Management > Groups. Once the group is created, you can add the two devices as members.
So, in the end, you’ll end up with three files in your repository – a Jinja2 template, a GraphQL query, and a .infrahub.yml file that ties everything together. Commit and push these changes to your remote repository, then add the repository to InfraHub.
├── config.gql
├── config.j2
└── .infrahub.ymlTo add this repository to Infrahub, navigate to Unified Storage > Repository and provide the Git remote repository link, login credentials (for example, if you use GitLab, create an access token and use it as the password for Infrahub), and a unique name for the repository. Infrahub will then connect to your remote repository and import the components defined within it—in our case, the Jinja2 template, GraphQL query, and artifact definition.
Once you add the repository to Infrahub and everything is set up correctly, you should see the artifact under the Artifact tab. If you open the artifact, you’ll find the generated configuration, as shown below.
You can also test your transformation by using infrahubctl render. When you use infrahubctl you need to pass the name of the transformation and any required variables. Here is an example, of using access-01 as the device.
infrahubctl render device_config device=access-01
!
vlan 30
name admin
!
vlan 40
name cctv
!
vlan 10
name finance
!
vlan 20
name sales
!
interface Eth5
description cctv_01
switchport mode access
switchport access vlan 40
!
interface Eth6
description device-02
switchport mode access
switchport access vlan 30
!Nornir-Infrahub Plugin
We’ve now completed about 75% of the process, with the remaining steps focusing on Nornir and how to use Nornir/Napalm to retrieve and apply these artifacts (configs). If you remember, our ultimate goal is to store all necessary information in Infrahub, generate the configurations, and push them to the devices. Infrahub will act as the inventory source for Nornir and also provide the artifacts.
First, you need to install the nornir-infrahub and nornir_napalm plugins. Use the following command to install it. As always, use a virtual environment for installing pip modules.
python3 -m venv venv
source venv/bin/activate
pip install nornir-infrahub
pip install nornir_napalmOnce installed, the following Nornir configuration file (config.yml) initializes Nornir with the Infrahub inventory plugin, fetching the required inventory and configuration details from Infrahub.
config.yml
- - -
inventory:
plugin: InfrahubInventory
options:
address: http://10.10.10.40:8000
token: 1811f38d-feb8-24da-2f6c-c51a2af588c8
host_node:
kind: DcimDevice
schema_mappings:
- name: hostname
mapping: primary_address.address
- name: platform
mapping: platform.napalm_driver
group_mappings:
- platform.name
group_file: groups.yml- We configure Nornir to use DcimDevice nodes from Infrahub as the host, which acts as the source of the device inventory. If you’re using a different schema in Infrahub, you need to specify the appropriate node type that represents a device in your setup. This ensures Nornir retrieves the correct inventory data based on your schema structure.
- We define schema mappings to enable Nornir to correctly interpret the data from Infrahub. For example, we map platform.napalm_driver to the platform field, ensuring that Nornir identifies the correct driver for each device.
- When using Napalm, the platform names must match the expected values (e.g., eos for Arista). However, if you’re using Netmiko, platform names might differ (e.g., arista_eos for Arista). These differences need to be accounted for when setting up Nornir.
- A groups.yml file is used to define group-specific attributes, such as the username and password for the eos platform. The plugin automatically creates groups based on the group_mappings specified in the configuration. Here, we use platform.name, so Nornir creates a group for each host based on the value of platform.name. In this case, it creates a group named platform__eos, with platform__ prefixed to the platform name.
- You can then define attributes for this group in the groups.yml file, such as credentials or other platform-specific settings. If you’re using a different schema, ensure your group mappings align with your schema structure to reflect the appropriate group names.
groups.yml
platform__eos:
username: admin
password: adminWith the prerequisites out of the way, we can now move on to pushing the generated configurations to the devices.
In the following Python script (main.py), get_artifact function retrieves the artifact (in this case, the rendered configuration) associated with each device from Infrahub. It uses the Infrahub API to fetch the artifact’s content, which is then stored in the Nornir’s result object.
main.py
from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir_napalm.plugins.tasks import napalm_configure
from nornir_infrahub.plugins.tasks.artifact import get_artifact
def main(task):
# Fetch artifacts from Infrahub
artifacts = task.run(task=get_artifact, artifact="config_file")
# Configure devices using Napalm with the fetched artifacts
task.run(task=napalm_configure, configuration=artifacts[0].result, dry_run=False)
if __name__ == "__main__":
nr = InitNornir(config_file="config.yml")
results = nr.run(task=main)
print_result(results)Once the artifacts are successfully fetched, the next step is to apply these configurations to the devices using the napalm_configure task. This task takes the fetched configuration (artifacts[0].result) and pushes it to the target devices.
So, just to recap, our directory structure will look like this – main.py is our Python script, config.yml specifies that Nornir should use the InfraHub inventory and defines how it interacts with it, and groups.yml is used to provide the login credentials.
.
├── groups.yml
├── main.py
├── config.ymlLet’s Make Some Changes
To demonstrate how changes are made and pushed to devices using Nornir, let’s create a new VLAN, assign it to one of the interfaces, and update the interface description.
The process begins by creating a branch in Infrahub. This branch allows us to isolate and manage the changes without affecting the main configuration. You can create a new branch (called vlan_40) in Infrahub GUI by clicking the ‘+’ button as shown below.
Once the branch is created, select the branch to work on and create a new VLAN.
After the VLAN is created, choose an interface (e.g., access-01, Eth5 in this example) and update its configuration to use the newly created VLAN 40. Additionally, update the interface description to reflect the changes. This ensures that both the VLAN assignment and description are consistent with the new configuration requirements.
Once you’ve made the changes, navigate to Change Control > Branches and select the branch you just created. Under the Data tab, you can view exactly what has been modified.
You can expand each field to see the specific changes in detail. For example, you’ll notice that a new VLAN was added (highlighted in green), and the description and untagged VLAN for the interface was updated (highlighted in blue).
At this point, you have the option to merge your changes directly. However, Infrahub offers a more robust way to manage changes using a feature called Proposed Changes. Let’s explore how to use this feature.
Infrahub Proposed Changes
Infrahub’s Proposed Changes feature takes automation a step further. Instead of merging directly from Change Control > Branches, you can create a Proposed Change. In this process, you provide a name, description, the person raising the change, and the source and destination branches.
Once the proposed change is created, navigate to Proposed Change and select the change you just raised. Here, you’ll find multiple tabs:
- Overview: Provides a general overview of the change, as the name suggests. You can also add comments here.
- Data: Similar to what you’ve seen before, this tab shows exactly what was modified.
- Artifact: This is the most important tab for this post. Infrahub detects the changes, renders a new configuration, and highlights exactly what has been updated.
If the changes look good, the reviewer can approve the proposal. As soon as the change is approved and merged, the artifact gets re-generated automatically.
Now you can use Nornir to run the job again, and the updated configuration will be pushed to the devices seamlessly. This workflow ensures changes are tracked, reviewed, and implemented efficiently.
Here are the outputs from Nornir showing what is being changed on the devices.
access-01
-- napalm_configure ** changed : True -------------- INFO
+vlan 40
+ name cctv
!
interface Ethernet5
- description device-01
+ description cctv_01
- switchport access vlan 10
+ switchport access vlan 40access-02
-- napalm_configure ** changed : True -------------- INFO
+vlan 40
+ name cctvIf we SSH into access-01, we can confirm that the changes have taken effect. The new VLAN 40 is present, and interface Eth5 is now using this VLAN with the updated description, as expected.
access-01#show run interfaces eth5
interface Ethernet5
description cctv_01
switchport access vlan 40
access-01#
access-01#show vlan
VLAN Name Status Ports
-- ---------- --- ------
1 default active Et1, Et2
10 finance active
20 sales active
30 admin active Et6
40 cctv active Et5Closing Up
Just to keep this post simple, we only covered the basics, but you can absolutely manage every aspect of the configuration, such as SNMP servers, NTP, trunk ports, uplinks, port channels, and more. All you need to do is input the relevant data into Infrahub, update your Jinja2 template and GraphQL query, and let Infrahub and Nornir handle the rest.
