In this part of our series on generators and services, we’re diving into the practical aspects of setting up a generator. In our last video, we covered the planning phase, including identifying use cases, defining deliverables, and mapping workflows. This time, we’ll focus on the actual coding and configuration needed in InfraHub to get our generator up and running.

Setting the Generator Up

To start, we need to define our generator within the .infrahub.yaml file. This is crucial as it outlines how the different components come together. Our definition file will specify the locations of the Python code and the GraphQL query required to pull the data we need for our generator to function.

Walking through our generator definition:

• name: We set a name for our generator, like site_generator.
• file_path: This points to where our generator code resides.
• targets: We define our targets, which will be set up in a future video.
• query: We specify the query name, which is used further in the file, and the path to the query file.
• class_name: We define a name for the class to use in the generator

And then a similar setup for the query:

• name: The name we use to point at the query
• file_path: This points to where our query file resides.

-

generator_definitions:

  - name: site_generator

    file_path: "generators/site_generator.py"

    targets: "generator_sites"

    query: "site_details"

    class_name: "SiteGenerator"

queries:

  - name: site_details

    file_path: "queries/site_details.gql"

GraphQL Query

The GraphQL query will filter out the necessary information for our generator. We will focus on returning the following variable values for our site:

• edges > node > name: This will return the name of the site we are targeting.
• edges > node > site_prefix: This return the site prefix for device and interface names.
• edges > node > homing: Whether the site is single or dual-homed.

query site_details($site_name: String!) {

  LocationSite(name__value: $site_name) {

    edges {

      node {

        name {

          value

        }

        site_prefix {

          value

        }

        homing {

          value

        }

      }

    }

  }

}

Generator File

This is where the magic happens.  The generator is coded with Python and uses the Infrahub SDK to bring the logic to life.  I’ll use comments within the Python code to walk through the flow:

import logging

from infrahub_sdk.generator import InfrahubGenerator

# Configure logging

logging.basicConfig(level=logging.INFO)

logger = logging.getLogger(__name__)

# Suppress httpx logs for ease of viewing on the console

httpx_logger = logging.getLogger("httpx")

httpx_logger.setLevel(logging.WARNING)

# Call our SiteGenerator as defined in our .infrahub.yml file

class SiteGenerator(InfrahubGenerator):

##### Receive the output of the GraphQL query

    async def generate(self, data: dict) -> None:

        logger.info("Received data for processing.")

        if not data["LocationSite"]["edges"]:

            logger.warning("No sites found in query result.")

            return

######### Extract site details we need to build our devices

        site = data["LocationSite"]["edges"][0]["node"]

        site_name = site["name"]["value"]

        site_prefix = site["site_prefix"]["value"]

        homing = site["homing"]["value"]

        logger.info(f"Processing site: {site_name} with prefix: {site_prefix}, Homing: {homing}")

######### Fetch the Management IP Pool

        management_pool = await self.client.get(

            kind="CoreIPAddressPool",

            any__value="Management Pool",

            raise_when_missing=False

        )

        if not management_pool:

            logger.error("Management Pool not found. Ensure it is created in the GUI.")

            return

        logger.info(f"Using IPAM Pool: {management_pool}")

######### Define device configuration based on homing type

        device_config = {

            "single_homed": [

                {"name": f"{site_prefix}-router-01", "role": "core", "type": "router"},

                {"name": f"{site_prefix}-firewall-01", "role": "firewall", "type": "firewall"},

                {"name": f"{site_prefix}-switch-01", "role": "end_user_switch", "type": "switch"},

            ],

            "dual_homed": [

                {"name": f"{site_prefix}-router-01", "role": "core", "type": "router"},

                {"name": f"{site_prefix}-router-02", "role": "core", "type": "router"},

                {"name": f"{site_prefix}-firewall-01", "role": "firewall", "type": "firewall"},

                {"name": f"{site_prefix}-firewall-02", "role": "firewall", "type": "firewall"},

                {"name": f"{site_prefix}-switch-01", "role": "end_user_switch", "type": "switch"},

            ],

        }

        devices = device_config.get(homing, [])

######### Begin creating the objects for our service of a new site

        for device in devices:

            device_name = device["name"]

            device_role = device["role"]

            device_type = device["type"]

############# Create device

            logger.info(f"Creating device: {device_name} ({device_role})")

            device_obj = await self.client.create(

                kind="InfraDevice",

                data={

                    "name": device_name,

                    "role": device_role,

                    "type": device_type,

                    "status": "active",

                    "site": site_name,

                },

            )

            await device_obj.save(allow_upsert=True)

############# Create interfaces for device

            for iface_num in range(1, 3):

                interface_name = f"{device_name}-iface{iface_num}"

                logger.info(f"Adding interface: {interface_name}")

                interface = await self.client.create(

                    kind="InfraInterfaceL3",

                    data={

                        "name": interface_name,

                        "device": device_obj,

                        "speed": 1000,

                        "status": "active",

                        "role": "management" if iface_num == 1 else "uplink",

                    },

                )

                await interface.save(allow_upsert=True)

############# Obtain and associate IP address for management interfaces

                ip_address = await self.client.allocate_next_ip_address(

                    resource_pool=management_pool,

                    identifier=f"{interface_name}-ip",

                    data={"description": f"IP for {interface_name}"}

                )

                ip_address.interface = interface

                await ip_address.save(allow_upsert=True)

                logger.info(f"Allocated IP: {ip_address.address.value} and linked to {interface_name}")

        logger.info("Device and interface creation completed.")

Let’s Run It!

Here we can see the output from the console of running the generator for site Chicago

infrahubctl generator --branch "generator-test" site_generator site_name=Chicago

And now we can see our devices from inside the Infrahub UI.

With the generator functioning as expected, we now look towards operationalizing it. This means integrating the generator into the application for easier consumption by users.

Conclusion

With our next installment in the series, we will implement a way to control when generators run and against which sites. This will allow for more flexibility and prevent unnecessary executions against all sites.

We will also be bringing the generator, and all of the required files, into a git repository so we can operationalize its usage.  We’ll also look at how to leverage our proposed changes feature to help teams even more.

To learn more about Generators today, check out the following links to our documentation:

• Documentation > Topics > Generator
• Documentation > Guides > Creating a Generator
• Documenation > infrahubctl > infrahubctl generator

In this second part of our series on generators and services, we delve deeper into the practical aspects of creating a generator that streamlines the delivery of new sites. This blog aims to provide a comprehensive overview of the process, focusing on the use case, design requirements, workflow, and a live demonstration of the generator in action. If you missed our first article and video in this series, be sure you check those out.

Recap of Generators

Generators are generic plugins that query data and create new nodes and relationships based on the results. They can play a crucial role in automating the creation of new objects, services, and other elements within infrastructure management and deployment frameworks. The logic behind these generators determines how effectively they can automate processes.

Defining the Generator Use Case

For our generator, the primary use case is to automate the delivery of new sites. This involves creating a consistent and repeatable process for setting up new infrastructure. Our deliverables will include the creation of devices within Infrahub and the artifacts necessary for launching new sites. We’ll also record our design requirements and map out the general generator workflow.

Design Requirements

Before diving into the workflow, we’ll clarify our design requirements. This includes decisions about hardware configurations, naming conventions, and the relationships between devices.

Hardware Configuration

We will implement two configurations:

• Single-Homed Configuration: This setup utilizes a single router, firewall, and switch, suitable for less critical applications and sites.
• Dual-Homed Configuration: For more critical infrastructures, we will set up two routers, two firewalls, and a single switch to ensure redundancy and fault tolerance.

Naming Conventions

• (site)-(device)-(incrementor)
  • (site) – will be a shortname for any given site
  • (device) – will be the type of device
  • (incrementor) – will be incremented for each device at the site

Relationships

• We will keep this simple and just map our devices to the sites

Map the Workflow

In our next article we’ll dive into the actual code that runs our generator. But for now we’ve outlined, in psuedocode, what this workflow should look like.

function generate(site):

validate site data

if homing == "single_homed":

create 1 router, 1 firewall, 1 switch

if else homing == "dual_homed":

create 2 routers, 2 firewalls, 1 switch

link devices to sites

return success

Running the Generator

Generators can be ran one of two ways: the command line or the UI. When developing a generator it’s best to run it from the command line. Using the UI works better once a generator has been completed and is ready to be used in production. While we only show one example below, we actually run this twice for the demo in the video.

Below we can see the generator running and creating our devices for the Chicago site. To help us track the generator running in the development stage, additional code has been added to output information to the console.

Once the generator has run successfully, we will see the 2 routers, 2 firewalls, and switch have been generated for the Chicago site.

Conclusion

In this article, and video, we’ve continued to expand our understanding on how a generator can start to help us deliver our services in a repeatable and reliable fashion. The continuing articles in the series will further enhance the delivery of our service of a fully functional site through our generator.

Stay tuned as we continue on this path in crafting our generator tailored to specific needs, enhancing our capabilities through Infrahub.

To learn more about Generators today, check out the following links to our documentation:

• Documentation > Topics > Generator
• Documentation > Guides > Creating a Generator
• Documenation > infrahubctl > infrahubctl generator

This article kicks off a multipart series where we’re diving into the concept of generators and services, exploring their significance, structure, and how they can streamline processes for teams. Whether you’re a developer or just curious about automation in IT, this guide will provide you with a comprehensive understanding of generators and their applications.

What is a Generator?

A generator, in the context of Infrahub, is defined as a generic plugin that queries data and creates new nodes and relationships based on the results. This definition encompasses several key components that work together seamlessly to facilitate the automation of repeatable tasks. Link to the Generator topic in our docs

To visualize a generator, think of it as a combination of various elements: targeting the intended group of objects, the logic that powers it, and the GraphQL query that bridges everything. This combination of components allows teams to automate the generation of service deliverable objects efficiently and effectively.

Why Use Generators?

Generators are particularly beneficial for teams that have repeatable tasks that require consistency and efficiency. These tasks can range from deploying new VLANs to creating infrastructures for new locations, such as remote offices of a healthcare organization or new construction of franchise buildings.

In scenarios where organizations are scaling operations—like a hyperscaler creating new data centers or a managed service provider deploying numerous applications—generators offer a solution to streamline these processes. By inputting new variables, teams can generate a fresh version of the task each time, saving time and reducing errors.

While these scenarios are hardly an inclusive list, they do serve as a point to kickstart the thought process. To ask the question “What services does my team deliver?”

Example Generators in Action

While this is best illustrated in the videos, we can briefly discuss the two provided examples.

Data Center Fabric Example

In our first example, we created a branch called generator demo to work on establishing a new data center fabric. The process began with adding fabric for a new location, in this case, Chicago. We set up the spines and leaf groups, specifying the number of Layer Two and Layer Three leafs required.

Setting up the fabric parameters

Next, we designated our target group within Infrahub, ensuring that our generator was aligned with the right site. After initiating the generator, we observed the tasks in progress, with successful outputs confirming the creation of our leaves and spines as per the defined parameters.

The generated objects

New Site Example

In our second example, we focused on creating a new site, specifically for Grand Rapids. This time, we selected a design type and specified the hardware—a Cisco 24 Port switch. The site was categorized under the automated sites group to ensure proper management and organization.

Setting up the new site parameters

Similar to the previous generator, once started the logic was processed which led to the successful creation of the new design, with the requested equipment. As we refreshed the data, we noted the interfaces, IP address consumption, and configurations that were automatically generated, showcasing the efficiency of the generator in preparing for a new site to come online.

The generated objects

Artifacts and Their Importance

As we explored the generators, we also encountered the concept of artifacts. Artifacts are essential as they represent the actual configurations needed for devices, such as boot-up configurations. They play a critical role in implementing changes in existing environments or introducing new services.

By utilizing artifacts, teams can ensure that the generated infrastructure aligns with operational requirements, making the automation process not only efficient but also reliable.

Looking Ahead: Building Our Own Generator

Generators are a powerful tool in the Infrahub ecosystem, enabling teams to automate repetitive tasks effectively. By understanding their components, applications, and the importance of artifacts, you can leverage generators to enhance productivity and streamline operations.

In the upcoming segments of this series, we will take a step further by creating our own generator. This endeavor will involve a design phase where we outline the generator’s functionality before diving into the coding aspect. Our goal is to simplify the process while ensuring clarity and understanding.

Stay tuned as we embark on this journey to craft a generator tailored to specific needs, enhancing our capabilities through Infrahub.

To learn more about Generators today, check out the following links to our documentation:

• Documentation > Topics > Generator
• Documentation > Guides > Creating a Generator
• Documenation > infrahubctl > infrahubctl generator