OpenStack Heat is the orchestration project of OpenStack, which allows to describe and provision OpenStack resources based on Heat templates, similar to Amazon CloudFormation. For a research project, I recently had the requirement to assign a floating IP address to every instance of a Heat autoscaling group (ASG). Given the fact that such setup is contradictory to the common practice having a load balancer in front of an ASG, this costed me some time to figure out.

Creating a Single Instance Using Heat

Setting up a single instance using a Heat template (HOT) is pretty easy, straightforward and also covered in the Heat templates repository:

# single_server_with_floating_ip.yaml
# (abbreviated example based on hot/servers_in_existing_neutron_net.yaml)

parameters:
  # ~~~snip~~~
  network:
    type: string
    description: The network for the VM
    constraints:
      - {custom_constraint: neutron.network}
  public_net:
    type: string
    description: ID of public network for which floating IP addresses will be allocated
    constraints:
      - {custom_constraint: neutron.network}

resources:
  server1:
    type: OS::Nova::Server
    properties:
      name: Server1
      image: { get_param: image }
      networks:
        - port: { get_resource: server1_port }
      
  server1_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: network }
      security_groups: [{ get_resource: server_security_group }]

  server1_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net }
      port_id: { get_resource: server1_port }

As we can see, the server1_floating_ip (of type OS::Neutron::FloatingIP) is NAT’ed to the server1_port which in turn is assigned to the server instance (type OS::Nova::Server) that we instantiate.

Creating an Autoscaling Group

In order to create an ASG of multiple instances, each only with a private IP, we also find help in the Heat templates repository:

# abbreviated example based on hot/asg_of_servers.yaml

parameters:
  # ~~~snip~~~
  image:
    type: string
    description: Name or ID of the image to use for the instances.
  network:
    type: string
    description: The network for the VM
    default: private

resources:
  asg:
    type: OS::Heat::AutoScalingGroup
    properties:
      resource:
        type: OS::Nova::Server
        properties:
            key_name: { get_param: key_name }
            image: { get_param: image }
            flavor: { get_param: flavor }
            networks: [{network: {get_param: network} }]
      min_size: 1
      desired_capacity: 3
      max_size: 10

We see the AutoScalingGroup defined with the Nova server instance being scaled up and down (between 1 and 10 instances). Using the scale up and down URLs provided, one can test the functionality of auto scaling.

Recalling our goal, namely to assign a floating IP address to every instance of the ASG (and also to the ones created during scale-out), it looks intuitively right to extend the resource block of the previous template. However, only one resource can be given to the AutoScalingGroup. For our use case, we also need the floating IP resources to be added and destroyed on demand.

Scaling a Complete Stack

The solution to my problem was finally hidden in one of the other examples, the ASG of stacks. Essentially, Heat can also scale a complete stack (defined by its own template file). So what we now do is essentially to treat the server with its floating IP as defined in the first listing (single_server_with_floating_ip.yaml) as entity of scale and pass through all parameters:

# asg_of_server_with_floating.yaml
# (abbreviated example based on hot/asg_of_stacks.yaml)
parameters:
  # ~~~snip~~~
  image:
    type: string
    description: Name or ID of the image to use for the instances.
  network:
    type: string
    description: The network for the VM
    constraints:
      - {custom_constraint: neutron.network}
  public_net:
    type: string
    description: ID of public network for which floating IP addresses will be allocated
    constraints:
      - {custom_constraint: neutron.network}
resources:
  asg:
    type: OS::Heat::AutoScalingGroup
    properties:
      resource:
        # this refers to the file previously described
        type: server_with_floating.yaml
        properties:
            image: { get_param: image }
            network: { get_param: network }
            public_net: { get_param: public_net}
      min_size: 1
      desired_capacity: 3
      max_size: 10

As the type of of resource, we can also supply a file name. That’s.. let me think.. not obvious? After long search, was able to find this behavior documented under Template composition.

Mind the public_net parameter defining the public network ID, from which floating IPs are assigned. Once we trigger a scale-out, we can see antoehr instance including a floating IP being added after some seconds:

I am not aware of a way to launch such stack using the Horizon dashboard, except when the inner template is referenced via an HTTP(S) URL. Instead, one can launch it using the command line client:

$ openstack stack create my-test-stack --template asg_of_server_with_floating.yaml -e environment.yaml

All definition of the required parameters, i.e., image and network IDs, are happening in the environment.yaml file.

While my use case was a little academic (why in the world should something scale inside an OpenStack cloud and be directly reachable from the outside world), I think this (to me still unknown) concept of template composition helps solving many more problems.

The complete code can be found in this gist.
(cover image by sipa on pixabay.com)