OpenStack Orchestration project Heat implements an orchestration engine to launch multiple composite cloud applications based on templates (1). A Heat template describes infrastructure resources (servers, networks, floating ips, etc) and the relationships between these resources, allowing Heat to deploy the resources in a correct order and to manage whole infrastructure lifecycle.
Today Heat provides compatibility with the AWS CloudFormation template format and has its own, native format called Heat Orchestration Template (HOT)(2).
In this blog-post I will talk about Flame, a tool that generates HOT Heat template from already existing infrastructure. Currently this project is developed by Thomas Herve (Heat core developer) and myself and provides support for Nova (key pairs and servers), Cinder (volumes) and Neutron (router, networks, subnets, security groups and floating IPs) resources.
Flame works as follows: using provided credentials (user name, project name, password, authentication url), the tool will list supported resources deployed in the project and will generate corresponding, highly customized HOT template.
Get the source code :
Flame uses the OpenStack clients to connect to the services and retrieve all information about resources that are deployed in the project.
Install all the libraries required to run Flame:
1usage: flame.py [-h] [--username USERNAME] [--password PASSWORD] 2 [--project PROJECT] [--auth_url AUTH_URL] [--insecure] 3 [--exclude_servers] [--exclude_volumes] 4 5Generate Heat Template 6 7optional arguments: 8 -h, --help show this help message and exit 9 --username USERNAME A user name with access to the project. Defaults to 10 env[OS_USERNAME] 11 --password PASSWORD The user's password. Defaults to env[OS_PASSWORD] 12 --project PROJECT Name of project. Defaults to env[OS_TENANT_NAME] 13 --auth_url AUTH_URL Authentication URL. Defaults to env[OS_AUTH_URL]. 14 --insecure Explicitly allow clients to perform"insecure" SSL 15 (https) requests. The server's certificate will not be 16 verified against any certificate authorities. This 17 option should be used with caution. 18 --exclude_servers Do not export in template server resources 19 --exclude_volumes Do not export in template volume resources
There are two possible option that can be used with Flame:
exclude_volumes. The first option makes Flame ignore all the instances deployed in project i.e. generated template will not contain any reference to them. Similarly, the second option will ignore all existing volume resources. If the two options are used, resulting Heat template will contain only network resources.
To use Flame you can provide yours OpenStack credentials as arguments :
1$ python flame.py --username user --password password --project project --auth_url https://identity.fr1.cloudwatt.com/v2.0
Or you can source your OpenStack RC file and use Flame without arguments:
The easiest way to understand how Flame works is to show some examples. So here they are.
Suppose that in your project you deployed a router, a network with corresponding subnet and an instance that is booted from an image. A public key was imported in the project and used to access the instance, a floating IP was associated with the instance and the default security group was modified in order to allow incoming ssh connection.
For this infrastructure, Flame will generate a template that will look like this:
1description: Generated template 2heat_template_version: 2013-05-23 3parameters: 4 external_network_for_floating_ip_0: 5 constraints: 6 - custom_constraint: neutron.network 7 description: Network to allocate floating IP from 8 type: string 9 flavor_server_0: 10 default: m1.small 11 description: Flavor to use for instance my_instance 12 type: string 13 image_server_0: 14 description: Image to use to boot instance my_instance 15 type: string 16 router_0_external_network: 17 constraints: 18 - custom_constraint: neutron.network 19 description: Router external network 20 type: string 21resources: 22 floatingip_0: 23 properties: 24 floating_network_id: 25 get_param: external_network_for_floating_ip_0 26 type: OS::Neutron::FloatingIP 27 key_0: 28 properties: 29 name: arezmerita 30 public_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8u8FIZjmhO+hM/f+2J9qYKgJPG16pQmBfQeUvFlC5u9xxf57eGKuq7xYMIoW63gGM8dnsXcQp9Lmp/+TacwPkis5Q8LKriJSxZUgwczM2ppwwJ/SOraRDHy+2bgbrrO2ZYNdoD5zBaiC5jh6YemrB+y5TtkiEo+llNZw+6e5TlZxEEGD4Zgid/Tfz4qwkKvoGwx34ltQ+XvT2Tv6kE7JWc8rR37wkCbLVQd3G3vAJFI3bWrYan3XNP5+wsVydWn3APF2l8FtLkSpE5Fkai7OWACPRZ9zNlQSBk6pRNlxfZ8jQL6Kuk3MU2tTrqw5g/jG7Hlu3vCeDIYOiFI2a8GUX 31 type: OS::Nova::KeyPair 32 network_0: 33 properties: 34 admin_state_up: true 35 name: network 36 shared: false 37 type: OS::Neutron::Net 38 network_subnet_0: 39 properties: 40 allocation_pools: 41 - end: 10.0.48.254 42 start: 10.0.48.242 43 cidr: 10.0.48.240/28 44 dns_nameservers:  45 enable_dhcp: true 46 host_routes:  47 ip_version: 4 48 name: network_subnet 49 network_id: 50 get_resource: network_0 51 type: OS::Neutron::Subnet 52 newdefault_0: 53 properties: 54 description: default 55 name: newdefault 56 rules: 57 - direction: egress 58 ethertype: IPv6 59 - direction: ingress 60 ethertype: IPv6 61 remote_mode: remote_group_id 62 - direction: ingress 63 ethertype: IPv4 64 port_range_max: 22 65 port_range_min: 22 66 protocol: tcp 67 remote_ip_prefix: 0.0.0.0/0 68 - direction: egress 69 ethertype: IPv4 70 - direction: ingress 71 ethertype: IPv4 72 remote_mode: remote_group_id 73 type: OS::Neutron::SecurityGroup 74 router_0: 75 properties: 76 admin_state_up: true 77 name: router 78 type: OS::Neutron::Router 79 router_0_gateway: 80 properties: 81 network_id: 82 get_param: router_0_external_network 83 router_id: 84 get_resource: router_0 85 type: OS::Neutron::RouterGateway 86 router_0_interface_0: 87 properties: 88 router_id: 89 get_resource: router_0 90 subnet_id: 91 get_resource: network_subnet_0 92 type: OS::Neutron::RouterInterface 93 server_0: 94 properties: 95 block_device_mapping:  96 config_drive: '' 97 diskConfig: AUTO 98 flavor: 99 get_param: flavor_server_0 100 image: 101 get_param: image_server_0 102 key_name: 103 get_resource: key_0 104 name: my_instance 105 networks: 106 - network: 107 get_resource: network_0 108 security_groups: 109 - get_resource: newdefault_0 110 type: OS::Nova::Server
It’s not so easy to write a template like this manually, right?
There are two major sections in this generated template: parameters and resources. The parameters section is used to customize each deployment, by specifying input parameters for template instantiation. In the resources section, are defined actual resources that will compose a Heat stack deployed from the HOT template.
In order to understand this generated template and to be able to modify it, I will explain here for each resource type, the possible parameters and its relationship with other resources.
Each resource declaration block is headed by the resource ID: floatingip_0 for this resource. Every resource ID must be unique within the resource section. Type attribute specifies the type of resource: OS::Neutron::FloatingIP in this case.
Since a floating IP is allocated from a network, we must indicate the network id. To be fully customisable, instead of hard coding a network id, we add a parameter external_network_for_floating_ip_0, that will be provided by the user during Heat stack creation.
1 key_0: 2 properties: 3 name: arezmerita 4 public_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8u8FIZjmhO+hM/f+2J9qYKgJPG16pQmBfQeUvFlC5u9xxf57eGKuq7xYMIoW63gGM8dnsXcQp9Lmp/+TacwPkis5Q8LKriJSxZUgwczM2ppwwJ/SOraRDHy+2bgbrrO2ZYNdoD5zBaiC5jh6YemrB+y5TtkiEo+llNZw+6e5TlZxEEGD4Zgid/Tfz4qwkKvoGwx34ltQ+XvT2Tv6kE7JWc8rR37wkCbLVQd3G3vAJFI3bWrYan3XNP5+wsVydWn3APF2l8FtLkSpE5Fkai7OWACPRZ9zNlQSBk6pRNlxfZ8jQL6Kuk3MU2tTrqw5g/jG7Hlu3vCeDIYOiFI2a8GUX 5 type: OS::Nova::KeyPair
Like floating IP resource, this key pair resource has an ID and a type. In addition, the name and the value of the public key are specified.
1 network_0: 2 properties: 3 admin_state_up: true 4 name: network 5 shared: false 6 type: OS::Neutron::Net 7 network_subnet_0: 8 properties: 9 allocation_pools: 10 - end: 10.0.48.254 11 start: 10.0.48.242 12 cidr: 10.0.48.240/28 13 dns_nameservers:  14 enable_dhcp: true 15 host_routes:  16 ip_version: 4 17 name: network_subnet 18 network_id: 19 get_resource: network_0 20 type: OS::Neutron::Subnet
Declaration of the network resource does not differ much from from two previous resources. However, the subnet resource that belongs to the network_0 resource requires its network ID. Since we do not know it, we will just reference network_0 resource using get_resource: network_0. At runtime, this reference will be resolved to reference ID of the network resource.
1 router_0: 2 properties: 3 admin_state_up: true 4 name: router 5 type: OS::Neutron::Router 6 router_0_gateway: 7 properties: 8 network_id: 9 get_param: router_0_external_network 10 router_id: 11 get_resource: router_0 12 type: OS::Neutron::RouterGateway 13 router_0_interface_0: 14 properties: 15 router_id: 16 get_resource: router_0 17 subnet_id: 18 get_resource: network_subnet_0 19 type: OS::Neutron::RouterInterface
These three resources are closely related. The router_0 resource declares a router. The router_0_gateway declares external network gateway for this router and expects a parameter from the user ( get_param: router_0_external_network ), that corresponds to the ID of the external network for the gateway.
The router_0_interface_0 resource declares an internal network interface to the router_0.
1 newdefault_0: 2 properties: 3 description: default 4 name: newdefault 5 rules: 6 - direction: egress 7 ethertype: IPv6 8 - direction: ingress 9 ethertype: IPv6 10 remote_mode: remote_group_id 11 - direction: ingress 12 ethertype: IPv4 13 port_range_max: 22 14 port_range_min: 22 15 protocol: tcp 16 remote_ip_prefix: 0.0.0.0/0 17 - direction: egress 18 ethertype: IPv4 19 - direction: ingress 20 ethertype: IPv4 21 remote_mode: remote_group_id 22 type: OS::Neutron::SecurityGroup
The default security group is created automatically for each project. The user can add new rules in this group, but the user is not allowed to delete this group or create another security group with the name default. For this reason, when we export this group, we rename it to _default.
1 server_0: 2 properties: 3 block_device_mapping:  4 config_drive: '' 5 diskConfig: AUTO 6 flavor: 7 get_param: flavor_server_0 8 image: 9 get_param: image_server_0 10 key_name: 11 get_resource: key_0 12 name: my_instance 13 networks: 14 - network: 15 get_resource: network_0 16 security_groups: 17 - get_resource: newdefault_0 18 type: OS::Nova::Server
The last resource corresponds to the instance. The flavor and image used to boot this server must be specified by the user : get_param: flavor_server_0 and get_param: image_server_0. The keypair name, network and security group will be automatically resolved.
The second example will focus on volumes. Like in previous example, in your project you deployed a router, a network with corresponding subnet and two instances. The first instance was booted from a volume that was created from an image. The second instance was booted from an image and a volume is attached to this instance.
For this infrastructure, generated template will look like this (only the major differences are showed):
1description: Generated template 2heat_template_version: 2013-05-23 3parameters: 4 flavor_server_0: 5 default: m1.small 6 description: Flavor to use for instance instance 7 type: string 8 flavor_server_1: 9 default: m1.small 10 description: Flavor to use for instance instance_from_volume 11 type: string 12 image_server_0: 13 description: Image to use to boot instance instance 14 type: string 15 volume_image_1: 16 description: Image to create volume volume_from_image 17 type: string 18 volume_type_0: 19 default: iscsi 20 description: Volume type for volume resource volume_0 21 type: string 22 volume_type_1: 23 default: iscsi 24 description: Volume type for volume resource volume_1 25 type: string 26resources: 27 server_0: 28 properties: 29 block_device_mapping: 30 - device_name: /dev/vdb 31 volume_id: 32 get_resource: volume_0 33 diskConfig: AUTO 34 flavor: 35 get_param: flavor_server_0 36 image: 37 get_param: image_server_0 38 key_name: 39 get_resource: key_0 40 name: instance 41 networks: 42 - network: 43 get_resource: network_0 44 security_groups: 45 - get_resource: _default_0 46 type: OS::Nova::Server 47 server_1: 48 properties: 49 block_device_mapping: 50 - device_name: vda 51 volume_id: 52 get_resource: volume_1 53 diskConfig: AUTO 54 flavor: 55 get_param: flavor_server_1 56 key_name: 57 get_resource: key_0 58 name: instance_from_volume 59 networks: 60 - network: 61 get_resource: network_0 62 security_groups: 63 - get_resource: _default_0 64 type: OS::Nova::Server 65 volume_0: 66 properties: 67 metadata: 68 attached_mode: rw 69 readonly: 'False' 70 name: volume 71 size: 10 72 volume_type: 73 get_param: volume_type_0 74 type: OS::Cinder::Volume 75 volume_1: 76 properties: 77 image: 78 get_param: volume_image_1 79 metadata: 80 attached_mode: rw 81 readonly: 'False' 82 name: volume_from_image 83 size: 10 84 volume_type: 85 get_param: volume_type_1 86 type: OS::Cinder::Volume
As in the previous example, each resource is identified by an ID. Since in this example we have two volumes and two instances, two
OS::Cinder::Volume and two
OS::Nova::Server resources are added.
In this example, for two server resources, the properties section
block_device_mapping is used to express the fact that
volume_0is attached to
volume_1is used as boot source for
And since an image is used to create bootable volume
volume_1, Flame will add
volume_image_1 parameter in template parameters section.
In this article we saw how to use Flame to automatically generate Heat template from existing infrastructure. Generated template is highly customized, can be easily modified and reused on every OpenStack installation (with Heat of cause).
Since Flame is still in development, there are some interesting features that we will add to improve it: enrich the set of supported resources, add stack data file generation, that will help users to “adopt” Heat stacks from already existing resources, and the last, but not the least, we would like to improve resource selection i.e. give the user the possibility to specify one by one what resource he/she wants to export in template.