Building VM images using Danube Cloud Factory (part 2)

Building VM images using Danube Cloud Factory (part 1)
2. March 2017
TOP 5 Trends in Cloud in 2017
10. April 2017

Building VM images using Danube Cloud Factory (part 2)

I assume that you have read the first part of this tutorial and already have a working esdc-factory build environment installed. In this part, we will learn how to create your own VM images. I have decided to create a GitLab CE image, which will be installed on CentOS 7.


The build files will be prefixed with the word “contrib” to indicate the fact that this is an image contributed by a user and not connected with other build logic (e.g. not related to building the Danube Cloud USB image).

Directory Layout

This is the list of newly created or affected files:

Documentation

Let’s start by writing some documentation for our new image. The documentation should include a description of the image and a list of supported metadata along with their description. Something like this: docs/contrib/gitlab-ce.rst.

Playbook

The main playbook is located in the ansible directory, and the file should be prefixed with the word “build“. Our playbook will be called build-contrib-gitlab-ce.yml. Every playbook for creating images is divided into four parts also called plays:

Play #1: Initial checkup

Although, this play is not required it is recommended to keep it here. There are tasks that will do some basic checks i.e. check whether the build web server is reachable and whether the build_base_dir is configured correctly.

- name: Check builder host
  hosts: builder
  tasks:
    - include: tasks/build/check.yml
      when: skip_check is not defined or not skip_check

Play #2: Create a build VM

This play will create a VM on buildnode and register it in the running Ansible playbook under a specific name (the hostname parameter). The pre_tasks section includes tasks that will make sure that a base image is installed on the buildnode and remove an old VM, which may still exist on the buildnode from a previously failed build.

- name: Create virtual machine
  hosts: buildnode
  vars_files:
    - vars/build/vm/contrib-gitlab-ce.yml
  pre_tasks:
    - include: tasks/build/cleanup.yml
    - include: tasks/build/prepare-base-image.yml
  roles:
    - smartos-vm
  tasks:
    - include: tasks/build/centos/register-host.yml
        hostname=contrib-gitlab-ce

The VM parameters should be configured in the ansible/vars/build/vm/contrib-gitlab-ce.yml file. These parameters start with the zone_ prefix and are used by the smartos-vm role. You will get the idea by looking at other VM vars files in the ansible/vars/build/vm folder.

Play #3: Install and configure the image

This is the main play that runs tasks in the VM created in the 2nd play. You should include all modifications and configuration stuff into this play. The last role here should be vm-image. This role runs a script inside the VM, which will clean up the VM and prepare it for a snapshot that will be used for creating the final image.

- name: Install and configure appliance
  hosts: contrib-gitlab-ce
  gather_facts: true
  vars_files:
    - vars/build/os/contrib-gitlab-ce.yml
  roles:
    - esdc-common
    - selinux
    - zabbix-agent
    - cloud-init
    - rc-scripts
    - iptables
    - mdata-client
    - qemu-guest-agent
    - contrib-gitlab-ce
    - passwords
    - vm-image

If a role requires some variables to be set, then these should go into the ansible/vars/build/os/contrib-gitlab-ce.yml file. For example, we will add a gitlab_ce_version and gitlab_ce_checksum variables here; these variables will be used by our new role – contrib-gitlab-ce.

The role contrib-gitlab-ce is based on the official GitLab CE install instructions and does the following:

  • installs all required packages;
  • downloads and installs GitLab CE;
  • installs an es-post-deploy.sh script.

You can see the details in the ansible/roles/contrib-gitlab-ce/tasks/main.yml task file, but let’s examine the es-post-deploy.sh script installed by the last task. The script will be installed into /var/lib/rc-scripts and run by the systemd rc-scripts.service during every VM boot. The script reads the VM metadata and uses them to configure the VM and services accordingly. During the initial VM boot, the script will perform the following operations:

  • update /root/.ssh/authorized_keys according to the root_authorized_keys metadata;
  • generate a self-signed SSL certificate;
  • update zabbix_agentd.conf according to the org.erigones:zabbix_ip metadata;
  • configure GitLab based on the gitlab:external_url metadata.

The last operation – GitLab configuration – will be performed during every VM boot. The es-post-deploy.sh is a simple script and configures just a few things. There are many other configuration options that can be included in such scripts to automate the deployment of new VMs. This also means that the power and usability of a VM image is related to scripts like this.

Play #4: Create and save the image

This final play creates image and metadata files on buildnode and copies them to the builder host. The image name and other metadata are configured in the already mentioned ansible/vars/build/vm/contrib-gitlab-ce.yml file (image_name, image_desc, image_homepage and builder_dir variables).

- name: Create and save image
  hosts: buildnode
  vars_files:
    - vars/build/vm/contrib-gitlab-ce.yml
    - vars/build/os/contrib-gitlab-ce.yml
  vars:
    image_tags: {internal: false, resize: true, deploy: false}
  tasks:
    - include: tasks/build/centos/create-image.yml

Building

In order to use the convenient Makefile, the contrib-gitlab-ce target must be added to the BUILD_TARGETS list at the begging of the file. That’s all. Let’s run it:

[user@builder ~/esdc-factory]$ make contrib-gitlab-ce

You can set the VERBOSE environment variable to make Ansible more verbose. This may come handy if you need to debug your tasks.

Conclusion

Although the resulting image is usable only for Danube Cloud / SmartOS, it should be pretty easy to convert it to other platforms. This could also be integrated into esdc-factory itself if there is such demand.

The esdc-factory is a very powerful tool for building not only Danube Cloud -related stuff, but also other VM images. It uses Ansible to automate the build process, and thanks to Ansible, lots of the code can be re-used. This turns the creation and management of VM images into a fairly simple process.

Daniel Kontšek
Daniel Kontšek
CTO, Danube Cloud

Leave a Reply

Your email address will not be published. Required fields are marked *