Opensource Configuration Management tools integration with vRealize Automation Cloud – Part 2

This is part 2 of opensource Configuration Management tools integration with vRealize Automation Cloud. In part 1 I wrote about using Puppet Bolt together with VMware Cloud Automation Services for installing and configuring software on a deployed VM. 

A lot has changed since, Cloud Automation Services is renamed to vRealize Automation Cloud (vRA Cloud) and the on-premises version, vRealize Automation 8, has just been released. Both share the same codebase and have similar features and use cases, giving customers choice how they want to consume their automation platform.

vRA Cloud has a lot Out-of-the-Box integrations. One of these is the Ansible opensource integration. In this post I’ll explain how to setup an Ansible Control node and how to configure the integration in vRA Cloud. Then with some Ansible code and a blueprint I’ll show how easy it is to install and configure software using this integration.

Setting up the Ansible Control node

The Ansible Control node is any Linux based machine with Ansible installed. Windows isn’t supported for the control node.

Once installed you can run commands and playbooks, invoking /usr/bin/ansible or /usr/bin/ansible-playbook, from the control node.

Playbooks are ordered lists of tasks, saved so you can run those tasks in that order repeatedly. Playbooks can include variables as well as tasks. Playbooks are written in YAML and are easy to read, write, share and understand. To learn more about playbooks, see Intro to Playbooks.

It is possible to install and configure the Ansible Control node by hand. You can also refer to Ansible documentation for detailed setup instructions here. In my case though I will use a vRA Cloud blueprint and a VM template with Cloud-init enabled to install packages and do the Ansible specific configuration using cloudConfig. In the blueprint, I do the following:

• Install Ansible
• Create an Ansible user including a ssh keypair
• Set some permissions
• Create some files
• Modifying Ansible specific configuration so that it can be added as Integration with vRA Cloud
• Enable Password Authentication
• Restart the sshd service

Here’s my blueprint YAML code. Notice that I use Ansible to install and configure the Ansible Control node ;-)

name: Ansible Control Machine
version: 1
formatVersion: 1
inputs:
  ansible_username:
    type: string
    description: The username for the ansible user account
    default: ansible
  ansible_user_password:
    type: string
    description: 'The password for the ansible user. The connection from Cloud Assembly currently uses password based auth, so this is a mandatory requirement.'
  ansible_vault_password:
    type: string
    description: The password that will be written to the Vault file.
  ansible_ssh_key:
    type: string
    encrypted: true
    description: The public half of SSH key used for authentication to this box. Not to be confused with the SSH key that will be used by the ansible user account to SSH into managed instances.
  slack_webhook:
    type: string
    description: A webhook for Slack notifications. This will be used to send the public key details of the generated keypair that ansible will use to connect to remote hosts.
resources:
  Ansible_Control_Node:
    type: Cloud.Machine
    properties:
      image: ddeswart-Ubuntu
      flavor: ddeswart-small
      constraints:
        - tag: 'platform:vsphere'
      cloudConfig: |
        #cloud-config
        repo_update: true
        apt:
          sources:
            ansible-ubuntu-ansible.list:
              source: "ppa:ansible/ansible"
              keyserver: 'keyserver.ubuntu.com'
              keyid: 7BB9C367
        packages:
          - ansible
        write_files:
          - path: /etc/ansible/playbooks/ansiblecn.yml
            content: |
              ---
              - name: Install and Configure Ansible Control Node for use with VMware vRA Cloud
                hosts: localhost
                gather_facts: true
                vars:
                  ansible_username: "${input.ansible_username}"
                  ansible_user_password: "${input.ansible_user_password}"
                  ansible_vault_password: "${input.ansible_vault_password}"
                  ansible_ssh_key: "${input.ansible_ssh_key}"
                  slack_notification_content: "{{ lookup('file', '/home/ansible/.ssh/id_rsa.pub') }}"
                  slack_notification_webhook: "${input.slack_webhook}"
                tasks:
                  - name: Create Ansible User
                    become: true
                    user:
                      name: "{{ ansible_username }}"
                      groups: sudo
                      shell: /bin/bash
                      generate_ssh_key: yes
                      password: "{{ ansible_user_password | password_hash('sha512') }}"
                      
                  - name: Set Authorised Key for Ansible User
                    authorized_key:
                      user: "{{ ansible_username }}"
                      key: "{{ ansible_ssh_key }}"
                      
                  - name: Set Ansible Directory Permissions
                    file:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible
                      recurse: yes
                      state: directory
                      
                  - name: Create Cleartext Vault Pass File
                    lineinfile:
                      create: yes
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/vault_pass.txt
                      line: "{{ ansible_vault_password }}"
                      
                  - name: Create Ansible Log file
                    lineinfile:
                      create: yes
                      owner: "{{ ansible_username }}"
                      path: /var/log/ansible.log
                      line: "ansible_log_file"
                      
                  - name: Update Config with Pass File Location
                    lineinfile:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/ansible.cfg
                      regexp: "vault_password_file"
                      line: "vault_password_file = /etc/ansible/vault_pass.txt"
                      
                  - name: Update Config with Private Key Location
                    lineinfile:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/ansible.cfg
                      regexp: "private_key_file"
                      line: "private_key_file = /home/{{ ansible_username }}/.ssh/id_rsa"
                      
                  - name: Update Config with Host Key Check Setting
                    lineinfile:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/ansible.cfg
                      regexp: "host_key_checking"
                      line: "host_key_checking = False"
                      
                  - name: Update Config with Roles path
                    lineinfile:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/ansible.cfg
                      regexp: "roles_path"
                      line: "roles_path = /etc/ansible/roles"
                      
                  - name: Update Config with Ansible Log location
                    lineinfile:
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/ansible.cfg
                      regexp: "log_path"
                      line: "log_path = /var/log/ansible.log"
                      
                  - name: Create localhost entry in Hosts File
                    lineinfile:
                      create: yes
                      owner: "{{ ansible_username }}"
                      path: /etc/ansible/hosts
                      line: "localhost"
                      
                  - name: Enable Password Based Auth
                    become: true
                    lineinfile:
                      path: /etc/ssh/sshd_config
                      state: present
                      regexp: "PasswordAuthentication no"
                      line: "PasswordAuthentication yes"
                      
                  - name: Restart sshd
                    become: True
                    systemd:
                      name: sshd
                      state: restarted
                      
                  - name: Send Ansible Public Key to Slack
                    uri:
                      method: POST
                      url: "{{ slack_notification_webhook }}"
                      body: {"text": "Your ansible public key is ```{{ slack_notification_content | regex_replace('ansible-generated on.*') }}```"}
                      body_format: json
                      
        runcmd:
          - ansible-playbook --connection=local --inventory 127.0.0.1, /etc/ansible/playbooks/ansiblecn.yml
      networks:
        - network: '${resource.VM_network.id}'
  VM_network:
    type: Cloud.Network
    properties:
      name: VM network
      networkType: existing

1. Login to vRealize Automation Cloud (vRA Cloud) and go to Blueprints and create a new blueprint. Copy and add the YAML code. Be aware that you have to change some settings to get it working in your vRA Cloud environment!

   

2. Click Deploy, enter a Deployment Name and select Current Draft.

   
3. In the next screen enter the Ansible username and password, the Ansible Vault password and your private SSH key for connecting remotely to this node. If you have Slack, create an App in Slack with a webhook enabled for receiving notifications in one of your channels. Enter the webhook details. This will be used to send the public key details of the generated keypair that Ansible will use to connect to remote nodes. Click Deploy.
4. After a successful deployment, connect remotely to the Ansible Control node using ssh. Check if Ansible is installed by entering  ansible --version. Then go to /etc/ansible and check of all the necessary directories and files are in place.
5. Copy the public ssh key you received in your Slack channel.

Setting up the Ansible integration in vRA Cloud

1. In vRA Cloud, go to Integrations, select Add new integration and select the Ansible tile. 
2. Enter the IP address of the Ansible Control node, select the location of the machine and if it runs on-prem select your Cloud proxy. Enter the Username and Password used during install. Click Validate. After validation, enter a name for the Integration.  

Adding content to the Ansible Control node

To show how Ansible works I created a simple Playbook that updates my deployed machine, installs Nginx and deploys a webpage. I also make use of a Role for setting up the firewall and opening tcp port 80. 

1. Use ssh to login to the Ansible Control node. Go to /etc/ansible and in the console enter ansible-galaxy install geerlingguy.firewall
2. Check if the Role was installed in /etc/ansible/roles by entering ansible-galaxy list 
3. Download my Nginx playbook including additional files for the webpage on https://github.com/ddeswart/ansible-examples
4. Copy the downloaded content over to the Ansible Control node. I used WinSCP for this task but other alternatives are available. 

Create a blueprint using the Ansible integration

The final step is to create a blueprint which deploys a VM and installs and configures Nginx using the Ansible integration.

Here’s my blueprint YAML code.

name: Ansible Nginx on Ubuntu
version: v1
inputs:
  hostname:
    type: string
    title: Machine name
    description: Name of the Virtual Machine
resources:
  Ansible:
    type: Cloud.Ansible
    properties:
      inventoryFile: /etc/ansible/hosts
      username: ansible
      playbooks:
        provision:
          - /etc/ansible/playbooks/install-nginx.yml
      osType: linux
      groups:
        - WebServers
      maxConnectionRetries: 10
      host: '${resource.Linux_VM.*}'
      account: vmguru-AnsibleCN
      privateKeyFile: ~/.ssh/id_rsa
  Linux_VM:
    type: Cloud.Machine
    properties:
      image: ddeswart-Ubuntu
      flavor: ddeswart-small
      constraints:
        - tag: 'platform:vsphere'
      cloudConfig: |
        hostname: ${input.hostname}

        users:
          - name: ansible
            ssh-authorized-keys:
              - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDY2Qlj4RFL5G5rGNlmGEvAuJ3ToXfVkFCmnVlMw2WNuSBeE8lQpfxH3WqurEdtapUxYcQDlH/xpy/220+9EJajUa3KGp8fGvtZFZzTaPPkOIR0bvTZe9x4Udiuwch7jULKpEhZAl75gNs9dBb0zO+DkBj85YKs4Wcap28kSSDNZvgNgJhcJJi5GGq2DjaROUl4rIrdrUIeVEM6j3XTvOoZ7Py1wgATDD1IxdUCe4UB4RzUUP1bdpV1EMdGfg00KZuX/xOJ1U2s7mtcdhB0yZe/moMsPFwjisdxHFO435ra54MlfKsk8WUVtR+LLb/Dol8LEcCmC8Y5XSBqIwrj9eRX
            sudo: ['ALL=(ALL) NOPASSWD:ALL']
            groups: sudo
            shell: /bin/bash
      networks:
        - network: '${resource.VM_network.id}'
  VM_network:
    type: Cloud.Network
    properties:
      name: VM network
      networkType: existing

1. In vRA Cloud, go to Blueprints and create a new blueprint.
2. Copy and add the YAML code. Be aware that you have to change some settings to get it working in your vRA Cloud environment! 
3. In the VM part of the YAML code notice that I’ve created a new user (ansible) with sudo privileges using cloudConfig. Change the ssh key to the key you received in your Slack channel!
4. In the Ansible part of the YAML code you can see settings as Inventory file location (etc/ansible/hosts), which Playbook to provision (install-nginx), if the node needs to be grouped in specific group (WebServers) and off course the name of the Ansible integration to use.
5. Click Deploy and enter a Deployment Name and select Current Draft.
6. Enter a Machine name and click Deploy.
7. If the deployment was successful go to your web browser and enter the IP address of the deployed machine.
8. You can also check if the Ansible Playbook was deployed correctly by logging in to the Ansible Control node using ssh and view the ansible.log file in /var/log/
9. And finally, check that the deployed node was added to the Ansible inventory under the WebServers group. View the /etc/ansible/hosts file

This concludes my two part blog about opensource Configuration Management integration with vRA Cloud. I hope that with this blog, and earlier blogs about Configuration Management, you have a good understanding of how to install and configure software with vRA Cloud using Cloud-init, Puppet Bolt, Puppet Enterprise and Ansible opensource.