Managing Puppet code with a source control repository

Last couple of months I’ve been working closely with Puppet. I wrote some blogs about it, recorded a VMware vRA Expert Talk video and had two successful events in the Netherlands where I presented and demo’ed the Puppet Enterprise integration with VMware vRealize Automation. For the demo part, Kevin Reeuwijk and I created customPuppet code and stored it on the local code repository of the Puppet Master. Over time the code changed somewhat and because we were not using source control we had no history who made what changes to the code and also had no possibility for a rollback if something went horribly wrong. Luckily that was not the case, but now I know the best way to manage your Puppet code is to use a source control repository. This control repository is where code management stores code and data to deploy your environments.

Puppet Code management uses Git repository branches to create environments. Environments allow you to designate a node or node group to use a specific environment. As you update the code in your control repository, code management tracks the state of that repository to keep each environment updated.

This blog post will learn you how to setup Gitlab CE and configure Puppet Enterprise to use it as a control repository for your Puppet code.

Some items are assumed to be installed and are not part of this guide:

• A virtual machine running Puppet Enterprise.
• A CentOS7 virtual machine to be used for Gitlab Community Edition.
• A virtual machine (or Docker container) running the Git tools.

Step 1 – Setup Gitlab Community Edition

SSH into the CentOS7 machine and login with root. First install some dependencies.

yum -y install postfix
systemctl start postfix
systemctl enable postfix

yum -y install curl openssh-server cronie

Optional, stop the firewall and disable it.

systemctl stop firewalld
systemctl disable firewalld

Then install the latest version of Gitlab CE.

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh |bash
yum install gitlab-ce

Edit /etc/gitlab.rb with nano or vim to change the external URL.

Change it in external_url ‘http://x.x.x.x or FQDN’

Configure and start Gitlab with the following command:

gitlab-ctl reconfigure

Check the status of Gitlab with:

gitlab-ctl status

Step 2 – Create a Puppet control repository in Gitlab

1. Browse to the Gitlab CE server page and login with an account that has permissions to create groups and projects.
2. Create a new group called ‘puppet’.
   1. Click the + icon on the top of the page.
   2. Click ‘New Group’.
   3. Set the group path to: http://<fqdn of your Gitlab server>/puppet
   4. Set the group name to: puppet
   5. Leave the visibility level at its default of ‘Private’
   6. Click ‘Create Group’.
3. Create a new control-repo project in the puppet group.
   1. Click ‘New Project’.
   2. Set the project name to: control-repo
   3. Leave the visibility level at its default of ‘Private’
   4. Click ‘Create project’.

Step 3 – Setup SSH keys for Puppet Code Manager and Gitlab

1. Login to the Puppet Enterprise server and create a set of SSH keys:

   ssh-keygen
   #follow the prompts and accept all the defaults (ensure NO passphrase is set)
   

2. Make the SSH keys available to Code Manager:

   mkdir /etc/puppetlabs/puppetserver/ssh
   cp /root/.ssh/id_rsa /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa
   chown pe-puppet:pe-puppet /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa

3. Copy the SSH public key to your clipboard:

   cat /root/.ssh/id_rsa.pub
   #copy the output to the clipboard
   

4. Browse to the User Settings page in Gitlab for the account you used to create the control-repo project:
   1. Browse to the Gitlab CE server page and login with the same account used in Step 2.
   2. Click on the user’s avatar icon on the top right-hand side of the page and click ‘Settings’.
   3. Go to the ‘SSH Keys’ section.
   4. Paste the clipboard contents into the ‘Key’ field.
   5. Enter a description in the ‘Title’ field. For example ‘Name of your Puppet server‘.
   6. Click ‘Add key’.
5. Check that you are now able to SSH from the Puppet Enterprise server into the Gitlab server with your SSH key:

   ssh git@<fqdn of your gitlab server>

6. This should provide output similar to the following:

Step 4 – Push Starter Content to the Gitlab Puppet control repository

1. Login to the virtual machine with the git tools installed, or use the following command to install the git tools on a fresh virtual machine:

   yum install –y git

2. Setup a git working folder:

   cd ~
   mkdir code
   cd code
   git init
   git config --global user.name "Your Name"
   git config --global user.email your.email@address.com
   

3. Clone the Puppet starter content and push it to your Gitlab server:

   git clone https://github.com/puppetlabs/control-repo.git
   cd control-repo
   git remote remove origin
   git remote add origin http://<fqdn of your gitlab server>/puppet/control-repo.git
   git push -u origin production
   #provide the credentials to the gitlab server when prompted

4. You should see output similar to the following:
5. Browse to the Gitlab CE server page and login with the same account used in Step 2.
6. Go to your Puppet control-repo, click on the + icon in the middle of the screen and select New branch to create an additional environment.
7. Set the Branch name to: dev
8. Select Create from production and click ‘Create Branch’.

Step 5 – Setup Puppet Code Manager

1. Open the Puppet Enterprise webconsole (https://<fqdn of your puppet master>) and login with username ‘admin’.
2. Browse to ‘Classification’ and click the + sign next to PE Infrastructure.
3. Click on the ‘PE Master’ group, the click the ‘Configuration’ tab .
4. Look for the section called ‘Class:puppet_enterprise::profile::master’.
5. Click on the ‘Parameter name’ dropdown box and select ‘code_manager_auto_configure’. Set its value to: true Then click the ‘Add parameter’ button to lock in this parameter.
6. Click on the ‘Parameter name’ dropdown box and select ‘r10k_private_key’. Set its value to: /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa Then click the ‘Add parameter’ button to lock in this parameter.
7. Click on the ‘Parameter name’ dropdown box and select ‘r10k_remote’. Set its value to: git@<fqdn of your Gitlab server>:puppet/control-repo.git Then click the ‘Add parameter’ button to lock in this parameter.
8. Finally, scroll down the page and click the button ‘Commit 3 changes’ to make the changes permanent.
9. Now you just need to run the Puppet agent on the server to have Code Manager set itself up. For this, login to the Puppet Master shell and execute the following command:

   puppet agent -t

10. When Code Manager is setup for the first time, some Puppet Master services will be restarted, this is normal.

Step 6 – Sync the control repository to the Puppet Master server

1. We first need to create an access token to allow Code Manager to push content to the Puppet master:

   #this example uses facter to get the local FQDN
   puppet access login --service-url https://`facter fqdn`:4433/rbac-api --lifetime 5y
   

2. Now let’s perform a dry run to ensure Code Manager is working correctly:

   puppet code deploy --dry-run

3. You should see output similar to the following:
4. Next, let’s retrieve the actual access token so we can setup a webhook in Gitlab:

   puppet access show && echo
   #the echo command is used to add a linefeed after the puppet access show command
   #copy the resulting output to the clipboard

5. Next, browse to your control-repo project in Gitlab and go to the ‘Settings’ section.
6. Click on the ‘Integrations’ section.
7. Enter the following URL in the ‘URL’ field:
   https://<fqdn of your master>:8170/code-manager/v1/webhook?type=gitlab&token=<paste access token here>
8. Clear the ‘Enable SSL verification’ button, as we are using a self-signed certificate.
9. Click the ‘Add webhook’ button to add the webhook.
10. Logout and log back in your Gitlab server with the root account.
11. Click on ‘Configure Gitlab’.
12. Under ‘Admin area’ go to the ‘Settings’ section and scroll all the way down to Outbound Requests. Make sure this option is enabled. Click the ‘Save’ button if you made any changes.
13. Logout and log back in your Gitlab server with the account used in Step 2.
14. Browse to your control-repo project in Gitlab and go to the ‘Settings’ section.
15. Click on the ‘Integrations’ section.
16. Next to the newly configured webhook, click the ‘Test’ button, then select ‘Push events’. You should get an HTTP 200 result to indicate success!

Step 7 – Manage the Puppetfile for additional modules

1. You now have your control-repo setup and Code Manager configured. You’re ready to start configuring your Puppetfile with modules from the Puppet Forge! As an example, use the following procedure to setup your Puppetfile.
2. Browse to your control-repo project on your Gitlab server. Select the Production branch.
3. Click the Puppetfile and click ‘Edit’.
4. Configure your Puppetfile as follows:

   forge "https://forge.puppet.com"
   
   # Puppet modules from Forge
   mod "puppetlabs/inifile",                   '1.6.0'
   mod "puppetlabs/stdlib",                    '4.25.0'
   mod "puppetlabs/concat",                    '4.2.1'
   mod "puppetlabs/acl",                       '2.0.1'
   mod "puppetlabs/chocolatey",                '3.0.0'
   mod "puppetlabs/dsc",                       '1.5.0'
   mod "puppetlabs/dsc_lite",                  '0.2.0'
   mod "puppetlabs/iis",                       '4.3.1'
   mod "puppetlabs/powershell",                '2.1.4'
   mod "puppetlabs/reboot",                    '1.2.1'
   mod "puppetlabs/registry",                  '1.1.4'
   mod "puppetlabs/wsus_client",               '1.0.3'
   mod "puppet/download_file",                 '2.1.0'
   mod "puppet/windows_env",                   '3.1.0'
   mod "puppet/windows_eventlog",              '2.0.0'
   mod "puppet/windowsfeature",                '3.2.0'
   mod "puppet/windows_firewall",              '2.0.0'
   mod "puppet/dotnet",                        '2.0.0'
   mod 'puppetlabs/apache',                    '3.1.0'
   mod 'puppetlabs/mysql',                     '5.3.0'
   mod 'puppetlabs/translate',                 '1.1.0'
   mod 'puppetlabs/pe_puppetserver_gem',       '0.0.1'
   mod 'puppet/staging',                       '3.2.0'
   mod 'puppet/hiera',                         '3.3.2'
   
   # Community modules from Forge
   mod 'liamjbennett-win_facts',               '0.0.2'
   mod 'crayfishx/firewalld',                  '3.4.0'
   mod 'reidmv/unzip',                         '0.1.2'
   mod 'stahnma/epel',                         '1.3.0'
   mod 'herculesteam/augeasproviders_core',    '2.1.4'
   mod 'herculesteam/augeasproviders_ssh',     '2.5.3'
   mod 'hunner/wordpress',                     '1.0.0'
   
   # Modules from Gitlab
   mod 'control-repo',
     :git    => 'git@<FQDN of your Gitlab server>:puppet/control-repo.git',
     :branch => 'production'
   
   # Modules from Github
   mod 'autosign_example',
     :git => 'https://github.com/puppetlabs/puppet-vro-autosign_example'
   
   mod 'vro_plugin_user',
     :git => 'https://github.com/puppetlabs/puppet-vro-vro_plugin_user'
   
   mod 'vro_plugin_sshd',
     :git => 'https://github.com/puppetlabs/puppet-vro-vro_plugin_sshd'

   *ensure you change the FQDN of the Gitlab server in the example to your actual FQDN!

5. Click ‘Commit changes’, this will kick off a code deployment to the Puppet Master and get the modules installed. You can view the progress of the deployment on the Puppet Master with this command:

   tail -f /var/log/puppetlabs/puppetserver/puppetserver.log

6. Finally, to confirm the modules have been deployed, run this command:

   puppet module list --modulepath /etc/puppetlabs/code/environments/production/modules

7. You should see output similar to the following:
8. Repeat the steps above for the ‘Dev‘ branch and off course you can use other modules in the Dev environment if you want.

That’s it! You are now ready to define profiles and roles in the /site section of your control-repo and store your Puppet code. Happy puppetizing!

Special thanks to Kevin Reeuwijk of Puppet who wrote the majority of this guide.