Outdated content! This is a snapshot of my website pretty much as it was before a new version replaced it some time in 2016. As far as I could recover, this is the earliest version of this site.

You can find the most up to date content at: https://romaricpascal.is

Playing with Puppet

And here is the first technical article in the journal! Before I started building the journal page, I realized I had changed my computer since I first built the website. The sources were versioned, of course, so no problems on that side. But building a WordPress site meant I needed an HTTP server (Apache in my case) with PHP, a MySQL database for the project, and because I don't like IP in my browser address bar, a host entry.

So here was my problem: I had nothing configured on my machine. It's no big configuration, but I find it really boring. Edit a config file here, another one there, connect to MySQL to run a set of command… I'd rather be designing or coding. It would be so much more awesome if I could checkout my project, run a command, and boom:

  • Have a new host entry on my machine so I can access my project at https://awesome-project.local (for example)
  • Have a new Apache VHost for the host just configured
  • Have a new MySQL database, with its own user

Scripting was an option, but Puppet seemed a better option to do the job. It is a tool for managing server configurations. You first describe the resources (could be files, packages, services…) your project needs in a manifest file. Then, when you run Puppet, it makes sure the resources are available on the machine: installs the packages (apt-get under the hoodcd .. if you run it on Ubuntu), create configuration files, or databases… The manifest can live along your project files in the source repository.

This sounds exactly like what I need! Add to that it's open source and I find the name awesome (decisive criteria :p), definitely good reasons to give it a try. So I installed Puppet, created a manifest.pp file at the root of my project (seems as good a location as anywhere else in the project), and was ready to roll.

Adding a new host

Adding a new host was a breeze. Puppet comes with a built in host resource, that does just the task I'm after: adding an entry to my host file. I just had to add this bit to my manifest:

Configuring a vhost

Apache configuration was not as simple (nothing too awful, though). It's no big surprise, Puppet doesn't have a built-in "Apache" resource. After all, you can't expect the project to have resources for every third party project on earth. Fortunately, Puppet has a module system to extend its functionalities. Even better, the guys at Puppetlabs provide a module for Apache management: puppetlabs/apache. It has a apache::vhost resource to configure a virtual host on an Apache server.

It looks all sweet, but after I installed the module, I realized I'll have to make a few tweaks to fit my needs:

  • The module wipes out the content of the directory with the vhost configuration. Handy when your project is hosted alone on its server, to make sure old configurations get removed. Not so cool on my development environment, with the Apache server hosting multiple projects at the same time.
  • I need my vhost configuration to have a couple of SetEnv directives to give WordPress the database configuration. The module does not give much control over it, with a standard default template.

But nothing's lost here, as the puppetlabs/apache module is open source too, hosted on Github. This means I could easily create a fork to add my custom options. After I replaced the module with my forked version, I completed the manifest with:

Creating a database and its user

Two configurations automated, just one to go! As for Apache, there's no built-in resource, but a puppetlabs/mysql module to help with the database configuration. Its mysql::db resource does just what I want: create a database with a custom user:

With that last snippet, the manifest looks compete. One command (puppet apply manifest.pp) and I have my host entry vhost and MySQL database sorted. Awesome!

A quick note about passwords

As it is, the manifest does the job. But there's something not quite right to me about it: it contains all the passwords in plain text :s I don't want one manifest per development environment, and moreover, I don't think the source repository is the place to store the production database password. That's where Hiera, another of Puppetlabs projects, comes in. It is a text (YAML or JSON) database you can use to store information out of your Puppet manifests (you can also use it without Puppet if you want). Perfect for machine/environment specific information!

This article on the puppetlabs blog describes how to use Hiera with Puppet probably better than I'll ever do (they are the guys behind the tool after all ;)). It led me to a pretty reusable manifest where I can choose a name for my project and pull all sensitive information from Hiera.

It's kind of a step back from the "pull the project, run command, boom!" workflow I was hoping, as I have to populate the Hiera database beforehand. But it's only one file to create, not multiple operations to do on the different services. Plus, the manifest as become way more reusable. New environment you need to deploy your project on? Same manifest, just change the data. Same thing if you want to bootstrap a new project: copy the manifest, add entries in the Hiera database. I'd say it's a win!

In the end…

… Puppet was a big help. The pre-made resources or modules to manage hosts, Apache and MySQL definitely me time compared to writing a script from scratch. Even more if you consider that puppet already handles what to do when there's already an entry in the host file, or the MySQL user already exists. I had to customize a few things for Apache, true, but handling that properly in a script would've been such a pain.

I can't tell yet if the time saved by that reusable manifest when starting projects or deploying them on new servers will be worth the time learning Puppet and writing it — it's proved useful already when creating a staging instance of the site, though. But it was good fun, I learnt a lot and had material for an article. Not too bad a trade off :)

Thanks for reading it till the end! If you have any thoughts about that kind of automation, I'll be happy to hear about them (do you do it? with which tools? is it worth it?).

comments powered by Disqus