Pro Puppet (35 page)

When the Example.com operator brings new systems online, he needs only to ensure they have the
nagios::target
class included in their catalog, and Puppet will automatically take care of reconfiguring the central Nagios monitoring system. In addition, if the operator would like more than one system to monitor all of these nodes, he only needs to include the
nagios::monitor
class in the catalog of additional monitors and they'll automatically collect all of the host and service resources from the stored configuration database.

In the next section, we'll cover methods to scale stored configuration to support a large number of nodes and reduce the amount of time each Puppet agent requires to submit a copy of its configuration to the Puppet master.

Puppet stored configurations require the Puppet agent to upload the configuration catalog after each catalog run. This process introduces a potential bottleneck when many Puppet agents are running concurrently. In this section, we cover enabling thin stored configurations, an option added in Puppet 0.25.x to reduce the amount of information stored in the SQL database.

We also cover the Puppet queue daemon and queue service for stored configurations, which enables Puppet agents to operate asynchronously with regard to database updates.

The default behavior of stored configurations is to store a complete copy of every catalog in the SQL database. If only a small number of resources are being exported and collected, there may be too much overhead associated with storing the complete catalog. Thin stored configurations were added in Puppet 0.25.0 to address this problem. With thin stored configurations, only exported resources, node facts, and tags are stored in the SQL database. This limited set of information greatly reduces the number of synchronous database updates required for each Puppet agent run.

Thin stored configurations are very easy to set up. The option is located in
puppet.conf
on each Puppet master system. See how the Example.com operator enables thin stored configurations for his network in
Listing 6-23
.

Only the line
thin_storeconfigs = true
needs to be added into the master section of
puppet.conf
. The Puppet master should then be restarted to reflect this change.

Alternatively, the
--thin_storeconfigs=true
command line argument may be passed to the
puppet master
application. No client-side configuration settings are required to enable thin stored configurations.

Queue support is intended to take much of the load related to stored configurations off the Puppet master systems themselves. To accomplish this task, all database updates are moved to a separate process, named
puppetqd
in 0.25.x and
puppet queue
starting with version 2.6.0.

Puppet queue uses the ActiveMQ middleware service to handle message passing and queuing. In this section, you'll see how the Example.com operator sets up ActiveMQ on one system and uses the stomp Ruby gem to connect all of the Puppet master systems to the message bus.

The first step while setting up setting up queue support for stored configurations is to install ActiveMQ. Active MQ requires a Java runtime, available for most Unix platforms at
http://www.oracle.com/technetwork/java/
. The information presented here uses Java 6 update 16. Please refer to the Apache ActiveMQ documentation for the recommended version of Java for the version of ActiveMQ you're installing.

Once Java and ActiveMQ are installed, the Stomp protocol must be enabled in the ActiveMQ configuration, and then the ActiveMQ service will be started. We'll see how the operator configures and starts ActiveMQ on Debian and Enterprise Linux based systems in the following two sections. With ActiveMQ up and running, we'll also see how the operator configures the Puppet Master and Puppet Queue applications to connect to the message bus using the Stomp ruby library.

For an Enterprise Linux-based system, the Example.com operator first installs the Java 6 runtime from Oracle. Official packages for Java are not available in most online package repositories due to restrictions in the distribution terms of the license. The operator has chosen to download the RPM to the local system as a result, as shown in
Listing 6-24
.

Once Java is installed and available, the next step is to install the ActiveMQ packages. These packages are available online at
http://puppetlabs.com/downloads/mcollective/
. We create a new directory and download them in
Listing 6-25
.

As you can see in
Listing 6-25
, the Example.com operator creates a temporary directory named
/tmp/activemq
and downloads two packages required for ActiveMQ into this directory. Using the rpm command installs both packages. For a production environment we recommend staging these packages in a local YUM repository to simplify and automate management of these packages using Puppet.

Once Java and Apache ActiveMQ are installed, we're ready to proceed with the configuration of the stomp protocol. The stomp protocol is supported but not enabled by default in ActiveMQ. Before connecting the Puppet queue process to the message bus, stomp support must be enabled in the ActiveMQ configuration as we can see from
Listing 6-26
.

The Example.com operator has added a single line specifying the address and port ActiveMQ should bind to and listen for stomp messages on. This line should be added to the
transportConnectors
section of the
/etc/activemq/activemq.xml
file.

Once ActiveMQ has been configured to handle stomp messages, the service needs to be started (
Listing 6-27
).

The Example.com operator uses the ActiveMQ init script to start the service. Since ActiveMQ is a Java service, the operator verifies that the service is actually up and running by calling the status method of the init script. If there is a problem with the configuration file, the service may fail to start up properly and would not give an indication of the problem in the start command. Finally, checking the log files to make sure ActiveMQ is listening for stomp messages on port 6163 is a sensible final verification that things are working as expected. If there is a problem starting the server, a listing of the problem will be present in the file
/var/log/activemq.log
.

The Java JDK is easy to install on Debian-based systems by adding the “non-free” Apt repositories for Java to the
/etc/apt/sources.list
configuration file:

Once this additional repository is enabled, the operator uses the aptitude executable to install the Java Development Kit:

With Java installed, the operator downloads the ActiveMQ release archive and starts the service. ActiveMQ archives are available online at
http://activemq.apache.org/download.html
. ActiveMQ packages are not available in Debian Lenny, so the operator installs the service into
/var/lib/activemq/opt/activemq
.

First, he creates the
activemq
service account so the service doesn't run as the
root
user:

With the user and group in place, the operator unpacks the archive and moves it into the activemq home directory. These commands create the directory
/var/lib/activemq/opt/activemq
. In addition, the operator makes sure to give ownership to the
activemq
user.

Before starting the message service, the operator must generate a default configuration and make a small change to enable the stomp messaging service. Using the setup command, they write a default configuration file into the home directory of the service account.