Basic Nagios support was recently added to pallet, and while very simple to use, this blog post should make it even simpler. The overall philosophy is to configure the nagios service monitoring definitions along with the service itself, rather than have monolithic nagios configuration, divorced from the configuration of the various nodes.

As an example, we can configure a machine to have it's ssh service, CPU load, number of processes and number of users monitored. Obviously, you would normally be monitoring several different types of nodes, but there is no difference as far as pallet is concerned.

We start by requiring various pallet components. These would normally be part of a ns declaration, but are provided here for ease of use at the REPL.

(require
  '[pallet.crate.automated-admin-user
    :as admin-user]
  '[pallet.crate.iptables :as 'iptables]
  '[pallet.crate.ssh :as ssh]
  '[pallet.crate.nagios-config
     :as nagios-config]
  '[pallet.crate.nagios :as nagios]
  '[pallet.crate.postfix :as postfix]
  '[pallet.resource.service :as service])

Node to be Monitored by Nagios

Now we define the node to be monitored. We set up a machine that has SSH running, and configure iptables to allow access to SSH, with a throttled connection rate (six connections/minute by default).

(pallet.core/defnode monitored
  []
  :bootstrap [(admin-user/automated-admin-user)]
  :configure [;; set iptables for restricted access
              (iptables/iptables-accept-icmp)
              (iptables/iptables-accept-established)
              ;; allow connections to ssh
              ;; but throttle connection requests
              (ssh/iptables-throttle)
              (ssh/iptables-accept)])

Monitoring of the SSH service is configured by simply adding (ssh/nagios-monitor).

Remote monitoring is implemented using nagios' nrpe plugin, which we add with (nagios-config/nrpe-client). To make nrpe accessible to the nagios server, we open the that the nrpe agent runs on using (nagios-config/nrpe-client-port), which restricts access to the nagios server node. We also add a phase, :restart-nagios, that can be used to restart the nrpe agent.

Pallet comes with some configured nrpe checks, and we add nrpe-check-load, nrpe-check-total-proces and nrpe-check-users. The final configuration looks like this:

(pallet.core/defnode monitored
  []
  :bootstrap [(admin-user/automated-admin-user)]
  :configure [;; set iptables for restricted access
              (iptables/iptables-accept-icmp)
              (iptables/iptables-accept-established)
              ;; allow connections to ssh
              ;; but throttle connection requests
              (ssh/iptables-throttle)
              (ssh/iptables-accept)
              ;; monitor ssh
              (ssh/nagios-monitor)
              ;; add nrpe agent, and only allow
              ;; connections from nagios server
              (nagios-config/nrpe-client)
              (nagios-config/nrpe-client-port)
              ;; add some remote checks
              (nagios-config/nrpe-check-load)
              (nagios-config/nrpe-check-total-procs)
              (nagios-config/nrpe-check-users)]
  :restart-nagios [(service/service
                    "nagios-nrpe-server"
                    :action :restart)])

Nagios Server

We now configure the nagios server node. The nagios server is installed with (nagios/nagios "nagiospwd"), specifying the password for the nagios web interface, and add a phase, :restart-nagios, that can be used to restart nagios.

Nagios also requires a MTA for notifications, and here we install postfix. We add a contact, which we make a member of the "admins" contact group, which is notified as part of the default host and service templates.

(pallet.core/defnode nagios
  []
  :bootstrap [(admin-user/automated-admin-user)]
  :configure [;; restrict access
              (iptables/iptables-accept-icmp)
              (iptables/iptables-accept-established)
              (ssh/iptables-throttle)
              (ssh/iptables-accept)
              ;; configure MTA
              (postfix/postfix
               "pallet.org" :internet-site)
              ;; install nagios
              (nagios/nagios "nagiospwd")
              ;; allow access to nagios web site
              (iptables/iptables-accept-port 80)
              ;; configure notifications
              (nagios/contact
              {:contact_name "hugo"
               :service_notification_period "24x7"
               :host_notification_period "24x7"
               :service_notification_options
                  "w,u,c,r"
               :host_notification_options
                  "d,r"
               :service_notification_commands
                 "notify-service-by-email"
               :host_notification_commands
                  "notify-host-by-email"
               :email "my.email@my.domain"
               :contactgroups [:admins]})]
  :restart-nagios [(service/service "nagios3"
                     :action :restart)])

Trying it out

That's it. To fire up both machines, we use pallet's converge command.

(pallet.core/converge
  {monitored 1 nagios 1} service
  :configure :restart-nagios)

The nagios web interface is then accessible on the nagios node with the nagiosadmin user and specified password. Real world usage would probably have several different monitored configurations, and restricted access to the nagios node.

Still to do...

Support for nagios is not complete (e.g. remote command configuration still needs to be added, and it has only been tested on Ubuntu), but I would appreciate any feedback on the general approach.

FluidDB, a "cloud" based triple-store, where the objects are immutable and can be tagged by anyone, launched about a month ago. As a another step to getting up to speed with Clojure, I decided to write a client library, and clj-fluiddb was born. The code was very simple, especially as I could base the library on cl-fluiddb, a Common-Lisp library.

I have some ideas I want to try out using FluidDB. It's permission system is one of it's best features, together with the ability to use it for RDF like triples means that it could provide a usable basis for growing the semantic web. My ideas are less grandiose, but might take as long to develop, we'll see...

I have spent the last few months with my latest start-up, Artfox, where I have been trying to push home some of the lean start-up advice expounded by Eric Lie's and Steve Blank. I was hoping that "The Principles of Product Development Flow", by Donald Reinertsen, might help me in making a persuasive argument for some of the more troublesome concepts around minimum viable product and ensuring that feedback loops are in place with your customers as soon as possible. Unfortunately, I don't think that this is the book if you are looking for immediate, practical prescription, but it is a thought provoking, rigorous view of the product development process, that pulls together ideas from manufacturing, telecommunications and the Marines.

Perhaps Reinertsen's most accessible advice is that decisions in product development should be based on a strong economic foundation, pulled together by a concept of the "Cost of Delay". Rather than on relying on prescriptions for each of several interconnected metrics, such as efficiency and utilisation, Reinertsen suggests that economics will provide different targets for each of these metrics depending on the costs of the project at hand.

His proposition that product development organisations should measure "Design in Process", similar to the idea of "Intellectual Working In Process" proposed by Thomas Stewart in his book "Intellectual Capital", is what allows him to make the parallels to manufacturing and queueing theory and enables the application of the wide body of work in these fields to product development.

His practical advice, such as working in small batches and using a cadence for activities that require coordination, will come as no surprise to practitioners of agile development, and Reinertsen provides clear reasoning of why these practices work.

During my time at Alcan, and later Novelis, I gave a lot of thought to scheduling, queues and cycle times in a transformation based manufacturing environment, and I found that this had many parallels to his view of the product development process, and little in common with what Reinertsen describes as manufacturing, which seems to be limited to high volume assembly type operations. I found many ideas that could be usefully taken back to a manufacturing context.

If you look at this book as an introduction to scheduling, queueing theory and the reason's behind some of agile development practices, then you will not be disappointed.

The facility of Ruby on Rails' test, development and production environments is one of those features that goes almost unremarked, but which makes using rails more pleasant. No doubt everyone has their own solution for this in other environments, and while I am sure Common Lisp is not lacking in examples, I have not seen an idiomatic implementation. In developing cl-blog-generator I came up with the following solution.

Configuration in Common Lisp usually depends on using special variables, which can be rebound across any block of code. I started by putting the configuration of my blog into s-expressions in files, but got tired of specifying the file names for different blogs. Instead, I created an association list for each configuration, and registered each using a symbol as key. I can now switch to a given environment by specifying the symbol for the environment.

The implementation (in src/configure.lisp under the GitHub repository) consists of two functions and a special variable. SET-ENVIRONMENT is used to register an environment, and CONFIGURE is used to make an environment active. The environments are stored in *ENVIRONMENTS* special as an association list. An example of setting up the configurations can be seen in the config.lisp file. In creating the configurations I drop the '*' from the special names.

I'm relatively new to CL, so let me now if I have overlooked anything. Writing this post makes me think I am missing a WITH-ENVIRONMENT macro ...

I have now added a comment system to cl-blog-generator. My requirements were for a simple, low overhead, commenting system, preferable one that could possibly be fully automated.

The comment system was inspired by Chronicle's, with a slight modification in approach - the comments are never saved on the web server, and are just sent by email to a dedicated email address. Spam filtering is delegated to the whatever spam filtering is implemented on the mail server, or in your email client. The comment emails are then processed in CL using mel-base and written to the local filesystem. Moderation can optionally occur on the CL side, if that is preferable to using the email client.

There is still some work left to do - I would like to be able to switch off comments on individual posts, either on demand on after a default time period - but I thought I would let real world usage drive my development.