Apr 23

Centralized Configuration Management: At What Cost

One of the hot-button technology topics is the desire to automate system administration tasks for organizations with medium to large scale server installations.  Obviously, not an issue for the small concern with a server or two, but as the number of servers increases, so do the effort and costs required to install, manage and maintain them.

Organizations looking to control manpower and licensing costs, therefore, are not comfortable with the ‘variable cost’ aspect of system administration and the per server licensing of commercially available tools.  To reduce growing labor and licensing costs, variable costs associated with manpower and licensing need to be converted to fixed costs.  Theoretically, by doing so, a growing number of servers will not immediately translate into a correspondingly growing number of system support employees and configuration management (support) licenses.

Some recognized Open Source (i.e. free) configuration management tools include:  CFEngine, Puppet, DACS (Distribution and Configuration System), SmartFrog, LCFG ( Large Scale System Configuration), BCFG, Arusha, and many, many others.  The most intriguing aspect of most of these configuration tools (except for SmartFrog) is they require centralized management.   This is because it is erroneously assumed that to be effective, the management tool must have (and therefore, rely on) centralized coordination.

In discussing configuration management theory, I will use the terms ‘parameter’ and ‘aspect’, as defined by Burgess and Couch in their paper “Modeling Next Generation Configuration Management Tools”.  Parameters are ‘units of configuration information’, such as the static IP address of a server.   Aspects are the combination of the configuration parameter and its corresponding constraints, dependencies and preconditions required for the parameter to be functional, such as the fully qualified domain name of the server.

In the above example, the IP address of a server is a simple configuration parameter because it has almost no dependencies beyond the topology (e.g. network segment) in which it is networked.  Once the interface is configured and default route added, the device will be successfully ‘networked’.  The fully qualified domain name, however, has additional requirements for the correct resolv.conf and/or nsswitch.conf domain definitions, and also require external DNS records (likely on a separate DNS server) to match.

The strength of a tool like CFEngine or Puppet is information hiding: users need not cope with the complexity of aspects.  Thus, a good Puppet ‘recipe’ might use parameters such as host name and IP address to figure out what network it is on, and therefore, what the fully qualified domain name should be.  The main problem with these tools are they require a lot of human labor to set up, and require the centralized hosts to generate configurations based on intimate knowledge of each and every server it manages.  Though feasible, it does not scale well if the variety of local parameters and corresponding aspects increase with each new server.

Similarly, the centralized planning functions of these tools cannot react automatically to distributed changes.  Thus, these tools require manual changes to fairly complex configuration scripts, fairly often.

Given today’s generation of configuration management tools, I conclude that a centralized management tool like CFEngine or Puppet can be a cost effective alternative to similar commercial products, but only if your environment has the following qualities: 1) a relatively short list of hardware and OS types to support; 2) a relatively stable and predictable network environment; 3) control over all aspects of service provisioning; 4) and a large installation of servers of ‘similar or identical’ services (e.g. little or no variation) to grow and maintain.

However, if every server is different due to application and/or service it provides; different due to OS and hardware configurations, and/or you lack control over significant aspects of external service provisioning, these tools may well cost you more time in set up and maintenance; and in dollars for hardware purchases for centralized servers, than you yield in savings due to automation and/or fewer commercial client licenses.

For instance, if you are provisioning services on the Internet, you will lack control over certain service provisioning, like DNS servers.  If you cannot control DNS records or available DNS servers, for instance, then ‘scripts’ or ‘recipes’ to generate the nsswitch.conf file may be prone to failure due to changes you cannot predict or control.

In another example, if you are provisioning local printers to individual users, you may have to maintain a centralized database of ‘nearest printers’, which is an additional database management task required to keep your centralized configuration management tool working.

Conversely, if you moved towards a Service Oriented Architecture (SOA), you are not bound by the constraints of the ‘centralized’ server to be all knowing about your environment and local clients.

In the case of configuring DNS on a local client, we could create a local agent not part of the configuration server; but, instead installed on different servers or clients already in that domain, or pre-installed in our new client, to generate the correct nsswitch.conf and resolv.conf files at the time of installation.  As the domain names, servers and related DNS environment changes, dedicated agents in that domain can use available tools (e.g. dig or nslookup) to determine the correct entries in a configuration file for new clients. The CFEngine or Puppet ‘Master’ does NOT need to store this detailed information, nor even generate (cook) the final configuration file.

Similarly, for printer configurations, we could implement a printer configuration ‘agent’ in each local network.  That agent could reside on a local server; already configured local client, or the local print server itself.  This agent can communicate with other ‘local clients’ physically close to itself as determined by network subnet, IP range, MAC address range, or other criteria and provision a ‘local printer’ to the local client.

In this latter example, we did not need a centralized server, or a centralized database of ‘closest printers’ in order to provide the local client with a  local printer.  Instead, we relied on a distributed SOA model to provide the service of ‘local printer configuration’ to devices connecting to the network at that geographic location.

The strength of this type of SOA configuration provisioning is that, as the printer models, names and IP addresses change on that local network segment, the local agent can be responsible for provisioning those changes dynamically and automatically.  We do NOT need a centralized service for this, nor do we need to “inform” a centralized database authority of a printer change, at all.  In fact, as we spend efforts building comparable SOA methods of configuration provisioning, the need for a centralized provisioning ‘master’ diminishes rapidly.

Jeff Hamilton, Technology-Leadership

Technology-Leadership is a leading consultancy company in financial and business management consulting strategies, focusing on issues involving corporate strategy, technology, financing, planning and implementation.

Leave a Reply