How we handle application configuration

We recently overhauled the way we handle configurable settings within our application (server names, email addresses, polling frequencies, etc) . I’m going to present our solution below, but its new enough that I’d like to hear feedback on how others approach the problem.

The goal

We want a configuration solution with the following traits:

  • No magic strings. The configurable values will ultimately be stored with an identifying string key, but we didn’t want client code to have to know and use that string. The potential for typos is high, and maintaining a parallel list of constants is tedious.
  • Strongly-typed values. The configured values will likely be stored as strings in a configuration file, but we never want client code to know that. If the configurable value is numeric, the consumer should get a number. Forcing the consumer to deal with type conversion (and related error handling) would create a lot of duplicate, error prone, “ceremony” code that would obscure the essence of what it is trying to accomplish.
  • Test friendly. Client code that depends on a configurable value should not be forced to have any “out of process” dependencies (database, file system, etc.). Ideally, we would like to avoid having to stub expectations or use a test fake for configuration. We would also like to avoid adding any “simple type” (int, bool, string) constructor arguments to our classes, since the automocker we use cannot handle them.
  • Tooling friendly. We will have a large number of configurable values in many different places around the codebase. Our solution should make it easy to build tooling to generate a sample configuration to document all of the possible settings. We also want to be able to validate an existing configuration to make sure all necessary values have been provided.

Though not necessary, it would also be nice to have:

  • Default values. Users should only have to configure the values that have no sensible default, or values they want to override. It should not be left up to the client code to handle checking for a configured value, or falling back to a hardcoded default. That’s more “ceremony” code, not to mention the fact that multiple clients that use the same configurable settings would all have to know the default.
  • App.config/web.config Storing application configuration settings in an app.config or web.config is a well-established convention in the .NET world. It already has support for encryption, multiple file overrides, and tooling. We should use it if it doesn’t cause too much pain.

Our solution in action

To explain our solution, I’ll use the example of a service that associates an uploaded image with a user profile. Users might upload images of all sizes, and in many different formats, but we want to make sure they are all stored in the same size and format. We want to give owner’s of our application the ability to set the standard size and format, so they must be configurable. We also want to make sure the path where images are stored is configurable. Consider the following implementation:

public class AvatarService : IAvatarService
{
    private readonly AvatarSettings _settings;
    private readonly IFileSystem _fileSystem;
    private readonly IImageService _imageService;

    public AvatarService(AvatarSettings settings, IFileSystem fileSystem, IImageService imageService)
    {
        _settings = settings;
        _fileSystem = fileSystem;
        _imageService = imageService;
    }

    public void SaveAvatar(User user, Stream originalImage)
    {
        var extension = _imageService.GetImageExtensionFromMimeType(_settings.PreferredMimeType);
        var pathToSave = Path.Combine(_settings.AvatarStoragePath, user.Id + extension);
        user.AvatarFilePath = pathToSave;
        var scaledImage = _imageService.ScaleImageWithCrop(originalImage,
            _settings.PreferredMimeType,
            _settings.DefaultWidth,
            _settings.DefaultHeight);
        _fileSystem.Write(scaledImage, pathToSave);
    }
}

As you can see, in order for the class to make use of these externally configured values, it just needs to take an instance of AvatarSettings in its constructor. At runtime, the settings will be injected by our composition tool (StructureMap), just like the file system and image services. The AvatarSettings itself is a simple POCO class:

public class AvatarSettings : DictionaryConvertible
{
    public AvatarSettings()
    {
        PreferredMimeType = System.Net.Mime.MediaTypeNames.Image.Jpeg;
        DefaultWidth = DefaultHeight = 64;
    }

    [ExpandEnvironmentVariables]
    public string AvatarStoragePath { get; set; }
    public int DefaultWidth { get; set; }
    public int DefaultHeight { get; set; }
    public string PreferredMimeType { get; set; }
}

public abstract class DictionaryConvertible
{
    private readonly List<ConvertProblem> _problems = new List<ConvertProblem>();
    public IEnumerable<ConvertProblem> Problems { get { return _problems; } }

    public void AddProblem(ConvertProblem problem)
    {
        _problems.Add(problem);
    }
}

I’ll discuss the DictionaryConvertible base class later, but I included it here to show that it does not bring along any extra baggage that would hinder the testability of derived classes. When testing client code like the AvatarService, we can prepare a specific test context by setting configuration values directly on an AvatarSettings instance. This is more natural than stubbing values on a fake. If we are testing a scenario that is not impacted by the configuration values, we can just let the automocker pass in a default instance.

To set the configurable values for the application, we just need to add corresponding keys to the appSettings section of the app.config/web.config file:

<appSettings>
  <add key="AvatarSettings.AvatarStoragePath" value="%ALLUSERSPROFILE%DovetailCRMAvatars"/>
  <add key="AvatarSettings.DefaultWidth" value="96"/>
  <add key="AvatarSettings.DefaultHeight" value="96"/>
</appSettings>

There are a few things worth noting here. First, I have not included a value for AvatarSettings.PreferredMimeType in the config file. If you look at the source for AvatarSettings, you’ll notice it is set in the constructor. That is how we declare default values. It allows us to avoid cluttering up the config file with a bunch of settings that have reasonable defaults, while still allowing the flexibility to override them if desired. This is illustrated by the DefaultWidth and DefaultHeight settings which have configured values that override the defaults from the constructor. Also notice that the DefaultWidth and DefaultHeight properties are declared as integers (even though the values are strings in the config file) so that client code does not have to do any type conversion.

The final thing to notice is the [ExpandEnvironmentVariables] attribute that decorates the AvatarStoragePath property. In addition to type conversion, the code that populates a settings object can use additional metadata to do further manipulation of the configured values. In this case, the value “%ALLUSERSPROFILE%DovetailCRMAvatars” will be converted to “c:ProgramDataDovetailCRMAvatars” before any client code ever sees it. This not only simplifies the client code, but eliminates duplication of logic when a settings object has more than one client.

How it works

So how does this all work? The Settings object needs to be populated from the configuration file, so we define an interface for this responsibility:

public interface ISettingsProvider
{
    DictionaryConvertible PopulateSettings(DictionaryConvertible instance);
}

Implementations take in an instance of a DictionaryConvertible derived type, and should return an instance of the same type with all of the settings populated. We have a single implementation, AppSettingsProvider, which gets values from the <appSettings/> section of the app.config/web.config. Under the hood it makes use of a class that knows how to cycle over the properties of an object and set them using values from a dictionary (very similar to the DictionaryConverter in fubumvc). Any problems encountered during the conversion are recorded using the AddProblem method of the DictionaryConvertible, hence the original reason for the base class. We were already using this code to populate the input models for our MVC controller actions (similar to the ModelBinders that were introduced in ASP.NET MVC), so it made sense to re-use it.

The final step is to tell StructureMap how to create instances of the settings classes so that they are properly injected into consumers. In our Registry, we add:

For<AvatarSettings>()
  .Use<AvatarSettings>()
  .EnrichWith((session, original) => 
      session.GetInstance<ISettingsProvider>().PopulateSettings(original));

This is a good example of making use of the advanced power of your composition tool (and a good argument against trying to make a Common Service Locator equivalent for service registration). We use the EnrichWith statement to tell StructureMap “when I ask you for an AvatarSettings instance, before returning it to me, you should first pass it through the PopulateSettings method of my configured ISettingsProvider.”

Of course, it wasn’t long before we started adding a number of these Settings classes to our codebase, and it became a tedious extra step to add this same registration code for each class. The next logical step was to create a custom type scanner:

public class SettingsScanner : ITypeScanner
{
    public void Process(Type type, PluginGraph graph)
    {
        if (!type.Name.EndsWith("Settings") || !typeof (DictionaryConvertible).IsAssignableFrom(type)) return;
        graph.Configure(r => r.For(type).EnrichWith((session, original) =>
        {
            return session.GetInstance<ISettingsProvider>().PopulateSettings((DictionaryConvertible)original);
        }).TheDefaultIsConcreteType(type));
    }
}

We could now remove the previous code from our Registry that registered a single class (AvatarSettings), and replace it with this code which has the same effect for ALL Settings classes in our codebase:

Scan(x =>
{
    x.TheCallingAssembly();
    x.With<SettingsScanner>();
});

Tooling support

One of our goals was that the configurable settings should be easy to document. With a growing number of Settings classes all over the codebase, it could be very easy to lose track of which values are needed in the configuration file. However, since all of the Settings class follow a convention (derive from DictionaryConvertible, and named with the “Settings” suffix), it is trivial to write a little utility to reflect over the code and write out a sample <appSettings /> section.

public void Execute(string[] args)
{
    var settingTypes = _container.Model.PluginTypes
        .Where(t => t.PluginType.Name.EndsWith("Settings") && t.PluginType.BaseType == typeof(DictionaryConvertible))
        .Select(t => t.PluginType);

    dumpSettings(settingTypes, Console.Out);
    Console.WriteLine();
}

private void dumpSettings(IEnumerable<Type> settingTypes, TextWriter output)
{
    var xml = new XmlTextWriter(output) {Formatting = Formatting.Indented};
    xml.WriteStartElement("appSettings");
    settingTypes.Each(t =>
    {
        var settings = Activator.CreateInstance(t);
        var properties = t.GetProperties(BindingFlags.Instance | BindingFlags.DeclaredOnly | BindingFlags.Public);

        properties.Each(p =>
        {
            var key = AppSettingsProvider.DefaultNamingStrategy(p);
            var value = p.GetValue(settings, null);

            xml.WriteStartElement("add");
            xml.WriteAttributeString("key", key);
            xml.WriteAttributeString("value", value == null ? "" : value.ToString());
            xml.WriteEndElement();
        });
    });
    xml.WriteEndElement();
    xml.Close();
}

Summary

All of this comes together to allow a very natural workflow when we decide we need to make a value in our code configurable:

  1. Create a new Settings class (or append to an existing class) that derives from DictionaryConvertible  and add the properties for each of the configurable values we need. Optionally initialize the properties with a default value in the constructor.
  2. Add the Settings class as a constructor parameter in the client class that needs the configurable value.
  3. After the client class is done and tested, add an <appSettings /> entry to the application configuration file for each setting without a default value.

Related Articles:

This entry was posted in infrastructure, structuremap. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • DaRa

    very nice solution. i like all your design decisions especially using the poco approach. I also like the setting provider.

    I’ve been contemplating about a solution like this one because i have the .net framework way of doing it.

    This can be very well an open source project that many can benefit from. I agree that it’s a simple problem but it’s being solved wrong over and over by a lot of people. are you planning on sharing your code?

    I have one question regarding the way you are registering the settings with EnrichWith, does that mean every time the settings get resolved they’re constructed from scratch? if so then it’s unnecessary and inefficient and i suggest populating it once and then registering it as singleton.

  • http://www.lostechies.com/members/phatboyg/default.aspx Chris Patterson

    I really like how you just have multiple POCOs with the settings and then resolve them via the container. we have a single root settings object (IConfiguration) and then have children off of it (ISubcomponentConfiguration) which is clearly a violation of several SOLID principles. I think I’m going to get this refactored soon based on your approach.

    @Dara:
    If you have global configuration, you could modify the lifestyle of the settings object once it has been created. In our case, the configuration is on a per-client basis so we actually cache the configuration objects using the HttpRuntime.Cache provider instead of making it a singleton. nearly as fast, and allow for forced expiration if there is a configuration setting change.

    Our configuration is also backed by key/value tables in the database, so using this approach keeps our magic strings to a single specific point in the codebase, eliminating those nasty spelling errors.

    Great post man, really a mind tickler!

  • http://reshefmann.com/blog Reshef Mann

    I came up with a very similar approach to this issue. I placed it on google code as an open source: http://code.google.com/p/configreader/wiki/GettingStarted
    We are using it for the projects in the company I work for.

  • Marco

    How do handle updates? You release version 1 with some settings and for version 2 there are new settings.. How do you update the config file with the default settings without changing the (possible changed) values from version 1

  • Dara

    @Reshef Mann

    your API is not very initiative. why is there config reader and config browser? also how do you handler different sources for configuration data like database or separate xml files or anything else?

  • http://pitadeveloper.com PitaDevAj

    Very cool approach. Question, though: Is there a reason you aren’t using Custom Configuration classes to define your settings rather than using ? I’ve had quite a bit of success with writing classes that inherig System.Configuration.ConfigurationSection, ConfigurationElement, ConfigurationElementCollection, etc.

  • http://blogs.dovetailsoftware.com/blogs/kmiller Kevin Miller

    I love that in this case a convention approach is helping to keep things simple when configuration is necessary.

    Excellent write up. I like where you’ve take the technique including providing container setup automation and generation of default application settings.

    @marco RE: updates

    I think that the application configuration example emitted is just a guideline and would not overwrite the customer’s existing settings.

    If you wanted to automate the merge of new and customer existing settings it would be the job of the installer. This step could be as easy as: if there is an existing setting leave it be. For the situation where settings have been removed in theory the customer’s settings would just go unused. If desired existing settings that are no longer in use could be flagged with an XML comment.

  • http://miscjibberish.blogspot.com Jeremy Wiebe

    Very nice Joshua! In our current project we’re using a single ISettings (I think it’s similar to what @ChrisPatterson described.

    One of the nice things that your approach has over the one we’re currently using is that no stubbing/mocking of settings is required. I love that. POCO makes many things so much easier.

    I have a feeling that these ideas may find their way into our current project at some point. :-)

  • http://www.lostechies.com/members/jflanagan/default.aspx Joshua Flanagan

    @PitaDevAj – no reason we aren’t using a custom section, other than we didn’t need it at the time. If we decide we prefer a custom section, its a one line change.

  • http://www.lostechies.com/members/jflanagan/default.aspx Joshua Flanagan

    @DaRa – We considered registering them as singletons, but decided that was premature optimization. The built-in ConfigurationManager already caches the config file, so its really just the population of the classes that is repeated. The fact that the settings wont change at runtime is an implementation detail of AppSettingsProvider, and not necessarily part of the ISettingsProvider contract.
    But of course, if performance becomes an issue, that is an easy fix.

  • http://reshefmann.com/blog Reshef Mann

    @Dara:

    The job of the configreader is to set up the the various settings type and then the configbrowser is can be passed to get the settings, however a refactoring might be an option – this is just the initial api and it can evolve :)

    In order to define a different config data one can implement the IConfigurationSource interface and pass it. The default implementation is for the application configuration file.

  • andyclap

    What was your decision process behind DictionaryConvertible?

    Naively I’d see the detection and reporting of config parse errors as a responsibility of the ISettingsProvider; with a marker interface/attribute on the class to indicate it’s a settings holder.

  • http://www.lostechies.com/members/jflanagan/default.aspx Joshua Flanagan

    @andyclap – There were historical reasons for the DictionaryConvertible design. Starting fresh designing just for the configuration scenario, it would probably look like you describe.

  • http://schotime.net/blog Adam

    How would you do user specific settings?
    Stored in Session?
    Static Dictionary object with user name as key?

    Not quite sure where do go on this?

  • Gustavo

    Hello Joshua, nice read!!!
    I like your style…
    Do you have this code for example?

  • VirtualStaticVoid

    I discovered your blog today, and found it to be very informative.
    You make mention of “We” in most of the posts, and the “project” we are working on.
    So I was wondering who is “We” and what is the “project”.

    Thanks

  • http://twitter.com/richardwhatever Richard Brown

    Hi Joshua,

    Its been 3 years since you posted this, so plenty of time for real world testing :-) I was wondering – do you still use this solution for your new projects?  Also, do you have any source code of this?

    Thanks!