I recant my IoC! IoC containers in dynamic languages are silly.

After a year or so of solid Alt Dot Net infection (as far as infections go its a pretty awesome one to have), I decided to give Python a try again for more than one off sysadmin tasks, and to actually dive into it as a newly minted “Agilista”. 

However, I had  a problem..there were no non-painful IoC containers in Python (sorry to the other authors of IoC frameworks in Python like Spring Python and Snake Guice, I know you try and I respect the effort).  Ultimately, I could not  imagine coding anymore without something to handle all my registration for me, that’d dynamically inject in my dependencies, give me hooks for contextual resolution, and give me rich interception support.

So I built my own, it was painful still, but I had some ideas to move it in an Convention Over Configuration direction, and ultimately get within shooting range of what fancy stuff we’ve come to expect in .Net that Windsor, StructureMap and many others provide.

Now as I got into the fancier aspects of dynamic languages, open classes and the ability to override all behavior easily I get it…a dynamic languages runtime is like a bit IoC container.

Now I’ve heard other people say this many times but rarely with explanation or example, or focus on frankly silly things like “Dependency Injection adds too many lines of code” which is a bit melodramatic. Two arguments to a constructor, two fields ,and then avoiding having to call new is not “adding too many lines of code”, especially with templates, IDE’s, scripting, etc to cut down the actual typing load.Today I’m going to actually try to explain how languages like Python, Ruby etc give us the same awesomeness we’ve come to expect in things like Windsor..but at a much cheaper cost of learning and dev time.

Take a typical resolution scenario where you want to output to a different file format depending on command line switches.  With an IoC container you can either:

Change the resolution argument to load a different concrete type:

if(arg == “XML”)
{
  container.Register(Component.For<IOutput>().ImpmentedBy<XmlOutput>());
}
else if(arg == “HTML”)
{
  container.Register(Component.For<IOutput>().ImpmentedBy<HtmlOutput>());
}
else
{
  container.Register(Component.For<IOutput>().ImpmentedBy<NullOutput>());
}

or resolve different arguments or keys using a service locator approach in later client code (thereby depending on the container)

public void output(IKernel container, string key)
{
var output =container.Resolve<IOutput>(key);
output.save();

}

or implement a custom implementation of the resolver system (which I’ll leave out for the sake of brevity, but it’s not instant code).  Also to do all this you have to depend on interfaces, add all your interchangeable code to the constructor and life is grand.  You do this for many reasons in static languages, its the only way to get easy testability and code that is open to extension.  In dynamic languages its always open for extension and easy to test . Let me demonstrate:

import outputlib as o

def outputselect(arg):
if arg == “XML”:
o.Output = XmlOutput
elif arg == “HTML”:
o.Output = HtmlOutput
else:
o.Output = NullOutput

def saveoutput():
o.Output().save()  #will save whichever

Contextual resolution in a nutshell, and throughout your code if need be.  “Interception” is even easier, take a look at http://docs.python.org/dev/library/functools.html and then start playing you’ll see you can trivially apply logging and security to methods without explicitly adding it. A short logging example follows:

import types 
import functools

#applies a cepter to each non-underscored method.
def wrapcls(cls, cepter):
    publics  = [ name for name in dir(cls) if not name.startswith("_")] 
    methods = [getattr(cls,method) for method in publics if type(getattr(cls,method)) == types.MethodType ]
    for method in methods:
        intercepted_method = cepter(method)
        setattr(cls, method.__name__, intercepted_method) #attaches intercepted_method to the original class, replacing non-intercepted one

#the magic all happens in the functools.wraps decorator
def loggingcepter(func):
    @functools.wraps(func)
    def logafter(*args, **kwargs): #for csharp devs view this as an inline delegate
        result = func(*args, **kwargs) #invoking function
        print “function name: ” + func.__name__
        print “arguments were: ”
        for a in args:
            print repr(a)
        print “keyword args were: ”
        for kword in kwargs:
           print repr(kword) + “ : ” + repr(kwargs[kword])
        print “return value was: ” + repr(result)
        return result
    return logafter
    
#default boring repository class
class StorageEngine(object):
    @property
    def connections(self):
        pass
    
    def _sessioncall(self):
        pass
        
    def create(self,user):
        print “running create now from the method”
         
    def delete(selfid):
        print “running delete now from the method”
        return “deleted from the database”
        
    def get(selfid):
        pass
        
    def __init__(self):
        pass

#placeholder storage object
class User(object):
    pass
            
wrapcls(StorageEngine ,loggingcepter)
repo = StorageEngine()
print “calling count this should not be intercepted”
cnncount = repo.connections
print “now get should be intercepted”
repo.get(1)
print “we should see keyword arguments here”
repo.delete(id=2)
print “session call should not be intercepted”
repo._sessioncall()
print “create should be intercepted and we should see a User object”
repo.create(User())

 

Running the above script should result in the following output

 

Screen shot 2009-11-16 at 4.20.56 PM

So this is all very cool, but what does it mean or why do I care?  For those of us used to using a proper IoC container like Windsor or StructureMap, we’ve gotten used to have capabilities like the above easily available to us when we’ve needed them. It’s nice to find that in Python (or really any dynamic language ) we can easily build our own similar functionality that we’ve come to depend on.  We’re never coupled and we’re always able to test and mock out behavior.  It was a long time coming but I think I finally get it now.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Ryan Svihla

I consider myself a full stack polyglot, and I have been writing a lot of JS and Ruby as of late. Currently, I'm a solutions architect at DataStax
This entry was posted in Dynamic Langs, IoC, Python. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://joshuaflanagan.lostechies.com Joshua Flanagan

    You lost me on the outputselect example. Can you explain how that demonstrates dynamic extensibility? My initial impression is that it is just a hardcoded switch statement that could easily be replicated in C#.
    I think you might be showing how you are modifying outputlib after the fact, but that isn’t immediately clear. Am I on the right track?

  • http://www.lostechies.com/members/rssvihla/default.aspx Ryan Svihla

    You got it. It replaces what we’re used to with containers and _all_ new instances of Output will use the type it’s been assigned.

    You have “strategy” without explicitly
    making it clear, it’s like one big global IoC container.

  • http://blog.coreycoogan.com Corey Coogan

    Great article Ryan. I’ve been using Castle and StructureMap, but I know I’m still not fully utilizing the containers to their fullest potential.

    Can you explain how you achieve this?

    “give me hooks for contextual resolution”

  • http://blog.coreycoogan.com Corey Coogan

    Oops, I should have read your post first. I see how you are achieving it, and that’s what I first suspected. I do this with certain keys/types to resolve the correct interface.

    When you call it “contextual” it sounded so much fancier and thought there might be some cool magic I didn’t know about.

  • http://blog.sprettur.is Petar Shomov

    Hi,

    I am also in the process of learning Python/Django and trying to let go of my “static thinking” but I am not sure I completely understand your point. Looking at the registration sample with the renderers I can see the registration part but I am not seeing how is the dependency injection part going to happen. Let’s forget about the service locator for now, shall we ;)

    Mental Note to self: self, stop fooling around and check out python spring.

    Besrt regards,

    Petar

  • http://www.lostechies.com/members/rssvihla/default.aspx Ryan Svihla

    @Petar.
    Ignore service locator, it was an easy example to show for simple contextual resolution. Doing a proper override of Dependency Resolution in Windsor has been done many times in other blog posts and is worthy of a blog post itself.

    What is the point of dependency injection and Inversion of Control? We get testability, easy ability to override behavior in our _actual_ codebase, If you’re doing DI and IoC for any other reason I’m glad to recant my recant and hear what it is.

    Why do we use IoC containers themselves? having a nice thing to prevent us from having to write a ton of factory classes, since we have to use DI to keep ourselves from tightly coupling our code. Also, we get for low cost nice things like AOP, and contextual resolution.

    In dynamic languages you get all that _even when you call new_ . You still get testability and mockability, look at the examples above, I’m actually overriding the entire class object, so now _every_ time you call new on that class that code you will get the new value. This sort of functionality you get by using DI, dynproxy, and intercepters. Which involves a big chunk of code.

    Now if I can _STILL_ have testability that’s even stronger than the best mocking libraries in static languages, have low coupling always, and I can in a few lines of code apply logging, security, to entire modules and code bases (which is why I’m using an IoC container in the first place) why would I want to have another IoC container?

  • http://blog.sprettur.is Petar Shomov

    Hi Ryan,

    I think I phrased my question the wrong way. I will try again, but this time I will list the issues I have with the code above:

    1. the code is hard-coded to instantiating the type stored in outputlib.Output. If I have to reuse this code again I have to make this module and variable existent and store the type there

    2. if I want to make two instances of the code (in this case it looks like a method) that use two different methods of rendering I have to change renderers in outputlib.Output in between (or something like that) the method calls

    3. In multi-threaded scenarios (no idea how is threading in python) point #2 becomes something I am not ready to deal with

    4. This one is a bit on the side, but you are also instantiating the type in your method assuming it takes no parameters in the constructor.

    My current thinking is that DI takes care of these things no metter the language. You could do exactly the same thing you have done here but in C# and it will be equally mockable and testable, it’s just that I think it will be a lot cleaner if you just passed that renderer instance as a parameter ;)

    What am I missing Ryan? ;)

    Best regards,

    Petar

    P.S. Let’s leave AOP on the side for now, python decorators are interesting approach but I am still not completely sure how I feel about them

  • http://www.lostechies.com/members/rssvihla/default.aspx Ryan Svihla

    @Petar good questions

    1) make a module call it “IoCConfig” and use it the same way you’d do your IoC registration. This same issue could be related to how you use an IoC container. Use it in a dumb way and you’ll have to recall the code over and over again.

    2)Again this is the same issue you have with IoC controllers, you have multiple implementations of the same interface, you have to make special allowances for custom resolution. _in either approach you have to do custom code to switch which interface implementation is in use_. I made Output a specific empty class for a reason, its a good stand in for an interface (and when people see Output has no code body it especially hits home that a “strategy” needs to be selected”. Finally in this case I’d suggest a factory that hands off the correct instance of said class at a given time..which I’d be inclined to use as a solution in static languages in complex enough cases.

    3) See #2 , again custom factories are more suitable here, in python or C#. The reason IoC containers tend to handle this is because its the only place you typically get access to your “entire run time” barring some magic hackery in the CLR with the profiler. When you _already_ have access to your entire runtime, you can just use the natural methods to do these things.

    4) No I’m assuming my types have a constructor with the same signature (important distinction,and when not using DI, your constructors are much more manageable). If I really need to do dynamic resolution of types with differing signatures I again wrap this in a custom factory and it would have a similar algorithm to what you use in an IoC container (if you’ve never written one I highly encourage it, its not hard and it gives you a unique perspective), but I’d have no desire to put my entire runtime in it, or maintain placing my entire runtime in the IoC. only the bits I care about that need this special custom functionality, and its rare at best.

    PS I know you said ignore AOP but as a side note I’m not advocating decorators, and i think by themselves are a poor use of AOP in my opinion (same way attributes are in C#). I just used the wrap decorator in my code to apply interceptors on the fly (which can be done by rule matching and I only wrap what I want), otherwise the approach is I feel a Python equivalent to Windsor’s internal implementation of DynProxy and Interceptors.

  • http://blog.sprettur.is Petar Shomov

    Hi Ryan,

    Thank you for taking the time to asnwer my concerns. I find that all of the resolutions you offered were too much sacrificy for my taste and actually summarize why I always prefer DI over Service Locator(I am not sure what I was thinking but I realized outputselect and saveoutput methods are actually participatng in a service locator pattern. Have to read blogs in the morning, when fresh …). Anyways, thanx for the discussion, I am always curios about other people’s perspective, so keep these posts coming ;)