Semantics, Modules And Testing: Why (And How) I Test Internal Components, Not Private Methods

I saw someone tweet about a new-ish JavaScript library called Autooc. recently. I had not heard of it, so I clicked the link to find out more. It looks like an interesting project. I’ll have to look at it further when I have time. But one thing that stuck out and made m spidy-sense tingle was this line just after the first example:

“Hey, this function is private but I want to test it anyway.”

To which, I responded with a tweet: 

This sparked a question on from the library’s author, Dan Tao, where he asked me to elaborate. At the time I couldn’t elaborate much because I was traveling. But Dan put together a great post on why he tests internal methods, and I promised him I would write up something too – this post. Before you read the rest of this post, though, you really need to read Dan’s post. It provides context for this post, and it is very well written. In fact, I agree with Dan’s reasoning, 100%. It’s at the point where even Dan is asking:

I don’t think we disagree, when it comes to the practice of our craft. I think we use different words for the same things and our initial “argument” was over semantics and minor details that don’t matter. But I do want to explore my thoughts on testing private things in code, when I call things “internal” instead of private, and showing some of the things I’ve done to solve the problems of needing to test “private” or “internal” code. I think these things are important, and I’ve been trying to find a way to write about some of these details for a long time now. So, this discussion is only the impetus for me to write some of these thoughts, and not the generation of the ideas or a continuation in some inane internet fight over what to test. :)

The Boilerplate Arguments

Dan starts his post by correctly boiling the typical “don’t test private methods” argument down in to two points:

  1. Private methods are implementation details and shouldn’t be tested
  2. If a method does need tests, it should be public (and potentially refactored into a separate class)

I believe that private methods and other private things should not be tested directly. Private things exist only to support other things that can (and should) be tested. Private things are tested transiently – through the testing of non-private behaviors, we are inherently testing the private things that support the non-private thing.

Dan goes on to say he generally agrees with #2 but that he doesn’t apply this as a hard and fast rule. He takes this and heads down the path of testing private methods, and does so in a very nice way with his Autodoc library. I know a lot of people that would kill for the ability to directly test private methods in JavaScript apps, and this looks like it has potential for being really good at doing that. Again, I haven’t used it yet so I’m just speculating based on the samples. 

Where I differ from Dan in my reaction, though, is not by testing “private” methods. Rather, I prefer to modify point #2

  1. Private methods are implementation details and shouldn’t be tested
  2. If a method is sufficiently complex to warrant it’s own test directly, it might need to be refactored into a separate thing

The real difference in my version of #2 is that I don’t say anything about exposing it publicly. Like Dan, I think exposing something to the public just for testing is a bad idea.

Don’t Expose “Private” Things. Test “Internal” Things.

There are a lot of options for solving the problem of needing tests on something that is private and Dan lists several of them in his post (my modifications noted in [square brackets] to make the idea more generic):

  1. create a separate [library] and depend on that from within [yours]
  2. create a separate [library] but pull it [in] as a build step to keep zero run-time dependencies
  3. just test the publicly exposed sites where it’s used

All three of these ideas are great, and valid. Each has a strength and weakness, and I use all of them at various times. MarionetteJS, for example, does all three. I have many separated files that each have some chunk of code – much of which is “internal” or “private” code. I build them all in to one deployable file at the end, so you don’t have to rely on multiple files for Marionette (of course there are dependencies on jQuery and Backbone, so you do have to reference those). Marionette also has a few libraries spit out from it’s original code: Backbone.BabySitter and Backbone.Wreqr. Both of these are dependencies, and are built in to Marionette at build time. I generally approach testing private things by testing the behavior of the public things that use the private things. You don’t have to expose the private thing as something public to test it.

There is one thing that Dan is missing, though… something else that I do in Marionette which could be extracted in to a more general purpose tool quite easily:

use an internal module system to allow testing of internal components

I know this doesn’t sound like much, off-hand. But there are some important semantic differences and functional differences in how I approach my testing, evidenced by the specific wording that I am using (and not using). 

The Fallacy Of Testing Car Engines

One of the examples that I consistently see when talking about testing private things is that of a car and an engine. I guess Dan ran in to it, too, as he put the same basic analogy in to his post. This analogy falls apart with any cursory scrutiny, but most people don’t bother so they miss how this falls apart.

The basic logic of this analogy/argument is:

  • you have to test the engine
  • the engine is “private” within the car
  • therefore you must be able to test private things

I believe this is fallacy propagated by people who have never worked on cars, or at least by people who have never bothered to scrutinize the analogy. I have worked on cars – my dad is a mechanic and an engineer (among many other things) and I grew up with a wrench in my hand, helping him rebuilt most of the cars that I rode around in or drove until I moved away, after college. I have first hand knowledge of how engines are tested, and the first time they are tested is not after the car is assembled. Engines are not “private” components, in my experience. They are “internal” components of the car. They are complex, prone to faults from many parts, and must be rigorously tested prior to mounting them in the vehicle. Failure to do this leads to cars that have faulty engines.

The fallacy of testing car engines doesn’t apply to just cars, though. It goes far beyond that, and is something I’ve written about before. The core concept is that of defect prevention, instead of quality inspection. The gist of it is to prevent defects further down the assembly line by rigorously measuring and testing at every step of the process. If a part is off by more than the allowed variance, early in the manufacturing stages, then it will have a large ripple effect later on and cause huge problems. If early stages are very tightly controlled, with very small margins of error and variance, and with rigorous testing and measurement, later problems can be avoided entirely. 

This same concept is just as applicable in software development as it is in manufacturing. It’s part of the lean software development culture. It’s part of TDD / BDD. It’s part of other software development life cycle tools, too. I believe it’s part of Dan’s processes (he says so in his post), and my processes. Most importantly, though, it should be part of your processes.

Unfortunately, most people get the process wrong because they believe the fallacy of testing the engine after it’s installed in the car. I don’t think Dan believes this, based on what I am reading in his post and his code. I think it is unfortunate that he chose to repeat this analogy, though. It’s fallacy to think of testing engines like this. Of course you test the car as a whole and make sure the engine turns the transmission and everything else, correctly. But waiting to test the engine until it is mounted is a bad idea. 

But then, where Dan and I really disagree is in word choice – semantics. I can almost guarantee that Dan’s intention of using that analogy was not the way I read the analogy, only because of the difference in how I say “internal” when he says “private”. 

Semantics And Languages (Human And Computer)

“Semantics will continue to be important, until we learn how to communicate in something other than language” – this is a quote from Sharon Cichelli that I use A LOT. I love this quote as it expresses the importance of semantics in our languages – both human and computer. In human language, the difference between “apple” and “Apple” is a fruit vs a multi-billion dollar electronics corporation. Semantics are meaningful.

In computers, the difference between “internal” (my word) and “private” (Dan’s word). To me, “internal” is a component or thing that should not be exposed publicly. It may still need tests and other support code, though. In Dan’s example, the “Set” type for Lazy.js is an internal type. It needs to be tested because it is very important, and does some complex things. It should not be exposed publicly for those tests, though. Dan calls the Set “private”, which I think is a bad name for the type of code that it is. Private, to me, is an implementation detail and part of something else that is already tested / supported.

Again, semantics are meaningful. 

On Modules, Private And Internal Components

To understand my definition of “internal” vs “private”, you have to look at other languages. C#, for example, allows us to have what I call “internal” vs “private” classes and other constructs. When you build code in C#, you produce an assembly. The assembly is a module of deployment. It allows you to package up many individual classes, interfaces and other constructs in to a single (or multiple) deployable file(s). One of the advantages that you get in this is the notion of access modifiers for classes. Access modifiers allow us to explicitly say who can access the code in question:

  • public: any code
  • private: no one other than this class
  • protected: this class and any inheriting code
  • internal: this class and any code in this assembly
  • protected internal: any inheriting code and other code in this assembly

Sadly, JavaScript does not have any of these access modifiers built in to it at this point. JavaScript is improving and getting better scope management in upcoming versions, but I still haven’t seen much on access modifiers (if any – someone correct me, please!). My point, though, is that we need to take advantage of some of the ideas that other languages provide, even if we don’t have an explicit construct for these ideas. This is what we are doing with modules, after all. 

The AMD (Asynchronous Module Definite) module format, the “module pattern”, and all of the other types of “modules” (including the new ES6 module, and NodeJS modules) are borrowed ideas. Other than ES6′s and NodeJS, though, modules are an idiom and pattern more than an explicit construct. But even at that, we can take advantage of them to create “internal” vs “private” components and other code in our libraries and apps.

Without modules, we won’t be able to do “private” code. Without modules we won’t be able to do “internal” code. Without modules, we lose of things that other languages have in them. But that doesn’t mean modules are “one size fits all”. 

Module Of Development vs Module Of Deployment

A module of deployment is what most JavaScript developers are familiar with. Deployment modules allow you to say “hey, I’ve got this great library that hides all this complexity from you, and gives you this simple, elegant API from which you can use all of that complexity”. This is where the C# assembly comes in to play – it is the module of deployment. In JavaScript, AMD is a great example of a deployment module. It gives us the ability to deploy large chunks of code as multiple files, but have them all brought in through the “require” directives, thus simulating the behavior of an assembly in C#.  

A module of development, on the other hand, would be a module that is only available within the library. In C#, a module of development is achieved through having separate files that are compiled down in to a single assembly. The individual folders, files, namespaces, constructs, etc. in the assembly are the “modules” of development – the things on which we work when we are creating C# code. 

Unfortunately, JavaScript’s AMD does not have a separation of development vs deployment. A module is a module is a module. There is no way to say that a given file is a module for development purposes, but should not be part of the public modules that are exposed for use by 3rd parties.

NodeJS and NPM modules fix this by allowing each file to be a module and providing an explicit way to expose your public API through a single parent module that is referenced in the package.json config file for a module folder. You can work on individual NodeJS files (development modules) within your NPM package (deployment module) without fear of exposing your internal stuff to the external world. Within your NPM package codebase, you can write tests that require(“../whatever”) each file that needs to be included in the tests. Then for deployment through NPM, you create a file that only exports the things that the rest of the world needs to see. Specify that file as the primary file for the NPM package and you’re done. You now have development vs deployment modules – but only in a NodeJS environment (maybe with Browserify – not sure how that works, honestly). 

What we need, to be able to test our internal components – is any JavaScript environment (NodeJS, Browser, etc) –  is a development module that is completely separate from a deployment module, like the NodeJS and NPM combination. We need this so that we can test our “internal” components before package up the deployment module and ship it. 

Using Modules Of Development To Test Internal Components

As I’ve already said, NodeJS + NPM gives us development modules and deployment modules, already. In a browser, though, it’s not so clear-cut. There are no built in constructs for any module system, so we have to use idiomatic and pattern based development techniques to provide what we need.

In MarionetteJS, as I mentioned previously, I use development modules and deployment modules. The deployment module is a very simple module pattern – the IIFE that exports a single object which contains references to all the public things. My development modules are actually namespaced objects contained in most of the files. If you open the layout.js file, for example, you’ll see this  code:

Marionette.Layout = Marionette.ItemView.extend({
  // ... 
});

 

Using a simple namespace in each file gives me both a development module that I can use to test internal things, and an external deployment module exposed through the built backbone.marionette file, when I return the “Marionette” namespace from the IIFE. Truthfully, though, this isn’t a very good example of how development modules can be done. It’s a hard coded, somewhat convoluted mess when you look at the build scripts. A better way to do this would be to have a very small module definition / require setup that is only internally used by your code. Think micro-AMD-like modules with NodeJS-like require statement that can be used to pull internal code in to a test or another file that needs it. Then a build step would come along as compile all of the code in to a single file, wrapped up in a deployment module like AMD. 

For some pseudo-code with Jasmine and an internal module system might look like this:

MySys.module("Foo", function(){
return {
doFoo: function(){
return true;
}
}
});

describe("my test goes here", function(){
var foo = MySys.require("Foo");
var result;

beforeEach(function(){
result = foo.doFoo();
});

it("should be true", function(){
expect(result).toBe(true);
});
});

So that module looks an awful lot like AMD, eh? The difference is that this would be an internal module system that doesn’t have all the whistles and bells of AMD, RequireJS or the other AMD loaders. In fact, I would say that that this code should not include a loader. Use a script loader for loading scripts. The concern of module definition and script loading should not be stuck together, IMO. But the point is, having an internal module system would allow you to write tests against internal components without exposing those components to the public world. 

I once wrote an internal module system that worked like this, and used it in an open source project. I loved it. It made my life easier. Then I abandoned it because I thought people would be confused by it. Looking back, my abandoning it was a mistake. I need to re-build this internal module system, because I need a clear difference between modules for development vs modules for deployment. It is important.

Testing, Semantics And Modules, OH MY!

When it comes down to it, there are a few primary points that I really want to make, here:

  • Pay attention to semantics, and learn better semantics by stepping outside of your current knowledge base
  • Test “internal” components, not private methods
  • Use a module of development that is separate from the module of deployment

Hopefully you’ll be able to pull something useful out of these three points and the epic length post that tried to create an interesting journey on how we get there.


Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Derick Bailey

Derick Bailey is an entrepreneur, problem solver (and creator? :P ), software developer, screecaster, writer, blogger, speaker and technology leader in central Texas (north of Austin). He runs SignalLeaf.com - the amazingly awesome podcast audio hosting service that everyone should be using, and WatchMeCode.net where he throws down the JavaScript gauntlets to get you up to speed. He has been a professional software developer since the late 90's, and has been writing code since the late 80's. Find me on twitter: @derickbailey, @mutedsolutions, @backbonejsclass Find me on the web: SignalLeaf, WatchMeCode, Kendo UI blog, MarionetteJS, My Github profile, On Google+.
This entry was posted in AntiPatterns, Behavior Driven Development, CommonJS, Community, Javascript, Marionette, Modules, NodeJS, NPM, Principles and Patterns, Testing, Unit Testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://philosopherdeveloper.com/ Dan Tao

    This is great! A couple of thoughts.

    I really like your insight about development modules vs. deployment modules, which is something I hadn’t really put much explicit thought into before.

    As for the central issue: it looks like we really were on the same page from the start and, as you say, our original disagreement was over semantics (though I’m glad that happened, because I think this is an interesting topic!). This has motivated me to update the project README to read: “This function is not exposed publicly but I want to test it anyway.”

    With respect to the car analogy: I’ve never worked on cars; but I don’t think the argument falls apart as much as you say. In fact the interpretation in which it falls apart is the one based on your semantic distinction between “internal” and “private”; i.e. given that when I say “private” I mean what you mean by “internal” we are actually in agreement that you should test an engine before testing a car. (Funny how we actually make the same point and yet it feels deceptively like we’re disagreeing… curse you, semantics!)

    The only other point I want to make is about access modifiers. It seems to me that your distinction between “internal” and “private” is a useful one; however, I am also wary of the idea of incorporating the access modifiers of stricter languages like Java or C# into JavaScript because I believe these carry some unfortunate conceptual baggage. Or rather, they are blunt tools where greater precision is needed.

    Consider the very semantic misunderstanding that prompted this discussion. From my experience the “private” keyword has multiple applications. One is to signify “implementation detail” (your meaning); another is to simply mean “not exposed publicly” (mine). There is nothing inherent in the keyword to enforce one or the other use; it simply modifies programmatic access.

    Same with the “internal” keyword. In C# an internal component is visible only within the same assembly. You could layer additional meaning onto that—such as “internal components are beyond mere implementation details and are sufficiently complex to warrant testing”—but all the keyword itself does is restrict access. And interestingly enough what that means is that if you want to create a separate assembly for your tests, you actually need to mark all testable components as “public” anyway—or else use reflection or something crazy like that.

    I guess what I’m getting at is that if we’re striving for semantic clarity, then perhaps maybe rather

  • Kate Jo

    I just wanted to weigh in on the car example. I think that from a purist standpoint, the engine should have a public API of its own – it just shouldn’t be in the car’s public API.

    Consider using a 3rd party library to do something. The responsibility for testing that library falls to the library, not to you. Similarly, the responsibility for testing the car engine falls to the engine, not to the car.

    Technically, a car just needs to move – conceivably, it doesn’t have to have an engine. So you test that the car moves, and if it does then you’re done testing that aspect of the car.

    Say you then tweak the engine to improve its efficiency, and then you run your tests of the car – the car still moves! Great! Unfortunately, it also explodes after 30 seconds. Of course, you have a test for non-explosion of the car which has now failed. A billion things could have caused the car to explode, though, and the only clue you have that it’s probably the engine is the fact that you just tweaked the engine.

    If I’m the driver, I don’t care if the engine works; if the car moves and doesn’t explode, I’m happy. But if I’m the transmission, then I certainly care whether or not my interface with the engine causes an explosion. I don’t care if the engine’s motors spin, though. But maybe something else in the engine cares about that.

    So it’s a matter of how you define public-ness. And that doesn’t always map cleanly to access modifiers in languages that have them. Some internal components are going to be more public than others, and the more public ones should be tested. The less public ones probably shouldn’t. How you go about defining which is which and the best approach to defining and testing them is what matters.

    As a general rule, I’d imagine that level of coupling (conceptually) is the biggest tell here. If Class B can really only exist in the context of Class A, then it probably isn’t very public. But if it exists on its own and does or could realistically facilitate Class C (and D, E, etc) as well, it’s up for consideration. You’re the consumer; you would be leery of using an untested engine or 3rd party library, so if your own code has that effect on you and development would be easier if it didn’t, then test it.