Non-DI code == spaghetti code?

A previous post describing how Dependency Injection differs from using a Dependency Injection container (largely in the context of static languages) contained the following table:

	No DI Container	DI Container
No DI	Spaghetti Code	Service Locator Anti-Pattern
DI	Manual Dependency Injection	Enhanced Dependency Injection

Some took exception with the characterization of non-DI code as spaghetti code. My intent was not to be derogatory but was to succinctly describe the increased complexity of the control flow and the increased coupling of dependencies that occurs without dependency injection (due to the mixing of creation and logic concerns).  This post examines the use of this label and also contains an invitation to suggest an alternate metaphor for describing the structure of non-DI code in the context of an explanation of dependency injection.

Wikipedia describes spaghetti code as "a pejorative term for source code that has a complex and tangled control structure."  I hadn't intended to be uncomplimentary, but I do assert that the mixing of concerns caused by the lack of DI does indeed create a more complex and tangled flow.  Without dependency injection:

- creation concerns are mixed with the control flow (that is, with the application logic)
- object lifetime concerns are mixed with the control flow

Because creation instructions unrelated to the application logic are mixed in with the control flow, the control flow is (in my opinion) more complex and tangled without DI than with DI.  However, 'more complex and tangled than X' doesn't necessarily mean 'complex and tangled'.  So at what point does the mixing of concerns or additional complexity move from acceptable to spaghetti code?  Is the increased complexity in flow without DI enough by itself to merit term spaghetti code?  How many responsibilities can a unit of code have before turning into spaghetti?   How tight can couplings become before they are too tight?  I don't think that these questions can be answered outside of an individual's judgement within a particular context.  For my context in the original post of a high level comparison of the structural differences between DI code and non-DI code, I found the metaphor illustrative.

I also consider it more likely that without DI there will be increased flow complexity and dependency complexity due to reliance on global state (such as GoF-style singletons) and service locators.  Without a context-independent dependency management strategy like DI provides, I feel that these problematic dependency management strategies are more likely to be used (and hence mixed in with the logic).  This is of course not a given, however.

Beyond control flow concerns, I find it illustrative to use a spaghetti code metaphor in relation to object graphs.  When the dominant paradigm was procedural, it made sense to define spaghetti code strictly in terms of control structure.  But do people really write (for example) many goto statements these days?  Now that object-oriented programming is the dominant paradigm, high coupling between components seems to be a more common problem than, say, unstructured branching.  Of course, complexity in OO control flow could be considered a side-effect of high-coupling between objects since the objects in a system express the control structure.  Use of a spaghetti code metaphor is entirely appropriate in my opinion for describing high, unnecessary coupling among objects.

Is it possible to write good code without DI?  Of course.  People have been doing that for a long time and will continue to do so.  Might it be worth making a design decision to accept the increased complexity of not using DI in order to maximize a different design consideration?  Absolutely.  Design is all about tradeoffs.  But the point is that this should be a conscious decision informed by an understanding of what is being lost and what is being gained.

It is of course possible to write spaghetti code with DI too.  I did not mention this in the original post because it was about contrasting properly-applied DI with other alternatives and was not about describing poorly-applied DI.  My impression is that improperly-applied DI leads to worse spaghetti code than non-DI code.  It's essential to understand guidelines for injection in order to avoid creating additional dependencies.  Misapplied DI seems to involve more problems than not using DI at all.

I suspect that much of the antipathy for dependency injection stems from experience with poorly-applied DI.  Someone might start on a project where DI has been misapplied or where there has been 'container abuse' and then conclude that DI is nothing but hype and that its practitioners are a cargo-cult.  It would be understandable in that situation to not distinguish between the baby and the bath water, but it would be a mistake to ignore a valuable tool because of the result of its misapplication.  If someone attempts to use a lawnmower to paint a house, should they then conclude that lawnmowers have no value after viewing the result?

In any case, my purpose in using the term 'spaghetti code' was to illustrate the structural differences between a non-DI approach and a DI approach for those who may not yet understand the difference.  I think the metaphor does work in this context, but the term is indeed pejorative.  If you have a metaphor that illuminates this difference without being uncomplimentary, please leave a comment.  I'm interested in suggestions.

How to write testable code

This post is my attempt to summarize the high level basics of writing testable code.  My approach is heavily influenced by the work of Misko Hevery and is focused on creating loosely-coupled components using Dependency Injection (but not necessarily using a DI container).  This post describes how to structure your code to use Dependency Injection effectively.  If you keep these guidelines in mind, you’ll reduce your testing pain and have a nice, loosely-coupled code base.

Note that these guidelines mainly apply to static languages and mildly dynamic ones (such as, say, PHP).  Dependency management is also an important topic for highly dynamic languages, but different practices may be appropriate in those cases.

Writing testable code basically comes down to three guidelines:

Guideline 1: Categorize classes according to how clients should obtain instances of the class
Guideline 2: Follow the injection rules for each class category
Guideline 3: Structure your services according to the Single Responsibility Principle

Let’s examine each in turn:

Guideline 1: Categorize classes according to how clients should obtain instances of the class

There are two broad categories of classes for testability purposes:  newables and injectables.  A newable is a class that can just be instantiated (e.g. with the ‘new’ operator) as needed in code.  When a client (such as a test) needs an instance, it just ‘new’s the instance with the appropriate values.

An injectable is a class that should not be instantiated as needed in the code.  That is, instances should not be created with the new operator on an as-needed basis.  Instead, an injectable should be injected (typically via constructor injection) in its client [hence the name].  In other words, an injectable will be supplied to its client (usually as a constructor parameter) rather than having the client being responsible for constructing or locating this dependency.

However, these two base categories are too coarse to use as the only categories when creating a system, so I find it useful to classify objects into the pattern categories popularized by Domain Driven Design and then map these DDD patterns to the newable and injectable categories.  These patterns are:

Value Objects:  A Value Object is an immutable object whose responsibility is mainly holding state but may have some behavior.  A Value Object has no conceptual identity separate from its attributes (but this feature is not relevant for the purpose of testability).  Example of Value Objects might be Color and Temperature.  Value Objects are newables.  A client just uses the new operator to create on an as-needed basis.

Entities:  An Entity’s job is also mainly holding state and associated behavior.  Entities differ from Value Objects in that an Entity does have an identity separate from its contents (but again for testability this feature isn’t relevant).  Examples of Entities could be Account or User.  Entities are newables.  Just create them as needed in tests or production code (where the 'new's would typically be done in a Repository, which are very briefly discussed below).

For the purposes of testability, Value Objects and Entities are identical.  They are both newables.

Services: A Service performs an operation.  It encapsulates an activity but has no encapsulated state (that is, it is stateless).  Examples of Services could include a parser, an authenticator and a validator.  Services are injectables.

DDD also allows for the Factory and Repository class categories, but for the test purposes of testability, they behave just like Services, so I don’t discuss them separately in order to try to keep things simpler.

Guideline 2: Follow the injection rules for each class category

Each class category has rules for whether injectables are allowed to be injected or not.  Failure to follow those injection rules is one reason why tests can become ugly and why needless complexity can be added to systems.

As Misko Hevery states, a newable can only ask for other newables (or primitives) in its constructor, while an injectable should only ask for other injectables in its constructor.

A corollary to this rule is of course that newables should not be injected with injectables.  Why should newables not be injected with injectables? Because it will lead to pain for the client of the newable. And tests of course are clients, so it will lead to testing pain also.  Why?  Because injecting an injectable creates a dependency for the newable.  And if the newable has a dependency, then it can no longer be tested in isolation easily.  Every time a client wants to create this newable, it’s going to have to create this dependency also.  So every test that uses an instance of this newable is going to have to either create a mock for the injectable or use the real injectable collaborator.  In other words, the setup for any test involving this newable is now more complex.  The complexity can spiral out of control quickly since dependencies are transitive.  Instead of just being able to easily ‘new’ these collaborators,  the tests now have extra complexity because the rule to not inject into newables was ignored.

It’s not only tests that are more complicated but also production code.  Because it’s likely that a Factory will be needed to create the newable since the Service that it requires must be injected (i.e. the Service can’t be instantiated at the point of use).  Thus the newable is no longer really newable (i.e. able to be easily instantiated with the new operator).  This is needless additional complexity for the system.  Do yourself a favor and don’t inject Services into newables.

To state this in terms of our DDD categories,  Value Objects and Entities should never be injected with Services.  Only Services should be injected with other Services.

As noted above, injectables should only ask for other injectables in their constructors.  In other words, injectables should not be injected with newables.  However, there is one case where this rule can be broken without causing testing pain.  An injectable can be injected with a newable or a primitive value when the injectable uses the value as a configuration constant.  So, for example, you may have a Service that needs one URL when in production and different URLs for other environments (such as staging, beta, dev).  Then it is perfectly acceptable to inject this Service with a string instance or string literal for the URL as long as the Service treats this value as a constant.  In other words, the value serves to configure the Service at creation and can not be changed or substituted after that.  Such injection of a newable/primitive into a service will not cause any testability problems, so it is the exception to the rule.

Guideline 3: Structure your services according to the Single Responsibility Principle

This guideline is really just a way of emphasizing the Single Responsibility Principle when creating the Services in your system.  This guideline isn’t necessary for those already practicing good OOP, but I believe that it deserves special emphasis for Service design for two reasons.  First, a lot of testing pain results from ignoring it.  Second, in my experience, there is sometimes a great deal of resistance to the SRP and the idea of breaking a responsibility out into its own class is sometimes treated as heresy to commonly-practiced OO.  So I think that special emphasis of its importance is warranted when discussion writing testable code.

I informally categorize Services into two categories when I am developing:

Task Services: Task Services have a single functional responsibility.  They perform a single task such as, for example, validation, authentication, or parsing.

Controller Services:  Note that ‘controller’ isn’t used here in the sense of, say, an MVC controller but is used in the more general sense of the ‘Controller’ object role stereotype of Responsibility Driven Design.  The only responsibility of a Controller Service is to control and coordinate other Services.  Controller Services build a higher level of abstraction composed of other Services.  In GOOS language, Controller Services create a composite that is simpler than the sum of its parts.

Note that in Responsibility Driven Development, the object role stereotype of what I call a Task Service is ‘Service Provider’.  But since ‘service’ is overloaded between RDD and DDD, and ‘Service Provider Service’ isn’t very clear or helpful, I’m sticking with Task Service.  The important part isn’t the name but is instead to separate out functional responsibilities from the control and coordination responsibilities.

So why is it important to distinguish between Task Services and Controller Services when writing testable code?  Because mixing the responsibility of performing an operation with the responsibility of controlling and coordinating operations causes testing pain and complex test setups.

By isolating these responsibilities, each operation can be tested separately from the control of the operations.  If an operation is tested along with the control, then the operation will have to be tested in combination with other operations.  That is when ugly test setups happen.  Controller Services are easy to test because all they do is call other Services and have some conditional logic (such as for error values).  The Controller Services contain little functionality because they’re just rolling up other Services that contain the functionality.  So mock creation and use is easy in Controller Service unit tests.

Summary of testability for each category

Let’s look at each of our categories from a testing perspective:

Value Objects and Entities: Easy to use and easy to test.  Just ‘new’ as needed with the needed values.  Value Objects are particularly easy to test since they are immutable.

Task Services:  Easy to test because they’re stateless and their functionality is for a single behavior.

Controller Services: Easy to test because they are stateless and contain little functionality since they’re just aggregating other Services.

So each every class category is relatively easy to test in isolation.

Building a system out of these testable components gives you a great head start on creating a composable architecture while helping to reduce unnecessary testing friction.

In summary, when I have experienced testing pain personally or have seen it elsewhere, it is in general because of one or more of the following:

- a lack of use of Dependency Injection
- a misunderstanding of or misuse of Dependency Injection
- not separating classes cleanly into newables and injectables
- injection of injectables into newables
- inappropriate injection of newables into injectables
- disregard for the Single Responsibility Principle

So that’s my whirlwind tour of writing testable code.  I hope that I’ve helped to explain how some design mistakes can lead to ugly tests and testing pain and how you can produce loosely-coupled components with less testing friction.  If you're just beginning on the path to testability, I recommend these Misko Hevery videos as a great starting point for learning more about writing testable code:

The Clean Code Talks: Unit Testing
The Clean Code Talks: Don’t Look For Things

The relationship between Testability and Test-Driven Development

What is the relationship between Test-Driven Development and testability?  Is doing TDD sufficient to guarantee testable code?

In short, no. TDD is neither necessary nor sufficient for testability. You can write testable code without doing TDD, and doing TDD doesn't guarantee testable (that is, easy to test) code. Doing TDD without a clear understanding of how to write testable code results in unnecessary testing pain and doesn't yield the expected benefits.

TDD does help to produce more testable code by creating warning signs (that is, testing pain) when there are testability problems with the code. But TDD is a means to an end and is not an end in itself. Testability is the end. TDD helps to achieve the end, but it doesn't guarantee it.

Nat Pryce summed the relationship between TDD and testability perfectly in a tweet today about TDD and design:

TDD does not drive towards a good design, it drives away from a bad design. If you know what good design is, the result is a better design.

In the same way, TDD drives away from a lack of testability by creating pain for testability problems.  If you approach TDD with an understanding of how to write testable code, your testing path will be much smoother with more effective results. So how does one write testable code? That's the subject of the next post in which I will give testability guidelines and discuss how failure to follow the guidelines creates testing pain and friction.

What is testability?

What is testability?  My definition is:

Testability is the quality that allows a component to be easily tested in isolation.

There are two important parts to the above definition.  First, the component must be easily testable in isolation.  If something is able to be tested in isolation, but (for example) the test setup is difficult due to, say, excessive test setup or difficult mocking, then it's not testable.  That would be indicative of a testability problem such as high coupling and/or low cohesion.

The second important part is that the component must be testable in isolation. Testable has to mean something beyond 'able to be tested'.  By that definition, there's nothing that's not testable, since code can always be driven from the user interface or from a higher-level API.  If the definition is only 'able to be tested', then the concept loses all meaning.  So the ability to be tested in isolation is an important part of the definition.

In practice however, I use the term testability as an umbrella term for a group of interrelated qualities - loose coupling, high cohesion, maintainability, modularity, good design, readability, and (of course) the ability to easily test components in isolation.  Using testability as a stand-in for all of these qualities is not entirely accurate, but I find it useful.  

What makes testability suited for this purpose?  Because it's the closest thing that we have to an objective test for these qualities.  To a certain extent, something is either easy to test in isolation or not.  It's not a black-or-white.  But since testability is strongly correlated with all of the above qualities, it's the best that we have for a litmus test for the above qualities.

Strictly speaking, testability is really a side-effect of modularity.  The 'in isolation' part of my testability definition speaks to modularity (as does the 'easily' part, actually).  But I find it difficult to speak of modularity in practical and concrete terms.  However, given a chunk of code, I can easily discuss specifics about what makes it simple or difficult to test.  

So I tend to use the term testability as a general proxy for overall quality.  It is of course not strictly accurate, but it is very practical. 

Dependency Injection != using a DI container

What is the relationship between Dependency Injection and Dependency Injection containers?  Is a Dependency Injection container required to do Dependency Injection?  Is using a DI container the same thing as doing DI?  These were some of the questions that I had when I first began trying to apply Dependency Injection.  Most of the sources that I found at the time treated Dependency Injection as being synonymous with the use of a Dependency Injection container, as if they were inseparable and a container was a prerequisite for doing DI.  This article is my explanation of the relationship between Dependency Injection and Dependency Injection containers (also known as DI frameworks and IoC containers) for those who might be wondering about these issues now (in the context of static and mildly dynamic languages).

Dependency Injection and Dependency Injection containers are two completely independent concepts.  You can do Dependency Injection without a Dependency Injection container, and you can use a Dependency Injection container without doing Dependency Injection.  Here’s how I classify the combination of values:

	No DI Container	DI Container
No DI	Spaghetti Code	Service Locator Anti-Pattern
DI	Manual Dependency Injection	Enhanced Dependency Injection

Let’s take a look at each quadrant:

Spaghetti Code (No DI & no DI container used): Consider the spaghetti code metaphor for a moment.   What does a metaphoric strand of spaghetti represent in software system?  A spaghetti strand is a unit of code [which likely serves as a dependency for some other unit(s) of code].  So spaghetti code then is where many units of code (i.e. dependencies) are intertwined throughout the system and are difficult to separate from the whole (just list spaghetti strands on a plate of spaghetti).  The system is highly coupled, and it’s difficult to change a single unit of code without changes rippling  throughout the system.  So why do I refer to code written without Dependency Injection as spaghetti code?  Without DI, construction (and lifetime) concerns are mixed with application logic concerns.  As a result, most units are likely to be highly coupled with several other units.  Often the coupling is so high that nearly everything is transitively dependent on nearly everything else.  DI is the antidote for highly-coupled code.

[Update: My use of the spaghetti code metaphor was not without controversy.  For further discussion of this issue, see this follow-up post.]

Service Locator Anti-Pattern (DI container used without DI):  As Mark Seeman notes, Service Locator is an anti-pattern.  A Service Locator causes a class to, as Misko Hevery says, lie about its dependencies.  The use of a Service Locator is the “look, don’t ask” school of design rather than Dependency Injection’s “ask, don’t look” philosophy.  In other words, with Dependency Injection, a component “asks” for its dependencies (typically by requiring them in the constructor) rather than “looking” for them (such as by querying a Service Locator / Registry / Context).  Service Locators make for a confusing experience for the client.  Just say no to the Service Locator abomination.

Manual Dependency Injection (DI used without a DI container):  DI at its core is about creating loosely coupled code by separating construction logic from application logic.  This is done by pushing creation of services to the entry point(s) and writing the application logic so that dependencies are provided for its components.  The application logic doesn’t know or care how it is supplied with its dependencies; it just requires them and therefore receives them.  The ‘Manual’ in the name means that dependency creation isn’t automatically handled.  In other words, you have to write code to instantiate the dependencies.

Enhanced Dependency Injection (DI using a DI container): The point to realize here is that Enhanced Dependency Injection is still Dependency Injection.  The important part is the “Dependency Injection” part, not the “Enhanced” part.  The enhancement is that the dependencies are automatically resolved (i.e. created) for you.  That is, you don’t have to write code to instantiate the dependencies. If you’re doing Dependency Injection right, whether you are doing Manual Dependency Injection or Enhanced Dependency Injection, almost all of your code looks *exactly* the same, differing only at the entry point(s).  References to your container should be very isolated, occurring only near the entry points.  The entry points are the only places that should be aware that you are using a container.  Everything else just gets supplied the required dependencies and doesn’t know or care where they came from.  Let me emphasize: a container shouldn’t affect the structure, design or implementation of your code except in a few isolated, well-defined places.  In other words, whether or not you are using a container is irrelevant for nearly all of your code.   A container does indeed buy you some convenience and some additional functionality.  But there’s no tool that can make your code Dependency Injection ‘compliant’; that is up to your design of the classes and their relationships, and that is the important part.

Getting started with Dependency Injection

My recommendation is to start with Manual Dependency Injection.  Forget about the tools (that is, DI containers).  Learn the basics, and understand the technique.   Learn Dependency Injection in a containerless way first.  As a side-benefit, when you work in a world where containers are not very prevalent (such as PHP, C++ or Objective-C), you’ll have no problem creating a loosely-coupled system by applying Manual DI.

Once you get how to apply the technique, then take a look at what a tool can do for you.  You’ll be able to apply to use the tools much more effectively by understanding the technique first.  I’m not knocking DI containers but simply stating that to make effective use of any tool, you have to know how and when to apply it.

When combined with good OO design and an understanding of what to inject and what not to inject, Dependency Injection is the best technique that I know to produce loosely-coupled, testable software.  Learn how to create loosely coupled systems with Manual Dependency Injection.  Don’t let a tool blind you to the technique.