Ideas and books forming me as a Software Engineer

This September it was 15 years since I joined university to study Software Engineering, the official name was “Informatics”, but I think software engineering (or authoring and gardening as argued in another post) is better describing what I was interested in and, luckily, what I'm currently doing. Since then I've got some 10+ years of experience from working in the industry and a whole 15 years of continuous learning. As an exercise for myself I've set out to go through the major ideas I've picked up and used along with the people and books that collaboratively have formed me to the software engineer I am today. Hopefully you find this post at least a bit interesting and inspiring as well.


Object Orientation (OO)
In 1996, and the following years, OO was the new hot thing in software design, at least at my university. I learned to model domains (even though I didn't know that word by then) in objects using the OMT (Object Modeling Technique) notation by James Rumbaugh (later on one of the main contributers to UML). This way of abstracting the real world into objects with attributes, methods and relations very much appealed to me as it offered the tools to unambiguously document the analysis result by drawing a diagram. I did also believe my teachers when they talked about OO as the enabler of universal libraries of small reusable objects. Now I, and I'm sure they too, know this is never going to happen, but the OO-paradigm still brings structure for reuse within a system and hence is a great tool for honoring the DRY-principle. 
Ever since those first courses I started to think of OO as the way to design and build software systems. Later on I've come to see OO as the way of designing and building transaction based information systems where manipulation of state is the primary focus. Other types of systems might be better suited to use some other paradigm, but this type of systems for administering information is what I've been working with almost the entire time since I graduated. In university I learned the theory of OO, but we didn't actually turn any of our OO-designs into working software. We sure learned programming, but then it was to solve more traditional algorithmic problems. When leaving university I was all eager to learn how to do OO in real world projects. And boy, was I disappointed!
For several years I was employed in projects where I learned lots of things about programming and projects in general but nowhere was OO used to anything but data diagrams (now drawn in UML notation which I learned through Martin Fowler's “UML Distilled”), i.e. objects with attributes and relations but no behavior, that was later turned into database tables and data structures. All behavior was programmed into transaction scripts, either directly in the GUI components or as free standing functions or stored procedures. I was desperately looking for a real world example of true OO implementation because on my own I couldn't really figure out what it would look like in order to work properly. The first piece of the solution came when I got to read Craig Larman's “Applying UML and Patterns” . In this book he describes all aspects of OOA/OOD, how the software can be structured with boundary classes, controllers and entities, and shows by example code how it all comes together. Still, this isn't the approach intuitively encouraged by the structure in popular frameworks such as Java EE and Spring Framework. Out-of-the-box they rather suggests a static service structure operating on DTO:s, and that is, I think, the main reason why most systems are built with data and behavior separated and not using the true OO-paradigm.


Domain-Driven Design (DDD)
In a Jfokus 2008 tutorial I was first introduced to Domain-Driven Design. The three hour tutorial only scratched the surface of DDD but it was enough for me to understand that this was the description of how the full OO-paradigm is used to build real world systems. Soon after, I got to read Eric Evans' book “Domain-Driven Design - Tackling complexity at the heart of software”, which I have posted about previously, and since then I firmly believe that DDD is the approach to use for designing and implementing complex software systems.
DDD is not a method, nor is it simply a technique or an architectural pattern. I would rather call it an approach to software development, including analysis, design and implementation.
Perhaps what most of my colleagues first think of when DDD is mentioned is the design sketches I use to draw with the business domain at the center and technical integration packages all around. The domain being built on the conceptual building blocks, entities, value objects, repositories and services, as introduced by Eric Evans. But even though that part of the approach might be the easiest to pick up, and also important in building working software with DDD, I think what makes the biggest difference to me is the ubiquitous language. This practice of building a model with a shared language based on, but with more precise definitions than, the language spoken by the domain experts is the basis of my philosophy to always build the software so that structure and logic follows the business domain. I think that is the only way to guarantee a flexible design such that a small change to the business is always a small change to the software, never a big one. My experience is that the second you deviate from that principle, most often due to time pressure or pure laziness, you are asking for trouble further down the road. It might take a year or so, but it will come back and bite you!


Test Driven Development (TDD)
If DDD provides the approach on analysis, design and implementation, TDD is clearly what integrates that approach with quality awareness and assurance. TDD is my way to ensure a testable design and a correct implementation that is easy and secure to improve further. In a previous post, "Are you testinfected?", I told the story of how I got introduced to TDD, how I was skeptical at first and how I later on proved the usefulness to myself. Now I rarely, and definitely reluctantly, writes any code without first writing a failing test.


A Clean Coder producing Clean Code
Ever since the start of my career I’ve wanted to produce code that I’m proud of. However, the benchmarks for that assessment have changed over the years. Now I consider well tested and easy to read code being the standard to achieve. Since any piece of code is read much more often than it is written I think readability is prime quality measurement for code. Lately I’ve been reading both Clean Code and The Clean Coder by Robert C Martin. Both are excellent books. The first one giving good advice on how to make the code readable, the second really demanding reflection on what it means to be a professional software developer.


Agile with Scrum and Kanban
Another interest of mine is software development processes and techniques. At first I learnt how to deliver in a world of waterfalls, later I got to experience RUP. Despite all bad things said about RUP, I’ve always found the core messages on iterative, risk driven development to be right and frankly quite agile. However, agile approaches such as Scrum, Kanban and developmenttechniques gathered in XP take this a long way further. The major inspiration here has been Henrik Kniberg, first in his Jfocus tutorial in 2008 and through his two books on Scrum and Kanban.
Currently I’m working in a world of mixing and matching. I’m picking the best parts out of all those above to create a process and way of working that supports being as agile and lean as possible in an otherwise RUP-ified world. I think, at the current state it is somewhat of coming success.


“All problems are not worth solving”
That was one of the advice I got from an experienced colleague of mine many years ago when I was aspiring to go from being a sheer hacker to become a CapgeminiCertified Software Engineer. He probably gave me a bunch of other advice too, but this is the one I still remember. And I also think, for me personally, it might be the most valuable advice I’ve ever got. Since problem solving is my trade I happily go of designing solutions, then it is good to remember thinking for a minute or two about whether the problem really deserves being solved or if it is something we can as happily live with. At the bottom of this is that all resources are limited and needs to be applied where they make the best return on investment. On the other hand, we need to be careful not to spend too much time on deciding if a problem is worth solving, i.e. it is not always worth solving the prioritization problem if the solution offered is cheap enough.


The future
This pretty much sums up where I am today, but of course I will continue to evolve and in the near future I see the following ideas as the most interesting.
- Strategic design – How DDD concepts such as Bounded Context is applied at a larger scale to make strategic decisions on how to factor architectures and apply different design and development strategies. In most discussions “architecture” seems to mean frameworks, databases, application servers and technical layering. I think architectures would be much more interesting and usable if they concerned business domains.
- CQRS – I have only touched the surface of this approach to structure systems with separated command and query sides and event stores and messaging instead of relational database schemas and object/relational mappers. However, I think is an interesting approach and I definitely plan to dig in deeper.
- DCI  – Even though DDD has come to be my preferred approach I must admit the structure of the code base with behavioral code localized in every domain object has its draw backs. With requirements based on use cases or user stories, it is sometimes hard to answer the not so infrequent question “how is this UC implemented?”. DCI (Data, Context, Interaction) solves this problem through keeping the behavioral code in role objects implemented within a context representing a single UC. The code in the role objects are then injected into the data objects representing the domain. To me, it sounds like an interesting add-on to the ideas of DDD and in the future I like to continue exploring the benefits of this approach and the demands it poses on the implementation language.
- Functional programming – In order to build simpler programs that are safe to parallelize on multiple cores functional programming has gained new interest in the last years. Up until now I’ve lived only in the world of imperative programming and grasping the concepts and underlying mechanics of functional programming will pose a challenge. A challenge I’m happy to take on.
- Scala – Of all the new languages having emerged on the JVM I think Scala is the most interesting. It combines functional programming with additional powers in OO-programming without losing the benefits of static typing. All of which I think are interesting qualities. I think Scala might be the enabling programming language for many of the ideas I listed here for future exploration.

The value of using Domain Driven Design

From time to time I come across the relevant question of the value of using Domain Driven Design. Are there any hard numbers to use in convincing management about the benefits of DDD?

Hard numbers are really hard to get in software development since you never ever develop the same system twice and even if you did, it wouldn't be the same since you would have learnt a lot from the first attempt. But I'll share some thoughts and observations on using DDD.

By emphasizing communication with domain experts in developing the ubiquitous language DDD helps get you started on the right foot. Any system is simple from the beginning. The first set of functionality is never complex to implement since you have no other code to conform to and you tend to start with something basic. It means it is easy to just hack something together mixing levels of abstraction and business logic with technical complexity. Since first requirements are simple, the first complexity tends to come from unfamiliar tools like ORMs or other technical frameworks. Since we all are technicians at heart these things often become most important and the architecture tends to be more about technical concerns than the domain. All of which, we know, lead to a mess when requirements get more complicated.

By using DDD we are turning the focus away from technical concerns and instead focusing our effort on the business domain. With a bit of domain modelling, including test driven implementations, we get to investigate a larger set of requirements before going into technical details of persistence and other layers. A lean principle is to delay hard to change decisions to the last responsible moment, e.g. not deciding on a hard to change persistence model until we know what the domain really looks like. All in all, this leads to better architecture because the architecture is centered around the domain, not around technical frameworks.

Another aspect is complexity. What you consider complex depends on what you already know. I once worked with a large system where some parts were implemented using DDD and other parts were, what I would call, "undisciplined" transaction scripts. I think TS have their proper place and use, but implementing them in a way mixing business logic with technical concerns is never right. Hence "undisciplined". In this project I heard people complain about different things they thought was complex. Those people being unfamiliar with DDD but having a long history of working on the system thought the TS code with mixed concerns and very few abstractions was fine since they saw exactly what was going on and they "knew" what the context was and how everything worked on a detailed level. And they thought the DDD approach of separating things into different layers and building abstractions for business concepts just added complexity.

Then we had the newcomers, that didn't have any prior DDD-experience either, that struggled with understanding the domain. Even though DDD was new to them they pretty quickly picked up on the concepts and thought it was a great help in understanding the domain. To them it was the TS code that was most complex since it didn't make any difference between lines of business logic and those of database interaction.

These are two aspects on "the value of DDD". In another post I wrote about DDD being so much more than a set of technical building blocks and I think the most value from DDD is gained when wisely deciding on what parts to apply in each unique situation.

Is Domain Driven Design always applicable?

Disclamer: This blog post discusses use of DDD in information centric systems, which in my experience is most of the systems out there. It does probably not apply to technical software such as compilers and operating systems.

In my opinion the answer to the question in the title depends on how you define DDD. Do you do DDD if you do not use all technical building blocks? I think so. Is DDD more than the tachnical building blocks? I definately think so.

Personally, I would opt for using DDD in almost any situation but those that I know for sure will not evolve for more than a day or so and where I'm dead sure of the requirements. In any other situation I would start building the Ubiquitous language together with the domain experts to get a common vocabulary. This doesn't sound so much, but it is huge to avoiding expensive misunderstandings.

For the systems architecture I would always deploy the strategy to keep business logic and infrastructure code separated. It makes wonders for readability and testability. My rule of thumb is that business logic should be easily testable using unit tests (and end-to-end tests independent of the run-time environment) and infrastructure code should be so simple you generally need no other testing but a few basic integration tests.

For design my strategy is to keep the code close to the business in terminology, structure and logic. This is important to make sure that the impact of a change is about equal in size in both business and software. Here the ubiquitous language is an absolute need.

Then we come to the technical building blocks and now it is time for some choices. In general I think you could talk about two types of systems, or parts of systems; those mostly concerned with changes in object state and those mostly concerned with processing data streaming through the system.

In the first case entities, aggregates and repositories are a natural fit, in the second I think transaction scripts (in DDD context called domain services, since they do only concern domain logic, no infrastructure code, as per discussed above) are a nice fit. When the most important feature is to crunch some data, perhaps modify it and then route it further to some recepient (like another system or some persistent store) I think it is the "processing pipeline", i.e the stateless service code, that should be emphazised. So in those cases the internals of the data isn't very interesting and might be better left in some simple DTO format.

Whether going with entities or servicies I think the practice of using typed value objects for input parameters give a lot in terms of clarity and IDE-support when writing and reading the code. From a technical perspective, this is the easiest DDD building block to use and I think it has great advantages even if it is the only part of DDD that you apply to the system.

In conclusion, for the types of information systems I've been developing over the years DDD is the way to do it, however that doesn't necessarily mean everything should be entities or value objects. DDD is so much more.