The Emergence Of Knowledge In Software Development
At this point in my career, I’m convinced that software development is an empirical process. That is, we cannot predict the exact shape, size, complexity, … or any of a number of other properties … of the software that we set out to write. We can, however, generalize our expectations and set a known range of expected results based on some known factors – the team working on the project; the requirements of the project; the technology used; the target platform; the target audience; and many others. I’m not alone in this line of thought and I’m certainly not the first person to recognize this. I came to this conclusion by my own experience and by learning from books, blogs, and other sources of information that were created by some of the luminaries in our industry, such as Ken Schwaber 
Learning From Explicit Knowledge
“It’s very very difficult to explain something that has to be experienced first-hand.” – Scott Bellware via interview on Hanselminutes.
Because we are working in an empirical industry, much of the knowledge that is gained by an individual is entirely experiential. The person who wishes to obtain the knowledge in question, must experience the effects of that knowledge first-hand. Only when a person has experienced the effect of that knowledge, will they be able to see the actual knowledge and understand it. In order to experience the knowledge, though, the person must be given one or more explicit examples of that knowledge. They won’t understand the theoretical or abstract portion of the knowledge in the examples, until they have experienced the concrete effect of that knowledge by seeing that solution applied, and ultimately by creating the solution themselves.
Creating Tacit Knowledge
When a teacher is able to clearly and concisely convert their own tacit knowledge into an explicit form for a specific scenario, the student may be able to see the effect of the knowledge. When that explicit knowledge is stated in a context that the student already understands, the student will have a high likelihood of seeing the knowledge that underlies the solution. Once the student is able to see and apply that knowledge in a different context, they will have internalized it, creating their own tacit knowledge. As the student begins to apply their tacit knowledge to more and more situations, their own tacit knowledge grows. When that student then attempts to share their knowledge with someone else, they become the teacher and must convert their tacit knowledge into explicit knowledge… and the cycle continues …
This empirical learning cycle is the primary reason for having so much sample code and so many sample scenarios when we teach and learn about software development. It is why there are so many software development books, articles, blogs, help files, and other sources of development knowledge that provide a description of a situation, a description of a solution or pattern used for a solution, and an example of implementing that pattern in a concrete solution.
Applying Tacit Knowledge
When a person is able to take the examples that they have seen and apply the foundational knowledge to their own specific situations, they are creating tacit knowledge within themselves. They are able to see the value of the underlying knowledge from the example and apply it to another context, with a potentially different implementation. Every time a developer applies a principle, pattern, or solution to a different context – no matter how great or small the context difference is – they are increasing their own tacit knowledge of that principle, pattern, or solution.
We often measure a developer’s value in a team by their ability to apply existing knowledge into new or slightly varied contexts. We call it ‘creativity’, or being able to ‘think abstractly’. We give titles such as ‘senior developer’, ‘technical lead’, and ‘architect’ to the people who show a great ability to expand their tacit knowledge in this manner. The tacit knowledge that they now have is why the ‘great’ software developers seem to have so many options to choose from when implementing a solution. There are multiple contexts under which knowledge can be applied – multiple platforms, technologies, architectures, etc – that will all drive the choices made for the actual implementation. However, even within those constraints, our ability to creatively apply a theory or principle is often the difference between a novice and an expert.
Having the tacit knowledge of a principle, pattern, or solution, though, is only half of the equation in the emergence of knowledge in the software development industry.
Making Tacit Knowledge Explicit
In the book, Extreme Toyota
“Tacit knowledge is converted to explicit knowledge every time someone verbalized or writes down the knowledge they have embodied, which is a deep understanding based on experience.”
The very act of attempting to teach our own tacit knowledge – our experience and understanding of a given situation and the principles, patterns, and solutions that can be applied to that situation – is how we convert our tacit knowledge into explicit knowledge. Every time we walk over to a whiteboard with a colleague and we draw out pseudo-UML boxes and arrows, write snippets of pseudo-code, discuss how a particular solution can be best applied to a specific scenario, or otherwise discuss a scenario and the possible solutions for it, we are creating a form of explicit knowledge. The person who is attempting to learn is then looking for the effects of that explicit knowledge in order to understand it and be able to convert it into their own tacit knowledge. They are now engaged in learning from explicit knowledge.
Expressions Of Explicit Knowledge
The ability for a person to apply their own tacit knowledge in a concrete example based on a specific context is no small feat. Many software developers utterly fail at this due to our tendencies to be introverts with poor social skills. We must learn to express our ideas and understandings in ways that are useful to others, though. If we don’t, then there will always be a significant disconnect between the person with the tacit knowledge and the person without. This can lead to poor team dynamics and even dysfunctional teams with bottlenecks – team members that say “I can’t do xyz. go talk to …”.
The good news is that we can improve our ability to make tacit knowledge explicit – all we need to do is continuously try to convert it into one form or another. Write a paper or a blog post, give lectures, talk near and draw on whiteboards – we do whatever it is that we are comfortable doing, and then expand our comfort zone by learning how to express our knowledge in a new medium. Every time we express our understandings in a concrete form – especially when that form is in a semi-permanent medium like a document or blog post – the next person that comes along and finds that knowledge may be able to learn that much faster. They may be able to convert our explicit example into their own tacit understanding based on our observations and writings, rather than having to meticulously observe their own experience and output, attempting to extract foundational knowledge on their own.
A Virtuous Cycle
The process of learning in an empirical system involves the constant conversion of tacit knowledge into explicit knowledge by the teacher, and conversion from explicit knowledge into tacit knowledge by the student. Every time we engage in the process of learning, we are building upon our experience and understanding by accumulating the experience and understanding of those that came before us. When we teach, we help others build upon their experience and understand, through our own experience and understanding.
If we are truly engaged in a process of continuous learning, then we must first and foremost be students. We must be able to internalize and convert the explicit knowledge of others into our own tacit knowledge, experience the effects of applying that knowledge in our own specific situations, and then convert our experience and tacit knowledge back into explicit knowledge for the next person. In this cycle of learning and teaching, then there is no end to what we can learn. We will spiral upward and outward in a virtuous cycle of learning and teaching, becoming more and more valuable and knowledgeable with every step.