The large-scale structure of software-intensive systems

Abstract

The computer metaphor is dominant in most discussions of neuroscience, but the semantics attached to that metaphor are often quite naive. Herein, we examine the ontology of software-intensive systems, the nature of their structure and the application of the computer metaphor to the metaphysical questions of self and causation.

1. Introduction

In the unfolding of the human experience, the meaning of the self, the nature of sentience, the dichotomy of the brain and of the mind are issues that have been deeply pondered and yet that remain profoundly without closure. One question in particular stands out: is the self emergent, or is the self a product of external causes? The journey of consideration for this question quickly leads to two other questions, even more general in nature: Can any significantly complex system exhibit interesting behaviour without some element of externally applied top-down causes? Indeed, is there even such a thing as a closed system?

This paper examines these issues from the perspective of the computer metaphor.

2. The nature of metaphor

As Dr Ellis has noted, ‘the computer metaphor is dominant in most discussions of neuroscience’¹. Now, I am not a neuroscientist: the only defensible qualifications I have in that domain are the possession of a brain, a mind and a singular instance that I self-identify as an I. However, I do have a modest understanding of computing, and thus as an outsider (coming from the domain of software-intensive systems) looking in (to the domain of the neurosciences), I suggest that one must be careful not to apply that metaphor too naively. To do so, if I may be direct, would be akin to my trying to reason about the New York Stock Exchange by studying the traces from a few digital logic analysers scattered about the globe. These may be the only things that I think I can measure. I undoubtedly would learn some things. But I doubt I would learn much about macroeconomics.

In every age, the then contemporary context of human life frames how we approach the universe and offers us a language whereby we may describe and reason about the structure and the dynamics of that universe. Indeed, it is likely that we are mental prisoners of our own making, for it is unusual—if not extraordinary—to be able to transcend the shackles of the metaphors that surround us. As Lakoff [1] so beautifully describes in Women, Fire, and Dangerous Things, ‘to change the very concept of a category is to change not only our concept of the mind, but also our understanding of the world’. Lakeoff & Johnson [2] elaborate in Metaphors We Live By by noting, ‘the way that we think, what we experience and what we do every day is very much a matter of metaphor’. In The Category of the Person, Maus [3] suggests that there are several identifiable epochs in the history of the meaning of the self. Recasting Maus's point of view into the context of Lakeoff's worldview, I would suggest that these epochs are a product of the metaphors of their time. Specifically, Maus enumerates the following categories of self: in the tribal cultural, the self as personage, playing a specific role in the myths of the time; in Greek and Roman culture, the self as persona, wearing particular masks as required of the society; influenced by the Stoics and early Christian practice, the self as a moral being; beginning with Kant, the self as a psychological person, whose existence derives from biological and physiological consequences, only to be codified in the social and political structures of the time.

The construction of Maus' historical timeline is open for considerable debate. His is a particular dead-white-guy-Western-centric point of view that sidesteps the rich legacy of Asian cultures and religions, for example; also, he stops short of exploring the idea of self-as-a-machine, a concept that flowed first from the Industrial Revolution and now finds an anchor in our self-styled Information Revolution. However, this debate is not the focus of our current discourse. Without fussing over the details, it is fair to say that in different times and in different cultures, our understanding of self has changed, and that our understanding is generally always couched in the language of the then contemporary metaphors.

3. The computer metaphor

In early-recorded history, citizen scientists such as Yan Shi and Hero of Alexandria, and then Al-Jazar built automata that mimicked some aspect of nature or of the human figure. In the flowering of the Renaissance and then the Enlightenment, catalysed by a deeper understanding of the mathematical and scientific foundations of the universe, the concept of man as a machine emerged. As the infrastructure for creating mechanical devices of greater precision matured, cunning devices such as von Kempelen's Mechanical Turk (which played a fair game of chess albeit by fully human means) and de Vaucanson's Canard Digerateur (which simulated the process of ingestion and excretion in a duck) were created. While these were little more than amusing objects of conversation, de la Mettrie's work, Man a Machine, introduced the highly controversial (at the time) philosophical idea that man was indeed nothing more than a machine.

This concept—of man as nothing more than a biological mechanism, without spirit—and its corollary—the computer-as-a-brain—gained considerable traction as computers first entered public view in the 1940s and 1950s. In particular, Grey Walter's autonomous robots and Norbert Weinter's writings on cybernetics brought substance to this point of view. Even more recently, this concept is echoed by many in contemporary philosophy and physics. Stephen Hawking, for example, writes ‘I regard the brain as a computer which will stop working when its components fail. There is no heaven or afterlife for broken down computers; that is a fairy story for people afraid of the dark’ [4].

Ignoring the philosophical or emotional reactions, one might have to such a proposition, a calm analysis would suggest that one might reasonably ask what semantics Hawking attaches to the term ‘computer’.

Turing [5] pondered that issue quite thoroughly in his exploration of the computability of the mind, propelled by a thought experiment he called the ‘imitation game’. That work in turn was an outgrowth of his thoughts on computability and the limits of what could and could not be calculated automatically. Turing's investigation led him to devise a Turing machine whose structure consisted of

an infinite memory capacity obtained in the form of an infinite tape marked out into squares, on each of which a symbol could be printed. At any moment there is one symbol in the machine; it is called the scanned symbol. The machine can alter the scanned symbol and its behaviour is in part determined by that symbol, but the symbols on the tape elsewhere do not affect the behaviour of the machine. However, the tape can be moved back and forth through the machine, this being one of the elementary operations of the machine. Any symbol on the tape may therefore eventually have an innings. Turing [6]

The importance of Turing's contribution to theoretical computing cannot be understated. Among other things, the notion of Turing completeness gives rise to a grounded meaning of computability. Specifically, the Church–Turing thesis asserts that any decidable effect that can be computed can always be computed by a universal Turing machine.² Furthermore, no matter the degree of perceived complexity at the top, if at the bottom of the abstraction stack there exists Turing completeness, all layers of abstraction above may be considered computable.

Ultimately, Turing drew the conclusion that it would indeed be possible to devise a machine that could pass the imitation test. By his semantics, the brain was indeed computable.

Obviously, however, his prediction of this coming to pass by the year 2000 has not been made manifest.

The computer metaphor is thus an understandable one—it does have many appealing elements that offer a predictive framework for reasoning about a variety of non-computer systems such as the mind, but the reality that underlies that metaphor is far more richly textured that one might first realize. Indeed, I assert that software-intensive systems are among the most complex artefacts ever created.

4. Systems

Note that I use the term systems rather than programs. In my world, all economically interesting computing thingies³ are very much systems—they have a unique identity, they have discernible boundaries, they are composed of reducible constituent parts, they exhibit certain behaviours as a whole and—unless they are closed systems—they live as a society of systems with which they dance in context with their peers. By software-intensive, I mean to suggest that these thingies are manifest in the co-dependent and co-evolved artefacts of software and hardware. While hardware represents the physical form of a computing thingy, any such device would be no more powerful than a rock were it not for the software that directs its actions; at the same time, any chunk of software without a hardware platform on which it may execute would be a spirit in search of a body, neither alive nor dead but rather an unrealized entity. Hardware is the visible platform, the place where computation takes place; software is the collection of instructions, hidden from view, that animate that hardware and draw it to action.⁴

Stated in a more lyrical fashion, one might say that software is the invisible thread, and hardware is the loom on which we weave the artefacts of computing.⁵

From the perspective of computability, whether something is made manifest in software or in hardware is irrelevant. As long as the programming language is Turing complete and as long as the hardware is Turing complete, all software-intensive systems constructed upon such a foundation are themselves Turing complete. Furthermore, it is possible to take a software artefact and move it to hardware and vice versa without impacting the completeness of the system as a whole.

The semantics of the term system is one that may exhibit dissonance when explained from the point of view of different domains, and so a more precise statement of its meaning here is in order.

In this context of this current investigation, a system has been defined as

a collection of individuals in which some of the intrinsic properties of the collection and their joint results are not strictly deducible, in principle, from the properties of an spatio-temporal relations among the composing individuals. Some feature or features of the interrelations among the parts give rise to novel systemic properties and forms of causal efficacy.

In systems engineering, we use a subtly different definition. As adopted by the International Council on Systems Engineering, using the definition from Rechtin

a system is a construct or collection of different elements that together produce results not obtainable by the elements alone. The elements, or parts, can include people, hardware, software, facilities, policies, and documents, that is, all things required to produce systems-level results. The results include system level qualities, properties, characteristics, functions, behaviours and performance. The value added by the system as a whole, beyond that contributed independently by the parts is primarily created by the relationship among the parts; that is, how they are interconnected (http://www.incose.org/practice/fellowsconsensus.aspx).

I prefer the latter definition; whereas both emphasize the importance of relationships in the semantics of a system (the whole is greater than the sum of its part, because of the reification of the relationships), the latter is more general (it attends to other than individuals) and more comprehensive (especially in regard to the meaning of system level behaviours).

5. Peopleware and bits at the bottom

I also use the terms artefact and created to acknowledge that humans directly or indirectly (though other computing thingies) make a computing thingy.⁶ Developing hardware is fully an engineering activity, involving today the creation of digital circuitry containing hundreds of millions of transistors, each transistor forged from a hundred or so atoms. Similarly, developing quality software thrives on the cusp of engineering and art, of mathematical formalisms and creative writing; it is not unusual to find tens of millions of lines of code in any given software-intensive system, each line of code originally crafted by some person. Furthermore, a quick back of the envelope calculation⁷ suggests that, worldwide, some 33 billion source lines of code (SLOC) of new or modified software are produced yearly, giving approximately one trillion SLOC produced since the late 1940s (when high-order programming languages began to gain traction).

Collectively, these artefacts have changed the way we live, and it is at once surprising (that relatively so few people have laboured to produce these artefacts) and humbling (that these artefacts have woven themselves into the interstitial spaces of the world).

Software-intensive systems are created through an astonishingly labour-intensive activity. Most software-intensive systems of value go through the incremental and iterative cycle of development, deployment, operation and evolution, each phase of which transforms the system's architecture. In that regard, the process of building software has certain parallels to the process of building things. Small software-intensive systems are like doghouses: they do not require any blueprints and are largely disposable. Modest to large software-intensive systems are like houses or skyscrapers: they entail more cost and risk and therefore best practices demand more rigorous blueprints, better tools, significant testing, a risk-driven process and accountability. Ultra-large, long-lived software-intensive systems exhibit obduracy, and thus are closer to the problem of the organic growth of a city and the attendant activities of urban renewal [7].

We can and do construct software-intensive systems whose behaviour surprises us. For example, there have been automatic theorem-proving systems that have yielded novel proofs that no human had ever thought to derive; projects such as Massachusetts Institute of Technology's (MIT) Kismet and Honda's Asimov offer robots that are astonishingly engaging from a social perspective; even IBM's Watson is able to make human-like leaps of ‘intuition’ when attending to the nuances and puns one finds in playing the game Jeopardy!

Nonetheless, every software-intensive system—no matter how complex or how surprising its emergent behaviour—is built by humans; there is always an outside force that brought these artefacts into being. Software-intensive systems do not spring up whole from the primordial soup of bits.⁸ Furthermore, every software-intensive system is simply bits at the bottom; there is no magic, there is no Deus ex Mechania necessary to explain the operation of a software-intensive system. If there is anything we may call magic, it arises only because the modular, hierarchical layers of abstraction formed upon those bits give us the illusion of simplicity (and which we name as magical because their mind and their matter are obscure). It may be that considerable complexity obscures the meaning or the operation of a software-intensive system, but ultimately, such Turing complete systems are also decidable.

It is important to note that this apparent complexity is imposed upon a software-intensive system from the outside, from the top-down. We may have a vision for a software-intensive system which we morph into more precise statements of requirements which in turn are made manifest in strings in our programming languages, which in turn are transformed (usually by other software-intensive systems) into some instance of the equivalence class we call software, as, for example, holes, or physical configurations, or charges. At each level in this chain from top to bottom, there is nothing that cannot be reduced to something simpler, something Turing complete.⁹

Nonetheless, as another lyrical aside, there is a curious inverse relationship between the concerns of theoretical physics and those of software-intensive systems. In physics, one considers the fierce complexity of the universe and works to tease apart each string and make transparent the simple models from which all things are made. In the engineering of software-intensive systems, we take very simple things (Turing complete machines are wonderfully simple mathematical constructs), manufacture huge, tangled, dripping hairballs containing millions of lines of code and then unleash them into the wild, expecting that they will fade into the shadows, the best ones ending up completely invisible.

6. Complexity

A deeper examination of complexity is in order [8]. One may use classical system measures to name the complexity of the hardware components of computing thingies, but measuring the complexity of its software components is less well codified.

As Brooks [9] has so eloquently noted, software-intensive systems embody an essential complexity, a complexity that is baked in to the very domain and is hence inescapable. Furthermore, software-intensive systems are discrete, and that brings along with at least four fundamental challenges that contribute to their complexity.

First, discrete systems may exhibit non-continuous behaviour. In an analogue world—the world in which we live—the laws of physics are continuous;¹⁰ there are no discontinuities. Thus, as one throws an apple into the air and watches it fall to the ground, classical physics can predict its parabolic path; we would not be surprised to watch its graceful arc. However, were we to throw a digital apple in a virtual world, no such laws of physics need intrude¹¹ and thus it would be possible—and it sometimes happens—to see said virtual apple zig and zag or stop in mid air, only to rapidly shoot off in some random direction. Now, it may be that this is not the behaviour we desired or that we thought we programmed, but the discrete nature of the software-intensive system that gives rise to that virtual apple does not rule out such behaviour.

Second, as the size of a software-intensive system grows, so does its state space. Indeed, there is often a combinatorial explosion of states that rises in such systems. A program with one variable has a simple state space; a program with millions of variables has an enormous state space. Indeed, the state space of a given problem may be so large that any solution is intractable: there may be an algorithm that yields an answer, but it would require more computational elements than there are particles in the known universe or more seconds than there are in the life of the known universe to compute. As it turns out, it is not difficult to reach the point of intractability. In computer science, this is the domain of complexity theory, which has given rise to a class of problems called non-deterministic polynomial (NP) time. The travelling salesman problem is the classic example of an NP-hard problem: given a set of cities with measured paths among them, determine a route that is less than some specified distance. For a small number of cities and paths, a solution is possible, but as the state space grows, the solution quickly becomes intractable.

In that light, complexity [10] may be measured by consideration of the number of possible states that a problem might embody [11]. By that measure, at many levels of abstraction, a boulder is arguably simple, recognizing that any state changes it may undergo are few in nature and spread across geological time. In contrast, even the most modest software-intensive system will embody an enormous number of possible states, and it is only by abstracting away a myriad of details that we may have any degree of confidence in the correctness of that system. Now, lest the formalists berate me, remember that I am also talking about the state of software-intensive systems in the real world, wherein the discrete, digital world collides with the harsh reality of a very continuous, very analogue real world. The fact that we may have to attend to a combinatorial explosion of possible states in the digital world is nothing compared with the context of possible states in the real world.

Speaking of computing in the real world, consider this third element: software-intensive systems may be corrupted by unexpected external events. In this regard, we may think that our software-intensive systems are closed systems whose behaviour is entirely predictable, but, alas, the ugliness of the real world sometimes intrudes. A cosmic ray may flip a bit in our processor or memory; a hardware device might fail. While these are elements that any reasonable systems engineering process must take into consideration, especially where human lives are involved, the reality is that that there is no pragmatic means of defending against all such gremlins in the system. Furthermore, building fault-tolerant mechanisms introduces further complexity into a system, such that even more points of failure spring into being and the state space is further expanded.

Finally, there is one other pragmatic challenge that limits our ability to attend to complexity: we lack the mathematical tools and intellectual capacity to model and reason about the behaviour of large discrete systems.¹² Ultimately, there is little one can do to reduce the essential complexity of software-intensive systems; the best we can do is hope to manage that complexity. The primary mechanism we as humans attend to complexity is by abstracting: we see this reflected in our programming languages, our methodologies, our platforms. It is fair to say that the entire history of software engineering is one of the raising levels of abstraction. By abstracting, we are able to focus on the essence of a problem and plaster over the implementation of that problem. Again, the notion of Turing completeness comes in play: each layer of abstraction is built upon another, and at the very bottom, we have a Turing complete machine. As such, as long as we maintain semantic integrity, each level is decidable.

It is these layers of abstraction, coupled with the notion of information hiding, that give rise to the organized complexity of a well-structured software-intensive system. Each such layer may be recursively layered (meaning that the modular hierarchy of such a system flows from the top of the system down to its non-divisible bits) and each such layer may embody its own decidable complexity (by building upon the semantics of lower levels, whose implementation is hidden from the levels above).

7. Measuring complexity

While we speak of a given system being more or less complex than another, there are—shockingly—few meaningful measures of complexity for large software-intensive systems. Classically, complexity may be measured in two ways: computational complexity and Kolmogorov complexity.

Computational complexity, in general, attends to the time and space needed to carry out a particular algorithm. Generally, one may name the best case, worst case and average times for execution, and this offers a measure of complexity. In sorting, for example, a bubble sort will run (at best) in n time (where n is the number of items to be sorted) and at worst n² time. On the other hand, the average case for the quicksort is on the order of n log n (and the worst case is on the order of n²). A bubble sort therefore has a higher computational complexity than a quicksort, for it requires more resources.

While these sorts of complexity measures are useful from an operational sense, the very production of a software-intensive system introduces its own nature of complexity, and therein Kolmogorov complexity comes into play. Given an problem involving a simple input–output mapping (meaning, for every given input, there is a desired output that is a function of that input), it is reasonable to say that a million line program that solves the problem is more complex than a 10 line program. Kolmogorov complexity is thus more of a descriptive complexity—assuming that we are comparing implementations for the same problem space—wherein it is a measure of the size of the artefacts required to carry out such a mapping. If we think of a software-intensive system as just a string—and the Turing machine is in effect a reduction of any such system into such a string—then we consider the minimum string necessary to carry out this mapping. An implementation that requires a string of length x would be deemed simpler than an implementation that requires a string greater than length x.

While theoretically interesting and relevant, calculating the computational complexity or the Kolmogorov complexity of anything but the simplest algorithm is impractical. How does one reduce the corpus of Facebook¹³ to a single string in order to compare its complexity against, for example, Adobe Photoshop?¹⁴

Elsewhere [12], I have posited that software architectural complexity may be measured in terms of mass,¹⁵ the enumeration of things, the enumeration of connections and the presence of patterns across views. By mass, I mean simply the number of lines of code in a solution. While this offers a gross level characterization of a system, it does suffer from a number of problems: what is a line of code? How does a line of code in one language compare with a line of code in another? Are not there multiple ways of varying efficiency—even within the same programming language—of expressing a solution using different lines of code? In short, measuring the complexity of a system by measuring its SLOC is a somewhat useful but easily misleading approach.

As we will see, however, measuring things, connections and patterns is a more promising approach, but we need to explain the large-scale structure of software-intensive systems first (and will do so shortly).

As it turns out, practical software-intensive systems are even messier than one might dream theoretically. Indeed, there are several triggers of complexity that flow from the nature of their implementation and the nature of their interaction with the real world.

All but the simplest software-intensive systems embody a large number of moving parts, each with significant degrees of freedom and a significant number of potential interactions among those parts. While we strive to design programming languages and machines that make it difficult for a programmer to do violence to the software, reality is that any memory location in a system may, in theory, be visible to a process. This reality is what makes it possible for various species of computer viruses to wreak havoc among normally well-behaved systems. Thus, even if we find a problem of state space explosion in the problem space, then complexity is further triggered by the increased state space introduced by the elements of the solution.

Continuing, because software-intensive systems are built by humans, and because such economically interesting systems tend to evolve in fits and starts over time,¹⁶ it is difficult to develop a system that is perfectly textured. The lack of regularity within a system is yet another trigger of self-made complexity. For example, when simulating a weather system, calculating the forces of wind on a flexible structure such as a building, or simulating the explosion of a nuclear device, there is an embarrassing degree of parallelism in the real world that can be mapped directly to physical parallelism in the computing space. Finite element analysis, which underlies each of these three examples, involves the nearly independent calculation of millions upon millions of partial differential equations. While full of hairy mathematics, there is considerable regularity in such systems. In contrast, there are many classes of problems for which there is no such natural parallelism and hence no opportunity for such regularity. For example, the processing of airline seat reservations, the scheduling of parcels or people across multiple destinations, and the analysis of speech are all examples that involve tightly coupled data and algorithms that make it difficult to map perfectly to multiple processors; difficult, because there is far less semantic independence among processing elements.¹⁷

The presence of non-holonomic constraints also introduces complexity. Basically, a non-holonomic system is one in which history plays a role:¹⁸ the journey that such a system takes impacts its current state. Non-holonomic systems are common in the real world; robotic motion control and path planning are common examples in the space of software-intensive systems. Although not strictly non-holonomic, many interesting software-intensive systems are stateful, meaning that a thread of control may behave as a function not only of its inputs but also of its current state.¹⁹ A very simple example is that of a website: visiting a site once might generate a cookie on the client machine, such that revisiting the site offers up a subtly different page.

By contrast, functional programming (FP) eschews state. The essence of FP is the creation of computations that evaluate mathematical functions that are stateless and immutable. Pragmatically, the class of problems that are best made manifest by FP is small, for it is easy to do hard things in FP but very hard to do many easy things.²⁰

8. On simplicity

We have examined what complexity is; what does a simple software-intensive system look like?

Building on the observations of Herbert Simon, my experience would posit that simple, quality software-intensive systems exhibit the following characteristics.

First, they are filled with crisp abstractions. By abstraction, I mean simply a simplification of reality. A software object that denotes a customer is not the customer itself; it is an abstraction. In software-intensive systems, one builds abstractions in layers, the bottom most layer being closest to the machine and the physical world, the highest being closest to the metaphor in the problems space. By crisp, I mean that such abstractions have clear boundaries and are conceptually nearly independent from others.²¹ In the history of computing, we have seen a shift from algorithmically oriented abstractions to object-oriented ones. Algorithmically oriented abstractions are a reflection of early programming languages (such as Fortran and Cobol) that were focused on the domain of mathematical and symbolic manipulations; object-oriented abstractions are a reflection of more contemporary languages (such as C++ and Java) that are focused on more abstract manipulations. In algorithmic design, abstractions focus on functions, with data existing external/global to that function. In object-oriented design, abstractions combine data with the operations that are appropriate do that data.

Second, there is a clear separation of concerns among these abstractions. This separation may be vertical through the system, yielding the layers of abstractions described earlier. A major force that drives such layering is that first proposed by Parnas, namely information hiding. In such a strategy, the interface of an abstraction is exposed to peers and layers above, whereas the implementation of that interface is hidden. In this manner, there is clear separation between structure and behaviour and their manifestation. This separation may also be horizontal. At a given layer of abstraction, semantically distinct concepts will be manifest as conceptually or physically distinct abstractions. This separation is induced by the strategy of building crisp abstractions.

Third, there is a balanced distribution of responsibilities among these abstractions. By responsibility, I mean the behaviour associated with a given abstraction; by balanced, I mean that the behaviour of a system is crisply divided among different abstractions, so that no one abstraction does more/is significantly weightier than others. For example, in user interfaces, it is a best practice to have three distinct abstractions, one representing the thing being modelled, another representing the projection of a view of that thing and the third responsible for controlling the flow of information and the flow of control among a model and its views.

Fourth, such systems are filled with patterns.²² The example above (of user interfaces) is an example of such a pattern, called model–view–controller. Naively, one may view a software-intensive system as just a bundle of bits in layers of abstraction. However, in practice, the most well-structured systems are made simple by the presence of these kinds of patterns, which name repeatable structures that are woven through a system. Implementing such patterns often involved scattering and tangling. With scattering, a pattern is not manifest as a single abstraction, but rather involves a society of abstractions that work collaboratively to carry out that pattern. With tangling, a given abstraction may participate in many patterns, and thus if we look at its interface (or its implementation), we may trace some elements that correspond directly to a given pattern but that most often exist to support multiple patterns simultaneously.

Weaving patterns into a complex software-intensive system is a significant element of conceptual simplicity that transcends the physical complexity because patterns represent element of reuse at a relatively high level of abstraction.

9. The large-scale structure of software-intensive systems

A software-intensive system may be considered Turing complete; it manifests no irreducible complexity; it is potentially time-invertible (assuming that the information entropy of the system is not increased owing to external factors, such as by deletion or the introduction of a random variable) [13]; it may even be introspective, reflective and self-healing.

If we determine something to be computable (say, for example, the brain²³), then theoretically it could be manifest as a software-intensive system.²⁴ The fact that a software-intensive system has no irreducible complexity leads to the implication that even if we execute something as (formally) unpredictable as a neural net or a genetic algorithm on a Turing complete platform, no Deus ex machina is needed to explain its behaviour. Logical reversibility means that a software-intensive system may, functionally, ignore the arrow of time.²⁵ Reflection means that a software-intensive system may aware of itself, through meta agents (themselves being software-intensive) that observe the system, and thus permit the system to ‘know thyself’ and change and evolve.²⁶

A naive view of computing often focuses on the circuits from which hardware is formed, structured at growing levels of hierarchical abstractions (and their connections) from transistors to gates to circuits to chips to subsystems and so on. A slightly more advanced yet equally simplistic view of computing focuses on the expressions of the digital domain from which software rises, also found structured in layers of abstraction.

My expertise is in software-intensive systems; so I will defer a discussion of the parallel considerations of hardware: ontological nature of computing hardware is already well understood [14].

At the lowest level of abstraction in software—concerning myself first to data—we find bits, which may be chunked into words, and those into primitive data structures, then those into composite ones and so on. When we project a suitable symbolic meaning on them—their semantics—then we begin to approach the realm of information. Now, while data forms the nouns of our systems, we have also the verbs: expressions formed into statements leading to control structures composing functions leading to components and then on to subsystems and finally systems themselves.

As we introduced earlier, for much of the history of computing,²⁷ data and processes—nouns and verbs—were treated as relatively distinct. However, as the essential complexity of software-intensive systems grew, the then contemporary algorithmic-oriented languages (such as Fortran) and methods (such as structured analysis and design techniques) grew tired, and were replaced over time by object-oriented ones (such as Java and the notation and processes associated with the unified modelling language, http://www.uml.org). Most contemporary software-intensive systems are object-oriented in some nature, and so at the lowest level of abstraction we have, not surprisingly, the object. An object has identity, state and behaviour, and thus represents the fusion of data and action. This is also a wonderfully recursive concept: objects may be composed of objects may be composed of objects and so on. No object is an island, however, and thus there are rich relationships that abound: associations, whole/part structures and inheritance, to name the three most important.

And yet, there is more than just these objects and relationships to be found in a well-tempered software-intensive system. Three in particular come to mind as particularly germane.

First, there are communicating sequential processes [15]. The most basic software-intensive systems are fully sequential, having a single line of control that weaves through it; all others tend to have multiple lines of control, which are the warp and the woof in the fabric of the data and processing structures of such a system. This sort of parallelism may be manifest at many levels, ranging from individual software agents to the multi-core chip to server farms to the entire computing cloud.²⁸ Be it just a few processes and threads or many, such concurrency introduces tremendous complexity associated with critical regions, synchronization, race conditions, and livelock or deadlock.

Second, as we explained earlier relative to simplicity, there are the issues of patterns (http://www.hillside.net). A pattern denotes a common solution to a common problem, and most often is manifest as a named society of objects that collaborate in certain ways. For example, the Iterator pattern offers a mechanism for traversing the parts of any complex object independent of the representation of that object. Design patterns in particular are part of the texture—be it accidental or intentional—of every well-tempered software-intensive system [16] as well as of beautiful physical structures [17].

Finally, there are issues of crosscutting concerns, commonly called aspects. Aspects are akin to patterns, in that aspects may be manifest as a pattern; aspects are subtly different than just patterns, for they represent the abstraction of structure or behaviour as seen from the perspective of specific sets of concerns.

There are a multitude of consequences of the presence of this ontology of software-intensive system elements, but three in particular are useful in this context.

First, most such systems have no top. One may identify certain places of interest, multi-dimensional cusps in the fabric of that system, but there is often no real top, no starting place. Even to the user of a personal computer, the notion of a top is an illusion that rises from the relationship of that stakeholder (the user) and the computer (the object that appears to have a top). The user may see the on/off switch, a command line or a start menu as the ‘top’ of the system, but that is only a conceptual convenience useful to the human; the software-intensive system itself sees any of these inputs as just one message that flows into the system from some port, a port that is distinctly not at any ‘top’ that is so named by the computing system.

Second, interesting software-intensive systems are still input–output mappings, but not in the pragmatic sense as we might envision from a theoretical point of view. In the real world, any interesting software-intensive system S may ingest information I from multiple sources simultaneously or in some partial ordering of time, metabolize I and then yield I', along with a potential state change in S. While this at one level of abstraction is an input–output mapping, it is a bit messier than that: I may be semantically and physically very large, the state space of S may be enormous to the point of intractability, and the very exportation of I' may change the state of the real world. In other words, we can make some interesting theoretical conjectures about the behaviour of a software-intensive system, but such conjectures are significant abstractions of what goes on in the system itself.

Third, to understand, to reason about, to visualize any such system that lies above a certain level of complexity, one must use multiple points of view.²⁹

The importance of views is a relatively recent yet very significant advancement in our understanding of software-intensive systems from the perspective of its architecture. As defined by International Standards Organization (ISO) 42010, ‘architecture is the fundamental conception of a system and its environment embodied in elements, their relationship to each other and to the environment, and principles guiding system design and evolution’. In that sense, as 42 010 describes it, architecture serves as a naming of the essence of a system and serves as a blueprint, as literature, as a language of expression, as a set of significant design decisions.

As the standard goes on to note, every system has an architecture,³⁰ and every architecture may be represented by one or more descriptions.³¹ Furthermore, every such description is best formed from different views, each view representing a set of concerns from the perspective of a given set of stakeholders.

The ISO standard does not specify what views should be used to describe a software-intensive system, but my experience is that Philippe Kruchten's 4 + 1 View Model [18] provides a necessary and sufficient set of views for virtually all classes of software-intensive systems. Kruchten's views include (http://en.wikipedia.org/wiki/4%2B1_Architectural_View_Model):

• — use case view (highlighting sequences of interactions between the system and its context);

• — logical view (concerned with end user functionality);

• — process view (focused on the needs of system integration and the characteristics of performance, scalability and throughput);

• — implementation view (concerned with the needs of programmers and the issue of configuration management); and

• — deployment view (focused on the concerns of systems engineering, with attention paid to system topology, communication and provisioning).

In my experience (http://www.handbookofsoftwarearchitecture.com/index.jsp?page=Main), each such view is best described by enumerating the key abstractions and the key mechanisms present, using the language of patterns to express all crosscutting abstractions. At the system level, my experience also shows that a system's architecture can be partially described by an enumeration of the significant design decisions that may cross view boundaries. While all architecture is design, not all design is architecture, and so here we have the naming of the most significant decision,³² each of which represents some load-bearing wall in the design of the system as a whole.

10. The limits of software-intensive systems

Software-intensive systems allow us to form new worlds (http://secondlife.com) and yet, in any universe, such systems must still constrained by the laws of that universe.³³ Indeed, while the human imagination may be unlimited, there are limits to software-intensive systems, from the fundamental (information may not be transmitted faster than the speed of light;³⁴ a computation must preserve the second law of thermodynamics) to the theoretical (for example, the problem of NP-complete class complexity) to the semiotic (as Terrence Deacon observes, a symbolic system must be sufficiently rich so as to permit the creation of complex sentences, but also as Godel notes, any such complex system will give rise to sentences that are true but that cannot be proved so in that symbolic system) to the organic (how does one best architect a system? How does one best architect the organization that architects that system?) to the social (economic, moral and ethical limits are often forces that weigh upon such systems).

11. The questions

What we hope to have achieved is to make it clear that what Hawking might have meant by computer is a fiercely complex concept. While by no means judging his semantics—we have had no conversations with him on the matter—to say that anything is ‘just’ a computer glosses over the elegance of this metaphor.

So, in light of that metaphor, let us attend to the questions we introduced at the start of this discourse.

Is the self emergent, or is the self a product of external causes?

Strictly from the perspective of the computer metaphor, there can be no emergence save for the initiation by some external cause. Even if we consider systems of systems of systems, we must always accept that such systems were constructed by humans and thus their ‘laws of physics’ in which they were conceived and operate were ultimately external causes.³⁵

There do exist self-organizing software-intensive systems. Research into swarms of robots has yielded some astonishing, emergent behaviour. Genetic algorithms and neural nets have been applied to attend to problems that are otherwise intractable. And yet, from perspective of a computing system, we may be surprised by the behaviour of a system, we may name some behaviour as emergent because we did not program it directly, but nonetheless there is no irreducible complexity in a software-intensive system that makes the operation of a Turing complete system undecideable.³⁶

As to the nature of the meaning of self in the context of a software-intensive system, it does beg the question of the semantics of self. For our purposes, let us suggest that self is a property of reflection, in which an entity—within the system of the entity itself—possesses a state that represents itself. A further analysis of this definition is far beyond the scope of this present discourse, but I shall posit that, under that definition, yes, a software-intensive system may embody a sense of self. Introspective (and self-healing) systems are not uncommon, and as Minsky [19] has described, the illusion of self may simply be an emergent property rising from a society of nearly independent agents. While such a conclusion may be surprising, it is, in my framing of the issue, simply a tangled loop, as Hofstadter [20] describes it.

Can any significantly complex system exhibit interesting behaviour without some element of externally applied top-down causes?

(Hofstadter 1999, pp. 709–710)

Again, from the strict perspective of software-intensive systems, the answer is yes—and no. Yes, in the sense that, at the limit, no software-intensive system is truly self-starting; no, in the sense that one must question what top-down means. If we interpret the semantics of top-down to mean simply external causes, then again we must answer yes; if we truly require there to be a top, then we must abstain, for we cannot offer any meaningful semantics to the concept of top within a software-intensive system except insofar as one layer may build upon a lower one.³⁷

Indeed, is there even such a thing as a closed system?

From a theoretical sense, perhaps yes; we can posit such a system for the sake of understanding and reasoning about said system. From a pragmatic sense, we can also assume that our software-intensive system is closed: it exists in a particular well-defined context and it behaves thusly. However, in both cases, we know that these are just abstractions necessary for us to wrap our human minds around a vastly more complex reality. Software-intensive systems are never truly closed: they consume energy; they are impacted by forces outside the context that we may imagine exist. Indeed, the real world has a nasty habit of intruding on our illusion of closed and context.

12. On the computability of the mind

Take all the squishy bits of a human, a cat or a plant. Physically and chemically and biologically you will find astonishing similarities at a given level of abstraction, and yet there are fundamental, subtle differences at the bottom. Now, the current Wikipedia, in all of its languages, consists of over 3500 million symbols, a healthy 3.5 Gb of information. As late as the 1960s, the collective capacity of every computer in the world together fell far short of a single gigabyte; today, I can carry far more information around in my hand, on my phone. By comparison, Ralph Merkel (http://www.merkle.com) has estimated that the storage capacity of the human brain is a strikingly modest value, numbering perhaps only a few hundred megabytes of information. Similarly, the Power7 chip, which lies at the heart of IBM's line of mainframe computers, can process over 264 gigiflops, or approximately 10³⁰ floating-point operations per second. Much of this performance can be attributed to astonishingly fast circuitry; processing multiple streams of data in parallel contributes to this high performance as well. Again, by comparison, Merkel estimates that the human brain—a very parallel computer indeed—has the ability to process only about 10¹³–10¹⁶ instructions per second.

Why, then, is there such a gulf between what we know of computability and what it is to live as a sentient being?³⁸