Learning Software Architecture

Software design/architecture is a strange beast. It feels that if you want to learn it, you should spend time in legacy systems and large codebases of rewrite a project 3 times to explore counterfactuals. A lot of books on the subjects are abstract and give such simple examples, they are useless.

The recommendations are often very good, for example Ousterhouts A Philosophy of Software Design, but seem to be on software development in general, not actually software architecture in particular.

For that, I would recommend the classic texts, such as Software Architecture: Perspectives on an Emerging Discipline (Shaw/Garlan) and really anything you can find by Mary Shaw. Including more recent papers that explore why the field of software architecture did not go the way they foresaw, for example Myths and Mythconceptions: What Does It Mean to Be a Programming Language, Anyhow? or Revisiting Abstractions for Software Architecture and Tools to Support Them

More practically: look at why Unix pipes and filters and REST are successful, and where they fall down and why. Hexagonal architecture is also key.

And a plug for my own contribution, linking software architecture with metaobject protocols as a new foundation for programming languages and programming: Beyond Procedure Calls as Component Glue: Connectors Deserve Metaclass Status. An answer to Mary Shaw's Procedure Calls Are the Assembly Language of Software Interconnection: Connectors Deserve First-Class Status.

Answering the question: if procedure calls are the assembly language, what might a high level language look like? And also maybe that software architecture might have a brighter and more practical future ahead of itself.

This made me think of the wisdom of learning between Confucius and Laozi in ancient Chinese philosophy.

Confucius treats learning as cultivation: you do not really know something just because you were instructed in it. You know it by practicing, reflecting, making mistakes, and gradually developing judgment.

Laozi gives the complementary warning: “In pursuing learning, every day something is added. In pursuing the Tao, every day something is dropped.” Mastery is not only accumulation. It is also subtraction: removing unnecessary abstraction, ceremony, cleverness, and control.

Software architecture seems to need both. You learn it in a Confucian sense, by doing real work and living with the consequences. You improve it in a Taoist sense, by noticing when the system has accumulated structure that no longer serves the people, incentives, and constraints that actually shape it.

That is why the article’s point about incentives resonates. Architecture is not just what you design on paper. It is what survives contact with the organization that produces and maintains it.

In this vein, I really recommend "Architecture of Open Source Applications."[1] It's a book series where you learn architecture by example, with each chapter written by a maintainer of the project in question. This lets you learn not only what the architecture is, but what are the constraints that shaped it, usually history and changing project visions.

Not all chapters are equally good or equally interesting, that's the curse of a multi-author book, and all of them are dated, but I think the book is worth reading nonetheless.

[1] http://aosabook.org/

I would like to spend my time more on gaining a mental model of the projects I work on, but I get very demotivated if I start disliking things like the programming language, certain arch. Choices or anything that gets too complex that doesn't seem like its worth my time

It's heavily dependent on the project, but I feel like working as a "fullstack dev" kind of removes the fun of programming. I'm already spending 40 hrs a week looking at the most dull project I can imagine

I think this really down plays the value of mental model or strategies for organizing code. Take a compiler: often described as a sequence of transformations on an AST, taken to the extreme in the nanopass framework. That's a really useful mental model, and you can extract that model and apply it in other contexts. For example, many business applications are a sequence of transformations on JSON. So they're basically compilers. That can be good architecture in the right situation.

You don't have to call a sequence of transformations a compiler. You can say your AST is an algebraic data type, and your transformations are folds (or structural recursions; same thing). Now you have an abstract model that isn't tied to a particular application, and you can more easily find uses for it.

If you know a bit of maths you might wonder about duals. You will find codata---objects---are the dual of algebraic data. Ok, now we're programming to interfaces. That's also useful in the right context. What's the dual of a fold? An unfold! So now we have another way of looking at transformations, from the point of view of what they produce instead of what they consume. At this point we've basically reinvented reactive programming. And on and on it goes.

You can find most of this in the literature, just not usually presented in a compact and easy to understand form.

(Note, the above description is very quick sketch and I'm not expecting anyone to understand all the details from it alone.)

Shameless self promotion: the book I'm writing is about all these concepts. You can find it here: https://functionalprogrammingstrategies.com/

Most codebases eventually take on similar shapes, depending on their size and on the IoC model imposed by the framework.

And when you try to prevent that IoC from leaking into the domain too much, the design often starts to look like hexagonal architecture.

Programming often feels like inventing a new form, but in the end we tend to converge on the shapes that previous programmers already discovered.

The Gary Bernhardt talk is truly special. Lots of concepts which will lead you to other interesting places