Posted 2022–04–19.
Audience: people interested in programming languages.
The following is a rambling attempt at transcribing my stream of consciousness while thinking about Marcel Weiher’s 2020 Conference paper, “Can programmers escape the gentle tyranny of call/return?”. It’s not particularly coherent, but I’m happy (at the time of writing) with where I got to at the end.
This probably shouldn’t be read as a primary source, but rather as notes or a companion to the original paper.
I read it here on the Universität Potsdam website.
The big idea
Weiher wishes for a world in which the right tool is used for the job at hand. Right now, we use call/return very widely, but it is not necessarily the best tool. He claims that call/return architecture is designed for performing a calculation and returning a result when complete.
While this call/return mechanism is the best tool for parts of most applications, there are other components of an application which are not best expressed with call/return. Many of our modern programs wish to express things like “these values are always related in this way”, or “these things wish to communicate remotely”.
Obviously we can express these things, otherwise we’d be unable to write many classes of applications. However, a common strategy in mainstream programming languages is to use call/return code to represent these other architectures. This may take the form of very complex “engine” written in call/return code, which is interacted with by using a “builder” interface.
Weiher wishes for a world in which interesting architectural relationships are expressed directly in code, rather than in abstract objects and builders providing a facade for a complex underlying engine.
Pragmatic
At the start of section 4, Weiher explicitly states that the solution is to “generalise from this one particular architectural style to a method of programming that allows multiple architectural styles”, but programming languages “tend to at most allow extension, not generalisation”. His solution is to propose a language that has a curated selection of other architectural styles, which are chosen for being “pragmatic”.
Pragmatism comes right down to the core of the problem, in my opinion. The reason that call/return is ubiquitous in industry is that it is pragmatic. In the past, languages using call/return (such as C) were found to be extremely useful across a wide variety of situations, and future mainstream languages built on top of C.
This doesn’t say anything for call/return being intrinsically better, but it does go some way to explain why industry continues to use similar tools, even when more appropriate tools may be possible.
- New learners, for the most part, learn the popular tools
- Companies want to have a codebase that is easy to hire for
- Even those who are interested in exotic programming paradigms probably still work primarily in a mainstream language
So how do new features or paradigms ever get utilised by industry?
We see gradual adoption of exotic paradigms, but it’s a slow process. There are two ways for some part of an exotic paradigm to be introduced into an existing language:
- A change in the language
- A library using the existing mechanisms of the language
Some programming languages do make use of the first option. For example, C# has been steadily including features that are popular in functional programming languages. Some of these include lambda expressions in C# 3 and record types. These features are now first-class features with their own syntactic structures in the language. However, each feature requires a huge amount of effort from the language’s designers to ensure usability, backward-compatibility, and deal with all the other complexities of doing something new.
The features I mentioned above aren’t even exotic by Weiher’s standard. After all, a lambda expression is just a call/return function, but anonymous. A record type is just a struct with semantics that are handy for writing code in a functional programming style. Weiher even explicitly calls out functional programming as being just another form of call/return.
Go To Statement Considered Harmful
In the Constraint Connectors section, Weiher shows us Objective-Smalltalk syntax for expressing dataflow relationships directly within the language. This is contrasted with the use of an object-oriented call/return interface for setting up dataflow relationships.
Weiher claims that the former should be preferred: it directly describes the desired architecture, whereas the latter meta-describes it. The call/return interface is merely describing a method call on an object which abstractly represents the relationship.
While the two are equivalent in their effects, an analogy with Dijkstra’s Go To Statement Considered Harmful illustrates the significance. A sentence from this essay provides what I think is the core of Weiher’s point of view: “For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible”
However, Dijkstra was talking about using structured programming (loop constructs, lexical scope, etc.) instead of unscoped jumps. Going from jumps to structured programming is an intentional design decision to move from a construct that is (too) powerful and difficult to read, to constructs that are less powerful but much easier to reason about.
I’m not sure that Weiher’s situation is actually analogous to jumps vs structured programming. Call/return is more powerful in the general case, but when it’s used to create a well-designed interface, that interface should be roughly as powerful and easy to reason about as a language-native version would be. The paper isn’t particularly old, so there have been plenty of libraries designed with constrained, reasonable interfaces and Weiher has surely interacted with them. Therefore, my assumption is that Weiher is concerned about call/return being more powerful and less readable in the general case. That is, you could create an interface that is constrained and easy to reason about, but you don’t have to, and doing so isn’t necessarily the option you’re most likely to turn to.
In the above paragraphs, when I say “powerful” or “constrained”, I mean that in the sense that less power can absolutely be a good thing. For example, it is pretty common wisdom that side effects should be managed carefully. Some languages (e.g. Haskell) actually constrain you, give you less power, by placing restrictions on when side effects can occur.
Another thought about “pragmatic”
Another reason why call/return is so well-used in industry: it’s more powerful/primitive/general and therefore the same language can be used for just about everything. For example, by using frameworks like CDK, Entity Framework and ASP.NET with Razor, we can define our whole stack using just one core programming language. We can then use that programming language to define any number of custom architectures within our application code. Very handy for hiring and training, but won’t necessarily lead to the best code, which may cost more in the long run. This is one of those hard problems, mostly due to how hard it is to measure the cost of bad code against the cost of delivering too late.
Being more primitive and therefore more powerful and generalisable is arguably true of assembly code too. However, in that case we seem to have decided that choosing the more constrained and readable paradigm is worth it. I’m sure there’s a whole history course to be said about why that is — maybe it will happen for call/return in time.
Clarity
A thought that came to me while reading the essay was this: “Well, 3 alternative architectures is great and all, but what about ones that haven’t been included in your language? Is there a mechanism that could be generalised to achieve your desired outcome?” For a while, this thought seemed so reasonable to me.
However, I realised that this is the wrong way to look at it. Of course there is a general mechanism — we’ve been talking about it the whole time. Any of these architectures can be, and routinely is implemented using piles of call/return code. It could be implemented using even larger piles of assembly code making heavy use of jumps. If we accept Weiher’s arguments, then we accept that doing so is not the best approach in all situations. To me, this way of thinking makes it apparent that those initial 3 architectures are not a misguided attempt at a solution, but rather a well-aimed part of the proper solution!
Languages
Here are some interesting languages that may implement some of the ideas that Weiher wishes to pursue.
Erlang and Elixir.
Erlang homepage
Elixir homepage
Two languages for the BEAM virtual machine. Deeply ingrained into the languages and conventions is asynchronous “actor model” semantics, to the point where running thousands of lightweight processes is a common activity in these languages.
Elm.
Elm homepage
A language for creating interactive web apps. Largely pure functional, but the language provides a small number of built-in app “harnesses”. These handle setting up an architecture, such that the programmer only needs to write handlers for each component of the architecture, and the built-ins will stick them together.
Wrap up
Key takeaways for me:
- Specific language features for expressing architectures can lead to beautiful code
- There is a general language that can be used to implement any architecture, but use of it leads to code that is harder to work with
- There are pragmatic reasons for software engineers to choose mainstream languages without these features
- There are pragmatic reasons for software engineers to make widespread use of a more primitive and powerful paradigm over one that is more constrained and easier to reason about
- I think and hope that, similar to how we have seen functional programming features appearing in mainstream languages, we will see these “architectural” features appear too
- I also think, however, that the non-generality of these features will be a roadblock in these features being implemented first-class in languages