Posts on Programming Linguistics

When Modules Are Not Just Namespaces

Mon, 25 Jul 2022 12:31:46 +1000

It is time for Cone to get a proper module system. This design space is complex and rife with historical missteps. In order to distill the topography of the landscape and clarify the key requirements, I felt it necessary to begin the journey with cross-language research and contemplation. I wound up pursuing three rabbit-chasing adventures in module wonderland: What do programmers want from modularity? You can read about this adventure in my earlier post on modularity, which captures what we want from modularity, and summarizes how the three modularity capabilities are surfaced across different layers of programming language features (and program decomposition).

Modularity in Programming

Mon, 27 Jun 2022 16:14:13 +1000

Cone’s module system has been on my mind of late. The best design for modules is neither easy or obvious, as evidenced by how much modules vary from one language to the next. To guide my approach for Cone, I went back to basics: What is the role (and benefit) that modularity plays in programming (languages)? What role do modules play within this larger picture. This post synthesizes my findings.

Dispatch Magic and Concurrency

Sun, 10 Oct 2021 13:40:17 +1000

I have long enjoyed the convenience of the “dot” dispatch syntax: instance.log("Ethereal cognistrands do not support quantum entanglement") The magic of Universal Function Call Syntax makes this sugar for a function call: log(instance, "Ethereal cognistrands do not support quantum entanglement") The broad popularity of method dispatch is driven by these benefits: Ad hoc polymorphism. When used in conjunction with method overloading, we can reuse the same easier-to-remember semantic names across multiple types and parameter configurations, with minimal-to-no ambiguity on which implementation to use.

Data Flow Analysis

Sat, 07 Aug 2021 10:50:48 +1000

Note: This is a heavily revised version of an earlier post The Cone compiler performs a data flow analysis pass after name resolution and type checking. Given that this sort of analysis is rarely covered by compiler literature, I thought it might be useful to jot down some thoughts about its purpose and intriguing mechanics. Trigger Warning: This blog post is highly technical and brief. It reads more like an organizing outline for a design spec than a typical essay-oriented post.

The Siren Song of Declarative Programming

Mon, 04 Jan 2021 07:48:18 +1000

Let’s continue our journey of discovery through the wild jungle of programming paradigms by clearing away more obstructive vines. This post tackles the taxonomic distinction that many attempt to make between declarative and imperative programming. What becomes clear, after some study, is that “declarative” and “imperative” do not have singularly simple and clear meanings. Wikipedia offers three incompatible definitions! The many strands of these confusing concepts entangle themselves into a Gordian knot that resists all attempts to neatly untie it.

What is a Programming Paradigm?

Sun, 22 Nov 2020 10:07:08 +1000

Have our conversations about programming paradigms grown stale? It seems so. Paradigms like object-oriented programming and functional programming, the two most talked-about, are decades-old. Nearly always, conversations about them devolve into reciting the same desiccated stereotypes. Is this because the notion of a programming paradigm has outlived its usefulness? Dr. Harper appears to think so. Even among those who find some value in one paradigm or another, there is a notable wariness to engage in fruitful discussions across diverse communities.

Region Modules: The Rest of the Story

Sun, 31 May 2020 14:55:29 +1000

The previous Lifecycle of a Reference post illuminates the working relationship between references and regions. It explains how a reference’s region annotations are programmer-definable struct-like types, which can specify an allocated object’s region state using fields, and region-based operations on an object (e.g., allocate and free) using methods. This, however, leaves out an important part of the story about region definitions: their global state and global runtime logic. Holistically, a Cone region is fully defined by an importable module, within which lies:

The Lifecycle of a Reference

Mon, 25 May 2020 11:08:02 +1000

Over two and half years ago, I described something I called Gradual Memory Management. Inspired by Rust and Pony, my paper proposed that it is feasible and desirable for a systems programming language to allow programs to exploit multiple memory management and permission strategies. Without putting memory and data race safety at risk, doing so would facilitate significant improvements to throughput and latency, even in multi-threaded architectures. It is easy to propose wild ideas in words.

Weakening Cycles So That Turing Can Halt

Mon, 11 May 2020 16:27:45 +1000

It is notable how often paradoxes arose in the historical journey that led to the Rise of Type Theory. Resolving Russell’s paradox led to his theory of types. Gödel’s Incompleteness theorems rested its proof around a variation of the Liar Paradox. Church and Turing recapitulated this result using their distinctive formalisms. Intuitionistic type theory was deliberately designed to avoid these paradoxes. And so it goes. As an undergraduate, I had the pleasure of studying Gödel’s ingenious proof as part of a course on Predicate Logic.

The Rise of Type Theory

Sat, 25 Apr 2020 18:02:18 +1000

After three dreary posts on syntax, let’s change the pace and pursue an entirely different, deeper and more fun topic! In our community’s #theory channel on Discord, someone asked: “Is there a basis for some of the mathematics related to type theory and its relationship to its usage in programming languages?” This triggered a spirited dialogue about the historical antecedents of type theory, how type theory evolved from that, and the interplay between programming languages and type theory.

Unifying Type and Value Expressions

Tue, 21 Apr 2020 11:27:31 +1000

I seem to be on a roll with syntactic concerns lately, which is not where I typically spend most of my time. Today’s issue is, at least, more unusual than stylistic preferences regarding semicolons and significant indentation. This post explores the issues that arise when teaching the compiler to be agnostic as to whether it is parsing a value expression or a type expression. A value expression computes some typed value when evaluated:

Significant Indentation

Sat, 18 Apr 2020 17:42:43 +1000

The last post focused narrowly on evaluating various techniques for semi-colon inference, thereby enabling a program to successfully compile even when the programmer omits statement-ending semicolons. This post broadens this theme, looding at how the compiler can take advantage of significant indentation when handling block inference and multi-line string literals. I wander into this topic knowing that for many programmers, compiler support for significant indentation is troubling and uncomfortable. They prefer the explicit clarity of required semi-colons to end statements and curly braces to delimit blocks.

Semicolon Inference

Fri, 03 Apr 2020 10:26:29 +1000

Many programming languages require that statements end with a semicolon. Some languages, such as Javascript, Go, Kotlin, Swift, Scala and Lua, make this requirement optional. Although a semicolon may be specified, it is not required to terminate a statement. This post explores the underlying rules various languages use to make semicolon inference possible. It also articulates the arguments for and against a language offering support for it.1 The Challenge The semicolon inference rules would be simple if every statement fit on a single line.

Practical Subtyping

Tue, 17 Mar 2020 17:50:31 +1000

The last two posts introduced the fundamental elements underlying Cone’s type system. They also distinguished between nominal vs. structural type equivalence. However, they were largely silent about subtyping relationships between types. Given how important subtyping is to Cone’s type versatility, I thought it might also be valuable to offer a detailed treatment of these mechanisms. My approach will be practical, rather than formal (alas! no subsumptions nor lattices), distilling lessons learned from implementing a rich, but type-safe collection of subtyping mechanisms.

Reference Type Semantics

Mon, 09 Mar 2020 18:19:19 +1000

The prior post described the semantics underlying Cone’s fundamental types, for all types except the all-important pointer and reference types. The presence of versatile pointers/references as an explicit and distinctly-separate kind of type is a distinguishing characteristic of systems programming languages. I choose to cover them here as a separate post, since the safety guarantees of Cone’s references make them semantically rich and complex. In harmony with the subatomic theme of the prior post, reference types are themselves composed of “quarks”: pointers, regions, permissions, and lifetimes.

Searching for Quarks in Systems Types

Sun, 01 Mar 2020 16:21:43 +1000

When I began work on the Cone programming language, I believed I had a confident handle on which primitive types this modern, safe systems language required: various sizes of atomic number types, fixed-size arrays, programmer-defined structs, raw pointers, safe references, discriminated (and raw) unions, generics, and limited support for void and tuples. My confidence was misplaced. Once or twice a year, implementation challenges and further research forced me to re-think aspects of the type system, each time deepening my understanding of what it required.

2030: Programming Language Trends

Mon, 24 Feb 2020 10:45:01 +1000

When the clock ticks over to a new decade, it is customary to look back, to reflect on how much we have accomplished, and then look forward, to sort out where we want to go. Ten years is long enough that substantive progress should be visible in the glacially-slow evolutionary pace of programming languages. One can see this by noticing how many now-influential languages had no notable marketplace presence only ten years ago: Rust, Go, Swift, Kotlin, Dart, and Julia.

Infectious Typing

Sun, 03 Nov 2019 07:21:16 +1000

Type theory has many kinds of typing mechanisms: product, sum, top, bottom, recursive, universal, existential, subtypes, linear, refinement, dependent, and so on. I have run across something different from all those, which I am calling “infectious typing”. If anyone knows a formal name and/or treatment for this mechanism, I would love to hear about it. Type composition allows us to create new types out of existing ones. A struct or class product types is composed of several fields, each with its own type.

The Fascinating Influence of Cyclone

Tue, 10 Sep 2019 07:29:57 +1000

In 2001, Trevor Jim (AT&T Research) and Greg Morrisett (Cornell) launched a joint project to develop a safe dialect of the C programming language, an outgrowth of earlier work on Typed Assembly Language. After five years of hard work and some published papers, the team (including Dan Grossman, Michael Hicks, Nik Swamy, and others) released Cyclone 1.0. And then the developers moved on to other things. Few have heard of Cyclone and almost no one has used it.

Unified Variant Type

Tue, 13 Aug 2019 10:05:24 +1000

The prior post, When Sum Types Inherit, shows that we can (and should) use the magic of inheritance to enrich sum types to offer as powerful a capability as traditional classes. Graph-based, variant node data structures benefit from variant nodes being able to support common fields, methods, and virtual dispatch. That result, however, raises a new question: do we really need two separate language abstractions that offer very similar capabilities?

When Sum Types Inherit

Wed, 31 Jul 2019 10:05:24 +1000

Inheritance makes it easier than any other mechanism (e.g. generics, macros, composition/delegation) to define a type that reuses the state and some methods of other types. After reading my inheritance posts, I hope you are convinced that simplifying inheritance to a namespace-based mechanism ensures we obtain this convenient reuse capability, while avoiding most of the complexity and coupling dangers of traditional inheritance. However, you might still wonder whether real-world code needs inheritance’s reuse capability.

Delegated Inheritance

Fri, 19 Jul 2019 07:42:08 +1000

After removing the interface, inversion of control, and protected access capabilities from traditional inheritance, what do we have left (besides composition)? This is what we have: placing a few extra tokens on a derived class causes all named fields and methods of one or more base classes to be absorbed as if explicitly incorporated. Further, certain inherited methods can be customized (overridden) with their own implementation. The primary selling point for inheritance has always been this sort of code reuse.

Disinheriting Abstract Classes

Thu, 18 Jul 2019 12:14:51 +1000

Anti-inheritance advocates are likely to enthusiastically support this post. It promotes the most useful feature of traditional inheritance (Interfaces), turning it into a more valuable abstraction that is largely independent of inheritance. It discards two other traditional features of inheritance, Inversion of Control and Protected Access, as both unnecessary and dangerous. Let’s examine each in turn… Interfaces An interface is “an abstract type that contains no data but defines behaviors as method signatures.

Favor Composition Over Inheritance?

Thu, 18 Jul 2019 06:07:56 +1000

I need to decide what sort of inheritance capability Cone will offer. None! I can hear some of you insist. “Inheritance has recently fallen out of favor as a programming design solution,” claims the Rust language book. “Favor object composition over class inheritance,” recommends the Design Patterns book in 1994. “Inheritance is Evil.” insists Nicolò Pignatelli. It is not hard to find cogent, hard-hitting critiques of inheritance, complaining about costs incurred from fragile base classes, excessive coupling, and broken encapsulation.

The Power of Lifetimes

Tue, 16 Apr 2019 07:20:05 +1000

The execution of a program unfolds over some interval of time. The lifetime of every temporary resource (e.g., variable or object) is the time span between that resource’s “creation” and “destruction”. This lifetime is wholly contained within the typically-longer lifetime of the program. The goal of this post is to explore how versatile lifetime analysis has increasingly become in managing memory efficiently, safely and with better performance. By the end of this post, we will explore exciting new ways to apply lifetime analysis, beyond their current support in Rust.

Compiler Performance and LLVM

Sat, 09 Mar 2019 07:50:45 +1000

Note: You can read Russian translation here. I have always wanted the Cone compiler to be fast. Faster build times make it easier and more pleasant for the programmer to iterate and experiment quickly. Compilers built for speed, such as Turbo Pascal, D and Go, win praise and loyalty. Languages that struggle more with this, like C++ or Rust, get regular complaints and requests to speed up build times.

Premature Optimization

Thu, 07 Mar 2019 07:31:33 +1000

The decades-long golden age of Moore’s Law is fading. Slow, bloated software will find it increasingly difficult to hide behind the skirt of ever-faster computer hardware. Given that businesses and users will not yield on their demand for speedy software, developers will need to get a lot better at architecting for performance. In his excellent article on Performance Culture, Joe Duffy begins: “Performance is one of the key pillars of software engineering, and is something that’s hard to do right, and sometimes even difficult to recognize.

Transitional Permissions

Sat, 12 Jan 2019 18:19:02 +1000

To complete our three-part series on permissions, which began with Race-Safe Strategies, let’s talk about the transitional nature of reference permissions. When are permissions transitional? When we can safely create a copy of a reference which has a different permission than the reference it copied from. There are several ways in which this can happen, which this diagram summarizes (and the following sections explain): The following sections describe the nature of several one-way transitions that flow downward in the diagram.

Interior References and Shared Mutability

Mon, 17 Dec 2018 08:24:17 +1000

In my last post, Race-safe Strategies, one footnote stated “safety issues which look suspiciously similar to race conditions can crop up when a language supports the creation of “interior references” to shared, mutable values of certain types”. Let’s explore that now. I will begin by recapitulating Manish Goregaokar’s excellent post “The Problem With Single-Threaded Shared Mutable”. His post clearly explains why the Rust language wishes to steer developers towards RefCell for shared references over use of Cell, its inflexible shared, mutable counterpart.

Race-Safe Strategies

Sun, 16 Dec 2018 10:21:16 +1000

I recently made the observation that many people seem unaware of the full collection of constraint mechanisms available for protecting race safety. Someone sensibly asked for a link to an article that provides a modern, comprehensive review. It turns out that the pickings are very slim; the best I could find is this Wikipedia article on thread safety. It’s accurate, but incomplete. To close that gap, let me take a stab here at more comprehensive treatment.

Can a compiler guarantee multi-owner memory safety?

Sat, 01 Dec 2018 12:37:15 +1000

Is it possible to improve on Rust’s single-owner strategy to support more complex data structures? Before digging into this challenge, let’s summarize the story so far… The Promise and Limitations of Single-Owner Rust’s single-owner memory management is a form of automatic memory management; a garbage collection strategy that is distinct from tracing and reference-counting. Fundamentally, it is an improvement on RAII, which automatically finalizes some defined resource at the end of its defined lexical scope.

Move Mechanics

Mon, 22 Oct 2018 21:28:32 +1000

Most programming languages support only copy semantics. A value passed to a function or stored in a variable is a copy of the original value. We know it is a copy, because any change we make to the copy has no impact on the original value. A few languages, like C++ and Rust, also support move semantics. Unlike a copy, a transfer moves the original value to its new home; that value is no longer accessible at its previous home.

The Challenge of Counting References

Sat, 13 Oct 2018 10:37:40 +1000

Note: The latter part of this post is outdated in terms of the mechanisms that the Cone compiler uses to de-alias ref-counted references. See this post for an updated description. In the world of automatic memory management, reference counting is considered to be one of the easiest to implement. The rules seem simple: When a reference is created to an allocated memory area, set its counter to 1 When the reference is copied (aliased), increment the counter When an alias is destroyed (de-aliased), decrement the counter When the counter reaches zero, free the reference’s memory area The simplicity of these rules does not always translate to a simple implementation.

Data Flow Analysis (Old Version)

Sun, 07 Oct 2018 10:50:48 +1000

Note: This is an outdated post, replaced by this post The Cone compiler performs a data flow analysis pass after name resolution and type checking. Given that this sort of analysis is rarely covered by compiler literature, I thought it might be useful to jot down some thoughts about its purpose and intriguing mechanics. Goals Like Rust (and unlike C), Cone applies constraints to references that ensure they can only access memory safely, even in the face of concurrency.

The IR Tree: Typed Nodes

Tue, 14 Aug 2018 17:01:25 +1000

As mentioned in the previous post, all typed nodes use the TypedNodeHdr common header. It only contains a pointer to the node’s type. This type field applies to every node in the “expression” group. This group holds all node types that return a value, including leaf nodes like literals and variables, function call nodes, and even the block and if nodes. The type check pass focuses largely on this type field, ensuring it specifies a valid, consistent type.

The IR Tree: Named Nodes and Namespaces

Thu, 09 Aug 2018 19:27:41 +1000

Working out effective name handling mechanisms in Cone’s IR took several tries. The key challenges: Although most node types don’t use names, the ones that do are spread out unevenly across every node group: a few expressions (variables and functions), most types, and some statements (e.g., module nodes). What is the best way to gain access to a node’s name information, given this disparity? Cone’s namespace rules vary depending on the semantic context a name is declared within.

The IR Tree: The INode Interface

Wed, 08 Aug 2018 09:04:42 +1000

Having spoken about the compiler’s IR tree in general terms, let’s focus in on an important detail: how to represent a node that could be any arbitary type. Cone’s IR makes use of dozens of different types of nodes, each defined using a different struct. However, sometimes the compiler needs to point to a node without restricting which type of node it must be. For example, consider an assignment node.

The IR Tree - Introduction

Tue, 07 Aug 2018 08:02:19 +1000

The Cone compiler uses a traditional pipeline to transform source programs into object files. The Intermediate Representation (IR) plays a central role in this design. It is the glue binding together the parser, semantic analysis passes, and the LLVM IR generator. The literature on effective IR design is relatively sparse as compared to other topics, such as parsing, type inference/theory, and optimization techniques. This is a shame. I have devoted far more time trying to get Cone’s IR “right” than I have on any other aspect of the compiler.

Gradual Memory Management

Sun, 06 Aug 2017 00:00:00 +0000

Realtime, quality 3D is punishing on hardware and software alike. At least every 17 milliseconds, the scene needs to be updated and redrawn. Managing memory efficiently across the many large and small data structures that make up an interesting 3D scene is a well-known and important part of that challenge.