Projecting Computer Languages for a Protean Interaction

Jury

• Amy KO Professor, University of Washington Rapporteure & Examinatrice
• Stéphane CONVERSY Professor, ENAC-LII & Université de Toulouse Rapporteur & Examinateur
• Alan DIX Professor, Swansea University Examinateur
• Anastasia BEZERIANOS Professor, Université Paris-Saclay Examinatrice
• Tomas PETRICEK Assistant Professor, Charles University Examinateur

Abstract

To interact with computers, we often rely on computer languages such as Python for creating programs and LaTeX for writing technical documents. In the past few decades, these languages have become increasingly used in a variety of fields ranging from science to arts. They are now being taught to millions of pupils and have become a staple skill in the job market, widening the gap between those who are computer literates and the others. However, for the most part, our interaction with these languages has remained similar to what it used to be fifty years ago: reading and writing code as plain text. Although various alternatives have been introduced, they have often been developed with a modernist approach, in isolation from popular languages and widespread workflows, resulting in systems that hardly cross the borders of academic circles and niche communities. As a result, our interaction with computer languages is now hampered by essentialist views that encourage us to move away from them, at the risk of losing some of the agency they give us, as if dealing with code was nothing but a burden from the past compared to ever more “simple” user interfaces and “intelligent” code synthesis. In this thesis, I argue against this view that I consider harmful to human-computer interaction research and computer-driven democracies. To do so, I introduce a number of arguments that show that it is indeed possible—and even beneficial—to make our interaction with computer languages more pluralist, including with the many languages that already exist that are inherent to the diversity of computing in the 21st century. I first develop a new theory of interaction with computer languages that shows that no such language is inherently bound to a specific representation or type of interaction. To that end, I deconstruct the notion of computer language into five fundamental aspects to isolate interaction from the other constituents of these languages, yielding a more holistic model than those that already exist. I then use this model to identify different levels of interaction with computer languages, which can be hybridised, and show that a single piece of code can very well be projected onto several representations to let end-users decide which representation supports the form of interaction most appropriate for them. I then apply this view of computer languages to two research problems using user-centred design methodologies: helping users author documents written in LaTeX and helping programmers appropriate their text editors by crafting their own projections. For each problem, I assess the limitations of existing solutions with the help of a formative study; I develop a prototype of a text editor equipped with additional projections; and I evaluate it with both qualitative and quantitative user studies. The results show that that complementing text with other representations helps us understand and modify code faster and with a lower workload and that these representations can be created by recomposing existing parts that can be reused from one projection to another. In conclusion, I show that considering interaction with computer languages as projections makes it more protean, an approach which, according to this thesis, is theoretically grounded, technically possible and empirically desirable. It lends itself to the urgent task of equipping an ever-growing public of citizens with new intellectual and technical tools to help them understand and appropriate the computer languages that rule the societies we live in.