Robot Languages

There's this problem in robotics, apparently. The problem is how to get all the different bits of the robot to connect together. You might wonder how it could be that you can have a robot whose bits are not connected together? Well, it turns out that physically connecting the bits of the robot together is the easy part.  The real problem is how the software that runs the different bits connects together. Ask the people at Intuitive Machines if you don't believe me.

But really, any computer user-interface is a robot of sorts. In order to get your computer to do useful things, you need extra software to let you communicate with the software that runs on the machine. That extra software is called the user interface, and it's a sadly neglected part of the problem of programming computers. Think about a typical menu bar in an application: that is a program that, when it gets the 'input focus' in some sense, reacts to keypresses or mouse-button presses or something else like a touch-screen event or gesture, or maybe the output of a speech-recognition system. The menu program will then display one of possibly several sub-menus and then respond to further events to allow you to select one particular option from that submenu, or at least to see all the entries in that submenu so that you can decide whether that is in fact the submenu you want, or whether in fact it is another submenu you should be looking for. When you select options from submenus the robot reports the event to the system and eventually that will cause something to happen to the part of the application program associated with that menu option.

The problem in robotics is similar: a robot is a bunch of different kinds of hardware devices, all somehow connected to a central computer which I suppose you can think of as the brain of the robot. So there are sensors which detect the orientation of different parts of the robot relative to each other, or to the external world. And these sensors are monitored in the brain and the results of changes these sensors detect are called events, so these events need to be connected via some kind of channel process like a nerve running from the sensor to the brain. Then the brain needs to be able react to the events the sensors represent by causing some kind of motion in the parts of the robot. These reactions to events are also communicated from the brain via channels to the different parts of the robot that move.

A laboratory is another kind of robot. There you have a collection of measurement instruments and signal generators and whatnot and you connect them to some system which you are studying. See the 1991 catalog of HP laboratory test instruments. Often you want the outputs of measurement devices to feed-back to the controls of signal generators. 

This is why Hewlett Packard became a computer company. They defined the channels for instruments to communicate together, called the GPIB/HPIB which later became IEEE-488.

So it seems to me that whatever software mechanism a robot uses could also be used by a user-interface component. But that is not the case in practice, as far as I can see. The problem seems to be in part that people don't think of user-interfaces this way. A user-interface is generally considered to be an integral part of any application program, not a separate semi-autonomous component. This is what makes software hard for users to use: typically, if you want to change some element of the user interface, you need to change the program. This creates problems when the user-interface is inadequate, as it seems to be for most blind or partially sighted people using Linux. See Using Pipewire to Make A Music Synthesizer.

The kinds of user-interfaces Bret Victor is creating in Dynamicland also have this structure of interconnecting semi-autonomous subsystems:

Subscribe to Dynamicland.