Content-Type: text/shitpost

I have a new bluetooth keyboard and often it works flawlessly. But under some circumstances using it seems to mess up my keyboard configuration. I have the useless ‘Caps lock’ key mapped to act like a second ‘Control’ key, and I have the right ‘Alt’ key set up to act like a dead compose key, so that typing (‘alt’ + ‘o’ + ‘=’) inputs the letter ‘ő’, without which I can't spell “Erdős”. The laptop's integrated keyboard is also set up this way.

Usually everything just works, but sometimes the the control or compose mappings stop working, depending on some sequence of events having to do with putting the laptop to sleep, turning off the bluetooth keyboard, waking up the laptop, and bringing back the keyboard. But I don't know what the sequence is. Sometimes both keyboards lose both bindings; sometimes only the bluetooth keyboard loses them; sometimes it loses only one.

Since it's important to be able to type “Erdős”, I want to fix this. I don't even care that much about tracking down the ultimate cause and fixing that. I just want to run a command that will fix it temporarily when it breaks.

And so began my epic journey into the wild and uncharted jungles of the Linux keyboard system. Or should I say systems.

I wish I could draw you a map here, but I don't have one. All I know is, I wandered about through many strange lands, having adventures. Please do not assume that anything in this article is technically correct.

There seem to be at least half a dozen places that keyboard mappings can go wrong. For instance, there are at least a couple of layers in the X server alone.

Voyage into the X server

X has an idea that the keyboard is producing keycodes, and it has a table (or several tables?) to map these to keysyms, and then keysyms to actual functions. I think.

You can in theory use the xmodmap command to control this but I tried once and I am not eager to try again. Messing around with the keyboard mappings not a place for casual experimentation and jolly hacking. it is a little more like removing your own appendix, because if you make a mistake, it is too late to fix it.

I think the control function of the “Caps lock” key is handled in the X server. There is an X event diagnostic program called xev and when I run it and tap x I see

KeyPress event, serial 40, synthetic NO, window 0x6600001,
    root 0xe9, subw 0x0, time 5380932, (75,68), root:(1441,105),
    state 0x0, keycode 53 (keysym 0x78, x), same_screen YES,
    XLookupString gives 1 bytes: (78) "x"
    XmbLookupString gives 1 bytes: (78) "x"
    XFilterEvent returns: False

KeyRelease event, serial 40, synthetic NO, window 0x6600001,
    root 0xe9, subw 0x0, time 5380977, (75,68), root:(1441,105),
    state 0x0, keycode 53 (keysym 0x78, x), same_screen YES,
    XLookupString gives 1 bytes: (78) "x"
    XFilterEvent returns: False

which are the two events that the X server sends to xev for the press and release. The state 0x0 I think means that no modifier keys are in force. Pressing ‘Caps lock’ generates a similar KeyPress event for the Caps_lock key, and then the events for tapping x have state 0x4 instead. The keycodes and the keysyms are the same, but the result of XLookupString becomes:

XLookupString gives 1 bytes: (18) "▒"
XmbLookupString gives 1 bytes: (18) "▒"
XFilterEvent returns: False

where 18 is some internal code, perhaps the Unicode codepoint, for the control-X character. (Note that 0x18 = 24 and ‘x’ is the 24th letter of the alphabet, which is why 0x18 is control-X in ASCII and in Unicode; similarly 0x78 is lowercase letter ‘x’ in ASCII and compatible mappings.) Using the real control key is similar, except that X recognizes it as the Control_L keysym instead of as Caps_lock.)

From the X point of view the keymap and keysym interpretations never change. What changes, when I map ‘Caps lock’ to the control function, is the mapping from keysyms to strings.

But the handling of the alt-to-compose mapping is handled differently. There, when the mapping is working, tapping the right-alt key produces the keysym Multi_Key, and when it isn't, it produces Alt_R.

On top of that, I think the actual compositions are handled in the client. When the X client receives Multi_Key, it knows that a compose sequence is coming up, and then it consults its private per-process composition map, which in my case was loaded from ~/.XCompose at some point. And in this case the client might be the window manager. Or maybe the display manager. Because having decentralized the composition mapping out into the client, it now has to be recentralized again so that the window manager can change the keyboard mappings for every client from English to Japanese when you type control-shift-space. Or maybe it's the display manager. Or both.

But wait, there's more!

But how does X decide which keycodes are for the keys being pressed? The OS tells it. And here things get even more exciting.

I know at least one of the problems is that the right-alt key is generating the Alt_R keysym instead of the Multi_Key keysym. Why? Rummaging the list of available commands suggests that setxkbmap might be useful. This command is a real piece of work. The name alone should tip you off, because why it is setxkbmap and not xsetkbmap? And why does it understand -help and -? but not --help or -h? And if you say setxkbmap -h why does it say Error! Option "-h" not recognized without printing out the options that it does recognize?

I could complain about this command all day. Its manual says:

      -device <deviceid>  Specifies the device ID to use

But there is no option to tell you what the allowed device IDs are. But that is all right! As far as I can tell this option is actually ignored because -device asdkjasd seems to make no difference to the output. Is there a way to address the two keyboards separately? I don't know.

Then there is also this:

  -print              Print a complete xkb_keymap description and exit
  -query              Print the current layout settings and exit

What is the difference between these two options? Well, the format, obviously. Except that sometimes they seem to print inconsistent results; one form will say that the keyboard options include some things, and the other form will say that they don't.

Yesterday after fucking around for forty-five minutes and looking at some of the 275 (!) xkb rules files, I guessed that maybe the ralt rule was the one I wanted to add; it looks like this:

 partial modifier_keys
 xkb_symbols "ralt" {
     key <RALT>  { type[Group1]="TWO_LEVEL", [ Multi_key, Multi_key ]
     };
 };

When I ran setxkbmap -option ralt it seemed to have fixed my compose key problem, and I was only somewhat disturbed that it also seemed to have fixed my caps lock key problem also. So I put it in a one-line shell script called fix-compose-key and hoped I had found the answer to my problem.

I had not. Today it appeared to do nothing. I went back to the database of xsetkbmap options that is in /usr/share/X11/xkb, and sought there. Looking again at the ralt option in symbols/compose, I wondered if perhaps I had been in some other directory yesterday when it appeared to work. “Okay,” I said. “Maybe it is failing to recognize ralt because of some path search issue, and it will find it if I explicitly say -config symbols/compose. So I ran it with that option and it said

  Couldn't find configuration file "symbols/compose"

which seems clear enough; maybe it doesn't know that it should look under the current directory. So then I ran it under strace to see where it was looking (Hi, Julia!) and it said:

open("./symbols/compose", O_RDONLY)     = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2303, ...}) = 0
read(4, "partial modifier_keys\nxkb_symbol"..., 4096) = 2303
close(4)                                = 0
write(2, "Couldn't find configuration file"..., 51) = 51

which means that it found, opened, read, and closed the file, and then claimed not to have found it. This is some quality software.

But anyway, if the problem is that it doesn't know what ralt means any more, why doesn't it just say so? Was I supposed to write it as compose:ralt? Maybe?

Interlude

While writing up this section, the mappings on the bluetooth keyboard went away. The integrated keyboard is still working properly. But on the bluetooth keyboard, the right-alt key now sends keysym Alt_R instead of Multi_Key. The caps lock key still sends Caps_Lock, but the X server is now interpreting it as a caps lock key,and one which shifts both keyboards into caps mode.

This happened to me before once; I had just put away the bluetooth keyboard and I realized that somehow the server had gotten itself into caps lock mode. Since the caps lock key on the integrated keyboard was still mapped to control, I had no way to turn it off. I couldn't even try to figure it out, because I couldn't type any lowercase letters.

I was going to say more about this, but somewhere in the previous paragraph the bluetooth keyboard started working properly again.

Fuck if I know.

But wait, there's still more!

Stymied by setxkbmap I tried looking under a different lamppost. In /var/log/Xorg.0.log there was the following suggestive paragraph:

[   122.475] (II) XINPUT: Adding extended input device "Anker A7726" (type: KEYBOARD, id 17)
[   122.475] (**) Option "xkb_rules" "evdev"
[   122.475] (**) Option "xkb_model" "pc105"
[   122.475] (**) Option "xkb_layout" "us"
[   122.475] (WW) Option "xkb_variant" requires a string value
[   122.475] (WW) Option "xkb_options" requires a string value

Those xkb things seem to be referring to what setxkbmap knows about, because some of those things are mentioned in the configuration files that it claimed not to be able to find. Maybe XINPUT, whatever that is, is failing to put the ralt option into xkb_options. The warning message was helpful, because I thought maybe I wanted to supply an xkb_options value, and I wouldn't otherwise have known that XINPUT was trying to supply one. Although it's not very reassuring that it is supplying an erroneous empty value instead of just omitting the option.

But what is XINPUT? I think it refers to another subsystem, which can be addressed with the xinput command. And this one, hallelujah, has an option to list the device IDs that it wants:

    % xinput --list
    …
    ⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
      …
        ↳ Anker A7726                               id=17   [slave keyboard (3)]

So maybe I can somehow use xinput to somehow redo whatever it was doing at 122.475, but this time with xkb_options set to ralt?

Maybe but if so I have not figured out how. The command has a relevant-seeming --set-prop option but none of the properties for the keyboard seem to be what I want, or even to have anything to do with what a keyboard is. They are things like:

    Device Accel Adaptive Deceleration (270):       1.000000

and

    Evdev Axis Inversion (272):     0, 0

and

    Evdev Scrolling Distance (277): 0, 0, 0

which sound to me like a trackball.

But that's not all!

Now how does xinput decide what to run? Maybe if I could find out what is running xinput I could see how it is run.

At this point I tried Google search for Adding extended input device and was led to the Kubuntu wiki page about X Input Configuration. This was a useful page. It says:

1. A hardware input device is present at boot, or gets hotplugged.

2. The kernel detects this, and creates a new "input" device … and a device node /dev/input/event3 …

3. udev picks up the "add" event and the new device. /lib/udev/rules.d/60-persistent-input.rules calls /lib/udev/input_id on it…

This seemed like paydirt: the kernel notices the bluetooth device and pokes udev, and then probably something in those /lib/udev/rules.d files tells it to kick off xinput.

I spent a while poking around in the rules.d directory looking at the rules. As promised there is a 60-persistent-input.rules and also 64-xorg-xkb.rules. Most of it is gobbledygook to me, and it is not tempting to try to understand it better, because each file begins with

    # do not edit this file, it will be overwritten on update

which means that even if the contents of the file are wrong, I would need to change something else somewhere else. But the 64-xorg-xkb.rules file mentioned /etc/default/keyboard, so I looked at that. It temptingly contained:

# KEYBOARD CONFIGURATION FILE

# Consult the keyboard(5) manual page.

XKBMODEL="pc105"
XKBLAYOUT="us"
XKBVARIANT=""
XKBOPTIONS=""

BACKSPACE="guess"

Aha, maybe that is the cause of the Option "xkb_variant" requires a string value message in Xorg.0.conf. And it has a reference to a man page! The man page is not that much help, but at least it commiserates:

The specification of the keyboard layout in the keyboard file is based on the XKB options XkbModel, XkbLayout, XkbVariant and XkbOptions. Unfortunately, there is little documentation how to use them.

The plan now is: hack /etc/default/keyboard to specify the XKBOPTIONS and then try to reinitialize the right part of the keyboard system. I hoped that the keyboard man page would mention which command does this. It has a SEE ALSO section but none of the commands there seemed to be what I wanted. There is a setupcon which seems to be for setting up the emergency console. And there is also a tantalizingly-named kbdcontrol command…

    % man kbdcontrol
    No manual entry for kbdcontrol
    % kbdcontrol
    kbdcontrol: command not found

Uh, okay, maybe there isn't.

There's still more!

Maybe I can get udev to redo whatever initialization it normally does when the new keyboard appears.

So I added ralt to XKBOPTIONS and then did man -k udev to see what might be about udev. There is a udevadm command that might work. Here's my history of poking at udevadm:

 2082  udevadm --help

This worked flawlessly and told me there was a udevadm info subcommand:

 2083  udevadm info

This told me that I needed to tell it what device I wanted info about:

udevadm info [OPTIONS] [DEVPATH|FILE]

and also gave me a summary of options. But what device do I want info about? I don't know. Does it have an option to produce a list of recognized devices or is this going to be another setxkbmap situation where I have to supplicate the Delphic Oracle? The next few commands are me trying to see if udevadm will disgorge a list of devices:

 2084  udevadm -q all
 2085  udevadm -x
 2086  udevadm info -x
 2087  udevadm info -a
 2088  udevadm info -q all
 2089  udevadm info -q all -a
 2093  udevadm info -e

Aha, that was it!

 2094  udevadm info -e|less

There are 774 devices. (!!) But only three of them mention “Anker” so perhaps I want one of those. I eventually figured out that this was not the case; the one I wanted looked like this:

P: /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/bluetooth/hci0/hci0:256/0005:291A:8502.0003/input/input20/event19
N: input/event19
E: BACKSPACE=guess
E: DEVNAME=/dev/input/event19
E: DEVPATH=/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/bluetooth/hci0/hci0:256/0005:291A:8502.0003/input/input20/event19
E: ID_INPUT=1
E: ID_INPUT_KEY=1
E: ID_INPUT_KEYBOARD=1
E: LIBINPUT_DEVICE_GROUP=5/291a/8502/11b:f4:8c:50:5c:aa:a7
E: MAJOR=13
E: MINOR=83
E: SUBSYSTEM=input
E: USEC_INITIALIZED=122466646
E: XKBLAYOUT=us
E: XKBMODEL=pc105

I no longer have any idea how I guessed this. There are many devices that mention bluetooth and many more that mention XKB, including what I think are the six virtual consoles, and also something called Chicony_Electronics_Co._Ltd._Integrated_Camera_0001. I don't know why an integrated camera has a pc105-model keyboard, but I guess if you can accept a keyboard that has an adaptive deceleration and an axis inversion setting, it's not too big a step to suppose that it might be attached to an integrated camera.

But somehow I did guess it — maybe I noticed that Xorg.0.log mentioned it:

[  7792.373] (II) config/udev: Adding input device Anker A7726 (/dev/input/event19)

After a very reasonable amount of consulting the manual and trying stuff I eventually did

   udevadm trigger  --action add --name-match /dev/input/event19 -v

which I think was the secret sauce, because at the end of Xorg.0.log there suddenly appeared:

[  7792.373] (II) XINPUT: Adding extended input device "Anker A7726" (type: KEYBOARD, id 17)
[  7792.373] (**) Option "xkb_rules" "evdev"
[  7792.373] (**) Option "xkb_model" "pc105"
[  7792.373] (**) Option "xkb_layout" "us"
[  7792.373] (WW) Option "xkb_variant" requires a string value
[  7792.373] (**) Option "xkb_options" "ralt"

and instead of being a warning, that last line says that it at least tried to set the option I wanted. You'll notice that although to udev the device ID is 19, to XINPUT it is 17. Whatever, that just means that someone somewhere has a secret decoder ring that says that 17 really means 19.

After that things were back to normal. Was that because of the udevadm trigger command I ran, or just a fluke? I don't know.

Can I find out what device file to use in the command, without needing a human being eyeball the log file? I don't know.

Will my change to /etc/default/keyboard persist past my next login? I don't know.

Why did it fix the control-key mapping as well as the compose key, when all I said to do was to add the ralt option? I don't know.

It's shit like this that makes me wonder if it's not too late to give up computer programming and become a roofer. In spring and fall this thought can persist for weeks. Fortunately, it's now December and so it's easy to remember why I'm not a roofer.

But I bet southern California is full of ex-programmers who are now roofers.

Now how much would you pay?

If you send me mail that tells me I was stupid because everyone knows that the way to do this is to simply extract the virtual device identifier from column 6 of /proc/hid/combined/kbrd and then just run

xcblfmd reset --config-path /etc/bluetooth/defs.d --vdev-ident=3 --no-connect

and that I would have known this if I had read the wiki like I was supposed to, I will take this Anker A7726 and I will find you and I will shove it up your ass.

Coming next week: The Linux sound system also sucks.

[ Addendum 2018-01-12: A less frustrated followup. ]