F/LOSS (in)activity, September 2017 Daniel Silverstone

In the interests of keeping myself "honest" regarding F/LOSS activity, here's a report, sadly it's not very good.

Unfortunately, September was a poor month for me in terms of motivation and energy for F/LOSS work. I did some amount of Gitano work, merging a patch from Richard Ipsum for help text of the config command. I also submitted another patch to the STM32F103xx Rust repository, though it wasn't a particularly big thing. Otherwise I've been relatively quiet on the Rust/USB stuff and have otherwise kept away from projects.

Sometimes one needs to take a step away from things in order to recuperate and care for oneself rather than the various demands on ones time. This is something I had been feeling I needed for a while, and with a lack of motivation toward the start of the month I gave myself permission to take a short break.

Next weekend is the next Gitano developer day and I hope to pick up my activity again then, so I should have more to report for October.

F/LOSS activity, August 2017 Daniel Silverstone

Shockingly enough, my focus started out on Gitano once more. We managed a 1.1 release of Gitano during the Debian conference's "camp" which occurs in the week before the conference. This was a joint effort of myself, Richard Maw, and Richard Ipsum. I have to take my hat off to Richard Maw, because without his dedication to features, 1.1 would lack some stuff which Richard Ipsum proposed around ruleset support for basic readers/writers and frankly 1.1 would be a weaker release without it.

Because of the debconf situation, we didn't have a Gitano developer day which, while sad, didn't slow us down much...

  • Once again, we reviewed our current task state
  • I submitted a series which fixed our test suite for Git 2.13 which was an FTBFS bug submitted against the Debian package for Gitano. Richard Maw reviewed and merged it.
  • Richard Maw supplied a series to add testing for dangling HEAD syndrome. I reviewed and merged that.
  • Richard Maw submitted a patch to improve the auditability of the 'as' command and I reviewed and merged that.
  • Richard Ipsum submitted a patch to add reader/writer configs to ease simple project management in Gitano. I'm not proud to say that I was too busy to look at this and ended up saying it was unlikely it'd get in. Richard Maw, quite rightly, took umbrage at that and worked on the patch, eventually submitting a new series with tests which I then felt obliged to review and I merged the series eventually.

    This is an excellent example of where just because one person is too busy doesn't mean that a good idea should be dropped, and I am grateful to Richard Maw for getting this work mergeable and effectively guilt-tripping me into reviewing/merging. This is a learnable moment for me and I hope to do better into the future.

  • During all that time, I was working on a plugin to support git-multimail.py in Gitano. This work ranged across hooks and caused me to spend a long time thinking about the semantics of configuration overriding etc. Fortunately I got there in the end, and with a massive review effort from Richard Maw, we got it merged into Gitano.
  • Finally I submitted a patch which caused the tests we run in Gitano to run from an 'install' directory which ensures that we catch bugs such as those which happened in earlier series where we missed out rules files for installation etc. Richard Maw reviewed and merged that.
  • And then we released the new version of Gitano and subsidiary libraries.

    There was Luxio version 13 which switched us to readdir() from readdir_r() thanks to Richard Ipsum; Gall 1.3 which contained a bunch of build cleanups, and also a revparse_single() implementation in the C code to speed things up thanks to Richard Maw; Supple 1.0.8 which improved wrapper environment cleanups thanks to Richard Ipsum, allowed baking of paths in which means Nix is easier to support (again thanks to Richard Ipsum), fixed setuid handling so that Nix is easier to support (guess what? Richard Ipsum again); Lace 1.4 which now verifies definition names in allow/deny/anyof/allof and also produces better error messages from nested includes.

    And, of course, Gitano 1.1 whose changes were somewhat numerous and so you are invited to read them in the Gitano NEWS file for the release.

Not Gitano

Of course, not everything I did in August was Gitano related. In fact once I had completed the 1.1 release and uploaded everything to Debian I decided that I was going to take a break from Gitano until the next developer day. (In fact there's even some patch series still unread on the mailing list which I will get to when I start the developer day.)

I have long been interested in STM32 microcontrollers, using them in a variety of projects including the Entropy Key which some of you may remember. Jorge Aparicio was working on Cortex-M3 support (among other microcontrollers) in Rust and he then extended that to include a realtime framework called RTFM and from there I got interested in what I might be able to do with Rust on STM32. I noticed that there weren't any pure Rust implementations of the USB device stack which would be necessary in order to make a device, programmed in Rust, appear on a USB port for a computer to control/use. This tweaked my interest.

As many of my readers are aware, I am very bad at doing things without some external motivation. As such, I then immediately offered to give a talk at a conference which should be happening in November, just so that I'd be forced to get on with learning and implementing the stack. I have been chronicling my work in this blog, and you're encouraged to go back and read them if you have similar interests. I'm sure that as my work progresses, I'll be doing more and more of that and less of Gitano, for at least the next two months.

To bring that into context as F/LOSS work, I did end up submitting some patches to Jorge's STM32F103xx repository to support a couple more clock configuration entries so that USB and ADCs can be set up cleanly. So at least there's that.

STM32 USB and Rust - Packet Memory Area Daniel Silverstone

In this, our next exciting installment of STM32 and Rust for USB device drivers, we're going to look at what the STM32 calls the 'packet memory area'. If you've been reading along with the course, including reading up on the datasheet content then you'll be aware that as well as the STM32's normal SRAM, there's a 512 byte SRAM dedicated to the USB peripheral. This SRAM is called the 'packet memory area' and is shared between the main bus and the USB peripheral core. Its purpose is, simply, to store packets in transit. Both those IN to the host (so stored queued for transmission) or OUT from the host (so stored, queued for the application to extract and consume).

It's time to actually put hand to keyboard on some Rust code, and the PMA is the perfect starting point, since it involves two basic structures. Packets are the obvious first structure, and they are contiguous sets of bytes which for the purpose of our work we shall assume are one to sixty-four bytes long. The second is what the STM32 datasheet refers to as the BTABLE or Buffer Descriptor Table. Let's consider the BTABLE first.

The Buffer Descriptor Table

The BTABLE is arranged in quads of 16bit words. For "normal" endpoints this is a pair of descriptors, each consisting of two words, one for transmission, and one for reception. The STM32 also has a concept of double buffered endpoints, but we're not going to consider those in our proof-of-concept work. The STM32 allows for up to eight endpoints (EP0 through EP7) in internal register naming, though they support endpoints numbered from zero to fifteen in the sense of the endpoint address numbering. As such there're eight descriptors each four 16bit words long (eight bytes) making for a buffer descriptor table which is 64 bytes in size at most.

Buffer Descriptor Table
Byte offset in PMA Field name Description
(EPn * 8) + 0 USB_ADDRn_TX The address (inside the PMA) of the TX buffer for EPn
(EPn * 8) + 2 USB_COUNTn_TX The number of bytes present in the TX buffer for EPn
(EPn * 8) + 4 USB_ADDRn_RX The address (inside the PMA) of the RX buffer for EPn
(EPn * 8) + 6 USB_COUNTn_RX The number of bytes of space available for the RX buffer for EPn (and once received, the number of bytes received)

The TX entries are trivial to comprehend. To transmit a packet, part of the process involves writing the packet into the PMA, putting the address into the appropriate USB_ADDRn_TX entry, and the length into the corresponding USB_COUNTn_TX entry, before marking the endpoint as ready to transmit.

To receive a packet though is slightly more complex. The application must allocate some space in the PMA, setting the address into the USB_ADDRn_RX entry of the BTABLE before filling out the top half of the USB_COUNTn_RX entry. For ease of bit sizing, the STM32 only supports space allocations of two to sixty-two bytes in steps of two bytes at a time, or thirty-two to five-hundred-twelve bytes in steps of thirty-two bytes at a time. Once the packet is received, the USB peripheral will fill out the lower bits of the USB_COUNTn_RX entry with the actual number of bytes filled out in the buffer.

Packets themselves

Since packets are, typically, a maximum of 64 bytes long (for USB 2.0) and are simply sequences of bytes with no useful structure to them (as far as the USB peripheral itself is concerned) the PMA simply requires that they be present and contiguous in PMA memory space. Addresses of packets are relative to the base of the PMA and are byte-addressed, however they cannot start on an odd byte, so essentially they are 16bit addressed. Since the BTABLE can be anywhere within the PMA, as can the packets, the application will have to do some memory management (either statically, or dynamically) to manage the packets in the PMA.

Accessing the PMA

The PMA is accessed in 16bit word sections. It's not possible to access single bytes of the PMA, nor is it conveniently structured as far as the CPU is concerned. Instead the PMA's 16bit words are spread on 32bit word boundaries as far as the CPU knows. This is done for convenience and simplicity of hardware, but it means that we need to ensure our library code knows how to deal with this.

First up, to convert an address in the PMA into something which the CPU can use we need to know where in the CPU's address space the PMA is. Fortunately this is fixed at 0x4000_6000. Secondly we need to know what address in the PMA we wish to access, so we can determine which 16bit word that is, and thus what the address is as far as the CPU is concerned. If we assume we only ever want to access 16bit entries, we can just multiply the PMA offset by two before adding it to the PMA base address. So, to access the 16bit word at byte-offset 8 in the PMA, we'd look for the 16bit word at 0x4000_6000 + (0x08 * 2) => 0x4000_6010.

Bundling the PMA into something we can use

I said we'd do some Rust, and so we shall…

    // Thanks to the work by Jorge Aparicio, we have a convenient wrapper
    // for peripherals which means we can declare a PMA peripheral:
    pub const PMA: Peripheral<PMA> = unsafe { Peripheral::new(0x4000_6000) };

    // The PMA struct type which the peripheral will return a ref to
    pub struct PMA {
        pma_area: PMA_Area,
    }

    // And the way we turn that ref into something we can put a useful impl on
    impl Deref for PMA {
        type Target = PMA_Area;
        fn deref(&self) -> &PMA_Area {
            &self.pma_area
        }
    }

    // This is the actual representation of the peripheral, we use the C repr
    // in order to ensure it ends up packed nicely together
    #[repr(C)]
    pub struct PMA_Area {
        // The PMA consists of 256 u16 words separated by u16 gaps, so lets
        // represent that as 512 u16 words which we'll only use every other of.
        words: [VolatileCell<u16>; 512],
    }

That block of code gives us three important things. Firstly a peripheral object which we will be able to (later) manage nicely as part of the set of peripherals which RTFM will look after for us. Secondly we get a convenient packed array of u16s which will be considered volatile (the compiler won't optimise around the ordering of writes etc). Finally we get a struct on which we can hang an implementation to give our PMA more complex functionality.

A useful first pair of functions would be to simply let us get and put u16s in and out of that word array, since we're only using every other word…

    impl PMA_Area {
        pub fn get_u16(&self, offset: usize) -> u16 {
            assert!((offset & 0x01) == 0);
            self.words[offset].get()
        }

        pub fn set_u16(&self, offset: usize, val: u16) {
            assert!((offset & 0x01) == 0);
            self.words[offset].set(val);
        }
    }

These two functions take an offset in the PMA and return the u16 word at that offset. They only work on u16 boundaries and as such they assert that the bottom bit of the offset is unset. In a release build, that will go away, but during debugging this might be essential. Since we're only using 16bit boundaries, this means that the first word in the PMA will be at offset zero, and the second at offset two, then four, then six, etc. Since we allocated our words array to expect to use every other entry, this automatically converts into the addresses we desire.

If we pop (and please don't worry about the unsafe{} stuff for now):

    unsafe { (&*usb::pma::PMA.get()).set_u16(4, 64); }

into our main function somewhere, and then build and objdump our test binary we can see the following set of instructions added:

 80001e4:   f246 0008   movw    r0, #24584  ; 0x6008
 80001e8:   2140        movs    r1, #64 ; 0x40
 80001ea:   f2c4 0000   movt    r0, #16384  ; 0x4000
 80001ee:   8001        strh    r1, [r0, #0]

This boils down to a u16 write of 0x0040 (64) to the address 0x4006008 which is the third 32 bit word in the CPU's view of the PMA memory space (where offset 4 is the third 16bit word) which is exactly what we'd expect to see.

We can, from here, build up some functions for manipulating a BTABLE, though the most useful ones for us to take a look at are the RX counter functions:

    pub fn get_rxcount(&self, ep: usize) -> u16 {
        self.get_u16(BTABLE + (ep * 8) + 6) & 0x3ff
    }

    pub fn set_rxcount(&self, ep: usize, val: u16) {
        assert!(val <= 1024);
        let rval: u16 = {
            if val > 62 {
                assert!((val & 0x1f) == 0);
                (((val >> 5) - 1) << 10) | 0x8000
            } else {
                assert!((val & 1) == 0);
                (val >> 1) << 10
            }
        };
        self.set_u16(BTABLE + (ep * 8) + 6, rval)
    }

The getter is fairly clean and clear, we need the BTABLE base in the PMA, add the address of the USB_COUNTn_RX entry to that, retrieve the u16 and then mask off the bottom ten bits since that's the size of the relevant field.

The setter is a little more complex, since it has to deal with the two possible cases, this isn't pretty and we might be able to write some better peripheral structs in the future, but for now, if the length we're setting is 62 or less, and is divisible by two, then we put a zero in the top bit, and the number of 2-byte lumps in at bits 14:10, and if it's 64 or more, we mask off the bottom to check it's divisible by 32, and then put the count (minus one) of those blocks in, instead, and set the top bit to mark it as such.

Fortunately, when we set constants, Rust's compiler manages to optimise all this very quickly. For a BTABLE at the bottom of the PMA, and an initialisation statement of:

    unsafe { (&*usb::pma::PMA.get()).set_rxcount(1, 64); }

then we end up with the simple instruction sequence:

80001e4:    f246 001c   movw    r0, #24604  ; 0x601c
80001e8:    f44f 4104   mov.w   r1, #33792  ; 0x8400
80001ec:    f2c4 0000   movt    r0, #16384  ; 0x4000
80001f0:    8001        strh    r1, [r0, #0]

We can decompose that into a C like *((u16*)0x4000601c) = 0x8400 and from there we can see that it's writing to the u16 at 0x1c bytes into the CPU's view of the PMA, which is 14 bytes into the PMA itself. Since we know we set the BTABLE at the start of the PMA, it's 14 bytes into the BTABLE which is firmly in the EP1 entries. Specifically it's USB_COUNT1_RX which is what we were hoping for. To confirm this, check out page 651 of the datasheet. The value set was 0x8400 which we can decompose into 0x8000 and 0x0400. The first is the top bit and tells us that BL_SIZE is one, and thus the blocks are 32 bytes long. Next the 0x4000 if we shift it right ten places, we get the value 2 for the field NUM_BLOCK and multiplying 2 by 32 we get the 64 bytes we asked it to set as the size of the RX buffer. It has done exactly what we hoped it would, but the compiler managed to optimise it into a single 16 bit store of a constant value to a constant location. Nice and efficient.

Finally, let's look at what happens if we want to write a packet into the PMA. For now, let's assume packets come as slices of u16s because that'll make our life a little simpler:

    pub fn write_buffer(&self, base: usize, buf: &[u16]) {
        for (ofs, v) in buf.iter().enumerate() {
            self.set_u16(base + (ofs * 2), *v);
        }
    }

Yes, even though we're deep in no_std territory, we can still get an iterator over the slice, and enumerate it, getting a nice iterator of (index, value) though in this case, the value is a ref to the content of the slice, so we end up with *v to deref it. I am sure I could get that automatically happening but for now it's there.

Amazingly, despite using iterators, enumerators, high level for loops, function calls, etc, if we pop:

    unsafe { (&*usb::pma::PMA.get()).write_buffer(0, &[0x1000, 0x2000, 0x3000]); }

into our main function and compile it, we end up with the instruction sequence:

80001e4:    f246 0000   movw    r0, #24576  ; 0x6000
80001e8:    f44f 5180   mov.w   r1, #4096   ; 0x1000
80001ec:    f2c4 0000   movt    r0, #16384  ; 0x4000
80001f0:    8001        strh    r1, [r0, #0]
80001f2:    f44f 5100   mov.w   r1, #8192   ; 0x2000
80001f6:    8081        strh    r1, [r0, #4]
80001f8:    f44f 5140   mov.w   r1, #12288  ; 0x3000
80001fc:    8101        strh    r1, [r0, #8]

which, as you can see, ends up being three sequential halfword stores directly to the right locations in the CPU's view of the PMA. You have to love seriously aggressive compile-time optimisation :-)

Hopefully, by next time, we'll have layered some more pleasant routines on our PMA code, and begun a foray into the setup necessary before we can begin handling interrupts and start turning up on a USB port.

Building a USB descriptor table set Daniel Silverstone

In order to proceed further on our USB/STM32 oddessy, we need to start to build a USB descriptor set for our first prototype piece of code. For this piece, we're going to put together a USB device which is vendor-specific class-wise and has a single configuration with a interface with a single endpoint which we're not going to actually implement anything of. What we're after is just to get the information presented to the computer so that lsusb can see it.

To get these built, let's refer to information we discovered and recorded in a previous post about how descriptors go together.

Device descriptor

Remembering that values which are > 1 byte in length are always stored little-endian, we can construct our device descriptor as:

Our device descriptor
Field Name Value Bytes
bLength 18 0x12
bDescriptorType DEVICE 0x01
bcdUSB USB 2.0 0x00 0x02
bDeviceClass 0 0x00
bDeviceSubClass 0 0x00
bDeviceProtocol 0 0x00
bMaxPacketSize 64 0x40
idVendor TEST 0xff 0xff
idProduct TEST 0xff 0xff
bcdDevice 0.0.1 0x01 0x00
iManufacturer 1 0x01
iProduct 2 0x02
iSerialNumber 3 0x03
bNumConfigurations 1 0x01

We're using the vendor ID and product id 0xffff because at this point we don't have any useful values for this (it costs $5,000 to register a vendor ID).

This gives us a final byte array of:

0x12 0x01 0x00 0x02 0x00 0x00 0x00 0x40 (Early descriptor)

0xff 0xff 0xff 0xff 0x01 0x00 0x01 0x02 0x03 0x01 (and the rest)

We're reserving string ids 1, 2, and 3, for the manufacturer string, product name string, and serial number string respectively. I'm deliberately including them all so that we can see it all come out later in lsusb.

If you feed the above hex sequence into a USB descriptor decoder then you can check my working.

Endpoint Descriptor

We want a single configuration, which covers our one interface, with one endpoint in it. Let's start with the endpoint...

Our bulk IN endpoint
Field Name Value Bytes
bLength 7 0x07
bDescriptorType ENDPOINT 0x05
bEndpointAddress EP2IN 0x82
bmAttributes BULK 0x02
wMaxPacketSize 64 0x40 0x00
bInterval IGNORED 0x00

We're giving a single bulk IN endpoint, since that's the simplest thing to describe at this time. This endpoint will never be ready and so nothing will ever be read into the host.

All that gives us:

0x07 0x05 0x82 0x02 0x40 0x00 0x00

Interface Descriptor

The interface descriptor prefaces the endpoint set, and thanks to our simple single endpoint, and no plans for alternate interfaces, we can construct the interface simply as:

Our single simple interface
Field Name Value Bytes
bLength 9 0x09
bDescriptorType INTERFACE 0x04
bInterfaceNumber 1 0x01
bAlternateSetting 1 0x01
bNumEndpoints 1 0x01
bInterfaceClass 0 0x00
bInterfaceSubClass 0 0x00
bInterfaceProtocol 0 0x00
iInterface 5 0x05

All that gives us:

0x09 0x04 0x01 0x01 0x01 0x00 0x00 0x00 0x05

Configuration descriptor

Finally we can put it all together and get the configuration descriptor...

Our sole configuration, encapsulating the interface and endpoint above
Field Name Value Bytes
bLength 9 0x09
bDescriptorType CONFIG 0x02
wTotalLength 9+9+7 0x19 0x00
bNumInterfaces 1 0x01
bConfigurationValue 1 0x01
iConfiguration 4 0x04
bmAttributes Bus powered, no wake 0x80
bMaxPower 500mA 0xfa

The wTotalLength field is interesting. It contains the configuration length, the interface length, and the endpoint length, hence 9 plus 9 plus 7 is 25.

This gives:

0x09 0x02 0x19 0x00 0x01 0x01 0x04 0x80 0xfa

String descriptors

We allowed ourselves a total of five strings, they were iManufacturer, iProduct, iSerial (from the device descriptor), iConfiguration (from the configuration descriptor), and iInterface (from the interface descriptor) respectively.

Our string descriptors will therefore be:

String descriptor zero, en_GB only
Field Name Value Bytes
bLength 4 0x04
bDescriptorType STRING 0x03
wLangID[0] en_GB 0x09 0x08

0x04 0x03 0x09 0x08

...and...

String descriptor one, iManufacturer
Field Name Value Bytes
bLength 38 0x26
bDescriptorType STRING 0x03
bString "Rusty Manufacturer" ...

0x26 0x03 0x52 0x00 0x75 0x00 0x73 0x00

0x74 0x00 0x79 0x00 0x20 0x00 0x4d 0x00

0x61 0x00 0x6e 0x00 0x75 0x00 0x66 0x00

0x61 0x00 0x63 0x00 0x74 0x00 0x75 0x00

0x72 0x00 0x65 0x00 0x72 0x00

(You get the idea, there's no point me breaking down the rest of the string descriptors here, suffice it to say that the other strings are appropriate for the values they represent - namely product, serial, configuration, and interface.)

Putting it all together

Given all the above, we have a device descriptor which is standalone, then a configuration descriptor which encompasses the interface and endpoint descriptors too. Finally we have a string descriptor table with six entries, the first is the language sets available, and the rest are our strings. In total we have:

    // Device descriptor
    const DEV_DESC: [u8; 18] = {
        0x12, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x40,
        0xff, 0xff, 0xff, 0xff, 0x01, 0x00, 0x01, 0x02,
        0x03, 0x01
    };

    // Configuration descriptor
    const CONF_DESC: [u8; 25] = {
        0x09, 0x02, 0x19, 0x00, 0x01, 0x01, 0x04, 0x80, 0xfa,
        0x09, 0x04, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x05,
        0x07, 0x05, 0x82, 0x02, 0x40, 0x00, 0x00
    };

    // String Descriptor zero
    const STR_DESC_0: [u8; 4] = {0x04, 0x03, 0x09, 0x08};

    // String Descriptor 1, "Rusty Manufacturer"
    const STR_DESC_1: [u8; 38] = {
        0x26, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x4d, 0x00,
        0x61, 0x00, 0x6e, 0x00, 0x75, 0x00, 0x66, 0x00,
        0x61, 0x00, 0x63, 0x00, 0x74, 0x00, 0x75, 0x00,
        0x72, 0x00, 0x65, 0x00, 0x72, 0x00
    };

    // String Descriptor 2, "Rusty Product"
    const STR_DESC_2: [u8; 28] = {
        0x1c, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x50, 0x00,
        0x72, 0x00, 0x6f, 0x00, 0x64, 0x00, 0x75, 0x00,
        0x63, 0x00, 0x74, 0x00
    };

    // String Descriptor 3, "123ABC"
    const STR_DESC_3: [u8; 14] = {
        0x0e, 0x03, 0x31, 0x00, 0x32, 0x00, 0x33, 0x00,
        0x41, 0x00, 0x42, 0x00, 0x43, 0x00
    };

    // String Descriptor 4, "Rusty Configuration"
    const STR_DESC_4: [u8; 40] = {
        0x28, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x43, 0x00,
        0x6f, 0x00, 0x6e, 0x00, 0x66, 0x00, 0x69, 0x00,
        0x67, 0x00, 0x75, 0x00, 0x72, 0x00, 0x61, 0x00,
        0x74, 0x00, 0x69, 0x00, 0x6f, 0x00, 0x6e, 0x00
    };

    // String Descriptor 5, "Rusty Interface"
    const STR_DESC_5: [u8; 32] = {
        0x20, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x49, 0x00,
        0x6e, 0x00, 0x74, 0x00, 0x65, 0x00, 0x72, 0x00,
        0x66, 0x00, 0x61, 0x00, 0x63, 0x00, 0x65, 0x00
    };

With the above, we're a step closer to our first prototype which will hopefully be enumerable. Next time we'll look at beginning our prototype low level USB device stack mock-up.

STM32 and RTFM Daniel Silverstone

I have been working with STM32 chips on-and-off for at least eight, possibly closer to nine years. About as long as ST have been touting them around. I love the STM32, and have done much with them in C. But, as my previous two posts may have hinted, I would like to start working with Rust instead of C. To that end, I have been looking with great joy at the work which Jorge Aparicio has been doing around Cortex-M3 and Rust. I've had many comments in person at Debconf, and also several people mention on Twitter, that they're glad more people are looking at this. But before I can get too much deeper into trying to write my USB stack, I need to sort a few things from what Jorge has done as demonstration work.

Okay, this is fast, but we need Ludicrous speed

All of Jorge's examples seem to leave the system clocks in a fairly default state, excepting turning on the clocks to the peripherals needed during the initialisation phase. Sadly, if we're going to be running the USB at all, we need the clocks to run a tad faster. Since my goal is to run something moderately CPU intensive on the end of the USB too, it makes sense to try and get our STM32 running at maximum clock speed. For the one I have, that's 72MHz rather than the 8MHz it starts out with. Nine times more cycles to do computing in makes a lot of sense.

As I said above, I've been doing STM32 in C a lot for many years; and fortunately I have built systems with the exact chip that's on the blue-pill before. As such, if I rummage, I can find some old C code which does what we need...

    /* Enable HSE */
    RCC_HSEConfig(RCC_HSE_ON);

    /* Wait till HSE is ready */
    HSEStartUpStatus = RCC_WaitForHSEStartUp();

    if (HSEStartUpStatus == SUCCESS)
    {
      /* Enable Prefetch Buffer */
      FLASH_PrefetchBufferCmd(FLASH_PrefetchBuffer_Enable);
      /* Flash 2 wait state */
      FLASH_SetLatency(FLASH_Latency_2);

      /* HCLK = SYSCLK */
      RCC_HCLKConfig(RCC_SYSCLK_Div1);
      /* PCLK2 = HCLK */
      RCC_PCLK2Config(RCC_HCLK_Div1);
      /* PCLK1 = HCLK/2 */
      RCC_PCLK1Config(RCC_HCLK_Div2);
      /* ADCCLK = PCLK2/6 */
      RCC_ADCCLKConfig(RCC_PCLK2_Div6);
      /* PLLCLK = 8MHz * 9 = 72 MHz */
      RCC_PLLConfig(RCC_PLLSource_HSE_Div1, RCC_PLLMul_9);

      /* Enable PLL */
      RCC_PLLCmd(ENABLE);
      /* Wait till PLL is ready */
      while (RCC_GetFlagStatus(RCC_FLAG_PLLRDY) == RESET)
      {}

      /* Select PLL as system clock source */
      RCC_SYSCLKConfig(RCC_SYSCLKSource_PLLCLK);
      /* Wait till PLL is used as system clock source */
      while (RCC_GetSYSCLKSource() != 0x08)
      {}
    }

This code, rather conveniently, uses an 8MHz external crystal so we can almost direct-port it to the blue-pill Rust code and see how we go. If you're used to the CMSIS libraries for STM32, then you won't completely recognise the above since it uses the pre-CMSIS core libraries to do its thing. Library code from 2008 and it's still good on today's STM32s providing they're in the right family :-)

A direct conversion to Rust, using Jorge's beautifully easy to work with crates made from svd2rust results in:

    fn make_go_faster(rcc: &RCC, flash: &FLASH) {
        rcc.cr.modify(|_, w| w.hseon().enabled());
        while !rcc.cr.read().hserdy().is_ready() {}
        flash.acr.modify(|_, w| w.prftbe().enabled());
        flash.acr.modify(|_, w| w.latency().two());
        rcc.cfgr.modify(|_, w| w
                        .hpre().div1()
                        .ppre2().div1()
                        .ppre1().div2()
                        // .adcpre().bits(8)
                        .pllsrc().external()
                        .pllxtpre().div1()
                        .pllmul().mul9()
        );
        rcc.cr.modify(|_, w| w.pllon().enabled());
        while rcc.cr.read().pllrdy().is_unlocked() {}
        rcc.cfgr.modify(|_,w| w.sw().pll());
        while !rcc.cfgr.read().sws().is_pll() {}
    }

Now, I've not put the comments in which were in the C code, because I'm being very lazy right now, but if you follow the two together you should be able to work it through. I don't have timeouts for the waits, and you'll notice a single comment there (I cannot set up the ADC prescaler because for some reason the SVD is missing any useful information and so the generated crate only carries an unsafe function (bits()) and I'm trying to steer clear of unsafe for now. Still, I don't need the ADC immediately, so I'm okay with this.

By using this function in the beginning of the init() function of the blinky example, I can easily demonstrate the clock is going faster since the LED blinks more quickly.

This function demonstrates just how simple it is to take bit-manipulation from the C code and turn it into (admittedly bad looking) Rust with relative ease and without any of the actual bit-twiddling. I love it.

Mess with time, and you get unexpected consequences

Sadly, when you mess with the clock tree on a microcontroller, you throw a lot of things out of whack. Not least, by adjusting the clock frequency up we end up adjusting the AHB, APB1, and APB2 clock frequencies. This has direct consequences for peripherals floating around on those busses. Fortunately Jorge thought of this and while the blue-pill crate hard-wires those frequencies to 8MHz, they are, at least, configurable in code in some sense.

If we apply the make_go_faster() function to the serial loopback example, it simply fails to work because now the bus which the USART1 peripheral is connected to (APB2) is going at a different speed from the expected power-on default of 8MHz. If you remember from the function, we did .hpre().div1() which set HCLK to 72MHz, then .ppre1().div2() which sets the APB1 bus clock to be HCLK divided by 2, and .ppre2().div1() which sets APB2 bus clock to be HCLK. This means that we'd need to alter src/lib.rs to reflect these changes in the clock frequences and in theory loopback would start working once more.

It'd be awkward to try and demonstrate all that to you since I only have a phone camera to hand, but if you own a blue-pill then you can clone Jorge's repo and have a go yourself and see that I'm not bluffing you.

With all this done, it'll be time to see if we can bring the USB peripheral in the STM32 online, and that will be the topic of my next post in this discovery series.

USB Device Stacks, on RTFM, part 2 Daniel Silverstone

Previously we talked about all the different kinds of descriptors which USB devices use to communicate their capability. This is important stuff because to write any useful USB device firmware we need to be able to determine how to populate our descriptors. However, having that data on the device is entirely worthless without an understanding of how it gets from the device to the host so that it can be acted upon. To understand that, let's look at the USB wire protocol.

Note, I'll again be talking mostly about USB2.0 low- and full-speed. I believe that high speed is approximately the same but with faster wires, except not quite that simple.

Down to the wire

I don't intend to talk about the actual electrical signalling, though it's not un-reasonable for you to know that USB is a pair of wires forming a differentially signalled bidirectional serial communications link. The host is responsible for managing all the framing and timing on the link, and for formatting the communications into packets.

There are a number of packet types which can appear on the USB link:

Packet type Purpose
Token Packet When the host wishes to send a message to the Control endpoint to configure the device, read data IN, or write data OUT, it uses this to start the transaction.
Data(0/1) Packet Following a Setup, In, or Out token, a Data packet is a transfer of data (in either direction). The 0 and 1 alternate to provide a measure of confidence against lost packets.
Handshake Packet Following a data packet of some kind, the other end may ACK the packet (all was well), NAK the packet (report that the device cannot, temporarily, send/receive data, or that an interrupt endpoint isn't triggered), or STALL the bus in which case the host needs to intervene.
Start of Frame Every 1ms (full-speed) the host will send a SOF packet which carries a frame number. This can be used to help keep time on very simple devices. It also divides the bus into frames within which bandwidth is allocated.

As an example, when the host wishes to perform a control transfer, the following packets are transacted in turn:

  1. Setup Token - The host addresses the device and endpoint (OUT0)
  2. Data0 Packet - The host transmits a GET_DESCRIPTOR for the device descriptor
  3. Ack Packet - The device acknowledges receipt of the request

This marks the end of the first transaction. The device decodes the GET_DESCRIPTOR request and prepares the device descriptor for transmission. The transmission occurs as the next transaction on the bus. In this example, we're assuming 8 byte maximum transmission sizes, for illustrative purposes.

  1. In Token - The host addresses the device and endpoint (IN0)
  2. Data1 Packet - The device transmits the first 8 bytes of the descriptor
  3. Ack Packet - The host acknowledges the data packet
  4. In Token - The host addresses the device and endpoint (IN0)
  5. Data0 Packet - The device transmits the remaining 4 bytes of the descriptor (padded)
  6. Ack Packet - The host acknowledges the data packet

The second transaction is now complete, and the host has all the data it needs to proceed. Finally a status transaction occurs in which:

  1. Out Token - The host addresses the device and endpoint (OUT0)
  2. Data1 Packet - The host transmits a 0 byte data packet to indicate successful completion
  3. Ack Packet - The device acknowledges the completion, indicating its own satisfaction

And thus ends the full control transaction in which the host retrieves the device descriptor.

From a high level, we need only consider the activity which occurs at the point of the acknowledgement packets. In the above example:

  1. On the first ACK the device prepares IN0 to transmit the descriptor, readying whatever low level device stack there is with a pointer to the descriptor and its length in bytes.
  2. On the second ACK the low levels are still thinking.
  3. On the third ACK the transmission from IN0 is complete and the endpoint no longer expects to transfer data.
  4. On the fourth ACK the control transaction is entirely complete.

Thinking at the low levels of the control interface

Before we can build a high level USB stack, we need to consider the activity which might occur at the lower levels. At the low levels, particularly of the device control interface, work has to be done at each and every packet. The hardware likely deals with the token packet for us, leaving the data packets for us to process, and the resultant handshake packets will be likely handled by the hardware in response to our processing the data packets.

Since every control transaction is initiated by a setup token, let's look at the setup requests which can come our way...

Setup Packet (Data) Format
Field Name Byte start Byte length Encoding Meaning
bmRequestType 0 1 Bitmap Describes the kind of request, and the target of it. See below.
bRequest 1 1 Code The request code itself, meanings of the rest of the fields vary by bRequest
wValue 2 2 Number A 16 bit value whose meaning varies by request type
wIndex 4 2 Number A 16 bit value whose meaning varies by request type but typically encodes an interface number or endpoint.
wLength 6 2 Number A 16 bit value indicating the length of the transfer to come.

Since bRequest is essentially a switch against which multiple kinds of setup packet are selected between, here's the meanings of a few...

GET_DESCRIPTOR (Device) setup packet
Field Name Value Meaning
bmRequestType 0x08 Data direction is IN (from device to host), recipient is the device
bRequest 0x06 GET_DESCRIPTOR (in this instance, the device descriptor is requested)
wValue 0x0001 This means the device descriptor
wIndex 0x0000 Irrelevant, there's only 1 device descriptor anyway
wLength 12 This is the length of a device descriptor (12 bytes)
SET_ADDRESS to set a device's USB address
Field Name Value Meaning
bmRequestType 0x00 Data direction is OUT (from host to device), recipient is the device
bRequest 0x05 SET_ADDRESS (Set the device's USB address)
wValue 0x00nn The address for the device to adopt (max 127)
wIndex 0x0000 Irrelevant for address setting
wLength 0 There's no data transfer expected for this setup operation

Most hardware blocks will implement an interrupt at the point that the Data packet following the Setup packet has been receive. This is typically called receiving a 'Setup' packet and then it's up to the device stack low levels to determine what to do and dispatch a handler. Otherwise an interrupt will fire for the IN or OUT tokens and if the endpoint is zero, the low level stack will handle it once more.

One final thing worth noting about SET_ADDRESS is that it doesn't take effect until the completion of the zero-length "status" transaction following the setup transaction. As such, the status request from the host will still be sent to address zero (the default for new devices).

A very basic early "packet trace"

This is an example, and is not guaranteed to be the packet sequence in all cases. It's a good indication of the relative complexity involved in getting a fresh USB device onto the bus though...

When a device first attaches to the bus, the bus is in RESET state and so the first event a device sees is a RESET which causes it to set its address to zero, clear any endpoints, clear the configuration, and become ready for control transfers. Shortly after this, the device will become suspended.

Next, the host kicks in and sends a port reset of around 30ms. After this, the host is ready to interrogate the device.

The host sends a GET_DESCRIPTOR to the device, whose address at this point is zero. Using the information it receives from this, it can set up the host-side memory buffers since the device descriptor contains the maximum transfer size which the device supports.

The host is now ready to actually 'address' the device, and so it sends another reset to the device, again around 30ms in length.

The host sends a SET_ADDRESS control request to the device, telling it that its new address is nn. Once the acknowledgement has been sent from the host for the zero-data status update from the device, the device sets its internal address to the value supplied in the request. From now on, the device shall respond only to requests to nn rather than to zero.

At this point, the host will begin interrogating further descriptors, looking at the configuration descriptors and the strings, to build its host-side representation of the device. These will be GET_DESCRIPTOR and GET_STRING_DESCRIPTOR requests and may continue for some time.

Once the host has satisfied itself that it knows everything it needs to about the device, it will issue a SET_CONFIGURATION request which basically starts everything up in the device. Once the configuration is set, interrupt endpoints will be polled, bulk traffic will be transferred, Isochronous streams begin to run, etc.

Okay, but how do we make this concrete?

So far, everything we've spoken about has been fairly abstract, or at least "soft". But to transfer data over USB does require some hardware. (Okay, okay, we could do it all virtualised, but there's no fun in that). The hardware I'm going to be using for the duration of this series is the STM32 on the blue-pill development board. This is a very simple development board which does (in theory at least) support USB device mode.

If we view the schematic for the blue-pill, we can see a very "lightweight" USB interface which has a pullup resistor for D+. This is the way that a device signals to the host that it is present, and that it wants to speak at full-speed. If the pullup were on D- then it would be a low-speed device. High speed devices need a little more complexity which I'm not going to go into for today.

The USB lines connect to pins PA11 and PA12 which are the USB pins on the STM32 on the board. Since USB is quite finicky, the STM32 doesn't let you remap that function elsewhere, so this is all looking quite good for us so far.

The specific STM32 on the blue-pill is the STM32F103C8T6. By viewing its product page on ST's website we can find the reference manual for the part. Jumping to section 23 we learn that this STM32 supports full-speed USB2.0 which is convenient given the past article and a half. We also learn it supports up to eight endpoints active at any one time, and offers double-buffering for our bulk and isochronous transfers. It has some internal memory for packet buffering, so it won't use our RAM bandwidth while performing transfers, which is lovely.

I'm not going to distill the rest of that section here, because there's a large amount of data which explains how the USB macrocell operates. However useful things to note are:

  • How IN OUT and SETUP transfers work.
  • How the endpoint buffer memory is configured.
  • That all bus-powered devices MUST respond to suspend/resume properly
  • That the hardware will prioritise endpoint interrupts for us so that we only need deal with the most pressing item at any given time.
  • There is an 'Enable Function' bit in the address register which must be set or we won't see any transactions at all.
  • How the endpoint registers signal events to the device firmware.

Next time, we're going to begin the process of writing a very hacky setup routine to try and initialise the USB device macrocell so that we can see incoming transactions through the ITM. It should be quite exciting, but given how complex this will be for me to learn, it might be a little while before it comes through.

USB Device Stacks, on RTFM Daniel Silverstone

I have been spending time with Jorge Aparicio's RTFM for Cortex M3 framework for writing Rust to target Cortex-M3 devices from Arm (and particularly the STM32F103 from ST Microelectronics). Jorge's work in this area has been of interest to me ever since I discovered him working on this stuff a while ago. I am very tempted by the idea of being able to implement code for the STM32 with the guarantees of Rust and the language features which I have come to love such as the trait system.

I have been thinking to myself that, while I admire and appreciate the work done on the GNUK, I would like to, personally, have a go at implementing some kind of security token on an STM32 as a USB device. And with the advent of the RTFM for M3 work, and Jorge's magical tooling to make it easier to access and control the registers on an M3 microcontroller, I figured it'd be super-nice to do this in Rust, with all the advantages that entails in terms of isolating unsafe behaviour and generally having the potential to be more easily verified as not misbehaving.

To do this though, means that I need a USB device stack which will work in the RTFM framework. Sadly it seems that, thus-far, only Jorge has been working on drivers for any of the M3 devices his framework supports. And one person can only do so much. So, in my infinite madness, I decided I should investigate the complexity of writing a USB device stack in Rust for the RTFM/M3 framework. (Why I thought this was a good idea is lost to the mists of late night Googling, but hey, it might make a good talk at the next conference I go to). As such, this blog post, and further ones along these lines, will serve as a partial tour of what I'm up to, and a partial aide-memoir for me about learning USB. If I get something horribly wrong, please DO contact me to correct me, otherwise I'll just continue to be wrong. If I've simplified something but it's still strictly correct, just let me know if it's an oversimplification since in a lot of cases there's no point in me putting the full details into a blog posting. I will mostly be considering USB2.0 protocol details but only really for low and full speed devices. (The hardware I'm targetting does low-speed and full-speed, but not high-speed. Though some similar HW does high-speed too, I don't have any to hand right now)

A brief introduction to USB

In order to go much further, I needed a grounding in USB. It's a multi-layer protocol as you might expect, though we can probably ignore the actual electrical layer since any device we might hope to support will have to have a hardware block to deal with that. We will however need to consider the packet layer (since that will inform how the hardware block is implemented and thus its interface) and then the higher level protocols on top.

USB is a deliberately asymmetric protocol. Devices are meant to be significantly easier to implement, both in terms of hardware and software, as compared with hosts. As such, despite some STM32s having OTG ports, I have no intention of supporting host mode at this time.

USB is arranged into a set of busses which are, at least in the USB1.1 case, broadcast domains. As such, each device has an address assigned to it by the host during an early phase called 'configuration'. Once the address is assigned, the device is expected to only ever respond to messages addressed to it. Note that since everything is asymmetric in USB, the device can't send messages on its own, but has to be asked for them by the host, and as such the addressing is always from host toward device.

USB devices then expose a number of endpoints through which communication can flow IN to the host or OUT to the device. Endpoints are not bidirectional, but the in and out endpoints do overlap in numbering. There is a special pair of endpoints, IN0 and OUT0 which, between them, form what I will call the device control endpoints. The device control endpoints are important since every USB device MUST implement them, and there are a number of well defined messages which pass over them to control the USB device. In theory a bare minimum USB device would implement only the device control endpoints.

Configurations, and Classes, and Interfaces, Oh My!

In order for the host to understand what the USB device is, and what it is capable of, part of the device control endpoints' responsibility is to provide a set of descriptors which describe the device. These descriptors form a heirarchy and are then glommed together into a big lump of data which the host can download from the device in order to decide what it is and how to use it. Because of various historical reasons, where a multi-byte value is used, they are defined to be little-endian, though there are some BCD fields. Descriptors always start with a length byte and a type byte because that way the host can parse/skip as necessary, with ease.

The first descriptor is the device descriptor, is a big one, and looks like this:

Device Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (18)
bDescriptorType 1 1 Constant Device Descriptor (0x01)
bcdUSB 2 2 BCD USB spec version compiled with
bDeviceClass 4 1 Class Code, assigned by USB org (0 means "Look at interface descriptors", common value is 2 for CDC)
bDeviceSubClass 5 1 SubClass Code, assigned by USB org (usually 0)
bDeviceProtocol 6 1 Protocol Code, assigned by USB org (usually 0)
bMaxPacketSize 7 1 Number Max packet size for IN0/OUT0 (Valid are 8, 16, 32, 64)
idVendor 8 2 ID 16bit Vendor ID (Assigned by USB org)
idProduct 10 2 ID 16bit Product ID (Assigned by manufacturer)
bcdDevice 12 2 BCD Device version number (same encoding as bcdUSB)
iManufacturer 14 1 Index String index of manufacturer name (0 if unavailable)
iProduct 15 1 Index String index of product name (0 if unavailable)
iSerialNumber 16 1 Index String index of device serial number (0 if unavailable)
bNumConfigurations 17 1 Number Count of configurations the device has.

This looks quite complex, but breaks down into a relatively simple two halves. The first eight bytes carries everything necessary for the host to be able to configure itself and the device control endpoints properly in order to communicate effectively. Since eight bytes is the bare minimum a device must be able to transmit in one go, the host can guarantee to get those, and they tell it what kind of device it is, what USB protocol it supports, and what the maximum transfer size is for its device control endpoints.

The encoding of the bcdUSB and bcdDevice fields is interesting too. It is of the form 0xMMmm where MM is the major number, mm the minor. So USB2.0 is encoded as 0x0200, USB1.1 as 0x0110 etc. If the device version is 17.36 then that'd be 0x1736.

Other fields of note are bDeviceClass which can be 0 meaning that interfaces will specify their classes, and idVendor/idProduct which between them form the primary way for the specific USB device to be identified. The Index fields are indices into a string table which we'll look at later. For now it's enough to know that wherever a string index is needed, 0 can be provided to mean "no string here".

The last field is bNumConfigurations and this indicates the number of ways in which this device might function. A USB device can provide any number of these configurations, though typically only one is provided. If the host wishes to switch between configurations then it will have to effectively entirely quiesce and reset the device.

The next kind of descriptor is the configuration descriptor. This one is much shorter, but starts with the same two fields:

Configuration Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (9)
bDescriptorType 1 1 Constant Configuration Descriptor (0x02)
wTotalLength 2 2 Number Size of the configuration in bytes, in total
bNumInterfaces 4 1 Number The number of interfaces in this configuration
bConfigurationValue 5 1 Number The value to use to select this configuration
iConfiguration 6 1 Index The name of this configuration (0 for unavailable)
bmAttributes 7 1 Bitmap Attributes field (see below)
bMaxPower 8 1 Number Maximum bus power this configuration will draw (in 2mA increments)

An important field to consider here is the bmAttributes field which tells the host some useful information. Bit 7 must be set, bit 6 is set if the device would be self-powered in this configuration, bit 5 indicates that the device would like to be able to wake the host from sleep mode, and bits 4 to 0 must be unset.

The bMaxPower field is interesting because it encodes the power draw of the device (when set to this configuration). USB allows for up to 100mA of draw per device when it isn't yet configured, and up to 500mA when configured. The value may be used to decide if it's sensible to configure a device if the host is in a low power situation. Typically this field will be set to 50 to indicate the nominal 100mA is fine, or 250 to request the full 500mA.

Finally, the wTotalLength field is interesting because it tells the host the total length of this configuration, including all the interface and endpoint descriptors which make it up. With this field, the host can allocate enough RAM to fetch the entire configuration descriptor block at once, simplifying matters dramatically for it.

Each configuration has one or more interfaces. The interfaces group some endpoints together into a logical function. For example a configuration for a multifunction scanner/fax/printer might have an interface for the scanner function, one for the fax, and one for the printer. Endpoints are not shared among interfaces, so when building this table, be careful.

Next, logically, come the interface descriptors:

Interface Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (9)
bDescriptorType 1 1 Constant Interface Descriptor (0x04)
bInterfaceNumber 2 1 Number The number of the interface
bAlternateSetting 3 1 Number The interface alternate index
bNumEndpoints 4 1 Number The number of endpoints in this interface
bInterfaceClass 5 1 Class The interface class (USB Org defined)
bInterfaceSubClass 6 1 SubClass The interface subclass (USB Org defined)
bInterfaceProtocol 7 1 Protocol The interface protocol (USB Org defined)
iInterface 8 1 Index The name of the interface (or 0 if not provided)

The important values here are the class/subclass/protocol fields which provide a lot of information to the host about what the interface is. If the class is a USB Org defined one (e.g. 0x02 for Communications Device Class) then the host may already have drivers designed to work with the interface meaning that the device manufacturer doesn't have to provide host drivers.

The bInterfaceNumber is used by the host to indicate this interface when sending messages, and the bAlternateSetting is a way to vary interfaces. Two interfaces with the came bInterfaceNumber but different bAlternateSettings can be switched between (like configurations, but) without resetting the device.

Hopefully the rest of this descriptor is self-evident by now.

The next descriptor kind is endpoint descriptors:

Endpoint Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (7)
bDescriptorType 1 1 Constant Endpoint Descriptor (0x05)
bEndpointAddress 2 1 Endpoint Endpoint address (see below)
bmAttributes 3 1 Bitmap Endpoint attributes (see below)
wMaxPacketSize 4 2 Number Maximum packet size this endpoint can send/receive
bInterval 6 1 Number Interval for polling endpoint (in frames)

The bEndpointAddress is a 4 bit endpoint number (so there're 16 endpoint indices) and a bit to indicate IN vs. OUT. Bit 7 is the direction marker and bits 3 to 0 are the endpoint number. This means there are 32 endpoints in total, 16 in each direction, 2 of which are reserved (IN0 and OUT0) giving 30 endpoints available for interfaces to use in any given configuration. The bmAttributes bitmap covers the transfer type of the endpoint (more below), and the bInterval is an interval measured in frames (1ms for low or full speed, 125µs in high speed). bInterval is only valid for some endpoint types.

The final descriptor kind is for the strings which we've seen indices for throughout the above. String descriptors have two forms:

String Descriptor (index zero)
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (variable)
bDescriptorType 1 1 Constant String Descriptor (0x03)
wLangID[0] 2 2 Number Language code zero (e.g. 0x0409 for en_US)
wLangID[n] 4.. 2 Number Language code n ...

This form (for descriptor 0) is that of a series of language IDs supported by the device. The device may support any number of languages. When the host requests a string descriptor, it will supply both the index of the string and also the language id it desires (from the list available in string descriptor zero). The host can tell how many language IDs are available simply by dividing bLength by 2 and subtracting 1 for the two header bytes.

And for string descriptors of an index greater than zero:

String Descriptor (index greater than zero)
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (variable)
bDescriptorType 1 1 Constant String Descriptor (0x03)
bString 2.. .. Unicode The string, in "unicode" format

This second form of the string descriptor is simply the the string is in what the USB spec calls 'Unicode' format which is, as of 2005, defined to be UTF16-LE without a BOM or terminator.

Since string descriptors are of a variable length, the host must request strings in two transactions. First a request for 2 bytes is sent, retrieving the bLength and bDescriptorType fields which can be checked and memory allocated. Then a request for bLength bytes can be sent to retrieve the entire string descriptor.

Putting that all together

Phew, this is getting to be quite a long posting, so I'm going to leave this here and in my next post I'll talk about how the host and device pass packets to get all that information to the host, and how it gets used.

Gitano 1.1 Daniel Silverstone

Today marks the release of Gitano 1.1. Richard(s) and I have spent quite a lot of time and effort on this release, and there's plenty of good stuff in it. We also released new versions of Lace, Supple, Luxio, and Gall to go alongside it, with bugfixes and improvements.

At this point, I intend to take a short break from Gitano to investigate some Rust-on-STM32 stuff, and then perhaps do some NetSurf work too.

F/LOSS activity, July 2017 Daniel Silverstone

Once again, my focus was on Gitano, which we're working toward a 1.1 for. We had another one of our Gitano developer days which was attended by Richard maw and myself. You are invited to read the wiki page but a summary of what happened, which directly involved me, is:

  • Once again, we reviewed our current task state
  • We had a good discussion about our code of conduct including adopting a small change from upstream to improve matters
  • I worked on, and submitted a patch for, improving nested error message reports in Lace.
  • I reviewed and merged some work from Richard about pattern centralisation
  • I responded to comments on a number of in-flight series Richard had reviewed for me.
  • We discussed our plans for 1.1 and agreed that we'll be skipping a developer day in August because so much of it is consumed by DebConf and so on.

Other than that, related to Gitano during July I:

  • Submitted some code series before the developer day covering Gall cleanups and hook support in Gitano.
  • Reviewed and merged some more Makefile updates from Richard Ipsum
  • Reviewed and merged a Supple fix for environment cleardown from Richard Ipsum
  • Fixed an issue in one of the Makefiles which made it harder to build with dh-lua
  • I began work in earnest on Gitano CI, preparing a lot of scripts and support to sit around Jenkins (for now) for CIing packaging etc for Gitano and Debian
  • I began work on a system branch concept for Gitano CI which will let us handle the CI of branches in the repos, even if they cross repos.

I don't think I've done much non-Gitano F/LOSS work in July, but I am now in Montréal for debconf 2017 so hopefully more to say next month.

Yay, finished my degree at last Daniel Silverstone

A little while back, in June, I sat my last exam for what I hoped would be the last module in my degree. For seven years, I've been working on a degree with the Open University and have been taking advantage of the opportunity to have a somewhat self-directed course load by taking the 'Open' degree track. When asked why I bothered to do this, I guess my answer has been a little varied. In principle it's because I felt like I'd already done a year's worth of degree and didn't want it wasted, but it's also because I have been, in the dim and distant past, overlooked for jobs simply because I had no degree and thus was an easy "bin the CV".

Fed up with this, I decided to commit to the Open University and thus began my journey toward 'qualification' in 2010. I started by transferring the level 1 credits from my stint at UCL back in 1998/1999 which were in a combination of basic programming in Java, some mathematics including things like RSA, and some psychology and AI courses which at the time were aiming at a degree called 'Computer Science with Cognitive Sciences'.

Then I took level 2 courses, M263 (Building blocks of software), TA212 (The technology of music) and MS221 (Exploring mathematics). I really enjoyed the mathematics course and so...

At level 3 I took MT365 (Graphs, networks and design), M362 (Developing concurrent distributed systems), TM351 (Data management and analysis - which I ended up hating), and finally finishing this June with TM355 (Communications technology).

I received an email this evening telling me the module result for TM355 had been posted, and I logged in to find I had done well enough to be offered my degree. I could have claimed my degree 18+ months ago, but I persevered through another two courses in order to qualify for an honours degree which I have now been awarded. Since I don't particularly fancy any ceremonial awarding, I just went through the clicky clicky and accepted my qualification of 'Batchelor of Science (Honours) Open, Upper Second-class Honours (2.1)' which grants me the letters 'BSc (Hons) Open (Open)' which, knowing me, will likely never even make it onto my CV because I'm too lazy.

It has been a significant effort, over the course of the past few years, to complete a degree without giving up too much of my personal commitments. In addition to earning the degree, I have worked, for six of the seven years it has taken, for Codethink doing interesting work in and around Linux systems and Trustable software. I have designed and built Git server software which is in use in some universities, and many companies, along with a good few of my F/LOSS colleagues. And I've still managed to find time to attend plays, watch films, read an average of 2 novel-length stories a week (some of which were even real books), and be a member of the Manchester Hackspace.

Right now, I'm looking forward to a stress free couple of weeks, followed by an immense amount of fun at Debconf17 in Montréal!

This blog is powered by ikiwiki.