Hello! It's clearly time for my yearly post, so this one will be over a recent interest and project- device tree overlays in FreeBSD.
Background
What are device tree overlays?
Device tree overlays are used to modify a device tree, generally (but not exclusively) used on Small Board Computers ("SBC"). The problem they usually solve is how to describe/list devices on a bus that doesn't support enumeration of attached devices. Your base DTB might describe an i2c/SPI bus or similar, then you can add overlays at runtime that describe peripherals attached to said i2c bus.
I won't go any further into describing overlays or how they are generated, because there are plenty of good resources that do so better than I could. The Raspberry Pi Foundation's Device Tree Documentation has some good information, and Adafruit has an article on device tree overlays; others can be found fairly easily.
Why do I care?
I've recently been working on Allwinner support on FreeBSD. A good amount of documentation for Allwinner SoCs can be found on the Sunxi Community Wiki, so they served as a pretty good starting point for getting into development on ARM platforms and kernel work in general. We recently flipped the switch to use DTS imported from mainline Linux releases directly, rather than our own DTS or modified DTS. This has had its ups and downs, but the results have been mostly positive.
However, this has also been somewhat inconvenient at times. Adding support for devices has to move with the pace of Linux releases, which is a roughly two month release cycle. We had some regressions in support because we weren't using stable bindings for some devices, and upstream (mainline Linux) hadn't adopted stable bindings yet, so the devices simply aren't in DTS. Regressions like ethernet going away on your SBC can be quite frustrating at times and were the main pain point, but all of this is happening in -HEAD so some breakage should be expected occasionally anyway.
This is where overlays come in. As a developer, I can easily pull the latest Linux -rc or patches going into Linux with the new bindings that will be coming in and build a new DTB from that. This is in fact necessary to try and keep ahead of the releases a little bit so we don't have major breakage when a new release comes in. The problem comes during any of the following scenarios:
- Testing compatibility with old bindings as well as new bindings
- Getting others to test the new binding support
- Helping users gain functionality of new bindings
Granted, none of these problems are particularly hard to solve. #1 is only a problem because it's inefficient to swap back and forth between old/new DTB frequently, and #2/#3 are basically the same problem from different perspectives. The latter problems are easily solved by just distributing new DTB as needed, but overlays can be pretty easily compiled and it's clear what's being changed from looking at an overlay.
Status
Where we were
When I started working in this area a couple of months ago now, our overlay support was fairly limited. We supported overlays to an extent, but there's no real spec for these things and it showed in both our dtc(1) and our loader implementation for overlay application. You could specify a comma-delimited list of overlays to apply as fdt_overlays in loader.conf(5), and loader would load and attempt to apply them. However, we had the following bugs/limitations:
- All fdt_overlays must be full filenames - this is in contrast to Linux land, where overlays were generally specified as the basename.
- All fdt_overlays must appear in one of the current module paths, with /boot/dtb being the logical path to place them in.
- fdt_overlays were not applied in all cases, including aarch64 boots and when U-Boot provided FDT on ARMv7 boots.
- A bug in the loading bits would discard previously loaded dtbo upon loading another one, effectively only allowing one overlay- the last one specified.
- When an overlay applied, /__symbols__ from the overlay would not get merged into the resulting FDT, so new symbols could not be referenced.
- Most importantly, because there was no spec on these things, our implementation would only work with overlays generated by BSDL dtc(1). This was due to the format of /__local_fixups__, which we were building in the exact same way as /__fixups__. This made sense, but was not the winning implementation in libfdt and everywhere else.
None of these were particularly serious, but it did hinder my ability to adopt overlays as a solution. gonzo@'s initial implementation did me a lot of good, and I really appreciate the work he did, but eventually I hit a point where it bit me pretty badly as I was getting neck deep in overlays. gonzo had already said we should use libfdt's implementation for overlays rather than continuing to maintain our own, so this was a clear next step.
Where we are now
- All but one (full filenames must be specified) of the above limitations are gone as of r328107 in -HEAD. Overlays may now be placed in /boot/overlays, which matches a convention from over in Linux land.
- A notable change, we now use libfdt for applying overlays as of importing libfdt 1.4.6 in r328106.
- All of the relevant changes should be MFC'd to stable/11 by 2018/01/25
- Ironically, our dtc(1) in-tree cannot currently generate overlays usable by our loader implementation. To generate overlays currently, either sysutils/dtc (GPL dtc) or an up-to-date BSDL dtc is required (get this from its GitHub project, you'll need to compile it). I will be importing BSDL dtc following some manpage revision, so hopefully this will not be the case for long.
Where we are going
As of time of writing, I have two reviews in flight: allow ".dtbo" to be dropped from fdt_overlays, and check /compatible on the overlay if it exists. The latter functionality is an interesting case: most examples of overlays will have a "compatible" specified on the root node, and it's expected to be checked and honored. This review checks it against /compateible on the base node while allowing a developer or someone just wanting to rapidly develop/prototype overlays to leave it off and still have their overlays apply.
I also consider /compatible checking a blocker for another feature I've been wanting lately: auto-loading of overlays present in /boot/overlays. The idea being that one could specify something in loader.conf(5), perhaps 'fdt_overlays_autoload="YES"', and the loader will attempt to load all overlays in /boot/overlays. In my opinion, this cannot work effectively unless we have a way in the overlay to prevent it from being loaded on a board that it isn't compatible with. This is mainly because I want to drop all of my sunxi overlays into /boot/overlays and have the loader just do the right thing no matter which board I've booted at the time. Autoloading overlays will likely require a slight re-working how we load overlays to check compatible at load time, rather than right before we attempt to apply it.
The next import of BSDL dtc(1) will leave us in a great position. The current master not only generates overlays properly, but it also adds support for the syntactic sugar recently supported by GPL dtc(1), an example of which you can find in the unit tests. This allows a syntax more common to DTS- the same syntax used to modify previously defined nodes, usually from a .dtsi. Internally, dtc takes &emac { prop = "value"; }; and generates a fragment with target = <&emac>; and the contents of the node are the contents of the fragment's __overlay__. This produces a much cleaner looking DTS for the overlay that is a little bit more consumable by humans.