From: Andrea Righi Date: Fri, 16 Jun 2023 09:42:54 +0000 (+0200) Subject: doc: update documentation and add README.md X-Git-Tag: v0.1~2 X-Git-Url: https://jxself.org/git/?a=commitdiff_plain;h=64353796cb4f5e48f3f96b24c219a13920ed0afc;p=annotations.git doc: update documentation and add README.md Update documentation with the same information published in the discourse post [1]. Moreover, convert README.rst into README.md. [1] https://discourse.ubuntu.com/t/kernel-configuration-in-ubuntu/35857 Signed-off-by: Andrea Righi --- diff --git a/README.md b/README.md new file mode 100644 index 0000000..a124612 --- /dev/null +++ b/README.md @@ -0,0 +1,259 @@ +# Ubuntu Kernel Annotations + +## Overview + +Ubuntu provides a wide variety of kernels for many different architectures and +flavours, such as generic, lowlatency, cloud kernels, kvm, etc. + +Maintaining a separate `.config` for each of these kernels would be extremely +challenging, Kconfig options need to always match the desired value in order to +have a kernel functional for the scope that it was designed for. + +Starting with Ubuntu 22.04 (Jammy Jellyfish) the kernel team introduced a new +format of annotations: a JSON-like file to store all the kernel config options +for the supported architectures and flavours. + +## Quick start + +Generate the Ubuntu kernel .config for a specific architecture & flavour +(i.e., `amd64`, `generic`): + +``` +$ annotations --arch amd64 --flavour generic --export > .config +``` + +Import your own kernel .config in the Ubuntu kernel for a specific architecture +and flavour (i.e., `amd64`, `generic`): + +``` +$ annotations --arch amd64 --flavour generic --import .config +``` + +After a new config is imported run `fakeroot debian/rules clean updateconfigs` +to update the dependencies, then you can review the actual changes doing a +simple `git diff`. + +## Ubuntu kernel annotations + +There is a main annotations file for each Ubuntu kernel in: +`debian./config/annotations` + +The main annotations file can include other annotations (recursively). + +The annotations has 4 different sections: + + - a header (that defines the format version, the list of supported + architectures, the list of supported flavours paired with each architecture + and a list of flavour inheritance rules) + + - a list of includes (annotations can include other annotations files to + create a hierarchy of configs with overrides) + + - a subset of mandatory config options that have a note associated (typically + config options that must be enforced for certain reasons, the note can be + simply a reference to a tracking bug, explaining why the specific config + needs to be set to a certain value, or just a description, like security + implications for that config value, etc.) + + - a subset of config options that are either generated as dependency of the + previous subset of configs or configs that have been set using the default + Ubuntu policy (enable everything as module when possible, except for debugging, + testing or deprecated features), so they don't need an explicit note. + +The annotations format looks like the following: + +``` +# Menu: HEADER +# FORMAT: 4 +# ARCH: ... +# FLAVOUR: - ... - +# FLAVOUR_DEP: {'-': '-', ..., '-': '-'} + +include +include +... +include + +CONFIG_ policy<{'[-]': '', ..., '[-]': ''}> +CONFIG_ note<''> +... + +# ---- Annotations without notes ---- + +CONFIG_ policy<{'[-]': '', ..., '[-]': ''}> +... + +``` + + - **ARCH**: contains the list of comma-separated values that defines all the + supported architectures for this kernel + + - **FLAVOUR**: contains the list of comma-separated values that defines the + list of flavours associated to each supported architecture + + - **FLAVOUR_DEP**: contains a mapping of `ARCH-FLAVOUR` pairs mapped to a + parent `ARCH-FLAVOUR` pair; this can be used to define flavours that derives + from other flavours, for example `amd64-lowlatency` derives from + `amd64-generic` (then the local annotations file will only contain an + include rule of the generic kernel plus the small subset of config options + to override the defaults from `amd64-generic`) + + - **FILE_1 .. FILE_N**: are the other included annotations (e.g., used by + derivative kernels that want to include the annotations from the kernel they + derive from) + + - **FOO, BAR**: kernel config options with the associated values across + architectures and flavours, represented by the `policy<...>` definition + +A simple example of an annotations file is the following (from the +`lunar/linux-lowlatency` kernel, that derives from the generic kernel +`lunar/linux`): + +``` +# Menu: HEADER +# FORMAT: 4 +# ARCH: amd64 arm64 +# FLAVOUR: amd64-lowlatency arm64-lowlatency arm64-lowlatency-64k +# FLAVOUR_DEP: {'amd64-lowlatency': 'amd64-generic', 'arm64-lowlatency': 'arm64-generic', 'arm64-lowlatency-64k': 'arm64-generic-64k'} + +include "../../debian.master/config/annotations" + +CONFIG_HZ_1000 policy<{'amd64': 'y', 'arm64': 'y'}> +CONFIG_HZ_1000 note<'HZ for lowlatency must be set to 1000 to provide better system responsiveness'> + +CONFIG_HZ_250 policy<{'amd64': 'n', 'arm64': 'n'}> +CONFIG_HZ_250 note<'Override default HZ used in generic'> + +CONFIG_LATENCYTOP policy<{'amd64': 'y', 'arm64': 'y'}> +CONFIG_LATENCYTOP note<'https://lists.ubuntu.com/archives/kernel-team/2014-July/045006.html, LP#1655986'> + +CONFIG_PREEMPT policy<{'amd64': 'y', 'arm64': 'y'}> +CONFIG_PREEMPT note<'Enable fully preemptible kernel'> + +CONFIG_PREEMPT_VOLUNTARY policy<{'amd64': 'n', 'arm64': 'n'}> +CONFIG_PREEMPT_VOLUNTARY note<'Disable voluntary preemption model'> + + +# ---- Annotations without notes ---- + +CONFIG_HZ policy<{'amd64': '1000', 'arm64': '1000'}> +``` + +## Managing the annotations file + +To help the management of annotations each Ubuntu kernel provides a helper +script in `debian/scripts/misc/annotations` (use `--help` for an overview of +the supported actions). + +Here some typical examples of what you can do with the annotations script: + + - Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across all the + supported architectures and flavours: + +``` +$ annotations --config CONFIG_DEBUG_INFO_BTF +{ + "policy": { + "amd64": "y", + "arm64": "y", + "armhf": "n", + "ppc64el": "y", + "riscv64": "y", + "s390x": "y" + }, + "note": "'Needs newer pahole for armhf'" +} +``` + + - Dump kernel `.config` for `arm64` and flavour `generic-64k`: + +``` +$ annotations --arch arm64 --flavour generic-64k --export +CONFIG_DEBUG_FS=y +CONFIG_DEBUG_KERNEL=y +CONFIG_COMPAT=y +... +``` + + - Update annotations file with a new kernel .config for `amd64` flavour + `generic`: + +``` +$ annotations --arch amd64 --flavour generic --import .config +``` + + - Enable `CONFIG_PROVE_LOCKING` on `amd64` for flavour `generic`: + +``` +$ annotations -c CONFIG_PROVE_LOCKING --arch amd64 --flavour generic --write y +{ + "CONFIG_PROVE_LOCKING": { + "policy": { + "amd64": "y", + "arm64": "n", + "armhf": "n", + "ppc64el": "n", + "riscv64": "n", + "s390x": "n" + }, + "oneline": false, + "note": "'prove locking enabled as a test'" + } +} + +$ git diff +diff --git a/debian.master/config/annotations b/debian.master/config/annotations +index 24cec55b1b20b..e331bbd6de640 100644 +--- a/debian.master/config/annotations ++++ b/debian.master/config/annotations +@@ -483,6 +483,9 @@ CONFIG_PPC_SECVAR_SYSFS note<'LP: #1866909'> + CONFIG_PREEMPT_NONE policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'y'}> + CONFIG_PREEMPT_NONE note<'LP: #1543165'> + ++CONFIG_PROVE_LOCKING policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> ++CONFIG_PROVE_LOCKING note<'prove locking enabled as a test'> ++ + CONFIG_PSI_DEFAULT_DISABLED policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'y'}> + CONFIG_PSI_DEFAULT_DISABLED note<'LP: #1876044'> + +@@ -9892,7 +9895,6 @@ CONFIG_PROFILE_ALL_BRANCHES policy<{'riscv64': 'n'}> + CONFIG_PROFILE_ANNOTATED_BRANCHES policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> + CONFIG_PROFILING policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> + CONFIG_PROTECTED_VIRTUALIZATION_GUEST policy<{'s390x': 'y'}> +-CONFIG_PROVE_LOCKING policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> + CONFIG_PROVIDE_OHCI1394_DMA_INIT policy<{'amd64': 'n'}> + CONFIG_PRU_REMOTEPROC policy<{'arm64': 'm', 'armhf': 'm'}> + CONFIG_PSAMPLE policy<{'amd64': 'm', 'arm64': 'm', 'armhf': 'm', 'ppc64el': 'm', 'riscv64': 'm', 's390x': 'm'}> +``` + +Every time the annotations file is changed we need to make sure that all the +dependent config options are refreshed. + +To do so we need to run the following command: + +``` +$ fakeroot debian/rules clean updateconfigs +... +``` + +This command will take care of interfacing with the kernel Kconfig subsystem +and it will interactively ask the value for all the potentially new enabled +dependent options. + +## Conclusion + +The annotations format introduced in Lunar provides an easier and more +efficient way of managing kernel .config's across the large variety of kernels, +architectures and flavours that Ubuntu supports. + +If you want to submit a patch to kernel-team@lists.ubuntu.com that needs to +change the kernel `.config`, or simply if you want to recompile the Ubuntu +kernel using a different `.config` you just need to modify the annotations file +and then you can simply upload your custom modified kernel to a ppa to test it, +or you can recompile it locally using the typical Ubuntu way of rebuilding +packages. + +## See also + + - https://lists.ubuntu.com/archives/kernel-team/2023-June/140230.html + - https://discourse.ubuntu.com/t/kernel-configuration-in-ubuntu/35857 diff --git a/README.rst b/README.rst deleted file mode 100644 index 220fac9..0000000 --- a/README.rst +++ /dev/null @@ -1,191 +0,0 @@ -================== -Config Annotations -================== - -:Author: Andrea Righi - -Overview -======== - -Each Ubuntu kernel needs to maintain its own .config for each supported -architecture and each flavour. - -Every time a new patch is applied or a kernel is rebased on top of a new -one, we need to update the .config's accordingly (config options can be -added, removed and also renamed). - -So, we need to make sure that some critical config options are always -matching the desired value in order to have a functional kernel. - -State of the art -================ - -At the moment configs are maintained as a set of Kconfig chunks (inside -`debian./config/`): a global one, plus per-arch / per-flavour -chunks. - -In addition to that, we need to maintain also a file called -'annotations'; the purpose of this file is to make sure that some -critical config options are not silently removed or changed when the -real .config is re-generated (for example after a rebase or after -applying a new set of patches). - -The main problem with this approach is that, often, we have duplicate -information that is stored both in the Kconfig chunks *and* in the -annotations files and, at the same time, the whole .config's information -is distributed between Kconfig chunks and annotations, making it hard to -maintain, review and manage in general. - -Proposed solution -================= - -The proposed solution is to store all the config information into the -"annotations" format and get rid of the config chunks (basically the -real .config's can be produced "compiling" annotations). - -Implementation -============== - -To help the management of the annotations an helper script is provided -(`debian/scripts/misc/annotations`): - -``` -usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG] - (--query | --export | --import FILE | --update FILE | --check FILE) - -Manage Ubuntu kernel .config and annotations - -options: - -h, --help show this help message and exit - --version, -v show program's version number and exit - --file FILE, -f FILE Pass annotations or .config file to be parsed - --arch ARCH, -a ARCH Select architecture - --flavour FLAVOUR, -l FLAVOUR - Select flavour (default is "generic") - --config CONFIG, -c CONFIG - Select a specific config option - -Action: - --query, -q Query annotations - --export, -e Convert annotations to .config format - --import FILE, -i FILE - Import a full .config for a specific arch and flavour into annotations - --update FILE, -u FILE - Import a partial .config into annotations (only resync configs specified in FILE) - --check FILE, -k FILE - Validate kernel .config with annotations -``` - -This script allows to query config settings (per arch/flavour/config), -export them into the Kconfig format (generating the real .config files) -and check if the final .config matches the rules defined in the -annotations. - -Examples (annotations is defined as an alias to `debian/scripts/annotations`): - - - Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across all the - supported architectures and flavours: - -``` -$ annotations --query --config CONFIG_DEBUG_INFO_BTF -{ - "policy": { - "amd64": "y", - "arm64": "y", - "armhf": "n", - "ppc64el": "y", - "riscv64": "y", - "s390x": "y" - }, - "note": "'Needs newer pahole for armhf'" -} -``` - - - Dump kernel .config for arm64 and flavour generic-64k: - -``` -$ annotations --arch arm64 --flavour generic-64k --export -CONFIG_DEBUG_FS=y -CONFIG_DEBUG_KERNEL=y -CONFIG_COMPAT=y -... -``` - - - Update annotations file with a new kernel .config for amd64 flavour - generic: - -``` -$ annotations --arch amd64 --flavour generic --import build/.config -``` - -Moreover, an additional kernelconfig commands are provided -(via debian/rules targets): - - `migrateconfigs`: automatically merge all the previous configs into - annotations (local changes still need to be committed) - -Annotations headers -=================== - -The main annotations file should contain a header to define the architectures -and flavours that are supported. - -Here is the format of the header for the generic kernel: -``` -# Menu: HEADER -# FORMAT: 4 -# ARCH: amd64 arm64 armhf ppc64el riscv64 s390x -# FLAVOUR: amd64-generic arm64-generic arm64-generic-64k armhf-generic armhf-generic-lpae ppc64el-generic riscv64-generic s390x-generic - -``` - -Example header of a derivative (linux-aws): -``` -# Menu: HEADER -# FORMAT: 4 -# ARCH: amd64 arm64 -# FLAVOUR: amd64-aws arm64-aws -# FLAVOUR_DEP: {'amd64-aws': 'amd64-generic', 'arm64-aws': 'arm64-generic'} - -include "../../debian.master/config/annotations" - -# Below you can define only the specific linux-aws configs that differ from linux generic - -``` - -Pros and Cons -============= - - Pros: - - avoid duplicate information in .config's and annotations - - allow to easily define groups of config settings (for a specific - environment or feature, such as annotations.clouds, annotations.ubuntu, - annotations.snapd, etc.) - - config options are more accessible, easy to change and review - - we can easily document how config options are managed (and external - contributors won't be discouraged anymore when they need to to change a - config option) - - Cons: - - potential regressions: the new tool/scripts can have potential bugs, - so we could experience regressions due to some missed config changes - - kernel team need to understand the new process (even if everything - is transparent, kernel cranking process is the same, there might be - corner cases that need to be addressed and resolved manually) - -TODO -==== - - - Migrate all flavour and arch definitions into annotations (rather - than having this information defined in multiple places inside - debian/scripts); right now this information is "partially" migrated, - meaning that we need to define arches and flavours in the headers - section of annotations (so that the annotations tool can figure out - the list of supported arches and flavours), but arches and flavours - are still defined elsewhere, ideally we would like to have arches and - flavours defined only in one place: annotations. - -See also -======== - - - https://lists.ubuntu.com/archives/kernel-team/2023-June/140230.html - - https://discourse.ubuntu.com/t/kernel-configuration-in-ubuntu/35857