Kconfiglib

Overview

Kconfiglib is a Kconfig implementation in Python. It started as a helper library but now has enough functionality to work as a standalone Kconfig implementation (including terminal and GUI menuconfig interfaces and Kconfig extensions).

The entire library is contained in kconfiglib.py. The bundled scripts are implemented on top of it, and creating your own scripts should be relatively straightforward if needed.

Kconfiglib is used extensively by projects such as the Zephyr Project and ESP-IDF. It is also employed for various helper scripts in other projects.

Because Kconfiglib is library-based, it can be used, for example, to generate a Kconfig cross-reference using the same robust Kconfig parser used by other Kconfig tools, instead of relying on brittle ad-hoc parsing.

Kconfiglib implements the recently added Kconfig preprocessor. For backward compatibility, environment variables can be referenced both as $(FOO) (the new syntax) and as $FOO (the old syntax). While the old syntax is deprecated, it will likely remain supported for some time in order to maintain compatibility with older Linux kernels. If support for it is ever dropped, the major version will be incremented accordingly. Note that using the old syntax with an undefined environment variable leaves the string unchanged.

See Kconfig: Tips and Best Practices for more information.

Installation

Installation with pip

A stable version of Kconfiglib is available on PyPI. To install the latest version directly from GitHub, run:

pip install git+https://github.com/sysprog21/Kconfiglib

Microsoft Windows is supported.

When installed via pip, you get both the core library and the following executables. All but two (genconfig and setconfig) mirror functionality available in the C tools.

• menuconfig
• guiconfig
• oldconfig
• olddefconfig
• savedefconfig
• defconfig
• alldefconfig
• allnoconfig
• allmodconfig
• allyesconfig
• listnewconfig
• genconfig
• setconfig

genconfig is intended to be run at build time. It generates a C header from the configuration and, optionally, additional data that can be used to rebuild only the files referencing Kconfig symbols whose values have changed.

Starting with Kconfiglib version 12.2.0, all utilities are compatible with both Python 2 and Python 3. Previously, menuconfig.py was limited to Python 3, so it now offers improved backward compatibility.

Note: If Kconfiglib is installed with the pip --user flag, ensure that the PATH variable includes the directory where the executables are installed. To list the installed files, use:

pip show -f kconfiglib

All releases have a corresponding tag in the Git repository (for example, v14.1.0 is the latest stable version).

Kconfiglib follows Semantic Versioning. Major version increments are made for any behavior change, regardless of scope. Most changes thus far have addressed small issues introduced in the early days of the Kconfiglib 2 API.

Manual installation

Just drop kconfiglib.py and the scripts you want somewhere. There are no third-party dependencies, but the terminal menuconfig will not work on Windows unless a package like windows-curses is installed.

Installation for the Linux kernel

See the module docstring at the top of kconfiglib.py.

Python version compatibility (2.7/3.2+)

Kconfiglib and all of its utilities run under Python 2.7 as well as Python 3.2 and later. The code primarily relies on basic Python features and does not depend on third-party libraries, making it relatively easy to maintain backward compatibility.

The Python 3.2 requirement stems from using the argparse module and the format() function with unnumbered braces ({}). If given a choice, using a more recent Python 3 release is recommended due to its improved Unicode handling.

Getting started

1. Install the library and the utilities.

2. Write Kconfig files that describe the available configuration options. For general Kconfig advice, see Tips and Best Practices.

3. Generate an initial configuration using menuconfig, guiconfig, or alldefconfig. The configuration is saved as .config by default.

   • For more advanced projects, use the defconfig utility to generate the initial configuration from an existing configuration file. Typically, this existing file is a minimal configuration generated by commands like savedefconfig.

4. Run genconfig to generate a header file. By default, this file is saved as config.h.

   • Normally, genconfig is run automatically as part of the build.
   • Before writing a header or any other configuration output, Kconfiglib compares the file’s old contents with the new contents. If they are identical, Kconfiglib skips writing. This avoids needlessly updating file metadata (like modification times) and can save build time.
   • Adding new configuration output formats is relatively straightforward. See the implementation of write_config() in kconfiglib.py. The documentation for the Symbol.config_string property also contains helpful tips.

5. Update an old .config file after changing the Kconfig files (e.g., adding new options) by running oldconfig (prompts for values for new options) or olddefconfig (assigns default values to new options).

   • Entering the menuconfig or guiconfig interface and saving will also update the configuration (the interfaces prompt for saving on exit if the .config file has changed).
   • Due to Kconfig semantics, loading an old .config file implicitly performs an olddefconfig, so building typically won’t be affected by an outdated configuration.

Whenever .config is overwritten, its previous contents are saved to .config.old (or, more generally, to $KCONFIG_CONFIG.old).

Using .config files as Make input

Because .config files use Make syntax, they can be included directly in Makefiles to read configuration values. For n-valued bool / tristate options, the line # CONFIG_FOO is not set (a Make comment) is generated in .config, allowing the option to be tested via ifdef in Make.

If you rely on this behavior, consider passing --config-out <filename> to genconfig and including the generated configuration file instead of .config directly. This ensures the included file is always a "full" configuration file, even if .config becomes outdated. Otherwise, you may need to run old(def)config, menuconfig, or guiconfig before rebuilding.

If you use the --sync-deps option to generate incremental build information, you can include deps/auto.conf instead, which is also a full configuration file.

Useful helper macros

The include/linux/kconfig.h header in the Linux kernel defines several helper macros for testing Kconfig configuration values. Among these, IS_ENABLED() is especially useful because it allows configuration values to be tested in if statements with no runtime overhead.

Incremental building

For guidance on implementing incremental builds (rebuilding only those source files that reference changed configuration values), refer to the docstring for Kconfig.sync_deps() in kconfiglib.py.

It may also be helpful to run the kernel’s scripts/basic/fixdep.c tool on the output of gcc -MD <source file>, to see how the build process fits together.

Library documentation

Kconfiglib includes extensive documentation in the form of docstrings. To view it, run, for example:

pydoc kconfiglib

For HTML output, add -w:

pydoc -w kconfiglib

This will work even after installing Kconfiglib with pip.

Documentation for other modules can be viewed the same way. For executables, a plain --help often suffices:

pydoc menuconfig/guiconfig/...

A good place to start is the module docstring, located at the beginning of kconfiglib.py. It provides an introduction to symbol values, the menu tree, and expressions.

After reviewing the module docstring, the next step is to read the documentation for the Kconfig class, followed by Symbol, Choice, and MenuNode.

Please report any issues if something is unclear or can be explained better.

Library features

Kconfiglib can do the following, among other things:

• Programmatically get and set symbol values

  See allnoconfig.py and allyesconfig.py, which are automatically verified to produce identical output to the standard make allnoconfig and make allyesconfig.

• Read and write .config and defconfig files

  The generated .config and defconfig (minimal configuration) files are character-for-character identical to what the C implementation would generate (except for the header comment). The test suite relies on this by comparing the generated files directly.

• Write C headers

  The generated headers use the same format as include/generated/autoconf.h from the Linux kernel. Symbol output appears in the order in which the symbols are defined, unlike in the C tools (where the order depends on the hash table implementation).

• Implement incremental builds

  This follows the same scheme used by the include/config directory in the kernel: symbols are translated into files that are updated when a symbol's value changes between builds, which can help avoid a full rebuild whenever the configuration changes.

  See the sync_deps() function for additional details.

• Inspect symbols

  Printing a symbol or other item (via __str__()) returns its definition in Kconfig format, and this also works for symbols that appear in multiple locations.

  A useful __repr__() is defined on all objects as well.

  All __str__() and __repr__() methods are deliberately implemented using public APIs, so all symbol information can also be retrieved separately.

• Inspect expressions

  Expressions use a simple tuple-based format that can be processed manually if needed. Kconfiglib includes expression-printing and evaluation functions, implemented with public APIs.

• Inspect the menu tree

  The underlying menu tree is exposed, including submenus created implicitly by symbols that depend on preceding symbols. This can be used, for example, to implement menuconfig-like functionality.

See menuconfig.py, guiconfig.py, and the minimalistic menuconfig_example.py example.

Kconfig extensions

The following Kconfig extensions are available:

• source supports glob patterns and includes each matching file. A pattern must match at least one file. A separate osource statement is available for situations where it is acceptable for the pattern to match no files (in which case osource becomes a no-op).

• A relative source statement (rsource) is available, where file paths are specified relative to the directory of the current Kconfig file. An orsource statement is also available, analogous to osource.

• Preprocessor user functions can be defined in Python, which makes it straightforward to integrate information from existing Python tools into Kconfig (for example, to have Kconfig symbols depend on hardware information stored in another format). See the "Kconfig extensions" section in the kconfiglib.py module docstring for more details.

• def_int, def_hex, and def_string are provided in addition to def_bool and def_tristate, allowing int, hex, and string symbols to have both a type and a default value at the same time. These can be useful for projects that define symbols in multiple locations and help address some Kconfig inconsistencies.

• Environment variables are expanded directly in statements such as source and mainmenu, making option env symbols redundant. This is the standard behavior of the new Kconfig preprocessor, which Kconfiglib implements.

  option env symbols are still accepted but ignored, provided they have the same name as the referenced environment variable (Kconfiglib will warn if they differ). This preserves compatibility with older Linux kernels, where the option env symbol's name always matched the environment variable. The main reason option env remains supported is to maintain backward compatibility. The C tools have dropped support for option env.

• Two extra optional warnings can be enabled by setting environment variables. These warnings cover cases that are easily overlooked when modifying Kconfig files:

• KCONFIG_WARN_UNDEF: If set to y, a warning is generated for any reference to an undefined symbol in a Kconfig file. The only caveat is that all hexadecimal literals must be prefixed with 0x or 0X so they can be distinguished from symbol references.

  Some projects (such as the Linux kernel) use multiple Kconfig trees with many shared Kconfig files, which can result in intentionally undefined symbol references. However, KCONFIG_WARN_UNDEF can be very useful in projects with a single Kconfig tree. KCONFIG_STRICT is an older alias for KCONFIG_WARN_UNDEF, retained for backward compatibility.

• KCONFIG_WARN_UNDEF_ASSIGN: If set to y, a warning is generated for any assignment to an undefined symbol in a .config file. By default, no such warnings are generated.

This warning can also be toggled by setting Kconfig.warn_assign_undef to True or False.

Other features

• Single-file implementation

  The entire library is contained in kconfiglib.py. The tools built on top of it are each contained in a single file.

• Robust and highly compatible with the C Kconfig tools

  The test suite automatically compares output from Kconfiglib and the C tools by diffing the generated .config files for the real kernel Kconfig and defconfig files across all ARCHes. Currently, this involves comparing output for 36 ARCHes and 498 defconfig files (or over 18,000 ARCH/defconfig combinations in "obsessive" test suite mode). All tests are expected to pass. A comprehensive suite of self-tests is included as well.

• Not horribly slow despite being a pure Python implementation

  The allyesconfig.py script runs in about 1.3 seconds on the Linux kernel using a Core i7 2600K with a warm file cache, including the make overhead from make scriptconfig. The Linux kernel Kconfigs are especially large (over 14k symbols for x86), and there is additional overhead from running shell commands via the Kconfig preprocessor.

  Kconfiglib is particularly efficient when multiple .config files must be processed, because the Kconfig files are parsed only once.

  For long-running jobs, PyPy provides a significant performance boost, although CPython is typically faster for short jobs since PyPy requires time to warm up.

  Kconfiglib also works well with the multiprocessing module, as it does not rely on global state.

• Generates more warnings than the C implementation

  Kconfiglib generates the same warnings as the C implementation, plus additional ones. It also detects dependency loops and source loops. All warnings indicate the relevant location(s) in the Kconfig files where a symbol is defined, if applicable.

• Unicode support

  Unicode characters in string literals within Kconfig and .config files are handled correctly, primarily thanks to Python’s built-in Unicode functionality.

• Windows support

  Nothing in Kconfiglib is specific to Linux. Universal newline mode is used for both Python 2 and Python 3. The Zephyr project uses Kconfiglib to generate .config files and C headers on both Linux and Windows.

• Internals that (mostly) mirror the C implementation

  While the internals are simpler to understand and modify, they closely track the logic of the C tools.

Menuconfig interfaces

Three configuration interfaces are currently available:

• menuconfig.py is a terminal-based configuration interface implemented using the standard Python curses module. It includes xconfig features such as showing invisible symbols and symbol names, and it allows jumping directly to a symbol in the menu tree (even if it is currently invisible).

  

  There is also a show-help mode that displays the help text of the currently selected symbol in the bottom help window.

  Starting with Kconfiglib 12.2.0, menuconfig.py runs under both Python 2 and Python 3 (it previously ran only under Python 3, so this was a backport). Running it under Python 3 provides better support for Unicode text entry (get_wch() is unavailable in the Python 2 curses module).

  There are no third-party dependencies on Unix-like systems. On Windows, the curses module is not included by default, but can be added by installing the windows-curses package:

  pip install windows-curses

  These wheels are built from this repository, which is based on Christoph Gohlke's Python Extension Packages for Windows.

  See the docstring at the top of menuconfig.py for more information about the terminal menuconfig implementation.

• guiconfig.py is a graphical configuration interface written in Tkinter. Like menuconfig.py, it supports showing all symbols (with invisible symbols in red) and allows jumping directly to symbols. Symbol values can also be changed directly in the jump-to dialog.

  When single-menu mode is enabled, only a single menu is displayed at a time, similar to the terminal menuconfig. In this mode, it distinguishes between symbols defined with config and those defined with menuconfig.

  guiconfig.py has been tested on X11, Windows, and macOS, and it works with both Python 2 and Python 3.

  Although Tkinter is part of the Python standard library, it is not always installed by default on Linux. The commands below install it on a few different systems:

  • Ubuntu/Debian:

    sudo apt install python-tk
    sudo apt install python3-tk

  • Fedora:

    dnf install python2-tkinter
    dnf install python3-tkinter

  • Arch:

    sudo pacman -S tk

  • Clear Linux:

    sudo swupd bundle-add python3-tcl

  • macOS:

    brew install python-tk

  Screenshot below, with show-all mode enabled and the jump-to dialog open:

  To avoid carrying around multiple GIF files, the image data is embedded in guiconfig.py. To use separate GIF files instead, set _USE_EMBEDDED_IMAGES to False in guiconfig.py. The image files are located in the screenshots branch.

  The included images might not be the most artistic. Touch-ups are welcome.

• pymenuconfig, created by RomaVis, is an older, portable Python 2/3 Tkinter menuconfig implementation.

  Screenshot below:

  While working on the terminal menuconfig implementation, a few APIs were added to Kconfiglib that turned out to be handy. pymenuconfig predates menuconfig.py and guiconfig.py and therefore did not benefit from these APIs.

Examples

Example scripts

The examples/ directory contains simple example scripts. Make sure to run them with the latest version of Kconfiglib, as they may rely on newly added features. Some examples include:

• eval_expr.py evaluates an expression in the context of a configuration.
• find_symbol.py searches expressions for references to a specific symbol and provides a "backtrace" of parents for each reference found.
• help_grep.py looks for a specified string in all help texts.
• print_tree.py prints a tree of all configuration items.
• print_config_tree.py functions similarly to print_tree.py but shows the tree as it would appear in menuconfig, including values. This is useful for visually diffing .config files and different versions of Kconfig files.
• list_undefined.py identifies references to symbols that are not defined by any architecture in the Linux kernel.
• merge_config.py combines multiple configuration fragments into a complete .config, similar to scripts/kconfig/merge_config.sh in the kernel.
• menuconfig_example.py demonstrates how to implement a configuration interface using notation similar to make menuconfig. This script is intentionally minimal to focus on the core concepts.

Real-world examples

• kconfig.py from the Zephyr project handles .config and header file generation, as well as configuration fragment merging.

• CMake and IDE integration from the ESP-IDF project, using a configuration server program.

  These examples use the older Kconfiglib 1 API, which was clunkier and less general (e.g., functions instead of properties, no direct access to the menu structure, and a more limited __str__() output):

• gen-manual-lists.py produced listings for an appendix in the Buildroot manual. (Those listings have since been removed.)

• gen_kconfig_doc.py from the esp-idf project generates documentation from Kconfig files.

• SConf builds an interactive configuration interface (similar to menuconfig) on top of Kconfiglib, for use with SCons.

• kconfig-diff.py by dubiousjim is a script that compares kernel configurations.

• In chapter 4 of Ulf Magnusson's master thesis, Kconfiglib was originally used to generate a "minimal" kernel for a given system. Some parts of that approach feel dated now, but that often happens with older work.

Sample make iscriptconfig session

The following log provides an overview of the functionality available in the API:

$ make iscriptconfig
A Kconfig instance 'kconf' for the architecture x86 has been created.
>>> kconf # Calls Kconfig.__repr__()
<configuration with 13711 symbols, main menu prompt "Linux/x86 4.14.0-rc7 Kernel Configuration", srctree ".", config symbol prefix "CONFIG_", warnings enabled, undef. symbol assignment warnings disabled>
>>> kconf.mainmenu_text # Expanded main menu text
'Linux/x86 4.14.0-rc7 Kernel Configuration'
>>> kconf.top_node # The implicit top-level menu
<menu node for menu, prompt "Linux/x86 4.14.0-rc7 Kernel Configuration" (visibility y), deps y, 'visible if' deps y, has child, Kconfig:5>
>>> kconf.top_node.list # First child menu node
<menu node for symbol SRCARCH, deps y, has next, Kconfig:7>
>>> print(kconf.top_node.list) # Calls MenuNode.__str__()
config SRCARCH
string
option env="SRCARCH"
default "x86"
>>> sym = kconf.top_node.list.next.item # Item contained in next menu node
>>> print(sym) # Calls Symbol.__str__()
config 64BIT
bool "64-bit kernel" if ARCH = "x86"
default ARCH != "i386"
help
Say yes to build a 64-bit kernel - formerly known as x86_64
Say no to build a 32-bit kernel - formerly known as i386
>>> sym # Calls Symbol.__repr__()
<symbol 64BIT, bool, "64-bit kernel", value y, visibility y, direct deps y, arch/x86/Kconfig:2>
>>> sym.assignable # Currently assignable values (0, 1, 2 = n, m, y)
(0, 2)
>>> sym.set_value(0) # Set it to n
True
>>> sym.tri_value # Check the new value
0
>>> sym = kconf.syms["X86_MPPARSE"] # Look up symbol by name
>>> print(sym)
config X86_MPPARSE
bool "Enable MPS table" if (ACPI || SFI) && X86_LOCAL_APIC
default y if X86_LOCAL_APIC
help
For old smp systems that do not have proper acpi support. Newer systems
(esp with 64bit cpus) with acpi support, MADT and DSDT will override it
>>> default = sym.defaults[0] # Fetch its first default
>>> sym = default[1] # Fetch the default's condition (just a Symbol here)
>>> print(sym)
config X86_LOCAL_APIC
bool
default y
select IRQ_DOMAIN_HIERARCHY
select PCI_MSI_IRQ_DOMAIN if PCI_MSI
depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI
>>> sym.nodes # Show the MenuNode(s) associated with it
[<menu node for symbol X86_LOCAL_APIC, deps n, has next, arch/x86/Kconfig:1015>]
>>> kconfiglib.expr_str(sym.defaults[0][1]) # Print the default's condition
'X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI'
>>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it (0 = n)
0
>>> kconf.syms["64BIT"].set_value(2)
True
>>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it again (2 = y)
2
>>> kconf.write_config("myconfig") # Save a .config
>>> ^D
$ cat myconfig
# Generated by Kconfiglib (https://github.com/zephyrproject-rtos/Kconfiglib)
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
...

Test suite

The test suite is run with:

python Kconfiglib/testsuite.py

pypy also works and is much faster for most tasks, except for allnoconfig.py, allnoconfig_simpler.py, and allyesconfig.py, where it has no time to warm up because those scripts are invoked via make scriptconfig.

The test suite must be run from the top-level kernel directory. It requires that the Kconfiglib Git repository has been cloned into it and that the Makefile patch has been applied.

To suppress warnings generated for the kernel Kconfig files, redirect stderr to /dev/null:

python Kconfiglib/testsuite.py 2>/dev/null

Note: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail.

Note: The test suite overwrites .config in the kernel root, so make sure to back it up.

The test suite consists of a set of self-tests and a set of compatibility tests that compare configurations generated by Kconfiglib with those generated by the C tools across various scenarios. See testsuite.py for available options.

The tests/reltest script runs the test suite and all example scripts under both Python 2 and Python 3 to verify that everything works as expected.

Occasionally, the C tools' output may change slightly (for example, due to a recent change). If the test suite reports failures, try running it again against the linux-next tree, which contains the latest updates. Any non-backward-compatible changes will be clearly stated.

A significant amount of time can be spent waiting for make and the C utilities to re-parse all Kconfig files for each defconfig test. Adding multiprocessing to the test suite could help reduce this overhead.

Notes

• This is version 2 of Kconfiglib, which is not backward-compatible with Kconfiglib 1. A summary of changes between Kconfiglib 1 and Kconfiglib 2 can be found here.

• To add custom output formats, it is fairly straightforward to do (see the implementations of write_autoconf() and write_config(), as well as the documentation for the Symbol.config_string property). If a user develops something that could be useful to others, the maintainers are happy to include it upstream—batteries included and all that. To contribute, please open an issue or submit a pull request.

• Kconfiglib assumes the modules symbol is MODULES, which is backward-compatible. A warning is printed by default if option modules is set on a different symbol. Report issues on GitHub if proper option modules support is needed. It would not be hard to add.

Thanks

• To RomaVis for creating pymenuconfig and suggesting the rsource keyword.
• To Mitja Horvat for adding support for user-defined styles to the terminal menuconfig.
• To Philip Craig for adding support for the allnoconfig_y option and fixing an obscure issue involving comments within choices. Although it did not affect correctness, it caused outputs to differ. The allnoconfig_y option is used to force certain symbols to y during make allnoconfig to improve coverage.

License

See LICENSE. SPDX license identifiers are used throughout the source code.