Config management in larger applications can become quite complex: Information needs to be loaded from different sources like environment variables, config files and command line arguments, the loaded data needs to be validated to ensure all expected information is present and in the correct format and then distributed to different locations in the codebase. This holds true especially in data science projects, having rich model and training configurations. To simplify this process, we developed and open-sourced ConfZ, a config management library for Python based on pydantic. It easily allows to load config values from heterogeneous sources, validates them and makes them accessible as Python dataclass-like objects with full IDE support. It furthermore supports in common use cases like having multiple environments, lazy loading and unit testing. Within two months, we already reached more than 100 stars on GitHub. In this talk, we show how ConfZ is used and how it compares to other config management solutions. We then dig into the pythonic details and see how meta classes drive the internals of the library.
Config management in larger applications can become quite complex: Information needs to be loaded from different sources like environment variables, config files and command line arguments, the loaded data needs to be validated to ensure all expected information is present and in the correct format and then distributed to different locations in the codebase. This holds true especially in data science projects, having rich model and training configurations. To simplify this process, we developed and open-sourced ConfZ, a config management library for Python based on pydantic. It easily allows to load config values from heterogeneous sources, validates them and makes them accessible as Python dataclass-like objects with full IDE support. It furthermore supports in common use cases like having multiple environments, lazy loading and unit testing. Within two months, we already reached more than 100 stars on GitHub. In this talk, we show how ConfZ is used and how it compares to other config management solutions. We then dig into the pythonic details and see how meta classes drive the internals of the library.