Plugins¶
edi comes with a few reusable plugins:
LXC/LXD Templates¶
During the root file system assembly edi adds templates to the container image (see LXD Documentation).
The following templates are already predefined:
LXC/LXD Profiles¶
With the help of profiles a container configuration can be fine tuned in a modular way (see LXD Profile Documentation).
The following profiles have proven to be useful for various projects:
Default Network Interface¶
This profile adds a default network interface to the container named according to the value of
edi_lxc_network_interface_name
. The interface is of type bridged
and its default parent is
lxdbr0
. To properly deal with legacy or emulated containers please use edibr0
instead of
lxdbr0
by setting the general parameter edi_lxc_bridge_interface_name
:
general:
...
edi_lxc_bridge_interface_name: edibr0
...
...
lxc_profiles:
...
100_lxc_networking:
path: lxc_profiles/general/lxc_networking/default_interface.yml
...
On the bridges matching edibr*
checksum offloading gets disabled (e.g. ethtool -K edibr0 tx off
).
This is to make sure that emulated and legacy containers can get an IPv4 address assigned.
Default Root Device¶
This profile makes sure that the container uses the default
storage pool as its
root device. Please note that newer LXD versions (>=2.9) require the configuration of a storage pool.
lxc_profiles:
...
200_default_root_device:
path: lxc_profiles/general/default_root_device/default_root_device.yml
...
Privileged Mode¶
This profile will make sure that the container is running in privileged mode.
lxc_profiles:
...
300_privileged:
path: lxc_profiles/general/security/privileged.yml
...
Please note that if a container has one or more shared folders configured it will automatically be turned into privileged mode.
Suppress Init¶
This profile will make sure that the container does not start using systemd but instead uses dumb-init. This is especially useful during the build of a distributable image. During such a build you just want to assemble the image without starting any services.
The following configuration snippet will conditionally enable the usage of dumb-init:
lxc_profiles:
...
400_suppress_init:
path: lxc_profiles/general/suppress_init/suppress_init.yml
skip: {{ not edi_create_distributable_image }}
...
dumb-init is not part of the default package set during bootstrapping. For this reason you have to add it within the bootstrap section (otherwise the launching of the container will fail):
bootstrap:
...
additional_packages: ["python", "sudo", "netbase", "net-tools", "iputils-ping", "ifupdown", "isc-dhcp-client", "resolvconf", "systemd", "systemd-sysv", "gnupg", "dumb-init"]
...
GUI Passthrough¶
Sometimes it is very useful to run an application with a graphical user interface (GUI) within a container and show it on the display of the host system. To achieve this setup a predefined LXC profile can be added to the configuration:
lxc_profiles:
...
500_gui_passthrough:
path: lxc_profiles/general/gui/passthrough.yml
skip: {{ edi_create_distributable_image }}
...
The passthrough template is a bit more complicated and looks like this:
name: gui_passthrough
config: {}
{% if edi_current_display is defined and edi_current_display %}
{% if edi_lxd_version is defined and edi_lxd_version.split('.')[0] | int >= 4 %}
description: edi graphical user interface passthrough
devices:
host-display:
bind: container
connect: unix:@/tmp/.X11-unix/X{{ edi_current_display }}
listen: unix:@/tmp/.X11-unix/X0
security.gid: {{ edi_current_user_gid }}
security.uid: {{ edi_current_user_uid }}
type: proxy
host-gpu:
type: gpu
{% else %}
description: edi graphical user interface passthrough - not supported by LXD version
devices: {}
{% endif %}
{% else %}
description: edi graphical user interface passthrough - no display
devices: {}
{% endif %}
edi
will automatically try to retrieve the current display setup from the DISPLAY
environment variable and pass it to the template as edi_current_display
. Please note that this
variable might change if multiple users are logged into the same workstation. In such scenarios you
can adjust the setup easily by re-applying the command edi lxc configure CONTAINERNAME CONFIG.yml
.
Furthermore this feature is only available for installations with LXD versions greater or equal than 4.0.
Please also note that this feature is only available for containers that run in privileged mode.
Once this profile has been successfully applied to the container, a GUI application can be launched as follows:
ssh IP_OF_CONTAINER
export DISPLAY=:0
someguiapp
To add even more convenience, the
development user facilities playbook can be configured to
automatically add the export DISPLAY=:0
statement to the ~/.profile
file of the container
user using the export_display
parameter.
Ansible Playbooks¶
edi ships with a few Ansible playbooks that can be re-used in many projects. This playbooks can also serve as an example if you want to write a custom playbook for your own project.
Please take a look at the comprehensive documentation of Ansible if you want to write your own playbook.
Here is a description of the built-in playbooks including the parameters that can be used to fine tune them:
Base System¶
The base system playbook tackles the following tasks:
Setup the lxc container network interface (optional).
Inherit the proxy settings from the host computer (optional).
Perform a basic apt setup.
Add a default user (optional).
Install an openssh server (optional).
The following code snippet adds the base system playbook to your configuration:
playbooks:
...
100_base_system:
parameters:
create_default_user: true
install_openssh_server: true
path: playbooks/debian/base_system/main.yml
...
The playbook can be fine tuned as follows:
Parameters
- apply_proxy_settings:
With this boolean value you can specify if the target system shall get a proxy setup. The default value is
True
and the standard behavior is that the target system will inherit the proxy settings of the host system. However, the proxy settings can be customized according to the table below. If you specifyFalse
the target system proxy setup will remain untouched.- configure_lxc_network_interface:
By default (boolean value
True
) the playbook will add a lxc network interface to the container. If this behavior is not desired, change the setting toFalse
.- lxc_network_interface_manager:
By default edi uses
ifupdown
to manage the default lxc network interface. Change this value tonetwork-manager
if you prefer to setup the default lxc network interface using NetworkManager. Please note that NetworkManager is currently not recommended for digital twin development containers as there are a few glitches that need to be ironed out. Furthermore edi will bring up the lxc network interface using low level commands in case systemd is overruled by dumb-init.- create_default_user:
By default (boolean value
False
) no additional user gets created. If you need an additional user switch this value toTrue
and fine tune the default user according to the table below.- install_openssh_server:
By default (boolean value
False
), no ssh server will be installed on the target system. Switch this value toTrue
if you would like to access the system using ssh.- disable_ssh_password_authentication:
By default password authentication is disabled for ssh (boolean value
True
). If you want to allow password based authentication then switch this value toFalse
but make sure to use a non standard password.- authorize_current_user:
By default (boolean value
True
) the current host user will be authorized to ssh into the account of the default user. Switch this value toFalse
if the current user shall not be authorized.- ssh_pub_key_directory:
All the public keys (ending with .pub) contained in the folder
ssh_pub_key_directory
(defaults to{{ edi_project_directory }}/ssh_pub_keys
) will be added to the list of authorized ssh keys of the default user.- install_documentation:
By default (value
full
) the documentation of every Debian package will get installed. Switch this value tominimal
if you want to deploy an image with a minimal footprint. Switch this value tochangelog
if you want to minimize the footprint but keep the changelog of all packages.- translations_filter:
By default all translations contained in Debian packages will get installed (empty filter:
""
). To reduce the footprint of the resulting artifacts the number of installed languages can be limited. By choosing the builtin filter"en_translations_only"
you can make sure that only English translations will get installed.- base_system_sources_list_template:
During bootstrapping a minimal
/etc/apt/sources.list
file gets added to the root file system. By specifying a template usingbase_system_sources_list_template
the initial sources list file will get removed and replaced by a file containing the content of the rendered template. The new file will be written to the/etc/apt/sources.list.d
subfolder.edi
provides the templatedebian.list
that can be chosen as a good starting point. In casedebian.list
does not meet the requirements a custom template can be specified. By default no template gets applied.
The proxy settings can be customized as follows:
Parameters
- target_http_proxy:
The http proxy that gets applied to the target system (defaults to
{{ edi_host_http_proxy }}
).- target_https_proxy:
The https proxy that gets applied to the target system (defaults to
{{ edi_host_https_proxy }}
).- target_ftp_proxy:
The ftp proxy that gets applied to the target system (defaults to
{{ edi_host_ftp_proxy }}
).- target_socks_proxy:
The socks proxy that gets applied to the target system (defaults to
{{ edi_host_socks_proxy }}
).- target_no_proxy:
The proxy exception list that gets applied to the target system (defaults to
{{ edi_host_no_proxy }}
).
The default user can be fine tuned as follows:
Parameters
- default_user_group_name:
The group name of the default user (default is
edi
).- default_user_gid:
The group id of the default user (default is
2000
).- default_user_name:
The user name of the default user (default is
edi
).- default_user_uid:
The user id of the default user (default is
2000
).- default_user_shell:
The shell of the default user (default is
/bin/bash
).- default_user_groups:
The groups of the default user (default is
adm,sudo
).- default_user_password:
The initially set password of the default user (default is
ChangeMe!
). You can adjust this password if needed. Set this password to"*"
if you would like to disable password based login. Please note that the playbook will then automatically create a sudoers file to not impair thesudo
command.
Base System Cleanup¶
The base system cleanup playbook makes sure that we get a clean distributable image by doing the following tasks:
It removes the openssh server keys (they shall be unique per system).
It removes cached apt data to reduce the artifact footprint.
It finalizes the proxy setup.
It sets the final hostname.
The following code snippet adds the base system cleanup playbook to your configuration:
playbooks:
...
900_base_system_cleanup:
path: playbooks/debian/base_system_cleanup/main.yml
parameters:
hostname: raspberry
...
The playbook can be fine tuned as follows:
Parameters
- hostname:
Set the hostname within the final artifact (default is
edi
).- regenerate_openssh_server_keys:
By default the playbook will make sure that the openssh server keys get regenerated (boolean value
True
). Switch this value toFalse
if you would like to keep the same openssh server keys for all instances that will receive this artifact.- ssh_host_key_backup_folder:
Optionally the ssh host keys can be restored from a folder on the first boot. The restore script will not overwrite existing valid keys in
/etc/ssh/
. By default this feature is turned off (ssh_host_key_backup_folder is""
). This feature is especially useful if ssh host keys shall be preserved during a complete OS update.- cleanup_proxy_settings:
By default the proxy settings of the resulting artifact will get cleaned up (boolean value
True
). If you would like to keep the same proxy settings switch this value toFalse
. When set toTrue
, the proxy settings can be fine tuned according to the table below.- document_build_setup:
To document the build setup of the artifact within the artifact set this value to
True
. As a result the file/usr/share/doc/edi/build.yml
will be generated. By default this feature is switched off (boolean valueFalse
).- document_installed_packages:
To document the packages of the artifact within the artifact set this value to
True
. As a result the file/usr/share/doc/edi/packages.yml
will be generated. The generated file will contain a list of all packages including version information. It is a snapshot of the available packages after the artifact build and will not get updated when new packages get installed usingdpkg
orapt
. By default this feature is switched off (boolean valueFalse
).- package_baseline_source_file:
In order to generate a differential changelog it is possible to add a package baseline file to the resulting artifact. The package baseline file has the same format as
/usr/share/doc/edi/packages.yml
. If a differential changelog between release n and n+1 is needed, you can copy the file/usr/share/doc/edi/packages.yml
from release n to{{ edi_project_directory }}/configuration/documentation/packages-baseline.yml
(default value for package_baseline_source_file). The playbook will then make sure that it gets added to artifact n as/usr/share/doc/edi/packages-baseline.yml
. The commandedi documentation render ...
will use this information to restrict the changelog to changes that happened between release n and n+1.
The final proxy settings can be customized as follows:
Parameters
- target_http_proxy:
The final http proxy settings (defaults to
""
).- target_https_proxy:
The final https proxy settings (defaults to
""
).- target_ftp_proxy:
The final ftp proxy settings (defaults to
""
).- target_socks_proxy:
The final socks proxy settings (defaults to
""
).- target_no_proxy:
The final proxy exception list (defaults to
""
).
Development User Facilities¶
The development user facilities playbook adds the host user (the user that runs edi
) to the target system.
In case the target system is an LXD container and shared folders are defined, the playbook will
make sure that the specified folders are shared between the host system and the LXD container.
The host user will automatically be authorized to ssh into the target system.
The password for the user (same user name as the host user) in the target system will be ChangeMe!
.
Please note that this playbook will get skipped entirely when a distributable image gets created
(when edi_create_distributable_image
is True
).
The following code snippet adds the development user facilities playbook to your configuration:
playbooks:
...
200_development_user_facilities:
path: playbooks/debian/development_user_facilities/main.yml
parameters:
export_display: True
...
The playbook can be fine tuned as follows:
Parameters
- export_display:
If
True
, add the statementexport DISPLAY=:0
to~/.profile
(default isFalse
).
Postprocessing Commands¶
Postprocessing commands can be used to gradually transform an exported LXD container into the desired artifacts (e.g. an image that can get flashed to an SD card).
A typical post processing command can be configured as follows:
postprocessing_commands:
...
100_lxd2rootfs:
path: postprocessing_commands/rootfs/lxd2rootfs.edi
require_root: True
output:
pi3_rootfs: {{ edi_configuration_name }}_rootfs
...
edi
will render the file postprocessing_commands/rootfs/lxd2rootfs.edi
using the Jinja2 template
engine and then execute it. It is a good practice to use this file as a thin shim between edi
and the scripts
that do the heavy lifting.
The statement require_root: True
tells edi that a privileged user (sudo) is needed to execute the command.
Each post processing command shall create at least one (intermediate) artifact that gets specified within the
output
node. The resulting artifact can be used as an input for the next post processing command.
The specified output can be either a single file or a folder (if multiple files get generated by the command).
The variable edi_input_artifact
can be used to locate the artifact that got generated before the post
processing commands get called. It contains typically the artifact created by the edi lxc export
command.
The post processing commands are implemented in a very generic way and to get an idea of what they can do please take a look at the the edi-pi configuration.
Documentation Steps¶
edi ships with a few Jinja2 templates that can be re-used in many projects. This templates can also serve as an example if you want to write custom templates for your own project.
To develop custom templates and learn more about the Jinja2 rendering context the documentation command can be executed in debug mode:
edi --log=DEBUG documentation render PATH_TO_USR_SHARE_DOC_FOLDER OUTPUT_FOLDER CONFIG.yml
The output of the provided templates is reStructuredText that can be further tweaked and then be transformed into a nice pdf document using Sphinx. For more details please take a look at the edi-pi example configuration.
Please note that you can generate other output formats such as markdown by providing custom templates.
The templates get applied chunk by chunk. The booleans edi_doc_first_chunk
and
edi_doc_last_chunk
can be used within the templates to add a header or a footer where needed.
Index¶
The index template can be used to generate an index file:
documentation_steps:
...
100_index:
path: documentation_steps/rst/templates/index.rst.j2
output:
file: index.rst
parameters:
edi_doc_include_packages: []
toctree_items: ['setup', 'versions', 'changelog']
...
Setup¶
The setup template can be used to document the build setup:
documentation_steps:
...
200_setup:
path: documentation_steps/rst/templates/setup.rst.j2
output:
file: setup.rst
parameters:
edi_doc_include_packages: []
...
Versions¶
The versions template can be used to document the package versions:
documentation_steps:
...
300_versions:
output:
file: versions.rst
path: documentation_steps/rst/templates/versions.rst.j2
...
Changelog¶
The changelog template can be used to document the changes of each package:
documentation_steps:
...
400_changelog:
path: documentation_steps/rst/templates/changelog.rst.j2
output:
file: changelog.rst
parameters:
edi_doc_include_changelog: True
edi_doc_changelog_baseline: 2019-12-01 00:00:00 GMT
edi_doc_replacements:
- pattern: '(CVE-[0-9]{4}-[0-9]{4,6})'
replacement: '`\1 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=\1>`_'
- pattern: '(?i)[#]*(Closes:\s[#])([0-9]{6,10})'
replacement: '`\1\2 <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=\2>`_'
- pattern: '(?i)[#]*(LP:\s[#])([0-9]{6,10})'
replacement: '`\1\2 <https://bugs.launchpad.net/ubuntu/+source/nano/+bug/\2>`_'
...