Peripherals and Device Profiles
How to ensure physically-attached peripherals (via, e.g., USB or serial interface) can be made available to containers launched on the device.
Peripherals are an essential part of edge-computing. Facilitating the ability to add sensors and actuators that act on a device's environment was and remains, one of the core ideas behind the CHI@Edge research testbed. In contrast to our previous peripheral support approach where we maintained a limited subset of peripherals on a best-effort basis; this updated documentation aims to provide a hollistic approach for contributing peripheral support on CHI@Edge.
To integrate peripheral devices into the CHI@Edge platform, follow these three essential steps:
Setting the Right Kernel Boot Options and Device Tree Overlays
Exposing the Right Devices Under
/dev
Using Whitelisted Device ProfilesInstalling the Right Software in the User Container to drive the peripheral
Setting the Right Kernel Boot Options and Device Tree Overlays
Kernel boot options and device tree overlays are critical for enabling and configuring hardware peripherals at the device level. These settings ensure that the hardware is recognized and properly managed by the edge device OS.
Example: Enabling a Pi Camera Module 3 on Raspberry Pi 4
The following are just some of the required boot options for enabling a Pi Camera Module 3 on a Raspberry Pi 4.
start_x=1 # Directive to enable legacy camera stack
camera_auto_detect=1 # Directive to detect Pi camera modules on a raspberry pi
dtoverlay=imx708 # Sensor driver for the Pi Camera module 3
Unfortunately, due to security reasons, we can not expose our device kernel boot options to direct modification from our users (We aim to support a framework for manually adding boot options to self-hosted user devices in the future). For now, to add boot options to our own hosted CHI@Edge devices, please open a ticket through our helpdesk with the following information:
Targeted type of peripheral support
The concerned boot option.
Documentation to support the purpose of the boot option and its addition
Exposing the Right Devices Under /dev
Using Whitelisted Device Profiles
/dev
Using Whitelisted Device Profiles By default, the testbed will auto-discover most devices available under /dev
. Currently this works against a list of known/allowed device names, which differs by platform:
Raspberry Pi
^snd$
^gpiomem$
^gpiochip[0-9]*$
^hci[0-9]*$
^i2c-[0-9]*$
^rtc0$
^video[0-9]*$
^vchiq$
^vcsm.*$
^ttyUSB[0-9]*$
^ttyACM[0-9]*$
^ttyTHS[0-9]*$
^ttyS[0-9]*$
NVIDIA Jetson
^snd$
^gpiomem$
^gpiochip[0-9]*$
^hci[0-9]*$
^i2c-[0-9]*$
^rtc0$
^video[0-9]*$
^vchiq$
^vcsm.*$
^ttyUSB[0-9]*$
^ttyACM[0-9]*$
^ttyTHS[0-9]*$
^ttyS[0-9]*$
nvidia-gpu
When launching a container on the testbed, end-users can specify a "device profile", which maps to some set of detected and available devices. The containers will then have permission to access these devices.
For security reasons, we currently whitelist the following device profiles
, which map a subset of the above devices to a key that can be specified during container launch.
Currently defined device profiles
jetson_camera (currently unsupported)
/dev/video0
pi_camera
/dev/dma_heap
: required for contiguous memory allocation used for capture buffers/dev/media0:4
/dev/v4l-subdev0
and/dev/v4l-subdev1
, both of which are only available after specifying theimx-708
Raspberry Pi device tree overlay in boot configs (imx-708
refers to the driver specific to the pi camera module 3)vchiq
vcsm-cma
video0, video1, video10:16, video18:23, and video31
pi_gpio
/dev/gpiomem
/dev/i2c-1
/dev/gpiochip0
/dev/gpiochip1
pi_serial
/dev/ttyACM0
pi_meter
/dev/ttyUSB0
Adding a device profile to CHI@Edge
Once again, due to security reasons, we refrain from providing free access to mounting devices under /dev on user containers. To package a subset of /dev devices into a device profile, contact us through the helpdesk with the following required information to support your device profile addition request:
Targeted type of peripheral support
/dev devices required
Documentation to support the various required /dev devices requested
Installing the Right Software in the User Container to drive the peripheral
Often times, peripherals require additional software to function as intended, in this section, we showcase our recent development effort to enable Pi Camera Module 3 support on our raspberry pi 4 devices as an example of a peripheral support container image
Example: Dockerfile for the Pi Camera Module 3 peripheral support image
FROM debian:bullseye
# Install necessary dependencies and tools
RUN apt-get update && \
apt-get install -y \
dkms \
build-essential \
git \
v4l-utils \
python3-dev \
python3-pip \
python3-yaml \
python3-ply \
cmake \
libboost-dev \
libgnutls28-dev openssl libtiff5-dev libjpeg-dev libpng-dev pybind11-dev \
qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5 \
libboost-program-options-dev libdrm-dev libexif-dev \
&& apt-get clean
# Install the required Python modules using pip
RUN pip3 install meson ninja jinja2
#ply pyyaml
# Clone and build libcamera
RUN git clone https://github.com/raspberrypi/libcamera.git /libcamera && \
cd /libcamera && \
meson setup build --buildtype=release -Dpipelines=rpi/vc4,rpi/pisp -Dipas=rpi/vc4,rpi/pisp -Dv4l2=true -Dgstreamer=disabled -Dtest=false -Dlc-compliance=disabled -Dcam=disabled -Dqcam=disabled -Ddocumentation=disabled -Dpycamera=enabled && \
ninja -C build && \
ninja -C build install
# Clone and build rpicam-apps
RUN git clone https://github.com/raspberrypi/rpicam-apps.git /rpicam-apps && \
cd /rpicam-apps && \
sed -i 's/platform = options_->GetPlatform();/platform = Platform::VC4;/' core/rpicam_app.cpp && \
meson setup build -Denable_libav=disabled -Denable_drm=enabled -Denable_egl=disabled -Denable_qt=disabled -Denable_opencv=disabled -Denable_tflite=disabled && \
meson compile -C build && \
meson install -C build
RUN usermod -a -G video root
# Ensure the container has access to video devices
ENV UDEV=on
ENV LD_LIBRARY_PATH=/usr/local/lib/aarch64-linux-gnu:/libcamera/build/src/libcamera:${LD_LIBRARY_PATH}
CMD ["sleep", "infinity"]
The above Dockerfile specifies an image that packages 2 essential dependencies for the Pi Camera Module3: libcamera
and the rpicam-apps
. Both of which we compile manually.
Example: Launching a container on CHI@Edge with access to the Pi Camera Module 3
In summary, to launch a peripheral support container image
with fully featured support for the Pi Camera Module 3. Ensure the following pre-requisites are satisfied
The device to which the camera is connected is supplied with the appropriate boot options. In this case:
start_x=1 # Directive to enable legacy camera stack camera_auto_detect=1 # Directive to detect Pi camera modules on a raspberry pi dtoverlay=imx708 # Sensor driver for the Pi Camera module 3
The
pi_camera
device profile is included in thecreate_container()
call as followsmy_container = container.create_container( container_name, image="ghcr.io/chameleoncloud/edge-picamera-image:latest", workdir="/home", reservation_id=lease.get_device_reservation(lease_id), platform_version=2, )
The appropriate
peripheral support container image
is used when launching the container. In this case, we supplyghcr.io/chameleoncloud/edge-picamera-image:latest
as shown in the abovecreate_container()
call code snippet.The right devices appear under
/dev
once the container is running# ls /dev dma_heap media3 random tty video0 video14 video21 fd media4 shm urandom video1 video15 video22 full mqueue stderr v4l-subdev0 video10 video16 video23 media0 null stdin v4l-subdev1 video11 video18 video31 media1 ptmx stdout vchiq video12 video19 zero media2 pts termination-log vcsm-cma video13 video20
We then use the
rpicam-apps
utility tools for capturing pictures and videos with the Pi Camera Module 3# rpicam-still -o image.png --width 1920 --height 1080 [1:29:40.425770158] [42] INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d [1:29:40.556087776] [43] WARN RPiSdn sdn.cpp:40 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise [1:29:40.559500703] [43] INFO RPI vc4.cpp:446 Registered camera platform/soc/fe205000.i2c/i2c-22/i2c-10/10-001a imx708 to Unicam device /dev/media2 a nd ISP device /dev/media3 [1:29:40.559686831] [43] INFO RPI pipeline_base.cpp:1102 Using configuration file '/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml' Preview window unavailable Mode selection for 2304:1296:12:P SRGGB10_CSI2P,1536x864/0 - Score: 3400 SRGGB10_CSI2P,2304x1296/0 - Score: 1000 SRGGB10_CSI2P,4608x2592/0 - Score: 1900 Stream configuration adjusted [1:29:40.666479417] [42] INFO Camera camera.cpp:1183 configuring streams: (0) 2304x1296-YUV420 (1) 2304x1296-SBGGR10_CSI2P [1:29:40.667590017] [43] INFO RPI vc4.cpp:621 Sensor: platform/soc/fe205000.i2c/i2c-22/i2c-10/10-001a imx708 - Selected sensor format: 2304x1296-SBGG R10_1X10 - Selected unicam format: 2304x1296-pBAA ## Frame capturing output... Mode selection for 1920:1080:12:P SRGGB10_CSI2P,1536x864/0 - Score: 2200 SRGGB10_CSI2P,2304x1296/0 - Score: 1150 SRGGB10_CSI2P,4608x2592/0 - Score: 2050 [1:30:54.605246722] [49] INFO Camera camera.cpp:1183 configuring streams: (0) 1920x1080-YUV420 (1) 2304x1296-SBGGR10_CSI2P [1:30:54.618294956] [50] INFO RPI vc4.cpp:621 Sensor: platform/soc/fe205000.i2c/i2c-22/i2c-10/10-001a imx708 - Selected sensor format: 2304x1296-SBGG R10_1X10 - Selected unicam format: 2304x1296-pBAA Still capture image received
Contributing Peripheral support to CHI@Edge:
With the recent effort to revive peripheral support on CHI@Edge, we aim to brew an environment where our community members can contribute on all the different layers involved in enabling peripherals on the platform.
If you developped support for a peripheral and would like to share it with the community, please refer to the following resources:
Chameleon Helpdesk for submitting special device configuration requests (boot options and device profiles)
How to use Trovi, our reproducible artifact sharing tool to share peripheral support container images and tutorials
Chameleon PowerUsers CHI@Edge slack channel, please first request access to the power users slack and motivate your request. (Note: this slack channel is not a support channel but rather a collaborative environment for our power users to contribute to Chameleon)
Last updated