DAIS Tutorials for Cosylab’s Support of FPGA-based AI
The DAIS Technology Partners for the AGV Use-case
Structured around eight supply chains, DAIS focuses on:
- Five supply chains dedicated to creating the necessary hardware and software for industrial-grade AI across various network topologies.
- Three supply chains demonstrating solutions to common AI challenges in different functional areas.
By embedding European values such as self-organisation, privacy by design, and low energy consumption into electronic components and systems, DAIS seeks to position Europe at the forefront of the global market in this critical domain.
One of the six DAIS demonstrators involved an innovative use case that integrated sensors and cameras on Automated Guided Vehicles (AGVs) to enable autonomous driving.
In the “AI-Driven AGV for Obstacle Detection and Avoidance” use case, Cosylab, TPV, and the Jožef Stefan Institute collaborated to develop a sophisticated solution. By combining LiDAR (Light Detection and Ranging) with vision data, the team created an environmental representation that enabled the AGV to detect obstacles, deviate from its magnetic tape-guided path, navigate around the obstacle, and return to the original path efficiently.
Cosylab’s responsibilities in this project included:
- Providing the hardware for the computing platform.
- Developing middleware, including custom firmware and an operating system for a System on a Chip (SoC) development board.
- Integrating a camera and its interface into the system.
Tutorials for Cosylab’s DAIS Hardware Platform Implementation
The primary hardware platform for the AGV control system was the Xilinx ZCU104 development board, built on the Xilinx UltraScale+ MPSoC (MultiProcessor System on a Chip). This versatile platform combines FPGA circuitry with a general-purpose CPU featuring four ARM Cortex-A cores. Cosylab leveraged this hardware to run an operating system and custom firmware designed specifically for the project.
The custom firmware provided fundamental input/output (I/O) connectivity on the development board. It also utilized the FPGA’s machine learning processors—Deep Learning Processors (DPUs)—to offload neural network processing tasks from the main CPU to the DPUs, which ran custom-compiled neural network models.
To support these capabilities, Cosylab developed a tailored version of the Petalinux operating system. This bespoke OS included:
- Application packages.
- Essential libraries.
- Development tools.
- Configuration files tailored to the hardware platform.
Additionally, the Linux image was equipped with a development installation of Robot Operating System 1 (ROS1) Noetic, a widely-used framework in robotics applications.
As part of the DAIS project, Cosylab produced several tutorials to guide users through deploying the hardware platform, enabling vehicle operations, and implementing neural networks within FPGA Systems on Chip. These tutorials serve as valuable resources for advancing the application of AI in edge computing and robotics.
1. Custom Petalinux platform for AI (Tutorial)
2. Vitis-Ai demo on ZCU104
Project website: https://dais-project.eu/
DAIS has received funding from the ECSEL Joint Undertaking (JU) under Grant agreement number: 101007273. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Sweden, Netherlands, Germany, Spain, Denmark, Norway, Portugal, Belgium, Slovenia, Czech Republic, Turkey.
This article reflects only the author’s view and the Commission is not responsible for any use that be made of the information it contains.
