Main Program

General Overview

The Linux installed on the DE1 SoC can be used to run the main program. This program is the interconnect between the neural network and the camera. As the network does not have 100% accuracy, filtering of the calculated output is needed. This filtering is also done in the main program.

Areas

  • DE1-SoC with Linux
    • Camera-Driver
    • Preprocessing
  •  CarVision System
    • Calculation of Center
    • Filtering
    • Providing Image to FPGA

DE1-SoC with Linux

To use the camera and run the main program, we set up our DE1-SoC board with a Linux Ubuntu distribution. The system also communicates over the operating system with the FPGA.

Camera-Driver

First Results

Input without preprocessing, no usable image.
At first it was not sure if the camera has problems, but after a while I figured out how to use the input.

 

Can you see whats in the picture?

First picture with actual data to figure out how to save the images in a right way.

  • clear output
  • greyscale and
  • correct image size
This is how the camera pictures are sent to the FPGA.

CarVision System

Structure of Program

To test the complete model, multiple dummy classes where created. These classes help with testing the complete program, even if some classes are not fully implemented yet.

Calculation of Center

The predicted image needs to be analyzed to find the detected lanes from the network. This is done by searching for the first white pixels from the left and right edges of the image. The center between these pixels is the center of the lane.

Filtering

Since there are some irregularities in the calculated signal, it needs to be filtered with a lowpass filter. This filter is implemented as two second order sections in series. Since the street lines are not continuous, there are a lot of spikes that occur in a somewhat regular pattern. This can be seen very clearly in the plot above. Furthermore, some interference like windscreen wipers and similar, can create wrong measurements which also need to be filtered.

Providing Image to FPGA

The image is provided to the FPGA via a memory mapped file. Since the program only has access to the virtual memory initially, the actual memory location has to be calculated.