2019年7月10日 星期三

Cross-Compile the Qt Libraries for Nvidia® Jetson TX2 and Set the QtCreator Environment

   

※ 2021, 5, 28, update to Qt version 5.15.2.
  
     For some high-end use cases, like car self-driving, medical devices, telemetry equipments and so forth, the specifications for the devices are usually peculiar.Nvidia® Jetson series are for filling the requirements : the embedded systems integrated with the CUDA function : a special system, with the special peripherals and sensors, with the special computation ability for the special purpose.
 
    However, the Nvidia® Jetson documents for building the Qt Libraries are limited. So, I write this post to show how to cross-compile shared libraries, and set the environment and set the QtCreator for Jetson TX2 Qt applications. The same procedures also works for TX1.

   Due to the application on an embedded system is only for a specific purpose, the GUI system is not necessary to be window form : the whole system is for an application only. Therefore, maybe it is necessary to build the Qt libraries supporting with the full-screen form.


零. Download Jetpack and use it to flash the TX2.
    The Jetpacks is here, you need to register a Nvidia account for the downloading. I use v. 3.3.
    There are some detail tutorials and videos for the Jetpack setting, like this or this or this.  I denote 2 precautions here:

   甲. Your host Linux must be installed with xterm, or the Jetpack will be hanging while it should start downloading (after Network layout page).

  乙. You could not run the Jetpack on LXDE environment, or the Jetpack program would be hanging in the component manager page.


After the flashing has been done, make sure your host is able to connect to the TX2. You could use ssh command in your host, to check is the connection workable or not.

ssh nvidia@your_tx2_ip
the password is also nvidia in default.

※ How to set TX2 UART could be found here.


一. Prepare the necessary libraries and headers on TX2.
   ※The works are all in the TX2.

  甲. install some packets to support the Qt build.
sudo apt-get install '.*libxcb.*' libxrender-dev libxi-dev libfontconfig1-dev
libudev-dev libxkbcommon-dev libxkbcommon-x11-dev

   乙. Extract the GLES and EGL headers, and put them under /usr/include.
  The commands are :
mkdir /home/nvidia/GLES
cd /home/nvidia/GLES
apt-get download libgles2-mesa-dev
ar x libgles2*.deb
tar -xvf data.tar.xz
mkdir /home/nvidia/EGL
cd /home/nvidia/EGL
apt-get download libegl1-mesa-dev
ar x libegl1*.deb
tar -xvf data.tar.xz
cd /home/nvidia/GLES/usr/include
sudo cp -r GLES2 GLES3 /usr/include
cd /home/nvidia/EGL/usr/include
sudo cp -r EGL KHR /usr/include

   丙. Create a symbolic link libGLESv2.so  :
sudo ln -s /usr/lib/aarch64-linux-gnu/tegra-egl/libGLESv2.so.2 \
 /usr/lib/aarch64-linux-gnu/libGLESv2.so

  丁. To leave the Nvidia-built Qt librares intact, we prepare a new directory to content the new build :
mkdir /usr/local/qt5
sudo chown nvidia /usr/local/qt5

install some packets to support the Qt build
sudo apt-get install '.*libxcb.*' \
 libx11-xcb-dev libxrender-dev libxi-dev \
 libfontconfig1-dev libudev-dev \
 libxkbcommon-dev libxkbcommon-x11-dev \
 libgles2-mesa-dev libegl1-mesa libegl1-mesa-dev \
 libglu1-mesa-dev libxrender-dev libxi-dev \
 libinput*  \
 mesa-utils  mesa-utils-extra libgles2-mesa-dev
 To here, the work on TX2 comes a stop.


二. Prepare the cross-compilation environment in the host.
       ※ The works are in the host machine.
   甲. Create a directory as the TX2's rootfs , and change your current directory to that, then copy the necessary folders from the TX2:
#cd your_tx2_rootfs_in_your_host
rsync -avz -e ssh nvidia@your_tx2_ip:/lib/aarch64-linux-gnu lib
rsync -avz -e ssh nvidia@your_tx2_ip:/usr/include usr
rsync -avz -e ssh nvidia@your_tx2_ip:/usr/lib usr
rsync -avz -e ssh nvidia@your_tx2_ip:/usr/aarch64-linux-gnu usr

   乙. Fix the necessary symbolic links :
ln -sf $PWD/lib/aarch64-linux-gnu/libz.so.1  $PWD/usr/lib/aarch64-linux-gnu/libz.so
ln -sf $PWD/lib/aarch64-linux-gnu/libm.so.6  $PWD/usr/lib/aarch64-linux-gnu/libm.so
ln -sf $PWD/lib/aarch64-linux-gnu/libdl.so.2 $PWD/usr/lib/aarch64-linux-gnu/libdl.so
ln -sf $PWD/lib/aarch64-linux-gnu/libpng12.so.0.54.0  $PWD/usr/lib/aarch64-linux-gnu/libpng.so
The libpng version or location may be changed with the Jetpack version, you should use find -name libpng* under folder TX2_rootfs_in_the_host_path to find out where it is by checking what soft-links to TX2_rootfs_in_the_host_path/usr/lib/aarch64-linux-gnu/libpng.so

  丙. Install the prerequisite packages.
sudo apt-get install -y 'libxcb.*' \
    libx11-xcb-dev libglu1-mesa-dev \
    libxrender-dev libxi-dev libinput* \
    mtdev* mesa-utils \
    mesa-utils-extra libgles2-mesa-dev
※ Some packets may be not necessary.

  丁. Prepare the toolchain:
sudo apt-get install g++-aarch64-linux-gnu
  and use aarch64-linux-gnu-g++ -v  to check the version, I use g++ 7.X;   
Linaro GCC 5.5 is here,  please download  the gcc-linaro-5.5.0-2017.10-x86_64_aarch64-linux-gnu.tar.xz file, and move it to an appropriate location, then extract it.
tar -xvf gcc-linaro-5.5.0-2017.10-x86_64_aarch64-linux-gnu.tar.xz
  I strongly recommend you should NOT use the aarch64-linux-gnu-g++ from apt-get, it will make the toolchain path setting complicated.

   
 戊. Download Qt everywhere.
    Qt everywhere is here. download a qt-everywhere tar ball, (I download qt-everywhere-src-5.13.0.tar.xz  qt-everywhere-src-5.15.2.tar.xz) and extract it.


三. Compilation Qt for TX2.

 甲.  There is no mkspecs file for TX2, we use the TX1's. But the TX1's mkspecs needs to be modified,  to avoid the compilation error.
※ The modification is necessary even your build is for TX1, as it is a bug of include path duplicate.

Comment out the line 29, in file where_the_qt_everywhere_extracted/qtbase/mkspecs/devices/linux-jetson-tx1-g++/qmake.conf.
include(../common/linux_device_pre.conf)

QMAKE_INCDIR_POST += \
#    $$[QT_SYSROOT]/usr/include \
    $$[QT_SYSROOT]/usr/include/aarch64-linux-gnu

QMAKE_LIBDIR_POST += \

 乙 Configuration for build.

 Move your directory to the root of where_the_qt_everywhere_extracted, and configurate Qt as below :
./configure -shared -c++std c++14 \
 -opensource -release --confirm-license -no-pkg-config \
 -device linux-jetson-tx1-g++ \
 -device-option CROSS_COMPILE=linaro_toolchain_folder_path/gcc-linaro-6.5.0-2018.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- \
 -sysroot TX2_rootfs_in_the_host_path \
 -nomake examples -nomake tests \
 -prefix /usr/local/qt5 \
 -extprefix qt_everywhere_path/JetsonTX2/qt5 \
 -hostprefix qt_everywhere_path/JetsonTX2/qt5-host \
 -skip webview -skip qtwebengine \ -opengl es2  


./configure -shared -c++std c++14 \
 -opensource -release --confirm-license \
 -no-pkg-config -device linux-jetson-tx1-g++ \
 -device-option CROSS_COMPILE=aarch64-linux-gnu- \
 -sysroot TX2_rootfs_in_the_host \
 -prefix /usr/local/qt5.15.2 \
 -extprefix TX2_qt5.15.2_libary_path_in_host \
 -hostprefix TX2_qt5.15.2_host_tool_path \
 -nomake examples -nomake tests \
 -opengl es2 -skip qtwebengine \
 -skip qtwebglplugin -skip qtwebsockets \
 -skip qtwebview -skip qtwinextras \
 -skip qtquick3d -skip qtquickcontrols \
 -skip qtquickcontrols2 -skip qtquicktimeline \
 -skip qtwayland -skip qtandroidextras \
 -skip qtdoc -skip translations \
 -skip qtdeclarative -skip qtpurchasing \
 -skip qtwebchannel -skip qtspeech \
 -skip qtlocation -skip qtremoteobjects \
 -skip qtscript -skip qtmacextras \
 -skip qtdoc -skip qtxmlpatterns
Below explain the arguments about the path.

-prefix: indicates where the built libraries will be placed in the TX2, my argument is /usr/local/qt5 /usr/local/qt5.15.2.

-extprefix :  It specifies where the built libraries will be placed in the host, I require the path is qt_everywhere_folder/JetsonTX2/qt5  /home/gaiger/JetsonTX2/qt5.15.2-device .

-hostprefix : the compilation tools path. The compilation tools would be imported into the QtCreator in the host. My argument is qt_everywhere_folder/JetsonTX2/qt5-host  /home/gaiger/JetsonTX2/qt5.15.2-host .

Denote : All the paths had better to be the absolute paths.
※ To save the TX2 disk space and build time, I skip the modules I will not use, you you check here to find out what modules are comprised of version 5.15.2.
 丙. build Qt libraries:
make -j4
make install

The build process costs a long time.
After "make install",  the Qt libraries for TX2 (qt_everywhere folder/JetsonTX2/qt5 TX2_qt5.15.2_libary_path_in_host) and the host tools(qt_everywhere folder/JetsonTX2/qt5-host TX2_qt5.15.2_host_tool_path) will be generated. Now we copy the libraries to the TX2 :
scp -r TX2_qt5.15.2_libary_folder nvidia@your_tx2_ip:/home/nvidia
# login into TX2, rename the folder as qt5.15.2
# and move it to be under /usr/local


四. Set Qt running environment on TX2.
  ※The works are all in the TX2.

The window mode, on XCB :

 Create a file /etc/profile.d/qt_xcb.sh in your TX2, the file contents :
export DISPLAY=:0.0
export QT_QPA_PLATFORM=xcb
export QT_QPA_FONTDIR=/usr/share/fonts/truetype
※ The variable DISPLAY may be :1.0

And bring the values into the current shell :
source /etc/profile.d/qt_xcb.sh

Test if the Qt application works or not :

 /usr/local/qt5/bin/qdbusviewer




The fullscreen mode,  on EGLFS:

Create a file /etc/profile.d/qt_eglfs.sh in your TX2, the file contents :
export DISPLAY=:0.0
export QT_QPA_PLATFORM=eglfs
export QT_QPA_EGLFS_PHYSICAL_WIDTH=360 export QT_QPA_EGLFS_PHYSICAL_HEIGHT=240 export QT_QPA_FONTDIR=/usr/share/fonts/truetype export XDG_RUNTIME_DIR=/tmp/runtime-nvidia 
※ The variable DISPLAY may be :1.0 
※We specify the Qt applications running on EGLFS, therefore, we need to set the EGLFS environment variables. The description of all EGLFS variables is here.

Create a file under the directory /etc/udev/rules.d to enable the access permission of /dev/input/eventX. The file name could be as your wish, mine is 00-event.rules. Its content is:
ACTION=="add", KERNEL=="event[0-8]*", MODE="0666"
Then , reboot your TX2.

/usr/local/qt5/bin/qtbusviewer would appear like below figure, that is fullscreen application.

Attention :: The application is full screen, so there is no max, min or close button.


五. Integrate the toolchain and host tools into QtCreator.
※ I assume your TX2 is online and be with your host at the same network segment.
 Open QtCreator,  go to Menu -> Tools -> Options, then the row Build and Run.

  甲. Page Qt Versions, to add the qmake file, which located in the host_tool_path/bin (qt_everywhere_folder/JetsonTX2/qt5-host TX2_qt5.15.2_host_tool_path, the argument of -hostprefix).


  乙. Page compilers, to add the linaro C++ compilers, the compiler you just use it to build the Qt libaries, aarch64-linux-gnu-g++. Besides, you might also need to add the C compiler.


  丙. Page debugger, add the linaro debugger, aarch64-linux-gnu-gdb, set Multiarch GDB as path /usr/bin/gdb-multiarch. (you may need to install it ahead via sudo apt-get install -y gdb-multiarch)



丁. In row devices, add a device for TX2, and set its ip, user id and password (default: nvidia/nvidia).


  戊. Back row Build & Run, in page Kits, add a kit for TX2, and  set the device, rootfs, compiler, Qt versions and so forth for TX2.

己. Qtcreator will overwrite the environment varibles whilt it is debugging, thus you need to set up the variable DISPLAY value at Project->Run->Run Environment, also please check the value of QT_QPA_PLATFORM.

 


庚.   Create a Qt application in the QtCreator, select the kit for TX2. While the project has been generated automatically, add the 2 lines in the .pro file :
target.path = /home/nvidia
INSTALLS += target
The lines could be placed anywhere of .pro.

  辛. Press Run or Start Debugging button, the application will run in the TX2, and the debugger should work perfectly. You must login the TX2 locally (use the TX2's screen/keyboard/mouse to login), or there would be no screen to show the GUI and leads to crash.



1 則留言: