4.3 Installation in a Container
4.3.4 Creating an Inference Container Image
Mapping Between Host and Container OSs
Table 4-8 lists the mapping between host OSs and container OSs. (Ubuntu 18.04 is recommended as the container OS.)
Table 4-8 Mapping between host and container OSs Architec
ture Host OS
Version Host Kernel Version Container OS Version
x86 Ubuntu
18.04.1 4.15.0-29-generic Ubuntu 18.04 CentOS 7.6
NOTEThe driver needs to be installed in the container.
CentOS 7.6 3.10.0-957.el7.x86_6
4 CentOS 7.6
Ubuntu 18.04
ARM Ubuntu
18.04.1 4.15.0-29-generic Ubuntu 18.04 CentOS 7.6
NOTEThe driver needs to be
installed in the container.
CentOS 7.6 4.14.0-115.el7a.
0.1.aarch64 CentOS 7.6
Ubuntu 18.04
NO TE
● You do not need to install the driver in the container for an OS that does not have the note "The driver needs to be installed in the container" in Table 4-8.
● In the container image OS scenario, only services in the running state are supported, and services in the development state are not supported.
● Currently, the container image OS can be Ubuntu 18.04.1 and 18.04.4.
Prerequisites
● Obtain the software packages and the Dockerfile and script files required for packaging images by referring to Table 4-9.
In a software package name, {version} indicates the software package version, {arch} indicates the operating system architecture, and {gcc_version} indicates the GCC version.
● The user environment needs to connect to the network to pull images. If the network is not connected, see 5.2 Configuring a System Network Proxy.
● The container OS image can be obtained from Docker Hub. For example, to pull the Ubuntu 18.04 container image, run the following command:
docker pull ubuntu:18.04
The ARM CentOS 7.6 cannot be obtained from Docker Hub. If you need to use the CentOS 7.6 image, you can obtain it from AscendHub. After the image is obtained, run the following command to rename the image:
docker tag swr.cn-south-1.myhuaweicloud.com/public-ascendhub/centos:7.6.1810 arm64v8/centos:7
Table 4-9 Required software
Software Package Description How to Obtain
Ascend-cann-nnrt_{version} _linux-{arch}_{gcc_version}.run
Offline inference engine Table 4-8, if the driver needs to be installed in the container, obtain this software package.
● ARM: Link
● x86: Link
Dockerfile Required for creating an
image Prepared by users
Service inference
program package Service inference program package.
The .tgz format is supported.
install.sh Installation script of the service inference
program.
run.sh Script for running the
service inference program.
Procedure
Step 1 Log in to the server as the root user.
Step 2 Create a directory for uploading the software package (for example, /home/test).
mkdir -p /home/test
Step 3 Upload the software package to the /home/test directory.
● Ascend-cann-nnrt_{version}_linux-{arch}_{gcc_version}.run
● A300-30x0-npu-driver_{version}_centos7.6-gcc4.8.5-{arch}.run (Ignore this driver if it does not need to be installed in the container.)
● Service inference program package
Step 4 Perform the following steps to create a Dockerfile:
1. Go to the software package upload directory and run the following command to create a Dockerfile (for example, Dockerfile):
vi Dockerfile
2. Write the following content to the file and run the :wq command to save the file. The Dockerfile content is edited in the following two scenarios:
– The driver does not need to be installed in the container. The following uses Ubuntu 18.04 as an example. For details, see Compilation Sample.
# Obtain the container image ubuntu:18.04.
FROM ubuntu:18.04
# Specify the offline inference engine package and user ID.
ARG NNRT_PKG
# Specify the main directory of the working container and copy the installation package.
WORKDIR /root COPY $NNRT_PKG .
# Install the offline inference engine package.
RUN umask 0022 && \
if [ "$(uname -m)" = "aarch64" ] && [ ! -d "/lib64" ]; \ then \
mkdir /lib64 && ln -sf /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1; \ fi && \
groupadd HwHiAiUser && \
useradd -g HwHiAiUser -d /home/HwHiAiUser -m HwHiAiUser && \ groupmod -g $HWHIAIUSER_GID HwHiAiUser && \
usermod -u $HWHIAIUSER_UID HwHiAiUser && \ chmod +x ${NNRT_PKG} && \
./${NNRT_PKG} --quiet --install && \ rm ${NNRT_PKG}
# Copy the service program package and script file.
ARG DIST_PKG COPY $DIST_PKG . COPY install.sh .
COPY run.sh /usr/local/bin/
# Run the installation script.
RUN chmod +x /usr/local/bin/run.sh && \ sh install.sh && \
rm $DIST_PKG && \ rm install.sh
# Program that is run by default when the container is started.
CMD run.sh
– The driver needs to be installed in the container. The following uses CentOS ARM 7.6 as an example. For details, see Compilation Sample.
FROM arm64v8/centos:7
# Specify the offline inference engine package, driver package, and user ID.
ARG NNRT_PKG ARG DRIVER_PKG
ARG ASCEND_BASE=/usr/local/Ascend ARG HWHIAIUSER_UID
ARG HWHIAIUSER_GID
# Environment variables ENV LD_LIBRARY_PATH=\
$LD_LIBRARY_PATH:\
$ASCEND_BASE/driver/lib64:\
$ASCEND_BASE/nnrt/latest/acllib/lib64
# Specify the main directory of the working container and copy the installation package.
WORKDIR /root COPY $NNRT_PKG . COPY $DRIVER_PKG .
# Install the offline inference engine package and driver package.
RUN umask 0022 && \ groupadd HwHiAiUser && \
useradd -g HwHiAiUser -d /home/HwHiAiUser -m HwHiAiUser && \ groupmod -g $HWHIAIUSER_GID HwHiAiUser && \
usermod -u $HWHIAIUSER_UID HwHiAiUser && \ chmod +x ${DRIVER_PKG} && \
# Copy the service program package and script file.
ARG DIST_PKG COPY $DIST_PKG . COPY install.sh .
COPY run.sh /usr/local/bin/
# Run the installation script.
RUN chmod +x /usr/local/bin/run.sh && \ sh install.sh && \
rm $DIST_PKG && \ rm install.sh
# Program that is run by default when the container is started.
CMD run.sh
The procedure for preparing the install.sh and run.sh scripts is the same as that for preparing the Dockerfile. Compilation Sample shows the file content.
NO TE
– In this document, the driver running user is HwHiAiUser. If you specify another user as the driver running user, change the user name to the actual one.
– After creating the Dockerfile, run the following command to change the permission on the Dockerfile:
chmod 600 Dockerfile
Step 5 Go to the directory where the software packages are stored and run the following command to create a container image:
docker build -t image-name --build-arg NNRT_PKG=nnrt-name --build-arg DRIVER_PKG=drivepackage-name --build-arg HWHIAIUSER_UID=uid --build-arg HWHIAIUSER_GID=gid --build-arg DIST_PKG=distpackage-name .
In the preceding command, --build-arg DRIVER_PKG=drivepackage-name is the specified driver package. If you do not need to install the driver in the container, delete it. Do not omit . at the end of the command. Table 4-10 describes the parameters in the command.
Table 4-10 Command parameter description Parameter Description
image-name Specifies the image name and tag. Change them based on the actual situation.
--build-arg Parameters in the Dockerfile.
NNRT_PKG nnrt-name: specifies the name of the offline inference engine package. Do not omit the file name extension.
Replace it with the actual one.
DRIVER_PKG drivepackage-name: specifies the name of the driver package. Do not omit the file name extension. Replace it with the actual one.
HWHIAIUSER_UI
D uid: specifies the UID of HwHiAiUser. Replace it with the UID of HwHiAiUser on the host.
HWHIAIUSER_GID gid: specifies the GID of HwHiAiUser. Replace it with the Gid of HwHiAiUser on the host.
DIST_PKG distpackage-name: specifies the name of the compressed package of the service inference program. Do not omit the file name extension. Replace it with the actual one.
NO TE
Run the following command to query the UID and GID of the HwHiAiUser user on the host:
id HwHiAiUser
If "Successfully built xxx" is displayed, the image is successfully created.
Step 6 After the image is created, run the following command to view the image information:
docker images Example:
REPOSITORY TAG IMAGE ID CREATED SIZE workload-image v1.0 1372d2961ed2 About an hour ago 249MB
----End
Compilation Sample
Pay attention to the following points when compiling a Dockerfile:
1. Replace the statement for pulling the basic image:
FROM ubuntu:18.04
Obtain the ubuntu: 18.04 image. You need to replace the image based on the container OS. For details, see Table 4-11.
Table 4-11 Example
Container OS Version Image Example
CentOS 7.6 Arm FROM arm64v8/centos:7
CentOS 7.6 x86 FROM centos:7.6.1810
Ubuntu 18.04 x86 Ubuntu 18.04 ARM
FROM ubuntu:18.04
2. Software packages corresponding to the container OS
Obtain the offline inference engine package of the corresponding GCC version based on the container OS. For details, see Table 4-12.
Table 4-12 Software packages corresponding to the container OS Container OS
Version Offline Inference Engine
Package How to Obtain
CentOS 7.6 Arm
Example of compiling install.sh
#!/bin/bash
# Create log-related files.
mkdir -p /usr/slog
mkdir -p /var/log/npu/slog/slogd
chown -Rf HwHiAiUser:HwHiAiUser /usr/slog chown -Rf HwHiAiUser:HwHiAiUser /var/log/npu/slog
# Go to the working directory of the container and decompress the service inference program package.
cd /root tar xf dist.tar
Example of compiling run.sh
#!/bin/bash
# Verify the installation of the npu-smi tool.
npu-smi info
# Start the slogd daemon process.
su HwHiAiUser --command "env LD_LIBRARY_PATH=/usr/local/Ascend/add-ons:$LD_LIBRARY_PATH /usr/
local/Ascend/driver/tools/slogd &"
sleep 1
ps -ef | grep -v grep | grep "tools/slogd"
# Access the directory where the executable file of the service inference program is located.
cd /root/dist
#Run the executable file.
./main