Users that do not wish to extend an existing kernel image must be cognizant of a couple things.
1. Requirements of a kernel-based image to be used by Enterprise Gateway.
2. Is the base image one fromJupyter Docker-stacks?
22.2.1 Requirements for Custom Kernel Images
Custom kernel images require some support files from the Enterprise Gateway repository.
These are packaged into a tar file for each release starting in 2.5.0. This tar file (named jupyter_enterprise_gateway_kernel_image_files-VERSION.tar.gz) is composed of a few files - one bootstrap script and a kernel launcher (one per kernel type).
Bootstrap-kernel.sh
Enterprise Gateway provides a singlebootstrap-kernel.shscript that handles the three kernel languages supported out of the box - Python, R, and Scala. When a kernel image is started by Enterprise Gateway, parameters used within the bootstrap-kernel.sh script are conveyed via environment variables. The bootstrap script is then responsible for validating and converting those parameters to meaningful arguments to the appropriate launcher.
Kernel Launcher
The kernel launcher, as discussedheredoes a number of things. In particular, it creates the connection ports and conveys that connection information back to Enterprise Gateway via the socket identified by the response address parameter.
Although not a requirement for container-based usage, it is recommended that the launcher be written in the same language as the kernel. (This is more of a requirement when used in applications like YARN.)
22.2.2 About Jupyter Docker-stacks Images
Most of what is presented assumes the base image for your custom image is derived from theJupyter Docker-stacks repository. As a result, it’s good to cover what makes up those assumptions so you can build your own image indepen-dently from the docker-stacks repository.
All of the images produced from the docker-stacks repository come with a certain user configured. This user is named jovyanand is mapped to a user id (UID) of 1000 and a group id (GID) of 100 - named users.
The various startup scripts and commands typically reside in /usr/local/bin and we recommend trying to adhere to that policy.
The base jupyter image, upon which most all images from docker-stacks are built, also contains a fix-permissions script that is responsible for gracefully adjusting permissions based on its given parameters. By only changing the necessary permissions, use of this script minimizes the size of the docker layer in which that command is invoked durnig the build of the docker image.
22.2.3 Sample Dockerfiles for Custom Kernel Images
Below we provide two working Dockerfiles that produce custom kernel images. One based on an existing image from Jupyter docker-stacks, the other from an independent base image.
108 Chapter 22. Custom Kernel Images
Custom Kernel Image Built on Jupyter Image
Here’s an example Dockerfile that installs the minimally necessary items for a python-based kernel image built on the docker-stack image jupyter/scipy-notebook. Note: the string VERSION must be replaced with the appropriate value.
# Choose a base image. Preferrably one from https://github.com/jupyter/docker-stacks FROM jupyter/scipy-notebook:61d8aaedaeaf
# Switch user to root since, if from docker-stacks, its probably jovyan USER root
# Install any packages required for the kernel-wrapper. If the image
# does not contain the target kernel (i.e., IPython, IRkernel, etc.,
# it should be installed as well.
RUN pip install pycrypto
# Download and extract the enterprise gateway kernel launchers and bootstrap
# files and deploy to /usr/local/bin. Change permissions to NB_UID:NB_GID.
RUN wget https://github.com/jupyter/enterprise_gateway/releases/download/vVERSION/
˓→jupyter_enterprise_gateway_kernel_image_files-VERSION.tar.gz &&\
tar -xvf jupyter_enterprise_gateway_kernel_image_files-VERSION.tar.gz -C /usr/
˓→local/bin &&\
rm -f jupyter_enterprise_gateway_kernel_image_files-VERSION.tar.gz &&\
fix-permissions /usr/local/bin
# Switch user back to jovyan and setup language and default CMD USER $NB_UID
ENV KERNEL_LANGUAGE python
CMD /usr/local/bin/bootstrap-kernel.sh
Independent Custom Kernel Image
If your base image is not from docker-stacks, it is recommended that you NOT run the image as USER root and create an image user that is not UID 0. For this example, we will create the jovyan user with UID 1000 and a primary group of users, GID 100. Note that Enterprise Gateway makes no assumption relative to the user in which the kernel image is running.
Aside from configuring the image user, all other aspects of customization are the same. In this case, we’ll use the tensorflow-gpu image and convert it to be usable via Enterprise Gateway as a custom kernel image. Note that because this image didn’t have wget we used curl to download the supporting kernel-image files.
FROM tensorflow/tensorflow:2.5.0-gpu-jupyter
USER root
# Install OS dependencies required for the kernel-wrapper. Missing
# packages can be installed later only if container is running as
# privileged user.
RUN apt-get update && apt-get install -yq --no-install-recommends \
(continued from previous page)
# Install any packages required for the kernel-wrapper. If the image
# does not contain the target kernel (i.e., IPython, IRkernel, etc.,
# it should be installed as well.
RUN pip install pycrypto
# Download and extract the enterprise gateway kernel launchers and bootstrap
# files and deploy to /usr/local/bin. Change permissions to NB_UID:NB_GID.
RUN curl -L https://github.com/jupyter/enterprise_gateway/releases/download/vVERSION/
˓→jupyter_enterprise_gateway_kernel_image_files-VERSION.tar.gz | \ tar -xz -C /usr/local/bin
RUN adduser --system --uid 1000 --gid 100 jovyan && \
chown jovyan:users /usr/local/bin/bootstrap-kernel.sh && \ chmod 0755 /usr/local/bin/bootstrap-kernel.sh && \
chown -R jovyan:users /usr/local/bin/kernel-launchers