Reproducible Development
Environments!
with Docker
+ Visual Studio Code
+ Devcontainers
William Killian
Assistant Professor of Computer Science
Background /
Disclaimers
Origins
•
CPOSC theme: Grew up in Lancaster, PA
Education + Interests
•
BS Computer Science (Millersville)
•
MS+PhD Computer Science (U of Delaware)
•
Research
•
High-Performance and Scientific Computing
•
Programming Models
•
CS Education
Disclaimers
All opinions stated in this talk are my own.
Information generously sourced from open-source repositories and documentation. [included at end]
Outline of Talk
•
Why do this?
•
Docker
•
Building a container
• Dockerfile
syntax
•
Using a container
•
Visual Studio Code + Devcontainers
•
Installing the extension
• devcontainer.json
format + features
Why?
“Your scientists were so preoccupied
with whether or not they could, they
didn’t stop to think if they should.”
Dr. Ian Malcolm
Why use Reproducible Environments in
Education (at Millersville University)?
Primarily a response to COVID-19 + Online Modality
•
Learning remotely, not on campus
• Department computer labs not easily accessible
•
Internet Stability
• Remote connection to Linux lab not always stable • ~200 students using 27 machines à NOPE.gif
•
Software + Operating System Variances
• Debugging problems on Windows vs. Linux vs. macOS • Ensure grading + development platforms are consistent
Goals of Reproducibility
1. Minimize pain points for troubleshooting
• Originally a cross product of OS, software, and languages • Now a single configuration
2. Provide a usable environment for students
• Focus on course learning outcomes • Consistent for everyone
• Mirror / replicate software stack used for grading
Department Infrastructure
•
Linux Lab (27 Dell Precision Workstations)
• CPU: Intel Core i7-6700 • RAM: 16GB DDR4
• OS: ArchLinux (a rolling distribution with latest kernel + packages)
•
Autolab Server (Virtualized Infrastructure)
• Intel Xeon Gold 6230 • RAM: 16GB DDR4 • Host OS: ArchLinux
• Grading Images: any Docker container
Provide the same software stack for
Linux Lab + Autograding + Students
What is a Container?
Package of software, its dependencies, and supporting
system software and settings into a single entity
Enables a uniform development experience!
•
Operating system files
•
Language toolchains
•
Libraries and tools
Defining Our Own Containers
Dockerfile
•
A recipe for how to build a container
•
Contains a list of commands
FROM
– define a base container to build from
ENV
– set / modify environment variable
WORKDIR
– change the directory inside of the container
ADD
– copy a file (internet or local) into the container
RUN
– execute a command inside of the container
Dockerfile
example
FROM archlinux/base
MAINTAINER Will Killian <[email protected]>
ADD autolab /usr/local/bin/
ADD add_user.sh /tmp
RUN /tmp/add_user.sh user password \
RUN sed -i '1s;^;Server = http://mirrors.rutgers.edu/archlinux/$repo/os/$arch\nServer = https://mirrors.rutgers.edu RUN pacman -Syu --noprogressbar --noconfirm \
emacs-nox vim nano base-devel curl wget git unzip rsync openssh mosh man-db
RUN bash -c 'yes | pacman -Scc' USER user
Building onto a Dockerfile
FROM mucsci/base
MAINTAINER Will Killian <[email protected]>
RUN sudo pacman -Syu --noprogressbar --noconfirm \
gcc clang openmp llvm cmake ninja boost catch2 gtest gmock \ gdb lldb valgrind cppcheck openmpi imagemagick yay
RUN yay -S --needed --noprogressbar --noconfirm lldb-mi
RUN yay -S --noconfirm --nopgpfetch --mflags '--skippgpcheck --nocheck' \ libc++ libc++abi libc++experimental
RUN bash -c "yes | yay -Scc”
Building onto a Dockerfile
FROM mucsci/base
MAINTAINER Will Killian <[email protected]>
RUN sudo pacman -Syu --noprogressbar --noconfirm \
jdk-openjdk openjdk-src openjdk-doc ant maven gradle junit
RUN bash -c "yes | yay -Scc” ENTRYPOINT /bin/bash -l
Building onto a Dockerfile
FROM mucsci/base
MAINTAINER Will Killian <[email protected]>
ENV PATH="/home/user/miniconda3/bin:$PATH" ENV MINICONDA_VERSION Miniconda3-py37_4.8.3
RUN wget https://repo.anaconda.com/miniconda/${MINICONDA_VERSION}-Linux-x86_64.sh
RUN bash ${MINICONDA_VERSION}-Linux-x86_64.sh -b
RUN rm -f ${MINICONDA_VERSION}-Linux-x86_64.sh
ENTRYPOINT /bin/bash -l
MUCS Container Image Hierarchy
archlinux
base
cpp
graphics
python
ai
java
ocaml
Core OS
Base
Software
Languages
Libraries
Running a Container
Option 1: Command Line
docker pull
mucsci/python
# download
docker run –it mucsci/python
# execute
•
Drops you into a command line within the container
•
No access to your “computer” files
Step 2: Load a Devcontainer Project
https://www.github.com/mucsci/devcontainers
•
Repository has a predefined projects configured for:
• C/C++ (gcc + clang + llvm)
• Java (openjdk 14 + maven + junit + gradle) • OCaml (4.10 + opam)
• JavaScript • Python (3.8)
• .NET (dotnet SDK 3.1) • … and several others
•
Devcontainers have additional features:
• Automatically install and configure extensions
• Mount directories on your computer in the container • Preserve settings and changes
devcontainer.json
Docker Arguments
Additional Environment Variables
VS Code Settings / Configuration Automatic Extension Installation
Automatic Directory Mounting
Statistics for Fall 2020
Course Language Students
• CSCI 330 (Programming Languages) OCaml ~45 • CSCI 362 (Data Structures) C++ ~45 • CSCI 380 (Operating Systems) C ~25 • CSCI 450 (Artificial Intelligence) Lisp, Python ~25
Autolab (grading platform) additionally used by:
• CSCI 161 (Intro to Programming I) Java ~75 • CSCI 162 (Intro to Programming II) Java ~50 • CSCI 370 (Computer Architecture) Assembly, C ~45
• CSCI 375 (Computer Graphics) C++ ~25
