Linux System Administration and
Shell Scripting
MENDOZA, John Robert T.
Science Research Specialist II
Advanced Science and Technology Institute October 1-5, 2012
House Rules
MENDOZA, John Robert T.
● 3 years of experience in Government service
● 3 years of experience in open-source networking
technologies
● Cloud and HPC operator
● Open-source software advocate ● Likes BSD, Linux
Virtualization
… allows multiple operating systems to run
simultaneously on a computer system.
Virtualization Terminologies
Virtual Machine (VM)
● Environment created by the virtualization software
which contains emulated hardware devices.
Host
● Physical machine / Operating System where the
Virtualization
Terminologies
Guest
● An operating system running on a virtualized platform.
● Has all the necessary hardware components to run (processor,
memory, storage, etc.)
Hypervisor (Virtual Machine Manager)
● software that manages hardware resources used by virtual machines. ● Allocates computing resources to avoid contention between VMs.
Virtualization
There are different kinds of virtualization technology:
● Container-based (Kernel-level) ● Paravirtualization
Container-based
(System-level)
● Multiple instances of the same operating system. ● Provides the required isolation and security to run
multiple applications or copies of the same OS (but different distributions of the OS) on the same server.
● Does not use VMMs. ● Examples:
Paravirtualization
● Requires modified version of the operating system
to run as guest.
Full Virtualization (HVM, Native)
● Allows unmodified versions of operating system to
run on the host.
● Can run almost all operating systems available. ● Uses a full set of emulated hardware devices
(storage, networking, etc.)
● Examples:
Importance of Virtualization
Virtualization use-cases:
●
Running multiple operating systems
simulaneously.
●
Consolidating hardware infrastructure.
●
Software testbed.
VirtualBox
VirtualBox
Supported host operating systems: (Minimum)
● Windows XP ● Mac OS X 10.6 ● Solaris 10 ● Linux Hosts ● Ubuntu 8.04 ● Debian 5 ● Oracle Linux 4
Get VirtualBox
VirtualBox
Documentation
VirtualBox:
VirtualBox:
Creating a VM
Supported Guest Operating Systems: ● Windows NT 4.0 ● Windows 2000 / XP / Server 2003 / Vista / Server 2008 / Windows 7 / Windows 8 / Server 2012 ● DOS / Windows 3.x / 95 / 98 / ME ● Linux 2.4 / 2.6 ● Solaris 10 (u6 and higher), Solaris 11 (including Solaris 11 Express) ● FreeBSD ● OpenBSD 3.7 or later
VirtualBox:
VirtualBox:
VirtualBox:
VirtualBox:
VirtualBox:
Richard Stallman
(March 16, 1953 - Present) ● Founded the Free Software
Foundation (FSF) in 1985
● Developed the GNU/Linux
Operating System
● Popularized the concept
of “copyleft”
● Created Emacs!
● Authored GNU Public License
Linus Benedict Torvalds
(December 28, 1969 - Present)
● Developed the Linux
kernel in 1991
● Owns the Linux
trademark
● Working in Open Source
Development Lab (OSDL) in Oregan
Anatomy of an OS
Operating systems consist of the following:
● Kernel
● System programs
Parts of the Kernel
The kernel has several components:
● Process Management ● Memory Management ● Hardware device drivers ● Filesystem drivers
Parts of the Kernel
User Programs / System Programs
User-mode Kernel Kernel System Calls Process Management Memory
Management NetworkServices ManagementVirtual FS Network Servides TCP/IP Drivers Ethernet Card Drivers Ethernet NIC File System Drivers SATA Disk Driver SATA Hard Drives Hardware
What is Linux?
●
Operating system developed in 1991 by Linus
Torvalds
●
Licensed under the GNU Public License (GPL)
●
2-nd Most popular Linux distribution
(Desktop)
●
Mostly used operating system for publicly
Features
● Free (as in free speech, and not as in 'free beer') ● Portability (to any hardware platform)
● Robust (high uptimes expected) ● Secure (no viruses!)
● Scalable (high-perfomance! Baby!) ● Community-supported (open-source)
Cons:
● Not user-friendly (only for pros)
● Not unified (too many distributions!) ● Open-source(?)
Linux Distributions
● Red Hat Enterprise Linux (RHEL) ● Debian
● Community Enterprise Operating System (CentOS)
● Ubuntu
● Fedora ● SuSe
Linux Distributions
● Red Hat Enterprise Linux (RHEL) ● Debian
● Community Enterprise Operating System (CentOS)
● Ubuntu
● Fedora ● SuSe
Ubuntu
ubuntu |oǒ'boǒntoō|
● Ubuntu is an ancient African word meaning 'humanity
to others'
● Founded by Mark Shuttleworth in 2004 ● Has a release cycle of every 6 months
● First release version was 4.10 (Warty Warthog);
Ubuntu
● Long-Term Support (LTS) version every 4th release,
i.e. every two (2) years
● Funded by Canonical Group Ltd.
● Has two editions: Desktop and Server
● Most popular operating system on the cloud: Amazon
and Rackspace
Hardware Requirements
Recommended Minimum Requirements:
● 300 Mhz x386 processor ● 128 MB of RAM
● 500 MB Storage (Base System)
● Monitor, Graphics card, Optical Drive/USB and
Where to Get
http://www.ubuntu.com/download
Ubuntu
Ubuntu is distributed in different types of media images:
● Desktop ● Server ● Alternate
Procedure
● Download an ISO image of Ubuntu Server 12.04
64-bit.
● Mount the ISO image on the CD/DVD Drive of the
virtual machine.
Installation Screens
● Boot Menu
● Install Ubuntu Server
● Select a Language
● English
● Select Location
● Philippines
● Configure the Keyboard
Installation Screens
● Configure the network
● Do not configure the network at this time ● Specify PREFERRED hostname
● Setup Users and Passwords
● Full name of the new user (optional) ● Username for the account
● Password of the new user (at least 8 characters with at least 3
character classes: upper case, lower case, and symbols, enter twice)
Hard Disk Structure
● Hard disks consists of platters or discs.
● These platters have concentric bands which are called tracks. (Seagate boasts of 340,000 tracks per linear
inch width in their 3TB disks)
● Sections within each track is called sectors.
● Sectors are the smallest physical storage unit in a disk and is always 512 bytes in size.
Disk Partitions
● Partitions are contiguous set of blocks on a drive
treated in an independent disks.
● Partitioning is the method of dividing a single hard
drive into multiple logical disks.
● Partition table is an index that relates the sections of
the hard drive to partitions.
● Contains the number of partitions and identifier of what
operating systems are installed in the partitions.
● Contains information on the beginning and end of the
Disk Partitions
There are different types of partition:
● Primary
– Original design of the partition table permits only 4 primary partitions per disk
● Extended
– A subpartition of the primary which is a workaround for the limitation – There can only be one extended partition in a disk
– Has its own file system identifier
● Logical
– Partitions created within the extended partition
Disk Partitions
Points to ponder:
● Partitions can't be moved but can be resized ● Partitions can't overlap
● Partitions should not have gaps between adjacent partitions
● Theoretically, a disk can have a maximum of 18 partitions ● At least one primary partition is require per disk
● There can only be one type of filesystem per partition ● Always place the swap partition at the end of the disk
Disk Partitions
● The first sector in every partition is reserved as the
“boot sector”
● The first sector of the disk is called the Master Boot
Disk Nomenclature
There is a special nomenclature used when referring to disk drives
hd refers to IDE drives
sd refers to SATA, SCSI and USB drives cd refers to optical drives
Disk Nomenclature
Letters are suffixed to the device names to indicate their order of detection by the operating system (for IDE devices, this
depends on the controller where the disk is connected)
hda – hdd IDE hard disk drives
sda – sdz SATA hard disk drives
Disk Nomenclature
Numbers are then suffixed to indicate the partitions and its type
hda{1-4} first four primary/extended partition
Filesystems
● data structures that an operating system uses to
keep track of files on a disk or partition; that is, the way the files are organized on the disk
● Relationship of Disks, Partitions and Filesystems ● Filesystems need partitions
Types of Filesystems
Most common filesystem types:
File System Original OS Max File Size Max Vol Size
UFS2 FreeBSD 512 GB to 32 PB 1YB NTFS Microsoft
Windows NT 16 EB 16 EB
Filesytem Mounting
● Before a filesystem is used, it should be mounted
onto a directory called mount points.
● Typical mount points: /root
/home /var /tmp
Quiz
● What filesystem allows sharing of filesystem
between computers for file and data access?
● What filesystem did Windows 95 use?
Installation Screens
● Partition Disks
● Partitioning Method: Manual ● Select disk to Partition: sda
– Create a new partition table on the device: YES
● Select FREE SPACE
– Select “Create a new partition” – Follow these file system layout:
/ 8 GB primary ext4 bootable=YES /var 6 GB logical ext4
/home 4 GB logical ext4 swap 1 GB logical
Installation Screens
● Installing the Base System (may take some time) ● Configuring TaskSel
● No automatic updates ● Software selection
● Software to install:
– OpenSSH server
Installation Screens
● Install the GRUB Boot loader on a hard disk ● Install the GRUB Boot loader to the
master boot record: YES
● Finish the installation ● Continue and reboot. :)
Logical Volume Manager (LVM)
● LVM provides a higher-level view of the disk storage
on a computer system than the traditional view of disks and partitions
● Allows management of storage volumes with
user-defined volume groups
Benefits of LVM
● Storage consolidation
● Modes of operation: Linear and Striping ● Dynamic storage allocation
● Snapshots
Elements of LVM
● Volume Group (VG) ● Physical Volume (PV) ● Logical Volume (LV) ● Physical Extent (PE) ● Logical Extent (LE)
Redundant Array of
Inexpensive Disks (RAID)
● combine multiple small, independent disk drives into an array of disk drives which yields performance exceeding
that of a Single Large Expensive Drive (SLED)
● MTBF of the array is equal to the MTBF of the individual disk.
● Mean Time Between Failure (MTBF) is the predicted elapsed
time between inherent failures of a system during operation
● Implementing RAID benefits performance and
Redundant Array of
Inexpensive Disks (RAID)
There are three ways to create RAID volumes:
● Hardware ● Software ● FakeRaid
Redundant Array of
Inexpensive Disks (RAID)
There are different types of RAID architecture:
● Striping (RAID 0) ● Mirroring (RAID 1) ● Parity (RAID 5)
Redundant Array of
Inexpensive Disks (RAID)
RAID 0 (Striping)
● Data are striped to member disk devices
● Provides the best performance in read and write operations
● Needs minimum of 2 disks
● No redundancy (can't tolerate disk failure) ● Capacity can be computed as S * N
Where:
S = single disk capacity (smallest)
Redundant Array of
Inexpensive Disks (RAID)
RAID 1 (Mirroring)
● Data is mirrored or replicated on member disk
● Read and write is comparable to a single disk performance ● Requires a minimum of 2 disks
● Provides 1+1 redundancy (able to survive a disk failure) ● Capacity equals S/N
Redundant Array of
Inexpensive Disks (RAID)
RAID 5 (Parity) and RAID 6 (Double Parity)
● Parity is calculated and stored across the disk members ● Read performance is excellent; write performance is
somewhat slower (parity calculation)
● Requires a minimum of 3 disks (for RAID5); 4 disks (for
RAID6)
● Can tolerate 1 disk failure (RAID5); 2 disk failure (RAID6) ● Capacity equals:
– RAID 5: (N-1) * S
Redundant Array of
Inexpensive Disks (RAID)
Configure LVM on Installation
● Partition Disk
● Create a partition table on the disks ● Create a 100MB partition on the disk
– Mount as /boot, ext4 formatted, bootable
● Create a Physical Volume (PV)
– Create a logical partition on the disk – Use it as “physical volume for LVM”
● Create Volume Group (VG)
– Name it as “UbuntuTraining”
– Select the created logical partition to include in the volume group
● Create Logical Volumes (LV)
– Create this logical volumes
UbuntuRoot 8 GB UbuntuHome 3 GB UbuntuVar 4 GB UbuntuSwap 2 GB
Configure LVM on Installation
● Format and mount the LV to filesystems
UbuntuRoot / ext4
UbuntuVar /var ext4
UbuntuHome /home ext4 UbuntuSwap swap
Configure LVM on Installation
● Extending the volume group
● Create a partition table on the second disk ● Delete the UbuntuRoot LV
● Extend the volume group UbuntuTraining: Add the partition of the second disk
● Create the new UbuntuRoot logical volume with all the capacity available
Configure RAID on Installation
● Partition Disks
● Create the prescribed partitions on the first disk instead of formatting them with ext4, select “use physical volume for RAID”
● Create the same layout on the second disk
● Create RAID volumes by matching the partitions from the first and second disk to create a new device, md.
Configure RAID on Installation
● Detach the disk drives from the virtual machine one
Booting Up For The First Time
(Post-Installation)
Boot Sequence
● Basic Input/Output System (BIOS) ● Boot Loader (Grub)
● Kernel and Init ● Runlevels
Basic Input/Output System (BIOS)
● Tests and checks the available devices in the system ● Looks for the devices to boot the system
● When booting from a disk drive, reads the content of
the first sector (MBR)
● The MBR contains instructions on how to load the
GRUB Boot Loader
● First software program that runs when a computer
starts and is responsible for loading and transferring control to the operating system kernel software
● GRUB stands for GRand Unified Bootloader
● Supports two boot methods: Direct loading and Chain
Loading
● Provides a CLI, pre-OS environment which displays
the available operating systems installed in the disk partitions
Kernel and Init
● First application process executed by the kernel after
the boot-loader passes on the boot sequence
● Located in /sbin
● Initializes its configuration file located in
/etc/inittab
● Spawns all other processes in the operating system
Init Runlevel
● runlevel is a configuration of what
applications/processes will be running on the operating system at boot time
● The init process looks in the inittab for the
configured runlevel.
● It then looks into the script directory configured in
the inittab, /etc/rcX.d
Init Runlevel
The run levels are:
0 - halt (Do NOT set initdefault to this) 1 - Single user mode (runlevel S)
2 - Multiuser, without NFS (The same as 3, no networking) 3 - Full multiuser mode
Getty and Login Console
● Getty is a process which prints tty lines and executes
the login process from the virtual consoles or virtual
terminals (VT).
Virtual Consoles
● interface for the user to interact with the operating
system provided by two main peripheral devices, keyboard and the monitor display.
● By default, there are 7 virtual terminals available for
the user.
● Switching virtual terminals are done by these
keystroke combinations, <CTRL> + <ALT> + <F1-F7>
