Igor Seletskiy
CEO, CloudLinux
Advanced Settings Oct 10, 2011
• Linux OS based on RHEL source RPMs • Binary compatible with RHEL 5.x/6.x
and CentOS 5.x/6.x
• Made for Shared Hosting Companies • Focus on Stability and Security
• Excellent, free 24/7 support
• Affordable for Companies of any Size
CloudLinux delivered patches for several local exploits days before RHEL and CentOS
• Lightweight resource limits
– CPU/Concurrent Connection/Memory limits • Transparent to administrator
• Easy to deploy to CentOS/RHEL servers • No need to setup per customer limits
• One customer usually the cause of majority of downtimes
– Limiting resource usage for a customer his usage spikes will prevent issues for other customers
– Hard & expensive to track
– Takes time to track, which results in downtime for the server.
• Better stability
• No server slow downs
• No need to suspend customers due to resource abuse
– Simplifies up sell to higher plans / VPS
• Ability to track usage on per customer bases • Less support
• Improved security • Better density
• We deployed CloudLinux on our shared hosting servers, and we have seen immediate improvement in reliability.
– Ditlev Bredah, CEO UK2 Group
• We deployed CloudLinux on our shared hosting
servers, and we have seen immediate improvement in reliability. Now we want to offer the same solution to our dedicated server customers.
• Resource Limits
– CPU
– Memory
– Connections
– IO Priorities (limits coming soon)
• Ability to change limits on the fly
• Ability to see current & historic usage • Easy way to pin-point abusers
• Kernel level technology
– modified CPU scheduler – cgroups/bean counters
• Apache module talks to kernel
– Works with any technology: cgi, mod_php, fcgid, MPT ITK, ruid2, LiteSpeed, etc…
• /etc/container/ve.cfg – lvectl • ubc – /proc/lve/list – /proc/user_beancounters – /etc/sysconfig/lve
CloudLinux 5.x
• Based on 2.6.18 kernel • UBC – memory limits
disabled by default
• Migration thread per core • Live ncpu change since
lve0.8.42 kernel
• gresecurity TPE patch
CloudLinux 6.x
• Based on 2.6.32
• cgroups – memory limits always enabled
• No migration processes • Live ncpu changes
• No grsecurity TPE patch (yet)
• Performance optimization still in progress
• /etc/sysctl.conf
– fs.proc_can_see_other_uid=1 – fs.proc_user_gid=0
• Different way to calculate
– 1 per LVE, instead of per process
• # sysctl -w kernel.full_loadavg=1
• 0 means disabled
• mod_php, LiteSpeed not supported
– Needs to fork to start counting
• Virtual memory vs real memory usage. • What happens when user hits memory
• Number of cores per LVE
– 1 migration thread per enabled core
• # lvectl set XX –ncpu 2 –force
• Smallest limit wins
• 1 core server, ncpu=1, cpu=25% -- true limit = 25% • 2 core server, ncpu=1, cpu=25% -- true limit = 25% • 4 core server, ncpu=1, cpu=25% -- true limit = 25% • 8 core server, ncpu=1, cpu=25% -- true limit ~ 12% • 16 core server, ncpu=1, cpu=25% -- true limit ~6% • 16 core server, ncpu=2, cpu=25% -- true limit ~12% • 16 core server, ncpu=4, cpu=25% -- true limit = 25%
• No more NCPU
• CPU will automatically adjust NCPU parameter
• --core – specifying speed on per core bases. 50% -- half a core, 150% -- 1.5 cores.
# lveinfo --from='2010-10-10' --to='2010-10-15' -o cpu_max --display-username
ID aCPU mCPU lCPU aEP mEP lEP MemF MepF
web2 5 10 10 14 25 25 4 342
web1 3 8 10 6 18 25 0 0
ID User or LVE id
aCPU Average CPU usage
mCPU Max CPU usage
lCPU CPU Limit
aEP Average Number of Concurrent Connections
mEP Max Number of Concurrent Connections
lEP Limit on Concurrent Connections
MemF Out Of Memory Faults
• 508 – Concurrent connections (maxEntryProcs)
• 500 – Either server/script error – or memory limits
• Use lve-stats –by-fault to figure out the issue
• CPU limit will slow down the site, but it will not cause it to fail.
• physical memory instead of virtual memory
– Processes share memory
– Allocated memory doesn’t mean used memory
– Copy on write
– OOM killer – will kill processes if LVE hits memory limit
• Number of Processes • CPU Weight
• lvestats-server binary
• Polls data ever 5 seconds
- Updates /var/lve/info
• Writes to database every minute
– Sqlite db /var/lve/lveinfo.db
– Can write to centralized MySQL/PostgreSQL
• Data is read via lveinfo
• /etc/sysconfig/lvestats
– db_type: postgresql, mysql, (sqlite) – server_id – unique id for the server
• Required DB server
• Part of lve_wrapper package
• Allows to run anything inside LVE
– # lve_suwrapper 300 service mysql restart
• Cron
– Vixie-cron
– /etc/sysconfig/crond
• LVE_DISABLE • LVE_START_UID
• session required pam_lve.so 500
– /etc/pam.d/sshd
– /etc/pam.d/crond
• works on PAM level
When you su/sudo to root from regular user you will still be inside LVE
• httpd.conf, modhostinglimits.conf • AllowedHandlers
– Support regexp
• Daemon based software:
– mod_fcgid, mod_cgid, other
• Tries to use all the cores on the server
– fix by limiting the number of threads to the available
processors to 1. This can be done by editing config.xml & policy.xml files
/usr/local/lib/ImageMagick-[version]/config/policy.xml /usr/local/lib/ImageMagick-[version]/config/config.xml Setting:
# yum install yum-protectbase Edit /etc/yum/pluginconf.d/rhnplugin.conf Add: [cloudlinux-x86_64-server-5] protect = 1 [cloudlinux-base] protect = 1
You can meet us at booth #4
If you rate my survey, I'll hook you up with $20 cPCache. Go to this address to take
the survey: http://go.cpanel.net/b33, and come up to the podium once you've
