Exploiting hardware management subsystems

(1)

19/03/2014

Exploiting hardware management subsystems

"iLO, iLO, it’s off to work we go!" CRESTCon 2014 – Simon Clow

(2)

 Introduction to Hardware Management Systems (HMS)  Identification of HMS Whilst Testing

 Tools of the Trade  Penetration Testing

 Initial Exploitation  Secondary Actions  Demonstrations

 Demo Part 1: Basic skills

 Demo Part 2: a 15 year-old 0-Day?

Overview

(3)

 Why Should We Know or Care about HMS?

 Modern computing systems implement a variety of remotely

accessible, instrumented management interfaces.

 Not all of them are obvious!

 As professional penetration testers it is important to understand

the capabilities of such interfaces

 Otherwise we cannot exploit them!

 How do we identify the management interfaces if we are not

aware of them?

 How do we advise customers how to protect the management

interfaces if we are unaware of the security considerations?

(4)

(5)

 Baseband Management Controllers  Service Controllers (SC’s)

 Intel™ AMT

 Active Management Technology  vPro - KMS

 Computrace™

 IP Management Interface (IPMI)

(6)

 Lights out Management (LOM)  Dell DRAC

 HP iLO

 Oracle (Sun) iLOM , aLOM and serial LOM  Supermicro IPMI

 An IPMI 2.0 reference client?  Various Others

(7)

(8)

 Baseboard Management Controller

 A dedicated “computer” that can manage the host system  Monitors the physical state of a device

 Highly Instrumented  Temperature  Power State

 Network Connectivity  Case Intrusion

 Fan State / Speed

 BMC’s communicate with the administrator through an

independent connection or API.

(9)

 BMC Firmware Rarely Updated

 At least not as part of the normal patch update cycle  BMC Upgrades may improve host performance

 BMC has direct interfaces to key (security) related component's:  DMA – Direct Memory Access

 I2C Bus  Controllers

 BIOS / UEFI Configuration  Raw Device Access

(10)

(11)

 Service Controller (SC)

 Many BMC’s implement a service controller (but not all!)  Often termed ‘service console’ or “service processor”

 BMC may require dedicated authentication

 SC’s typically provide a “shell” from which the BMC can be

managed.

 However, not all SC’s are accessible

 Laptops often have SC, generally integrated into ACPI

(12)

 Each Individual SC has their access method:  Serial (RS-232 or RJ45)

 Telnet  SSH

 Proprietary API

(13)

 Monitors the instrumentation in BMC and then schedules actions  E.g. CPU Watchdog, restart host if CPU hung

 Works in conjunction with BMC to “tune” the host.  ACPI is often a SC function

 CPU Tuning when on battery power

(14)

(15)

 Advanced Management Technology

 Intel Specific technologies implemented into both:  Processor *and*

 Chipset

 Uses IPv6 Broker and SSL to Connect to Management Center  Allows BIOS reconfiguration

 Virtual Media Support  Allows OS Re-installation



Think of it as RealVNC integrated into BIOS..

(16)

 Advanced Management Technology  Remote KMS – vPro

 Intel Management Command Toolkit (MCT)  Optional Web Interface

 Delivers custom VNC Client  Older Versions (Pre 2013 / AMT 6.0)

 Require Authentication (password only) to access  Must be exactly 8 characters!

 Unless it is one of the defaults (username is admin):  admin

 P@ssw0rd

(17)

 Newer Versions (2013+ / AMT 7.0 +)

 Connects back to Intel vPro Platform Solution Manager  PKI based authentication

 “Boundary-less” connection (via IPv6 Broker)  Able to remotely execute AMT plugins

 Privileged (System) level code execution  Remote (graphical) control

 Is this the Ultimate Red-Teaming tool?

(18)

(19)

 Absolute Software, Computrace™  Sales Pitch….

 “Computrace is the only endpoint security solution in the

world that can remain installed on computers, laptops,

tablets, smartphones, and other devices regardless of user or location.

 If the software agent is removed (accidentally or on purpose)

it will automatically reinstall.”



Very Widely Deployed Supported by Well Known

Manufacturers

 https://www.absolute.com/en-GB/partners/bios-compatibility

(20)

 Context reviewed in 2010 as a customer research project:  Analysis:

 Persistent even with OS Rebuild!  Can Exfiltrate Data

 Supports Command Execution  3rd Rate CnC…

 Conclusion

 Looks and behaves like Malware  Recommendation:

 Customer not to enable it, and reject systems shipped with it

enabled.

(21)

 Kaspersky Labs (Feb 2014):



http://s1.securityweek.com/pre-installed-computrace-software-could-be-used-hijack-computers-kaspersky-lab

 “Computrace uses many tricks popular among malicious

software.

 For example, it uses anti-debugging and anti-reverse

engineering techniques, injects memory into other processes and keeps configuration files encrypted.

 The network protocol used by the Computrace Small Agent

provides basic features for remote code execution.

 The protocol does not require the use of any encryption or

authentication of the remote server, opening up avenues of attack.”

 http://www.securelist.com/en/analysis/204792325/Absolute_Co

mputrace_Revisited

Computrace™

(22)

(23)

 Intelligent Platform Management Interface (IPMI)

 A collection of specifications that define communication

protocols for:

 Access to Local System Bus’ (SMBus/BMC/I2C)  Network Communication (LAN / LANPLUS …)

(24)

 http://en.wikipedia.org/wiki/File:IPMI-Block-Diagram.png

IPMI

(25)

 Intelligent Platform Management Interface (IPMI)

 A collection of specifications that define communication

protocols for:

 Access to Local System Bus’ (SMBus/BMC/I2C)  Network Communication (LAN / LANPLUS …)  Two widely implemented variants:

 1.5 – Up to 2004  2.0 – 2004 onwards

(26)

 IPMI 1.5

 No Encryption

 At least not within standard Implementations  No Console redirection

 SOL – Serial Over LAN

 Text only console access (*NIX)  BIOS admin

(27)

 IPMI 2.0

 Introduced Encryption  … ish – more later 

 Provided Authentication Framework  Stronger User Privilege Separation  Challenge / Response

 Hashes can be cracked

 Passwords stored in clear text – explore SC!  Console redirection

 Virtual Media

(28)

 IPMI Networking

 IPMI can be configured to *share* first NIC on motherboard

 SC/BMC will respond to IPMI requests to *hosts* IP address.  Host will send UDP port unreachable

 client apps must be able to handle that.

 Will often “fallback” to first NIC if dedicated LOM card removed…  Heat Causes Motherboard Expansion - Cards “walking out”  Over eager administrators “removing” LOM as they know

about IPMI “security issues”

(29)

(30)

 Intended to Provide “Out of Band” Management  Historically present on “high end” kit:

 Solaris LOM Port – Serial over RJ45

 Solaris [a|i]LOM – HTTP / SSH / IPMI +++  HP iLO

 Dell DRAC (and now iDRAC)

 And pretty much every other enterprise manufacturer…

(31)

 Lights Out Management

 Generally provide “web” management interface

 Predominantly using ActiveX / Java “plugin’s” and browser

detection.

 Primary role is usually to provide “zero installation” client  Client is used to perform the actual management

 Often based on VNC  Advanced Functions

 Client to enable the use of Virtual Media

 Redirected Consoles – Provides GUI access to OS and BIOS

(32)

(33)

 What Should we be able to do?

 Identify common Hardware Management Subsystems  Identify security defects within the deployed subsystem

 Perform basic "false negative" detection  Exploit common security defects

 Post exploitation activities

(34)

 How To: Identify common Hardware Management Subsystems  Look for the management services:

 IPMI (UDP 623)  SSH (TCP 22)  Telnet (TCP 23)

 HTTP & HTTPS (TCP 80 / TCP 443)

(35)

 Identify common Hardware Management Subsystems  Perform basic "false negative" detection

 Be aware of NIC sharing!

 Connect to services, don’t just portscan!

 Remember OS may report port’s closed and SC/BMC still

respond.

 Remember Nessus is Imperfect!

 Dell iDRAC rarely reports “Cipher 0” despite being vulnerable.  When it is reported, the firmware is very OLD!

(36)

 How to: Identify security defects within the deployed subsystem  Check for Cipher 0

 Check whether HMS support the extraction of password hashes  Attempt Anonymous Access (IPMI + Web Service force browsing)  Recent Test:

 Management Application – Controlled backend SC’s

 App required current JSESSIONID + current “CLIENT-ATH”

value.

 CLIENT-ATH provided by connecting to port “8123” (?)  Could replay “add user” SOAP function with valid

(unauthenticated) JSESSIONID and CLIENT-ATH to add new admin user - 

 *or* Use an unauthenticated file upload function in order to

upload arbitrary /root/.ssh/authorized_keys

(37)

 Exploiting common security defects

 Extract and crack hashes (Metasploit / John the Ripper)  Use default credentials (lots of them!)

 If vulnerable to Cipher 0 “just” reconfigure

(38)

 Post exploitation activities

 Hardware is compromised!

 Direct Memory Access / Memory Corruption  Dependent on I2C

 Dependent on LOM functionality  Reboot into alternate OS?

 Credentials harvesting

 On Domain Controller - OphCrack live via virtual media  Deploy tools

 MetSVC replacing Anti-Virus service binary on Exchange

(39)

(40)

 Tools to Access LOM’s

 An Older browser (seriously!)

 Java Run Time (JNLP/Java 1.4 plugin’s)  ActiveX

 May need to degrade your browsers security to get it

working!

 Internet Explorer 10/11 very rarely supported  Telnet / SSH

 My experience is more often than not it will be telnet (!)

(41)

 IPMI Client Tools  ipmitool

 bmc-config

 Supermicro Java implementation  Limitations:

 Library miss-matches can cause false negatives  Libgcrypt

 Libcrypto  OpenSSL

 Tools are generally released just for *NIX systems

(42)

 Status of IPMI Support in Common Toolkits  Kali

 Broken at various times in 1.04 / 1.05  Working (as of 13/03/2014) in 1.06  CentOS / RHEL / Scientific Linux

 Native ipmitool compiled so as to not support Cipher 0 due

to “it being a security vulnerability”.

 Debian / Ubuntu

 ipmitool / bmc-config in most repo’s is currently broken.  RMCP connections silently fail (False negative).

(43)

(44)

 IPMI

 UDP – 623 (Alert Standard Format)  In the case of IPMI on a Shared NIC?

 OS Believes Port Closed, port scan fails

 BMC responds to *valid* IPMI queries, not port scans!

 Wireshark will allow you to see both OS and HMS responding.

(45)

 IPMI

 UDP – 623 (Alert Standard Format)  In the case of IPMI on a Shared NIC?

 OS Believes Port Closed, port scan fails

 BMC responds to *valid* IPMI queries, not port scans!

 Wireshark will allow you to see both OS and HMS responding.

(46)

 IPMI 2.0 Implemented Cryptography  We all know cryptography is hard (!)

 Therefore the standard mandates first Cipher (0) to be “null

crypto” (clear text)

 Obviously null crypto is bad for sending credentials on the wire…  Therefore we should disable authentication if using Cipher 0…

 And implicitly trust the username supplied by the client (?)  Really, it is required to be compliant with the RFC!

 No it wasn’t an April 1st RFC (12/02/2004)  … but it probably should have been!

 http://ctx.is/ipmi-demo-cipher0

(47)

 BMC-Config

 Not *technically* IPMI but does support LANPLUS for connections

(like IPMI 2.0 it also provides Cipher 0)

 LANPLUS implemented as driver, LAN_2_0  Ignore documents that show LAN_2.0  BMC-Config –checkout

 Get the controller to show you the syntax to reconfigure it!  http://ctx.is/ipmi-demo-bmc-config-1

(48)

 BMC-Config - reconfigure

 Easiest attack to do, simply use a config file (Context.ipmi)  Can do / undo more easily

(49)

(50)

 BMC-Config - reconfigure

 Easiest attack to do, simply use a config file (Context.ipmi)  Can do / undo more easily

 Make sure you view checkout first and get Manufacturer specific

configuration options, these are not equivalent:

 None

 No Access  NoAccess  No_Access

 http://ctx.is/ipmi-demo-bmc-config-2

(51)

 iDRAC

 Comes in various “flavours”:  Enterprise

 Express – aka “Lite” version

 A combination of SC + BMC + IPMI on dedicated interface  Administrative GUI (Web Based)

 Java / Active X plugin (depending on browser)  http://ctx.is/ipmi-demo-idrac-primer

(52)

 IPMI != iDRAC Enterprise

 IPMI Users are not the same as iDRAC users.

 iDRAC Enterprise add an additional authentication layer (iDRAC),

configurable locally but not remotely!

 Newly added “context” user wont work on Enterprise iDRAC   http://ctx.is/ipmi-demo-idrac-enterprise

 This technique will work on pretty much everything else though.  But don’t worry… there is a solution for iDRAC!

(53)

 IPMI can configure iDRAC “root” user

 We can use IPMI to set the “root” users password:  http://ctx.is/ipmi-demo-idrac-password

 Easy to confirm our changes:  Either using IPMI

 Or logon to iDRAC web console

 http://ctx.is/ipmi-demo-idrac-password2

(54)

 Do we actually need iDRAC?

 iDRAC Enterprise provides a “handy” pre-packaged client, its

in-browser:

 Java  ActiveX

 However we can use IPMI to perform a lot of the functions of

iDRAC

 Supermicro Java Client

 http://ctx.is/ipmi-demo-supermicro

(55)

 Surely we can use Metasploit?  Yes:

 Metasploit has re-implemented IPMI inside a ruby library, code

ported from xCAT.

 Extreme Cluster Administration Toolkit

 Metasploit library is currently limited to providing “read only”

access:

 Identification of users – IPMI “user list”  Cracking of passwords

 http://ctx.is/ipmi-demo-metasploit

(56)

 IPMI

 Cygwin compiled ipmitool and bmc-config  No worries about library linking issues.

 (Unset PATH && LD_LIBRARY_PATH)

 Excellent for “Pivoting” through Windows boxes; take one and

then ‘sploit the others through shared MGMT VLAN’s

 http://ctx.is/ipmi-cygwin-binaries  Intel Management Command Toolkit (AMT)



http://software.intel.com/en-us/articles/download-the-latest-version-of-manageability-developer-tool-kit/

(57)

(58)

 Is this an #0day or is it just #0ldsk00l ?  Assuming Oracle (Sun) SPARC Systems:

 [a|i]LOM ->

 Cipher 0 to seize control of LOM

 Can break from LOM into Service Controller  Service Controller ->

 “console” and “#.” access to OpenBoot PROM  OpenBoot PROM ->

 “break” command

 OBP Written in Forth, can extend without recompiling…  Direct Memory Modification

 Hacking in Forth - Phrack Magazine - 7 Aug 1998

(59)



Seizing [a|i]LOM



Cipher 0 to seize control of LOM user account

 ipmitool -H 1.2.3.4 -v -I lanplus -C0 -U admin

-P BadPass user list

-P BadPass user set password 3 abc123



Verify Control

 ipmitool -H 1.2.3.4 -v -I lanplus -U admin

-P abc123 user list

 telnet –l admin 1.2.3.4

 ssh [email protected]

(60)



Seizing [a|i]LOM (cont.)



OR Try Default Credentials

 ALOM: “admin” + last 8 characters of serial number

-P BadPass fru print

 ILOM: “root” / “changeme”



OR Just use Cipher 0 to bypass authentication

 IPMI Support in early ALOM is very limited.

(61)



Migration to Service Controller



From Sun (Oracle) documents we know we can break from

LOM into Service Controller

 “console” -> Into SC

 “#.” -> Return to LOM



Alternatively we can bypass the migration step and go

straight to SC using IPMI:

-P BadPass sunoem cli

(62)



Migrating from SC to OpenBoot PROM (OBP)



“break -y” command halts Solaris OS (and doesn’t prompt!)



“console -f” drops us to the running OS console

 but actually into OBP if OS is halted

 “-f” forces a read-write connection (disconnects existing

session)



OBP Written in Forth

 We can extend functionality without recompiling, including

direct memory access



“go” command resumes execution of Solaris OS

(63)



Direct Memory Modification



Hacking in Forth - Phrack Magazine - 7 Aug 1998

 Assumed 32bit Solaris structures and the ability to extract

base memory from userland



Solaris 10(+) 64Bit OS / 32Bit Userland (by default)

 64Bit base memory addresses to large to express as 32Bit

numbers;

 Therefore OS masks base address from 32Bit processes

 However, we can get base memory address of processes if we

explicitly use 64Bit calls.

(64)



Direct Memory Modification (cont.)



Credentials structure has changed in Solaris 10(+)

 But by inspecting OpenSolaris we can find the offset for both

Effective UID (EUID) and Real UID (RUID)

 Credential structure no longer Basemem + 0x18 -   Now it is Basemem + 0x20 - 

(65)



Sploit Time!



This assumes you have already pwned [a|i]LOM via IPMI or

credential guessing using your new skillz.



We will be using telnet to connect to SC (dirty)



IP Addresses Used:

 192.168.1.2 – SSH Session to Solaris server (as “simon” – low

privileged)

 192.168.1.18 – Telnet session to ALOM on SunFire v240



http://ctx.is/ipmi-0day-or-0ldsk00l

(66)



Mitigation:



Follow Oracle’s best practice security advice:

 LOM as a “dedicated” Management Network  Use Firewall

 Change default credentials

 Enable OpenBoot PROM Security Mode (Advice from 1994!)



It is Context’s experience that whilst OS security is

considered, HMS security as a wider concept is not!

 Solaris system was chosen purely to demonstrate HMS access

is equivalent to physical access (e.g. at the console)

 It provides a very visual demonstration as to the impact of

direct memory modification!

(67)

Exploiting hardware management subsystems