Transparent fileservices for Windows,
Unix and Mac
Leveraging ProLiant Storage Servers
and Enterprise Virtual Array together
with Windows Storage Server,
ExtremeZ-IP and Cluster Extension EVA
Monday, 10-Nov-08 Heinz-Hermann Adam
Agenda
•
Who we are and what we do
•
Initial Situation
•
Goal
•
Components and Challenges
•
Implemented Solution
•
Migration Process
Who we are
•
WWU Münster is one
of the three major
universities in
Germany
– ~ 40,000 students – ~ 5,000 scientists and staff – Over 100 fields of study•
The Natural Sciences
Department is ~
¼
of
the university
– Major user and
provider of compute resources
– IT is a Volunteer
driven operation
• Not much dedicated
What we do
•
Provide and maintain
resources for
students, scientists
and staff in Biology,
Chemistry and
Physics
– ~ 4,000 Computers – ~ 12,000 Users•
Compute resources
– Scientific Computing • SMP and Clusters • Development Environment – Desktop Applications • Windows • Linux • Mac OSInitial Situation 2005/2006
•
Replacement of IT Infrastructure in Operation
since 1998: Overdue
•
Isolated Data-Silos of Direct Attached Storage
– OpenVMS
– Windows
– Tru64 UNIX
– Linux
Goal
•
Consolidation
– Versatile Storage System
• Storage Capacity
• Data Protection
• Reliability, Availabilty, Fault-Tolerance
– Highly Available Fileservice
• Transparent to client operating systems
– Unified Computersystem
• Scientific Computing ( HPC)
• Infrastructure Services (Active Directory etc.)
– Manpower
• OpenVMS 7.3-2 Cluster running Advanced Server 7.3A ECO-4 (Pathworks)
• Transparent Filesystems
– OpenVMS
– Windows
• Several Windows based
Fileservers
A first step (2005) – a proof-of-principle
Prior to 2005 Beginning in 2005
• ProLiant Storage Server
Cluster attached to an EVA 3000 storage array
• Transparent Filesystems
– Windows
– Linux
• Single Windows Storage
Architectural move in 2005
Second step (2006) – maturing the solution
•
Two „independent“
sites
•
More storage
– Mirroring of essential file systems•
Larger NAS-System
– Performance – AvailabilityComponents of the Solution
•
Microsoft Active Directory
•
Windows Server 2003 R2/
Microsoft Services for Unix
•
Windows Storage Server 2003 R2 Cluster
•
Continous Access & Cluster Extension EVA
•
Linux and Samba 3
Active Directory
•
X.509 based Directory Service with an
extensible Schema
– Can hold information not only for Windows, but also
for e.g. Unix/Linux users, groups and computers
•
Windows Server 2003 R2 or Microsoft Services
for Unix Schema extension necessary
– Forest-wide operation
•
Leverages industry standard LDAP and
User management for non-Windows
platforms
•
Linux/Unix
– Pluggable Authentication Module
• Uses Kerberos
– Name Service Switch
• Uses LDAP
•
Macintosh
– Open Directory Framework
• Uses LDAP and Kerberos
Windows Server 2003 R2/Microsoft Services
for Unix
•
Schema and Userinterface Extension on Domain
Controllers
•
Server for NFS on Fileservers (NAS)
– Exports Windows Directories as „Network File
Schema extension
Users
• msSFU30NisDomain
– No need for NIS on
Windows • msSFU30UidNumber • msSFU30LoginShell • msSFU30HomeDirectory • msSFUGidNumber – Primary Group Groups • msSFU30NisDomain
– No need for NIS on
Windows
• msSFU30GidNumber
• msSFU30PosixMember
– Beware the storage
limitation for an Active Directory attribute/object
Windows Storage Server Cluster
• Microsoft Cluster Service
• Consists of Cluster
Groups (= „virtual Servers“)
– Default Cluster Group
• Contains Quorum ressource
– Additional Groups for
production Resources
• One per node in the cluster
• Disks, Shares, VSS Tasks
– Loadbalancing
Windows Storage Server Cluster
•
No real
(active-active) cluster
– Failover cluster
•
No load balancing
– Static load distribution
between nodes, based on cluster group
configuration
– One cluster group per
Continous Access & Clusterextension EVA
• Stretched cluster
– Two SAN connected locations
• Continous Access
– Synchronous writes to mirrored Vdisks on both EVAs
• If connection between EVAs is broken,
changes are logged
• After re-establishing connection, changes
are commited to remote EVA
• Quorum
– Odd number of nodes in the cluster and at a minimum a third location
• Majority node set cluster
• Clusterextension
– Failover between EVAs at different sites – Automatic, no operator intervention
Clusterextension EVA
• Resource in MSCS
– One per cluster group
– Talks to EVA Storage
Management Appliance (one per EVA required) – Cluster node only talks to
EVA local to its site – SMA changes Vdisk
presentation etc.
automatically upon Offline and Online Operation of the CLX resource specific to a certain Cluster node
Multi-Protocol Challenges – Part I
• Access for Unix Servers
– NFS on ACL secured VLANs
• Access for Unix Clients
– NFS no option for Clients (No File
Security)
– CIFS (native Windows
Implementation)
• No support for special files, e.g. sockets
• Limitation to allowed characters in a file name, e.g. „:“
• Filesystem behaviour prevents some „features“, e.g. start of a KDE session
– CIFS (SaMBa/Linux
Implementation)
• Linux Server mounts file systems via NFS and re-shares them via Samba 3
Server for NFS on Fileservers
• File Name Handling
– Allows otherwise impossible file names
• Unix: .DCOPserver_myhost_:0
• Windows: .DCOPserver_myhost_20
• C:\SFU\common\__Translate__NFS_File_Names__.txt
• 0x00 0x3a : 0x00 0xb2 ; replace client : with 2 on server
– NFS created files beginning with a „.“ are hidden files on Windows as well
(via the DOS hidden flag)
• For multi-protocoll access, e.g. sharing a directory simultaneously
to Windows and NFS clients
– Microsoft Knowledge Base Article 321049
• HKLM\Software\Microsoft\Server forNFS\Current Version\Mapping
– KeepInheritance = 1
• Otherwise NFS created files and folders do not inherit NTFS ACLs from parent
directories, rendering the inaccesible from Windows
Multi-Protocol Challenges
•
Samba in Active Directory
– Security = ADS
•
Import Windows Shares via NFS
– Windowscluster:/home /homes nfs auto 0 0
•
Export Windows Share via Samba
– [homes]
• Browseable = no
• Writeable = yes
– Unix extensions = yes
Multi-Protocol Challenges – Part II
• Access for Macintosh Clients
– Compatibility Issues with CIFS Client on Mac OS X (file system
semantics)
– Microsoft Services for Macintosh
• Provide Apple Filing Protocol access to Windows files and
directories
• Not cluster-aware
– Manual Procedure (generic script cluster resource) takes more than two hours
to bring AFP shares online • Do not scale well
– Limited to 2.9 million files or 1.6 million directories combined on all AFP
volumes shared
– Only achievable with SFM having the systems paged pool on its own
• Ancient software, introduced with NT 3.x
– No longer maintained
GroupLogic ExtremeZ-IP
•
Native Apple Filing Protocol 3.1 Implementation on
Windows
– TCP/IP, no need for AppleTalk
– Microsoft Cluster Service aware
– Transparent to failover within the cluster
– Kerberos support
•
Does everything Microsoft Services for Macintosh
should do
– And more (e.g. TimeMachine support)
Moving the data from VMS to Windows
• 4 user disks as a VMS searchlist
– Disk$user_f, disk$user_k, disk$user_r, disk$user_z
• Analyzing current usage and size
– 5,000 – 6,000 users – 100 MB diskquota – Overcommitting • Planning (2005) for – 7,000+ users (currently ~12,000) – 650 MB diskquota (currently 2-10 GB) – Overcommitting
Moving data from VMS to Windows
• Data transfer Advanced Server Storage Server
– Robocopy
• Copying ISAM/indexed files (e.g. mail.mail) may crash Pathworks
• Exclude from copying, they are not useful under Windows , Linux or
Mac anyhow
– Multi-stage copying
• Full copy
– Test all services with production data
– Have some guinea pigs
• Incremental copy
– Update changes from production system, after successfull test
– Switch users to the new system
Our Way to Data Pools
• Versatile Storage System
– 1 GB units
• All Servers connected to the
SAN
• NAS-Cluster for Filesharing
• Partitionable SMP Shared Memory System – Itanium2 – 2-24 CPU • Bladesystem – X86-64
