Minimizing the costs and risks
involved in ECM migration
This SER white paper describes how the
Doxis4 Consolidation
Service
and its underlying migration model help minimize the
costs and risks involved in the migration of enterprise content
management (ECM) systems.
It takes real expertise to migrate systems of this type, particularly
because of the need to transfer not only data and documents but
also contexts and relations from the source to the target system
without any losses. Moreover, migration has to be carried out in an
audit-compliant way. The Doxis4 Consolidation Service fully
responds to these needs by enabling efficient, step-by-step data
migration with no disruption to operations.
The starting point
ECM systems typically manage large to very large amounts of data. Particularly with very large document collections (involving several billion items of data), the migration process can easily turn into a complex and lengthy project – in some cases lasting several years – that involves very high project costs and levels of risk. In such cases, exporting and importing data is simply not an option.
What's more, such migrations can only be cost-effective if the ECM/archiving system being replaced is decommissioned as quickly as possible while the new system is being introduced. In light of the high costs of maintaining legacy systems, their often high operating costs, and the need to eliminate the risks associated with operating an obsolete technological platform, it is vital to complete projects quickly.
Ultimately, if no change is made to the underlying storage technology, not even exporting and importing will be an option. Consider, for example, a scenario in which the data of a new archive is stored on an EMC Centera or IBM TSM platform that already houses the legacy data. In this case, you clearly want to preserve hardware resources and avoid copying onto the same system. This aspect, too, represents yet another argument for using the Doxis4 Consolidation Service.
Contents
3
4
6 7 8 89
The starting point
Solution – approach and architecture
Metadata transfer with the Doxis4 FIPS
The Doxis4 Storage Service's storage adapter for third-party archives Filing new documents in the Doxis4 Storage Service
Migrating BLOBs in a live system with the Doxis4 ILM Service
The starting point
ECM systems typically manage large to very large amounts of data. Particularly with very large document collections (involving several billion items of data), the migration process can easily turn into a complex and lengthy project – in some cases lasting several years – that involves very high project costs and levels of risk. In such cases, exporting and importing data is simply not an option.
What's more, such migrations can only be cost-effective if the ECM/archiving system being replaced is decommissioned as quickly as possible while the new system is being introduced. In light of the high costs of maintaining legacy systems, their often high operating costs, and the need to eliminate the risks associated with operating an obsolete technological platform, it is vital to complete projects quickly.
Ultimately, if no change is made to the underlying storage technology, not even exporting and importing will be an option. Consider, for example, a scenario in which the data of a new archive is stored on an EMC Centera or IBM TSM platform that already houses the legacy data. In this case, you clearly want to preserve hardware resources and avoid copying onto the same system. This aspect, too, represents yet another argument for using the Doxis4 Consolidation Service.
Contents
3
4
6 7 8 89
The starting point
Solution – approach and architecture
Metadata transfer with the Doxis4 FIPS
The Doxis4 Storage Service's storage adapter for third-party archives Filing new documents in the Doxis4 Storage Service
Migrating BLOBs in a live system with the Doxis4 ILM Service
Core ECM services
. . .
The most important components of the Doxis4 Consolidation Service
SOLUTION
Approach and architecture
The Doxis4 Consolidation Service is based on the tried and tested Doxis4 Content Service Bus (CSB) and its basic services such as the Doxis4 FIPS, the Doxis4 Storage Service, and the Doxis4 ILM Service – found in every installation of Doxis4. With the new storage adapter for third-party archives and the extensions in the Doxis4 FIPS to import bulk metadata, companies have a new and proven solution for rapidly migrating third-party archives. The Doxis4 Consolidation Service tackles the very issues described above and
takes the following approach to provide a solution that will significantly reduce project lead times – and therefore costs and risks.
Transferring metadata with Doxis4 FIPS (File Import Service/bulk import)
Instead of exporting all the data from the source system and importing it into Doxis4, “only” the metadata associated with the documents is transferred.
The Doxis4 Storage Service's storage adapter for third-party archives
There is no need to immediately migrate the actual content of the documents and its data bulk. Instead, with the aid of a special storage adapter, Doxis4 has read access to the data in the source system, which does not need to stay live. This means that the source system can be decommissioned much sooner than in a conventional migration process.
Filing new documents in the Doxis4 Storage Service
Any new documents are written straight to a new storage system. Doxis4 provides a variety of storage adapters for various systems. The platform can therefore simultaneously connect to a high number of storage systems and run them for the applications concerned in a transparent and storage-agnostic way.
Migrating content data in a live system using the Doxis4 ILM Service
Finally, the Doxis4 ILM Service can be used without disrupting operations to transfer the source system's content data to a new storage system – in exactly the same way storage technology would be switched on the Doxis4 platform.
1
2
3
4
Integrated apps
Or
ga Servic
e
E
v
en
t Servic
e
A
udit T
rail Servic
e
S
ec
urit
y Servic
e
DMS Servic
e
S
e
ar
ch Servic
e
R
epor
ting Servic
e
A
gen
t Servic
e
No
tif
ic
ation Servic
e
BPM Servic
e
s
Base servicesMetadata platform: Doxis4 Content Service Bus
Storage service
Copying the BLOBsto different storage systems without disrupting operations
Storage adapters for read-only access to third-party archives— which don't need to be operational! Abstracted from the storage system
FIP
S (impor
t)
ILM Servic
e
More ECM services
NAS/SAN Doxis4
safeLock EMC HP Hitachi IBM NetApp Cloud
Third-party archive
Core ECM services
. . .
The most important components of the Doxis4 Consolidation Service
SOLUTION
Approach and architecture
The Doxis4 Consolidation Service is based on the tried and tested Doxis4 Content Service Bus (CSB) and its basic services such as the Doxis4 FIPS, the Doxis4 Storage Service, and the Doxis4 ILM Service – found in every installation of Doxis4. With the new storage adapter for third-party archives and the extensions in the Doxis4 FIPS to import bulk metadata, companies have a new and proven solution for rapidly migrating third-party archives. The Doxis4 Consolidation Service tackles the very issues described above and
takes the following approach to provide a solution that will significantly reduce project lead times – and therefore costs and risks.
Transferring metadata with Doxis4 FIPS (File Import Service/bulk import)
Instead of exporting all the data from the source system and importing it into Doxis4, “only” the metadata associated with the documents is transferred.
The Doxis4 Storage Service's storage adapter for third-party archives
There is no need to immediately migrate the actual content of the documents and its data bulk. Instead, with the aid of a special storage adapter, Doxis4 has read access to the data in the source system, which does not need to stay live. This means that the source system can be decommissioned much sooner than in a conventional migration process.
Filing new documents in the Doxis4 Storage Service
Any new documents are written straight to a new storage system. Doxis4 provides a variety of storage adapters for various systems. The platform can therefore simultaneously connect to a high number of storage systems and run them for the applications concerned in a transparent and storage-agnostic way.
Migrating content data in a live system using the Doxis4 ILM Service
Finally, the Doxis4 ILM Service can be used without disrupting operations to transfer the source system's content data to a new storage system – in exactly the same way storage technology would be switched on the Doxis4 platform.
1
2
3
4
Integrated apps
Or
ga Servic
e
E
v
en
t Servic
e
A
udit T
rail Servic
e
S
ec
urit
y Servic
e
DMS Servic
e
S
e
ar
ch Servic
e
R
epor
ting Servic
e
A
gen
t Servic
e
No
tif
ic
ation Servic
e
BPM Servic
e
s
Base servicesMetadata platform: Doxis4 Content Service Bus
Storage service
Copying the BLOBsto different storage systems without disrupting operations
Storage adapters for read-only access to third-party archives— which don't need to be operational! Abstracted from the storage system
FIP
S (impor
t)
ILM Servic
e
More ECM services
NAS/SAN Doxis4
safeLock EMC HP Hitachi IBM NetApp Cloud
Third-party archive
Storage adapter for third-party archives
Even after the migration process is complete, the legacy archive data and documents continue to be stored unchanged in the source system. From now on, any individual document files in the storage system are retrieved using the Doxis4 storage adapter for third-party archives. The previous source system is no longer included in this process, nor does it require further maintenance. The storage adapter is based on a program library that allows users to retrieve documents from the files in the source system. Using this library, document files can be read with the help of information that is transferred into the new system during the metadata migration.
If required, the library can be expanded through an on-the-fly conversion option for the legacy archive documents, such as COLD data and annotations. In addition, the source system storage can be pre-scanned to identify any defective or missing files.
Transferring metadata with the Doxis4 FIPS
In contrast to a traditional migration scenario in which all of the data is exported and imported, the Doxis4 Consolidation Service only exports the metadata from the source system. Metadata includes all of the descriptive information associated with the documents but excludes the content data and the files themselves, which make up the real bulk of the data.
In the process, the metadata is exported from the source system in XML format into an XML schema, which can then be directly reimported by the Doxis4 FIPS. Also imported at this stage is the information that describes how the content data in the source system can be retrieved. This information is termed the storage locator in Doxis4.
Annotations can also be transferred, an option that is examined and planned at this stage.
This process has already been successfully implemented countless times, as SER's legacy archive ITA, used by hundreds of customers, was migrated to Doxis4 in precisely the same way.
As soon as the metadata migration process is complete, the legacy ECM system can be decommissioned and Doxis4 CSB can go live. Doxis4 CSB now has all of the information needed on the migrated documents and only needs to access the source system's content data in its data structures. The storage adapter for third-party archives carries out this task, as described next.
In addition, a caching process can also be activated in the Doxis4 Storage Service. If a document from the legacy data is identified, it can be immediately transferred on demand to a cache from which all further accesses will take
Metadata Export
Metadata import using the Doxis4 FIPS (incl. storage locator)
Storage Source system Migration environment Doxis4 CSB Metadata transformation Document data New/modified documents Storage Source system Doxis4 CSB Doxis4 Storage Service Adapter for source system (read access) Doxis4 Storage Service Cache Storage transparency
Metadata transfer from legacy system
XML
ECM systems:
n Easy Enterprise i (available)
n IBM Content Manager (in
development)
n IBM FileNet IS (available)
n Opentext IXOS (available)
n Saperion (in development)
n Others on request...
Storage:
n File system container files
for the supported ECM systems
n EMC2 Centera
n NetApp
n IBM TSM
The connector technology is available for the
following systems: Adapter for new storage system New storage system
Storage adapter for third-party archives
Even after the migration process is complete, the legacy archive data and documents continue to be stored unchanged in the source system. From now on, any individual document files in the storage system are retrieved using the Doxis4 storage adapter for third-party archives. The previous source system is no longer included in this process, nor does it require further maintenance. The storage adapter is based on a program library that allows users to retrieve documents from the files in the source system. Using this library, document files can be read with the help of information that is transferred into the new system during the metadata migration.
If required, the library can be expanded through an on-the-fly conversion option for the legacy archive documents, such as COLD data and annotations. In addition, the source system storage can be pre-scanned to identify any defective or missing files.
Transferring metadata with the Doxis4 FIPS
In contrast to a traditional migration scenario in which all of the data is exported and imported, the Doxis4 Consolidation Service only exports the metadata from the source system. Metadata includes all of the descriptive information associated with the documents but excludes the content data and the files themselves, which make up the real bulk of the data.
In the process, the metadata is exported from the source system in XML format into an XML schema, which can then be directly reimported by the Doxis4 FIPS. Also imported at this stage is the information that describes how the content data in the source system can be retrieved. This information is termed the storage locator in Doxis4.
Annotations can also be transferred, an option that is examined and planned at this stage.
This process has already been successfully implemented countless times, as SER's legacy archive ITA, used by hundreds of customers, was migrated to Doxis4 in precisely the same way.
As soon as the metadata migration process is complete, the legacy ECM system can be decommissioned and Doxis4 CSB can go live. Doxis4 CSB now has all of the information needed on the migrated documents and only needs to access the source system's content data in its data structures. The storage adapter for third-party archives carries out this task, as described next.
In addition, a caching process can also be activated in the Doxis4 Storage Service. If a document from the legacy data is identified, it can be immediately transferred on demand to a cache from which all further accesses will take
Metadata Export
Metadata import using the Doxis4 FIPS (incl. storage locator)
Storage Source system Migration environment Doxis4 CSB Metadata transformation Document data New/modified documents Storage Source system Doxis4 CSB Doxis4 Storage Service Adapter for source system (read access) Doxis4 Storage Service Cache Storage transparency
Metadata transfer from legacy system
XML
ECM systems:
n Easy Enterprise i (available)
n IBM Content Manager (in
development)
n IBM FileNet IS (available)
n Opentext IXOS (available)
n Saperion (in development)
n Others on request...
Storage:
n File system container files
for the supported ECM systems
n EMC2 Centera
n NetApp
n IBM TSM
The connector technology is available for the
following systems: Adapter for new storage system New storage system
Migrating BLOBs in a live system using
the Doxis4 ILM Service
At the point when the storage technology being used by the previous archive is due to be exchanged (or sooner than that), the Doxis4 ILM Service copies the content data over to a new storage system. This essentially moves data between different Doxis4 Storage Services (while, in turn, the Doxis4 Storage Service is abstracting data from the underlying storage system).
The system remains live during this process, which can be controlled using so-called time slices. Meaning, to prevent the read-only access to the legacy data from causing an undue system load during the day, copying can be set to take place at times when the system is under less pressure.
As soon as the data is available on the new storage system, read-only access takes place mainly on the new system – or, to be more precise, the administrator can specify, using access profiles and priorities, which system should be used. The data in the legacy storage system can still be accessed, something that would still be possible in the event of a system failover, at least until the legacy system has been completely deleted within Doxis4.
Filing new documents in the Doxis4 Storage Service
While the Doxis4 storage adapter for third-party archives allows read-only access to the data structures in the source systems, it cannot be used for making changes to those structures. Any versioning of an existing document or filing of new documents takes place instead in a new storage system, which is also connected via a Doxis4 Storage Service.
This is standard function of Doxis4, as different storage systems – even if they are located at different locations – are supported, while the physical storage of a document is separate from the logical filing system used for it. This forms the basis on which it is possible to transparently switch storage technologies within an archive.
Doxis4 Storage Service supports the following storage systems:
n File servers (NAS/SAN)
n Cloud storage (incl. Azure &S3) n Doxis4 safeLock n EMC VNX FLR n EMC ISILON n EMC Centera n HP iCAS n Hitachi HCP
n IBM TSM (SSAM API)
n NetApp SnapLock
n UDO-Jukebox
(via the SER ITA server)
Migration of data in a live system with the aid of the Doxis4 ILM Service can take place parallel to caching in the Doxis4 Storage Service. Similarly, documents that fulfill the specified criteria in terms of their metadata can first be migrated using a conditional ILM rule, if this is technically necessary.
Benefits
The Doxis4 Consolidation Service provides the following benefits, particularly compared to traditional migration methods:
n Shorter project lead times compared to conventional migration
n Reduced project costs and risks
n Delta migrations avoided thanks to shorter project duration
n Direct access to existing source system data and documents n Source system data is not modified (compliance requirements
fulfilled)
n Considerable savings in operation and maintenance costs (decommissioning the legacy archive)
n If the underlying storage technology is retained: Exporting and importing to the same storage system is avoided (especially relevant for EMC Centera and IBM TSM)
Transfer via ILM Service
Storage Source system Doxis4 CSB Doxis4 Storage Service Adapter for source system (read access) Doxis4 Storage Service Adapter for new storage system New storage system Cache ILM service
Migrating BLOBs in a live system using
the Doxis4 ILM Service
At the point when the storage technology being used by the previous archive is due to be exchanged (or sooner than that), the Doxis4 ILM Service copies the content data over to a new storage system. This essentially moves data between different Doxis4 Storage Services (while, in turn, the Doxis4 Storage Service is abstracting data from the underlying storage system).
The system remains live during this process, which can be controlled using so-called time slices. Meaning, to prevent the read-only access to the legacy data from causing an undue system load during the day, copying can be set to take place at times when the system is under less pressure.
As soon as the data is available on the new storage system, read-only access takes place mainly on the new system – or, to be more precise, the administrator can specify, using access profiles and priorities, which system should be used. The data in the legacy storage system can still be accessed, something that would still be possible in the event of a system failover, at least until the legacy system has been completely deleted within Doxis4.
Filing new documents in the Doxis4 Storage Service
While the Doxis4 storage adapter for third-party archives allows read-only access to the data structures in the source systems, it cannot be used for making changes to those structures. Any versioning of an existing document or filing of new documents takes place instead in a new storage system, which is also connected via a Doxis4 Storage Service.
This is standard function of Doxis4, as different storage systems – even if they are located at different locations – are supported, while the physical storage of a document is separate from the logical filing system used for it. This forms the basis on which it is possible to transparently switch storage technologies within an archive.
Doxis4 Storage Service supports the following storage systems:
n File servers (NAS/SAN)
n Cloud storage (incl. Azure &S3) n Doxis4 safeLock n EMC VNX FLR n EMC ISILON n EMC Centera n HP iCAS n Hitachi HCP
n IBM TSM (SSAM API)
n NetApp SnapLock
n UDO-Jukebox
(via the SER ITA server)
Migration of data in a live system with the aid of the Doxis4 ILM Service can take place parallel to caching in the Doxis4 Storage Service. Similarly, documents that fulfill the specified criteria in terms of their metadata can first be migrated using a conditional ILM rule, if this is technically necessary.
Benefits
The Doxis4 Consolidation Service provides the following benefits, particularly compared to traditional migration methods:
n Shorter project lead times compared to conventional migration
n Reduced project costs and risks
n Delta migrations avoided thanks to shorter project duration
n Direct access to existing source system data and documents n Source system data is not modified (compliance requirements
fulfilled)
n Considerable savings in operation and maintenance costs (decommissioning the legacy archive)
n If the underlying storage technology is retained: Exporting and importing to the same storage system is avoided (especially relevant for EMC Centera and IBM TSM)
Transfer via ILM Service
Storage Source system Doxis4 CSB Doxis4 Storage Service Adapter for source system (read access) Doxis4 Storage Service Adapter for new storage system New storage system Cache ILM service
Find out more about SER!
n
One of the world's top five software-only
*
ECM vendors
n
Included in Gartner's Magic Quadrant
**for Enterprise Content Management
n
Europe's largest independent producer
***
of ECM software
GERMANY - HEADQUARTERS
SER Solutions Deutschland GmbH Joseph-Schumpeter-Allee 19 | D-53227 Bonn Phone: +49 228 90896-0 | www.ser.de | info@ser.de SER 10x in Germany: Berlin, Bonn, Dresden, Dusseldorf/ Neuss, Hamburg, Frankfurt/Main, Munich, Muenster, Neustadt/Wied, Stuttgart
AUSTRIA
SER Solutions Österreich GmbH
Florido Tower | Floridsdorfer Hauptstraße 1 | A-1210 Vienna Phone: +43 1 5039555 | www.ser.at | office@ser.at SER 3x in Austria: Klagenfurt, Pasching/Linz, Vienna
SWITZERLAND
SER Solutions Schweiz AG
Leutschenbachstr. 95 | CH-8050 Zurich (Oerlikon) Phone: +41 44 308 36 66 | www.ser.ch | info@ser.ch SER 2x in Switzerland: Oberentfelden, Zurich
FRANCE
SER Solutions France SARL Paris Arc de Triomphe | F-75008 Paris 54/56 avenue Hoche | Phone: +33 1 58 04 27 12 www.ser-solutions.fr | info@ser-solutions.fr
GREAT BRITAIN
SER Solutions United Kingdom Ltd.
20 Primrose Street | London | EC2A 2EW London The Broadgate Tower - 12th Floor
Phone: +44 203178-6882
www.ser-solutions.com | info@ser-solutions.com
POLAND
S.E.R. Solutions Polska Sp. z o.o. ul. Przyokopowa 31 | PL-01-208 Warsaw Phone: +48 22 4365695
www.ser-solutions.pl | info@ser-solutions.pl
SPAIN
SER Solutions Iberia S.L.
Paseo de la Castellana, 18 – Planta 7 | ES-28046 Madrid Tel: +34 91 794 26 94
www.ser-solutions.es | info@ser-solutions.es
BENELUX
Muiderstraat 1 (Pinnacle Tower) | NL-1011 Amsterdam www.ser-solutions.nl | jose.perl@ser-solutions.nl
SHANGHAI
Phone: +86 21 54655517 | dennis.li@ser-solutions.cn SER is represented by a network of experienced partners in many other countries.
SER Solutions International GmbH
Kurfürstendamm 21 | D-10719 Berlin | Phone: +49 30 498582-0
Fax: +49 30 498582-299 | www.ser-solutions.com | info@ser-solutions.com
linkedin.com/company/ser-solutions fb.com/sergroup
twitter.com/sergroup xing.com/companies/sergroup sergroup.com
info@sergroup.com
* Measured by revenue according to the Market Share Analysis: Enterprise Content Management, worldwide, 2014 (Gartner) ** GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally,
