PHONE EMAIL WEB
602.801.2400 support@priasoft.com www.priasoft.com
Introduction
Many times when an Exchange migration project starts, a discussion about Public Folders arises. This document will attempt to define the complexities of Public Folders, their likely use, and how to make the best decision between using “free” Microsoft tools or the Priasoft Public Folder Migration tool.
Complexities
Public Folders in Microsoft Exchange are often initially thought of in only the context of the data in them. This is a very misleading approach to public folders. In cases where public folder environments are large (or very large), the value of the public folders is not solely due to the amount of data in them but is usually due to the other features and aspects to public folders can, and likely do, have.
There are 6 key points of value for public folders: 1. The folder hierarchy
This is the layout of the public folders and is important to consider and to keep intact. The structure of folders, especially complex folder trees, is important to end users as that is often how they remember where certain types of data is stored. The analogy is similar to remembering the numbers for a dial-faced combination lock. Usually one will remember the action of unlocking the lock and the numbers for it are immediately recalled while holding the lock in hand. Conversely, if someone asked for the combination and one was then required to recall the details of such, it becomes more difficult.
End users navigate public folders in the same way. Many times a user just “knows” where the data is but if asked to write down the specific path to a folder with important data he/she will struggle.
The challenge for the folder structure in a migration is the accurate transition of this structure. This is further
complicated by the MS tools in cases where a particular branch of a public folder tree is larger than what can fit into an Outlook PST file. In such a case, not all of the PF tree is copied to the PST file and one must physically compare the contents of the PST to the PF tree to determine which folders did not make it into the PST file. From there one must attempt to break apart the PF tree into multiple PST files for later import. This task involves starting an export deeper in the PF tree; but this is frustrating because the root folder in the PST file will be the selected Public Folder. One will not see the parent folder tree in the PST file. Very often this becomes a futile task since each time one must break apart the tree, reconstituting the tree on import becomes very difficult.
In contrast, the Priasoft PF migrator does not use PST files and copies the folder structure from source to target database directly. This type of approach guarantees that ALL public folders in the hierarchy are migrated.
2. The data
This is the content of each individual folder. If looked at superficially, the data may be seen as less than complex, but this is also misleading for large PF implementations.
When exporting PF data to a PST, there are several limits: data size on disk of the PST file, folder counts, and item counts per folder.
More often than not, one will run into the file size limit of a PST file when exporting data from public folders. When this limit is encountered, the only notification is that the operation must stop. No report is given to tell if the last folder copied has all the same items as the source folder; and very often there is a discrepancy in item count. When this happens (if you choose not to ignore it), a second export must be performed to capture the remaining items from that last exported folder – even if it is only a single item.
The real complexity with the above is that one must selectively copy specific items from the source folder to this second PST file (drag-and-drop or copy/paste). This means that one must first navigate to the folder in question in the PF tree (which may be very deep), then navigate the list of items in that folder (which also may be very long) to find the first item that was not copied to the first PST so that a selection from that item and below can be made. This also implies that one has already navigated to the bottom of the first PST file in order to determine the last item copied. This complexity is further increased for folder types like calendar and contacts. The default views in outlook for these type of folders do not present the items in a standard list-view presentation. Imagine navigating a PF calendar where the last item copied was 1 year ago. At best, one can switch to month-view and cycle back, month by month, looking for the last item copied. Contact folders have a similar challenge.
Contrast this effort with Priasoft’s tools where we are able to enumerate the contents of a folder, regardless of how many items the folder contains and regardless of the type of folder. In this fashion, we can guarantee that every item in a folder will be copied to the target PF database.
Some additional complexities with PF data are: read/unread status, large attachments, and calendar organizers. When items are copied to a PST file, the per-user read/unread status is lost. Large attachments are often corrupted or fail to copy, the latter of which is the most severe. Consider the risk of exporting to a PST file and an item only partially copied to the PST file. It will appear that the item was copied (the normal properties of the item like the subject will be seen), but the attachment may not have been copied. Such discrepancies are the most difficult to identify and it is highly unlikely that anyone would spend the time to analyze each and every item to ensure that the full content of the items were copied to the PST file. Lastly, when copying calendar items to a PST file, the “organizer” of the calendar item is copied as-is. This presents a unique problem in that when later imported to a target public folder that calendar item will be unmodifiable since there will be NO matching account in the new environment for the listed organizer. Priasoft’s solution will reset the organizer so that calendar items can be updated.
3. Permissions
Permissions add to the overall complexity since this is the feature that determines which folders are visible in the PF tree for a particular user. Using outlook to export public folders (or even exmerge) requires that the current user have sufficient permission to the folder. If the one executing the export does not have permission to see a folder, such
folder will not be exported; and unless the results of the PST are further analyzed with the folders seen in Exchange System Manager/Console the user that is responsible for the export may not even know that folders were missed. As such, prior to a exporting the public folders, a permission setting routine must be performed to ensure that the user that will perform the export has full permission to all the folders.
There are MS tools to manage PF permissions, but they are command-line, text-console tools with which you have to know very specific information about PFs before you can even start. Furthermore, the documentation on these tools are cryptic and as such will take much trial-and-error to figure out. Additionally, these tools, if not used properly can reset ALL PERMISSIONS on ALL FOLDERS without complaint. Priasoft has heard horror stories from customers that used such tools only to tell us that they had to restore from a backup because all the permissions were reset. The second complexity with permissions is with the application of them in the new PF database. In many cases, a matching routine must be made between permissions that exist on a source public folder and mailboxes that MUST exist in the target environment. Once a match is determined, then permissions can be set on the target public folder. This matching routine, when not using Priasoft’s tools, is a manual operation that has to be performed folder-by-folder, user-by-user. The common approach is to export the permissions to a file, then change the details of this file where for each source permission on a folder is modified to point to a mailbox account in the target environment, if one exists. This creates a complicated dependency in that mailboxes must exist in the target environment before permissions on public folders can be reapplied. This further means that for each batch of mailbox migrations a secondary step must be performed to reapply public folder permissions (consider that for each batch of mailboxes, one will have to fully reanalyze the exported permissions looking for matches). This becomes an extremely time consuming and error-prone task.
Priasoft’s solution provides a graphical user interface and source-to-target matching routines so that one can analyze the source to target matching and approve such before continuing. Furthermore, because the Priasoft solution is accessing public folders using SYSTEM permissions, there is no need to assign additional permissions to folder before starting; our solution will see ALL folders, regardless of any folder level permissions that are set.
A unique feature of Priasoft’s solution is our handling of folder permissions. Our solution does NOT require mailboxes to exist in the target environment. We are able to set permissions on folder solely by user account, regardless of whether said user has a mailbox, or not. This ability means that all folders migrate can have their permissions properly reapplied using existing user accounts in the target environment and eliminates the need to follow behind a mailbox migration with public folder updates. This approach means that when you choose to migrate a mailbox, the
permissions for that associated user will already be in place and ready for use. Ultimately this means you can do all the public folder permissions in one event hours, days, or weeks in advance of the mailbox migration. This simplifies the migration plan and process.
As a compliment to the above, our solution will also do the same for cases where Distribution Lists (aka AD Groups) are used to provide permission to a public folder. In such cases, the Microsoft tools fail miserably.
Public Folders can be mail-enabled. A mail-enabled Public Folder is one that has a primary SMTP address and zero or more additional secondary address and for which the Exchange mail transport is then allow to deliver mail directly into the public folder.
Management of email addresses on public folders is done via Microsoft’s PF management tools, one folder at a time. One of the first complexities about mail-enabled folders is the fact that discovery of which folders are mail-enabled is a frustrating and time consuming task. In non-powershell environments, an administrator would have to show
properties on EVERY SINGLE PF in the Exchange System Manger in order to analyze the email addresses for the folder. Unfortunately, this quickly proves to be a worthless task since by default ALL public folders in Exchange 2003 and earlier are mail-enabled. The act of creating a new public folder in these older environments automatically causes an email address to be assigned to the folder. However, the default in this case is to hide it from the Global Address List. However, just because a folder is hidden does not prevent it from accepting mail from a transport. If one can guess the email address for a folder (which is not hard since the address is contrived from the folder’s name), mail can be sent to this folder.
This complicates the analysis of folders since it would be a risky assumption to say that all (or even most) hidden folders are not being used (by real people) for the receipt of mail. In the end, the choice has to be made to migrate all mail-enabled folders, even if that means ALL folders. Once migrated, cleanup in the target environment can take place. Exchange 2007 and later, by default, do not automatically mail-enable a folder, but this is not immediately a cause for relief. Consider that most Exchange 2007 and later environments likely upgraded from Exchange 2003 or earlier and one begins to see that the issue is still there in a large part.
The next part of the complexity is the migration of email addresses. From a Microsoft tools standpoint nothing is provided for this. One is left attempting to develop scripts and manual routines to manage this effort. In Exchange 2003 and earlier, this is especially problematic since the only access to email addresses for a public folder is by finding the Active Directory proxy object. [Note that in all versions of Exchange, email addresses are stored in Active Directory and there is a binary link between the AD proxy object and the actual public folder].
The AD proxy object, unfortunately, does not have information about the path of the folder for which it relates. This means that any attempt to work with these proxy objects in order to catalog and analyze the email addresses becomes quickly frustrating as there is not apparent way to map the proxy object to the actual public folder. This is important to consider since migrating the email addresses into the target environment will require knowledge of the folder’s full path.
The Priasoft solution handles this work flawlessly and is able to map Exchange Public Folders to their AD Proxy complement. This saves many hours of work and possible errors.
The email address migration is likely one of the more complicated aspects of a Public Folder migration process, but one for which Priasoft has solved with great simplicity through smart and well developed tools.
One of the reasons that Public Folders have value is the fact that simple rules can be set on them. Very often it is the rule(s) placed on a public folder that makes the folder valuable to a business. The fact that a rule can be created that reacts to inbound mail allows for initial workflow patterns to be established. It is very common for a mail-enabled folder to have a rule that, if criteria matches, forwards the new item to an individual or a group. Other rules can perform auto-replies which are often used for customer support routines.
Of all of the aspects of a public folder described thus far, Rules are by far the most difficult topic for which to develop a process for migration. Unlike the previous topics, there are NO tools from Microsoft to manage or analyze rules. The only access to rules on a public folder are from Outlook – and with the dependency that you have appropriate permission (consider again the topic of permissions mentioned earlier).
This being the case, one would need to open properties of EVERY Public Folder in a source environment, one by one, from an Outlook client in order to record what the current rules are for a given folder, if any. The cataloging of this information would have to be by hand in the sense that there is no “export rule” option in the Outlook rule
management dialog for a public folder – as such, one would need to write down the rule name, criteria, and action. Also consider that a single public folder can have multiple rules with different criteria – this task alone can cause one to give up and ignore the task, and what a terrible risk that would be.
Even then, if the time were spent to capture all the rules, one would then have to reapply those rules to the respective target folders in the target environment – again, one folder at a time and by hand.
The Priasoft solution can migrate rules. No other tool available covers this very important (and often ignored) feature of a public folder.
6. Residence
The term “residence” refers to the physical storage of specific public folder data. In larger environments it is very common to have more than one public folder database for both point-of-presence and redundancy. However, it is also common to have only specific branches of a folder tree replicate between servers. There are very many times where particular folders are “homed” on only a single server or only a few servers of many.
Consider this fact: one can only get to the public folder data on a specific server if that user’s mailbox database points to that specific public folder database. There is a configuration in Exchange 2010 and below that identifies the “default public folder database” for a given mailbox database. This configuration basically says that all users with mailboxes on MailboxDatabase1 will read and write to PublicFolderDatabase1.
The complexity here that is often missed is the fact that some data can be isolated on only a specific public folder database. Using outlook (or even exmerge) to export public folder data will only be able to export data from the public folder database to which the mailbox (being used by outlook at the time) points. This means that it is critical to
understand of which mailbox to use (with outlook) before choosing to export data so as to be sure to get data off of the intended server.
Priasoft’s solution works from a server/database level first so that complete understanding of where data is being migration from is achieved. This eliminates the chance of missing important public folder data that is homed on only one server of many.
Cost Analysis
A common exercise decision makers must engage in is the cost/benefit analysis of an approach to a problem. This is completely relevant for Public Folder migrations, and becomes more critical the larger the Public Folder environment.
Priasoft believes that it is generally better for a business to use as much automation as possible when executing a transitional project, whether it be Public Folders or any other system. As such, Priasoft provides more automation for the aspects of Public Folders – for migration – than any other set of tools available, and in some cases we are the only one to provide such.
When looking at costs, the first routine is typically from an approach like this statement implies:
“We already have IT staff in place and we should just leverage their time in lieu of using a more automated solution”
Unfortunately, this is very misleading and often is more costly than a 3rd party solution. Consider the following mathematics: 1 – Skilled resource that will execute the public folder migration.
$65 – Hourly cost for that resource to dedicate time to the completion of such a task. 4 – Hours per work day this resource is like to have available to dedicate to this task. 50,000 - Number of folders in the Public Folder Environment
1TB – total data in all public folders
1gb/hour – net speed at which data can be copied from one public folder database to another using Microsoft Tools. 10min/folder – net speed at which rules for folders could be processed (remember this is a completely manual task) 1min/folder – net speed at which email addresses for folders could be processed. This is only really achievable at this rate when the source environment is Exchange 2007 or later and powershell scripting can be employed. There are other complexities that as well that come in to play such as directory replication and the effect of applying values too soon after a previous change.
1min/folder – net speed at which permissions could be matched to target mailboxes for restoration of permissions in the target folders. This is for the entire life of the project. Consider that after each batch of mailboxes is migrated, this operation has to be repeated in order to capture the relationship between exported permission information and new matches from the latest migration effort.
So, from a purely time of effort position:
+ 10 mins/folder; rules
+ 1 min/folder; mail addresses
+ 1 min/folder; permissions
12 total mins per folder of effort
x 50000 total folders
600000 total mins of effort
/ 60 mins/hour
10000 total hours of effort
x 65 dollars/hour
$650,000 total cost as man-hour labor, not including data
10000 total hours of effort
/ 20 available work hours per week
500 weeks of effort
/ 52 weeks per year
9.6 years of non-stop effort.
5 average years between new versions of MS Exchange
1000 total GB of data
/ 1 GB/hour to copy
1000 total hours to copy data to target environment 50 total weeks to copy data at 20 hrs/week
1000 total GB of data
/ 20 max PST size
50 total distinct copy operations
x 15 minutes to stop/start each new export import routine
750 total minutes of effort
/ 60 minutes/hour
12.5 total hours of effort
x 65 cost per hour labor
Additional Considerations
In addition to the above metrics and analysis, the following is not included:
Stop/start challenges. The above metrics assume interrupted effort for the entire duration.
Post-migration rework. There will be some level of rework necessary for things like calendar items, and recopying missing data from source due to changes that occur during the migration effort.
Monetary quantification of negative perceptions due to the migration process and duration
Time spent validating that the process is complete and that no data/feature is accidentally missed.
Time spent to develop scripts and process management framework and interfaces used to track progress.
Maintenance cost of having to maintain 2 mail systems for such a long period of time
Although it may be tempting to ignore some of the complicated features such as rules, doing so will only increase the intangible cost of negative perception. This is further exacerbated when this choice impacts sales and revenue.
