November 2013
From Chaos to Control:
Creating a Mature File Transfer Process
Moving files in the extended enterprise requires not only support for higher scale with fewer resources, but also dealing with the relentless challenges of security; compliance; errors, exceptions, and problems; and the time and opportunity cost of problem resolution. Aberdeen’s analysis found that mature, managed file transfer processes experienced 26% fewer issues and 4.8-times faster time to
Business Context: The Current Chaos of Moving Files
If ever there were an appropriate illustration for the well-worn phrase “do more, with less”, it would be a snapshot of how unstructured data — let’s just call it files, to keep things simple — is currently being moved hither and yon throughout
the extended enterprise. Aberdeen’s 4Q 2013 study of how more than 100 organizations are addressing the challenges of file movement confirms that:
• The number of end-users needing to transfer files is growing quickly —
between 6% and 9% year-over-year. This includes not only employees, but also temporary employees, contractors, business partners, and customers.
• The volume of file transfers is growing even faster than the number of
end-users — between 8% and 11% year-over-year.
• The size of files being transferred is also growing — between 6% and 7%
year-over-year.
• Meanwhile, growth in the number of full-time equivalent IT staff associated
with file transfers is not keeping pace, at just 1% to 2% year-over-year. But these are not the only challenges. Aberdeen’s study also shows that in addition to the challenges of supporting higher scale with fewer resources,
organizations must simultaneously deal with several other important issues related to file movements, including:
• Increasing risk — the number of security and compliance incidents related
to file transfers grew by an average of 4% year-over-year.
• Nagging inefficiencies — errors, exceptions, and problems affected 4% to
5% of total file transfer volume over the last year.
• Higher costs — the average time to correct errors, exceptions, and
As Aberdeen noted in Seeing Managed File Transfer as Strategic: Business Users Need Full Visibility (September 2013), it’s worth remembering that the business purpose for all of this activity is not merely the movement of files from one point to
another. On the contrary, what organizations really need is the convenient,
efficient, reliable, and secure means to enable collaboration between individuals, and to integrate business processes within and between organizations. Moving files is just the means to these greater ends.
The current chaos of moving files in the extended enterprise is reflected in Figure 1, which shows some of the many file delivery methods currently in use by the respondents in Aberdeen’s study. In broad categories, this includes:
• Physical methods — e.g., thumb drives, courier services, and
• Digital do-it-yourself methods — e.g., email attachments, non-commercial
FTP servers, custom programs and scripts, and consumer-oriented file synch or file share solutions.
Figure 1: File Delivery Methods Currently in Use (illustrative)
Source: Aberdeen Group, November 2013
Aberdeen’s study found that most companies (70%) currently support multiple
methods for moving files, such as those noted above. In addition, most companies
(63%) also support multiple modes of moving files, including person-to-person, process-to-person and person-to-process, and process-to-process.
P er cent ag e of A ll R es pond en ts , ( N =102) 100% 80% 60% 40% 20% 0% 9% 91% 78% 77% 77% 70% 61% 59% 58% 40% Email attachments FTP Secure FTP (FTPS, SFTP) Fax Physical media (e.g., thumb drive) Courier
services In-house /custom programs and scripts Overnight services Consumer file synch / file share solutions
From Chaos to Control: The Way Moving Files is Moving
Given this context, it’s easy to appreciate why companies are looking to gain more control over the current chaos of moving files throughout the extended
enterprise. Based on the research findings summarized in Table 1, the general direction that organizations are moving in over the next 12 months could hardly be more clear:
• For companies supporting multiple modes of file transfer, Aberdeen’s study indicates 100% growth in moving from multiple methods to a common method — i.e., to an enterprise-class managed file transfer solution.
• Even companies supporting just a single mode of file transfer (such as
person-to-person) indicate nearly 40% growth in moving from a mish-mash of multiple methods, to the visibility and control of a common method.
Table 1: Current Approach / Planned Approach in the Next Year
Multiple Methods Common Method Multiple
Modes
Current Planned Current Planned
51% 38% 12% 24%
One Mode
Current Planned Current Planned
19% 16% 8% 11%
One connection at a time, as needed: 7% Source: Aberdeen Group, November 2013 100% growth in MFT
The leading drivers for current investments in file movement initiatives are summarized in Figure 2. Note that in broad categories, these include:
• The pursuit of positive outcomes — e.g., productivity improvements,
collaboration with trusted third-parties, and alignment with industry standards and best practices.
• The reduction of negative outcomes — e.g., the previously noted problems related to security, reliability, compliance, audit, and cost.
Figure 2: Top Drivers for Investments in File Movement Initiatives
Source: Aberdeen Group, November 2013
Reduce total cost
25% 25% 32% 33% 37% 39% 65%
Replace other delivery mechanisms Address audit deficiencies Industry standards and best practices Improve reliability Collaborate with trusted third parties Security-related incidents (e.g., data loss) Improve productivity
0% 20% 40% 60% Percentage of All Respondents (N=102)
How Investments in Managed File Transfer Can Pay Off
This brings us to the heart of the matter: do investments made in an enterprise-class managed file transfer platform pay off? To gain insights into this question, Aberdeen analyzed and compared the responses from the following two groups:
• Managed File Transfer users — i.e., companies that have deployed a
standardized, supported “platform” for multiple modes of file transfer (N=22), and
• Managed File Transfer non-users — i.e., companies that have deployed
multiple supported solutions, unsupported solutions, or ad hoc solutions for multiple modes of file transfer (N=66).
The results are summarized in Table 2. Managed File Transfer users exhibited clear advantages over Managed File Transfer non-users, including:
• 26% fewer errors, exceptions, and problems with file movements
• 4.8-times faster time to address errors, exceptions, and problems with file
movements when they do occur
Table 2: Organizations Using Managed File Transfer Had Fewer Problems and Faster Resolution
Performance Metrics Related to File Transfer
(average, over the last 12 months)
MFT Users MFT Non-Users MFT Advantage
Errors, exceptions, and problems
(% of total annual file transfer volume) 3.3% 4.5% 26% Time to correct an identified error,
exception, or problem minutes 81 minutes 387 4.8-times
In addition, in the face of supporting higher scale with fewer resources, Managed File Transfer users were successful in making year-over-year improvements in security, compliance, reliability, and support for the organization’s business
objectives — while Managed File Transfer non-users, unfortunately, got worse in each of these areas (Table 3).
Table 3: Organizations Using MFT Made Y-o-Y Improvements
Performance Metrics Related to File Transfer (year-over-year change)
MFT
Users Non-Users MFT
Security incidents
(e.g., data loss or exposure) -2.4% +3.2%
Compliance incidents
(e.g., audit deficiencies) -1.9% +3.9%
Errors, exceptions, and problems -3.7% +1.4%
Calls and complaints -3.8% +0.2%
Creating a Mature File Transfer Process:
Capabilities that Correlate with Top Performance
A further comparison of the current capabilities of Managed File Transfer users and Managed File Transfer non-users provides additional insights into those that correlate most strongly with top performance, and can serve as a guideline for increasing the maturity of the file transfer process over time.
Organizational Capabilities: Have “One Throat to Choke”
Mature file transfer processes are more likely to have an executive or team with clear ownership and overall responsibility, and to explicitly assign technical
resources to file transfer activities. In many cases this includes dedicated
resources, as in a managed file transfer “Center of Excellence” (Figure 3). This demonstrates a clear understanding of the central role that file transfer plays in collaboration between people and in the integration of business processes. Mature processes are also more likely to provide real-time notification of policy violations — to end-users and managers, in addition to administrators — and when possible, to automate the remediation of policy violations. Many errors, compliance incidents, and security incidents are simply the result of
well-intentioned individuals trying to carry out their jobs, so using the capabilities of the managed file transfer platform to notify them in real-time can be an important part of communicating and enforcing the organization's established policies.
Figure 3: Selected Organizational Capabilities
Source: Aberdeen Group, November 2013
Policy and Process: Don’t Keep Fixing the Same Problems
File transfers are fundamentally about the flow of data in support of collaboration and business process integration — which means that best practices for data protection naturally apply. Mature file transfer processes are more likely to
classify the data being transferred, and to establish and communicate clear policies based on requirements for confidentiality, integrity, availability, and the
criticality of the business processes that the data supports (Figure 4).
Mature processes are also more likely to standardize implementation and rollout to new users and new use cases, and to have a standardized response for errors and exceptions when they do occur. Moreover, as Aberdeen has seen in study after study over the past several years, the top performers also seek to eliminate the root causes for errors and exceptions — to spend fewer of their limited resources addressing the same problems, over and over again.
Automatic remediation of policy violations Notification of policy violations to end-users Notification of policy violations to managers Notification of policy violations to administrators Formal documentation, awareness and training Dedicated resources assigned (e.g., “Center of Excellence”) Technical resources explicitly assigned Responsible executive or team
9% 23% 14% 32% 14% 33% 19% 48% 35% 41% 28% 68% 52% 82% 58% 86% 0% 25% 50% 75% 100% Percentage of Respondents MFT Non-Users MFT Users
Figure 4: Selected Capabilities for Policy and Process
Source: Aberdeen Group, November 2013
Percentage of Respondents MFT Non-Users MFT Users 34% 50% 33% 50% 38% 68% 56% 86% 63% 82% 51% 60% 42% 62% 32% 50% 0% 25% 50% 75% 100% Systematic elimination of root causes
Standardized response for errors, exceptions, security incidents, or audit deficiencies Systematic implementation / rollout processes Security policies communicated Security policies established Information classified by business processes Information classified by requirements for C.I.A. Formal risk analysis
Establishing New Connections: Help Users Help Themselves
Mature file transfer processes are more likely to move beyond a strict end-user dependence on IT staff, to provide end-users with a range of self-supporting options for making new connections. These include reusable templates, self-service
portals, wizards, and other automated setup tools as part of the managed file transfer
solution (Figure 5).
This is more than just a common sense response to the reality of the “do more, with less” business context previously described. As Aberdeen pointed out in Best Practices in Managed File Transfer (April 2013), if left to their own devices — and in the absence of well-defined policies, awareness and education, and enterprise-supported
alternatives — end-users over time will work around any dependence on IT to get
their jobs done. The predictable result is a hodge-podge of physical and digital do-it-yourself approaches for moving files, as we have already seen.
The obvious remedy: don't just keep telling end-users that they're doing something wrong; give them the enterprise-class tools that will help them to achieve the organization’s objectives, and support them in doing it right.
Figure 5: Selected Capabilities for Establishing New Connections
Source: Aberdeen Group, November 2013
Assured Delivery: Getting There is Important, But Expected
As described in Seeing Managed File Transfer as Strategic, reliable delivery of file movements is obviously important, but mechanisms to assure that files will
eventually be delivered can increasingly be considered table stakes — i.e., they are present in more mature initiatives, but they have relatively high presence other organizations as well. Aberdeen’s 4Q 2013 study confirms that mature file transfer processes are more likely to have capabilities such as guaranteed file delivery,
Percentage of Respondents MFT Non-Users MFT Users
0% 25% 50% 75% 100%
Wizards and other automated setup tools
Self-service portal
Reusable templates
IT supported (per end-user request, per-connection)
25% 32%
29% 45%
32% 55%
Figure 6: Selected Capabilities for Assuring File Delivery
Source: Aberdeen Group, November 2013
Visibility: Insist on Hearing Bad News Immediately
When file movement becomes recognized as critical to the achievement of critical business objectives — whether person-centric, process-centric, or both — it naturally means that end-users tend to want even greater visibility into what’s
happening throughout the file movement process, e.g.,:
• Were the files delivered?
• Were the claims submitted?
• Were the inventories updated?
Percentage of Respondents MFT Non-Users MFT Users 14% 36% 31% 41% 30% 50% 42% 50% 33% 55% 46% 55% 0% 25% 50% 75% 100%
Accelerators for large files Guaranteed file delivery Checkpoint restart (automatic restart at point of failure) Automatic retry after transmission failure Automatic deletion policies Confirmation of file receipt
• Were the transactions completed?
• Who has accessed this information, and when?
Mature file transfer processes are more likely to capture the information needed to let the organization know when there is a problem with mission-critical file transfers (Figure 7), before its collaborators, business partners, or customers can complain about it.
Figure 7: Selected Capabilities for Visibility into File Movements
Source: Aberdeen Group, November 2014
Percentage of Respondents MFT Non-Users MFT Users 34% 55% 38% 55% 23% 50% 39% 64% 40% 67% 56% 77% 0% 25% 50% 75% 100%
Capture performance metrics Detection of non-delivery events Monitor flow and status of business processes dependent on file transfers Monitor health and performance of the file transfer applications Monitor flow and status of file transfers Capture logs
Governance: Use Visibility to Make Decisions, Take Action
Visibility into file movements is necessary, but not sufficient. It’s one thing to be aware of errors, exceptions, and problems, but quite another to be proactive in actually doing something about it. (Solution providers often use the
poorly-conceived marketing term “actionable intelligence”, but the point is that having the intelligence without taking any action is meaningless.)
Mature file transfer processes are more likely to have capabilities such as
standardized monitoring and reporting, scorecards and dashboards, and analytics
(Figure 8). But more importantly, they are also more likely to review and use this information regularly — e.g., to optimize business processes that are dependent on file transfers, or to manage performance against service level agreements that have been established with internal or external users.
Figure 8: Selected Capabilities for Governance of File Movements
Source: Aberdeen Group, November 2013
Percentage of Respondents MFT Non-Users MFT Users
Service-Level Agreements
(e.g., reliability, error rates) 32% 36%
21% 38%
33% 45%
27% 55%
25% 59%
29% 64%
Correlation and optimization of business processes dependent on file transfers Regular review and response to reporting information Analytics Scorecards and dashboards Standardized monitoring, analysis, and reporting
Solutions Landscape (illustrative)
Before launching any evaluation process for managed file transfer solutions, it’s worth the reminder that companies should first inventory their existing file movement
activities to establish a baseline for the modes, methods, end-users, volumes,
frequency, applications, and systems currently in use. Solution providers for managed file transfer can range from smaller specialists to global, multi-billion firms; Table 4 provides an illustrative list.
Table 4: Solutions Landscape for Enterprise Managed File Transfer (illustrative)
Company Solution(s)
Attachmate File Express, Reflection for Secure IT
Attunity Attunity Managed File Transfer (formerly Repliweb)
Axway Axway SecureTransport, File Transfer Direct
Biscom Biscom Delivery Server (BDS)
Cleo Cleo Harmony, Cleo VLTrader, Cleo LexiCom
GlobalSCAPE Enhanced File Transfer, Managed Information Xchange
(MIX) SaaS
IBM IBM Managed File Transfer suite (includes IBM Sterling Connect:Direct, File Gateway, File Transfer Service, Control Center, Business Monitor, Secure Proxy, QuickFile)
Ipswitch File
Company Solution(s)
OpenText OpenText Managed File Transfer
PKWARE SecureZIP, SecureZIP PartnerLink
Seeburger Seeburger Business Integration Suite (BIS)
Software AG webMethods ActiveTransfer
(part of webMethods Integration Platform)
TIBCO TIBCO Managed File Transfer, TIBCO Slingshot
Summary and Key Takeaways
• Aberdeen’s research shows that moving files in the extended enterprise is a prime example of “doing more, with less”, including:
o Growth in the number of end-users needing to transfer files o Even higher growth in the volume of files being transferred o Growth in the size of files being transferred
o Less IT staff available to provide end-user support
• Aberdeen’s research also shows that moving files in the extended enterprise requires dealing with material risks and costs, such as:
o Growth in the number of security and compliance incidents related to file transfer, and their associated business impact
o Errors, exceptions, or problems with 4% to 5% of annual file transfer volume
o Time and opportunity costs to correct errors, problems, and exceptions when they do occur — at an average of 4 to 5 hours per event
• Most companies currently support multiple methods for moving files, as well as multiple modes or use cases. But there is a clear movement towards a common file transfer platform (i.e., towards managed file transfer
solutions) — Aberdeen’s research showed 100% year-over-year growth, across all respondents
• The leading drivers for current investments in this area are well-aligned with the context described above, i.e.,:
o Support business objectives for increased productivity and collaboration o Improve operational aspects of security, reliability, compliance, and cost
• Aberdeen’s comparison of 22 organizations that currently use Managed File Transfer solutions with 66 organizations that do not found that Managed File Transfer users realized several significant advantages, including:
o 26% fewer errors, exceptions, and problems with file transfers as a percentage of total annual volume
o 4.8-times faster time to correct errors, exceptions, and problems when they do occur
o Year-over-year reductions in errors, exceptions, and problems; security incidents (e.g., data loss or exposure); compliance incidents (e.g., audit deficiencies); and calls and complaints related to file transfers — in contrast to Managed File Transfer non- users, who reported year-over-year increases in each of these areas
• Comparison of Managed File Transfer users and Managed File Transfer non-users provides additional insights into the capabilities that correlate most strongly with top performance, and can serve as a guideline for increasing the maturity of the file transfer process over time. Aberdeen’s research highlights numerous capabilities of more mature, managed file transfer processes — in several dimensions:
o Organization (“People”) o Policy and Process
o Technology — including establishing new connections; integrating with and automating existing business processes; and assuring file delivery o Operations — including security; compliance; enterprise-wide visibility
into file movements; and effective management and governance For more information on this or other research topics, please visit
Related Research
Seeing Managed File Transfer as Strategic: Business Users Need Full Visibility;
September 2013
Best Practices in Managed File Transfer; April 2013
Protecting Your Intellectual Property: It Starts with a Single Click; April 2013
Three Steps to Successful Data Classification; February 2013 B2B Collaboration: The Impact and Advantages of Managed File Transfer; June 2012
Secure File Sharing in the UK; June 2012 The Role of Data Classification in Protecting Your Intellectual Property;
May 2012
Does Your Enterprise Classify Its Data?; January 2012
Left to Their Own Devices: Does Your Enterprise Have a “Dropbox Problem”?; January 2012
Secure / Managed File Transfer: Why You Should Be Looking More Closely Right Now; August 2011
Author: Derek E. Brink, Vice President and Research Fellow for IT Security and IT GRC (Derek.Brink@aberdeen.com)
For more than two decades, Aberdeen's research has been helping corporations worldwide become Best-in-Class. Having benchmarked the performance of more than 644,000 companies, Aberdeen is uniquely positioned to provide organizations with the facts that matter — the facts that enable companies to get ahead and drive results. That's why our research is relied on by more than 2.5 million readers in over 40 countries, 90% of the Fortune 1,000, and 93% of the Technology 500.
As a Harte-Hanks Company, Aberdeen’s research provides insight and analysis to the Harte-Hanks community of local, regional, national and international marketing executives. Combined, we help our customers leverage the power of insight to deliver innovative multichannel marketing programs that drive business-changing results. For additional information, visit Aberdeen http://www.aberdeen.com or call (617) 854-5200, or to learn more about Harte-Hanks, call (800) 456-9748 or go to http://www.harte-hanks.com.
This document is the result of primary research performed by Aberdeen Group. Aberdeen Group's methodologies provide for objective fact-based research and represent the best analysis available at the time of publication. Unless otherwise noted, the entire contents of this publication are copyrighted by Aberdeen Group, Inc. and may not be reproduced, distributed, archived, or transmitted in any form or by any means without prior written consent by Aberdeen Group, Inc.
