SURVIVAL GUIDE
Ideas to help
you land on
your feet
TABLE OF CONTENTS
3
12 Points to Consider Before Even Beginning Your Automation Project
5
Tips for Successful Project Development
7
Nine Tips for Automation Project Managers
9
Four IT Standards You Should Understand
10
Four Considerations for Upgrades & Migrations
11
Eight Ideas for Successful DCS Implementation
13
13 Suggestions for Control System Migrations
15
10 Steps to Creating the Perfect HMI
17
Safety: The Lifecycle Approach
19
Control System Security Tips
21
How to Avoid Mistakes with Control System Remote Access
23
Four Tips for Dealing with Wireless Latency and Bandwidth Issues
The first step in any automation project is the most critical one: Define your objectives. The more thorough and detailed this definition is, and the earlier in the process it can be achieved, the greater the likelihood that the project will be completed successfully.
1. Visualize success. Try to visualize what a project would look like if it were a stunning success. Take note of how it will affect all the people in-volved and write down any others you think it might touch. Take all of these people and put them on a spread-sheet column. Now in rows across,
write down the attributes they need in the machine/process. Use this when evaluating solutions and communi-cate shortcomings to those affected. Come up with workarounds or throw out the idea if the results won’t be acceptable.
2. What’s driving the project? You need to understand what is the most important motivation for doing this particular project and use that to guide your decision-making. 3. Helping people. Automation can do many things, but one must be aware that its purpose is to do real
things in a given ecosystem. Keep in mind that the goal is to have systems engineered to serve humans, not the other way around.
4. Project definition is critical. Without doing true engineering work, everything you learned in school and in your career up to this point, you are not doing any project properly or professionally. By creating definition for the project and then verifying that the project will answer the need, you are on your way to successful project management. It is only the start, but without a properly defined starting point, it is difficult to complete (or de-fend) a meandering, ill-defined project that is meant to resolve a problem, ad-dress a challenge or complement your company’s engineering resources. 5. Start with the objectives. Don’t even begin to select suppliers and service providers until you’ve estab-lished a project’s objectives. Make sure everyone on the team agrees on what the project needs to achieve before it starts. If you don’t know where you’re going, you’ll never get there.
6. Get a second opinion. It pays to get a second opinion from an in-formed outsider like a system integra-tor or machine builder before final-izing project objectives—they’ll often
12 Points to Consider Before Even
Beginning Your Automation Project
It’s essential to understand each person’s expectations before a project
starts. There are three parts to this definition process:
y
y
What outcomes or desired results does the project team want to
achieve?
y
y
What do they want the project experience to be like (for example,
no production line shutdowns during the project or communicate
updates by email)?
y
y
How will they define quality, such as on time/on budget or
in-creased production volumes or zero downtime, at the end of the
project?
Different people will have different expectations and they all have to
be satisfied.
do it for free. If you bring them in at this stage, so that they understand the history of the project, they can con-tribute to decisions that will improve the chances for a successful project. 7. Set rules for communication. De-fine what communications are expect-ed at the start of the project— what is to be communicated, how it is to be communicated, what the milestones of the project will be and how often things should be communicated. 8. Talk to everyone. Interview the stakeholders from various factory disciplines, such as operations, main-tenance, quality control, supply chain, shipping and management. They always have a stake in every automa-tion project.
9. Never assume. Don’t make as-sumptions about the ground rules— spell everything out in advance and define who is responsible for doing what.
10. Create a chart to keep objectives clear. Define the expected perfor-mance for each subsystem, and the expected steps to get there. Use Excel to list the task steps, and the hours/$
across a time matrix. Then that be-comes a calendar for the schedule, sort of a compressed MS Project. If you color the boxes, it becomes a Gantt chart. Putting all your objectives (the completion of functioning subsys-tems, integration) into one simple chart keeps those objectives clear to the whole team.
11. Spell everything out. If you want drawings in portrait vs. landscape mode, for example, or want certain brands to be used, such as for wire or PLCs or other components, state
that up front. If a requirement is not written down, then it likely will not happen.
12. Scope! Nothing is more impor-tant than a scope that reflects both the well-defined areas of the project and the gray areas of the project. The gray areas should have a gen-eral framework put together by the customer and the implementer, with benchmarks that clearly indicate when project reassessment should occur. This way scope creep can be managed to the benefit of both parties.
12 POINTS TO CONSIDER BEFORE EVEN BEGINNING YOUR AUTOMATION PROJECT
Never start by defining technology-driven objectives. Use the following
order:
1.
Business objectives. What will the business gain from this project?
2.
Operational objectives. How will this project impact operations—
greater efficiency / better quality / compliance, etc.?
3.
Integration objectives. Can data generated by this system be used
by other systems?
4.
People objectives. Skill development, ease in work pressures.
Only when all of these have been defined can you establish the
technology objectives.
Tips for Successful
Project Development
Project development is not an every-day occurrence at batch process facili-ties. To help ensure you are covering all the major issues involved in these infrequent work scenarios, here are some tips and considerations to facili-tate a successful project startup. 1. Clearly identify the project speci-fications. What do you want to do? What is your existing process? Define operator involvement, quality control issues, interface points with other sys-tems, and the technological capability available in-house.
2. Conduct a job risk assessment (JRA). Performing a JRA before the start of work highlights any hazards that could produce undesirable results to personnel or property. A safety assessment must be completed to ensure that the scheduled work can be performed in a safe manner and to address any hazards that are uncov-ered as a part of the review process. 3. Operator training is key. The operators must learn how to navigate and operate their process in the new control system. The training must be performed just in time (about two weeks before start-up) so that the information is fresh in their minds. During the instruction, it is critical that
the operators be trained using the operator interface graphics they will encounter.
4. Emphasize communications. Communicating with the site mainte-nance and operations departments is critical to the success of the project. Maintenance and Operations need to schedule their duties with enough lead-time to support the installation and startup activities. With enough time, maintenance can even contract back-fill support for the duration of the project startup activities. For op-erations, the work and vacation relief schedule will have to be organized so that enough operators are available to cut-over and startup the plant. This is especially important if a hot cut-over is involved.
5. Have a detailed cut-over plan. Planning is crucial to any stage of an automation project. By putting together a detailed cut-over plan, the personnel performing the work will have a clear directive of the activities that need to be completed each day. The cut-over plan will help keep the activities on task and allow the proj-ect manager to assess the progress of the work, create workarounds for problematic situations, coordinate with the plant operations, and drive
the project to completion. A cut-over plan, at minimum, should include the I/O to be cutover and tested (includ-ing the order in which they are to be tested), any water testing through the process to verify configuration on the live plant, and the actual order of the first products to be run on the unit.
6. Devise a roles-and-responsibil-ities matrix. Defining the roles and responsibilities of all personnel and contractors involved in the project is key to delivering a successful project. By putting together this matrix and using it as a pre- and post-training ref-erence for all staff, everyone involved will understand their responsibilities and perform the appropriate work. 7. Get management involved. Management at various levels, includ-ing upper management, needs to understand what is involved in the startup process and why it is criti-cal to delivering on management’s expectations of the batch process facility’s operations. Communica-tion and internal buy-in throughout the organization are very important aspects to a successful startup, and management’s visible support and connection to the project is critical to these aspects.
TIPS FOR SUCCESSFUL PROJECT DEVELOPMENT
8. Be thorough in examining out-side support. Be sure to determine if outside personnel, such as system integrators, have experience in your industry. Is their knowledge transfer-able to the project? Evaluate their background and capabilities. What is
the range of services they provide? Are there any commercial issues out-standing? Check references. Consider cost, but understand that the lowest bid is not always the best. A good resource for companies looking to hire control system integrators is the
Con-trol System Integrators Association,
www.controlsys.org. This organization not only validates industry expertise, but also supports dependable busi-ness practices by its system integrator members.
More than technical skills are required to successfully manage an automation project. It also requires communication and organizational skills, along with the ability to motivate a team of people from a variety of disciplines and differ-ent departmdiffer-ents.
Here are a few practical tips for auto-mation project managers:
1. Project management resource. There have been thousands of words written about project management. If you think you need a refresher course, or expect to be assigned to your first project, there’s an organization, Project Management Institute, dedicated to establishing standards, providing train-ing and certifytrain-ing individuals in project management skills.
2. Welcome the bad news. Every automation project has things that go wrong, but the earlier you find out what the problems are, the easier and cheaper they are to fix. Nobody wants to hear or deliver bad news, but it’s im-portant not to get defensive. Anybody on the team needs to be able to push the stop button if a project has gone off the tracks. Otherwise, you’re just gambling that things will come out all right at the end.
3. Keep simplicity top-of-mind. Engineers tend to make systems too
complex for non-engineers to deal with. Make sure expectations are es-tablished early that will keep the needs of the people who will have to operate and maintain the systems a priority. Include people from these functions on the automation team and consult them early in the design and testing stages for new systems and equipment. 4. Be ready to adjust. As with any project, unrealistic projections, poor ex-ecution and just plain bad design can cause a project to fail. What is impor-tant is that when you begin a project, understand that there will be modifica-tions necessary along with way. The final result is rarely as exactly planned. This is not considered a failure; it’s a realistic need to adjust and fine-tune as the project progresses.
5. Establish testing plans early. It isn’t enough to design a system. You
have to test it to prove that it works, not once but twice. It’s easy to get started on designing the tests by using a template. Equipment or systems should first be tested at the facilities of the integrator or OEM. This is called FAT (Factory Acceptance Testing), and its goal is to prove that the system design will work. Simulate various scenarios to find out how the system will react. The final testing stage, SAT (Site Acceptance Testing), is done when the system is delivered to the factory floor. Its objec-tive is to prove that the equipment actually does work as designed and is producing product at the level re-quired. Approve the testing plans early in the project so that everyone knows exactly what performance measures they need to achieve. Don’t rush the testing phase; make sure you leave enough time in the project schedule to accomplish the necessary tests. It’s
Nine Tips for
Automation Project Managers
Looking for training?
There’s a source to help you learn
the ropes of project management
or improve your skills.
http://awgo.to/028
also important to make sure the right people attend the FAT; that includes the lead operator and maintenance tech, not just the manager.
6. Follow programming standards. Make sure that in-house programmers, system integrators and OEMs use the same PLC programming standards, such as OMAC and PackML. There’s nothing worse than custom code that has to be reworked at the last minute to make it compatible with a plant’s existing systems. Multiple approaches to programming can cost a company millions of dollars.
7. Communicate often. Don’t make decisions without consulting the team. Unilateral decision-making alienates the team, creates confusion and fails to take advantage of the unique expertise of the team members. Foster open communication and communicate fre-quently, so that everyone on the team understands the issues and is aware of any problems that need to be resolved. Establish a communications roadmap for vendors; check with them soon into the project to make sure it’s working. 8. Don’t be a roadblock. As project manager, it’s your responsibility to respond to information requests and
approve various aspects of the project in a timely fashion. Stay involved and be responsive to prevent delays in the project’s timeline.
9. Make sure you have bench strength. There’s nothing that delays a project more than a team member who gets assigned to another project and no longer has the time to devote to your project. Identify alternative resources early and have them ready to fill in if needed. That same rule applies to the system integrator’s team; make sure they’ve identified people with equivalent skills who can be assigned to the project if required.
Lifecycle Methods
Waterfall
Agile
Spiral
Design V
Other
Development
Factory
Acceptance
Test
Site
Acceptance
Test
Requirements
Design
Development
Specification
Subsystem
Unit/Module
Unit/Module Test
Integration Test
Close
Internal
Kickoff
Traceability
Project management “V” model, courtesy Control System Integrators Association.
Imagine a world without electrical standards, such as 110 V at 60 Hz, or 220 at 50 Hz, or a world where every phone had a different type of connec-tion and required a different type of switchboard. Just as these standards are critical to the basic functioning of electrical equipment, there are also IT standards used daily to ensure optimal functioning of production systems in the process industries.
There are four production-related IT standards of special interest to the processing industries:
•
The ANSI/ISA 88 standard on batch control;•
The ANSI/ISA 95 standard for MES and ERP-to-MES communication;•
The ANSI/ISA 99 technical reports in industrial cyber security; and•
The new ANSI/ISA 106 technical report on procedure automation. These standards and technical reports define the best practices forimple-menting automated and manual con-trol on the systems that reside above the PLC (programmable logic con-troller) and DCS (distributed control system) level, and which perform the basic control that keeps production running. These four standards all share a common view of a production facil-ity, providing a consistent terminology that makes it easier to compare plants within a company and across compa-nies.
The ANSI/ISA 88 standard defines the most common and effective method for defining control systems for batch operations or for continuous and dis-crete startups and shutdowns. The ANSI/ISA 95 standard defines the most commonly used method for exchanging information between ERP systems, such as SAP or Oracle, and the multitude of shop floor systems. It has also become the de facto stan-dard for defining MES (manufacturing execution system) and MOM
(manu-facturing operations management) specifications.
The ANSI/ISA 99 reports define struc-tures and policies for designing effec-tive and secure networked production facilities.
The new ISA 106 reports define the procedural control strategy for continuous production during upsets, switchovers, and other types of pro-cess changes.
Because these standards establish a commonly accepted terminology, functions and process models by which technical professionals are trained, and upon which solution providers develop applications used in batch and process production operations (as well as discrete manu-facturing), they should be of particular interest to those who are new to the field and those who seeking a refresh-er on the fundamentals of industrial processes.
Four IT Standards
AUTOMATION PROJECT SURVIVAL GUIDE
Four Considerations
for Upgrades & Migrations
Regardless of whether you want to increase productivity or shorten time-to-market, attaining success in these areas depends on the application of suitable automation technologies in a batch processing plant. Following are the principal steps involved in assess-ing your plant’s technology to gauge whether a technology upgrade or migration is in order:
1. Consider the full range of aspects that relate to your existing systems, such as:
•
Risk of unplanned plant downtime and production stoppages;•
Ability to expand production or introduce new products;•
Ability to integrate with enterprise-level business software and at what cost;•
Ongoing maintenance costs;•
Need for continuing support of the legacy system; and•
Effect on the efficiency and produc-tivity of plant personnel.2. In each case of upgrade or migra-tion, return on investment plays a crucial role. A huge investment in hardware and application software is associated with the installed process control system, as well as the accu-mulated know-how of the operating, engineering and maintenance person-nel. For this reason, the prime objec-tive of any migration strategy should be to modernize the installed base gradually without any system discon-tinuity and, if possible, without any plant downtimes or loss of produc-tion that would negatively affect the investment return.
3. Assess the long-term security of existing investments. This assessment is important in order to maximize the return on assets (ROA). For this rea-son, every migration should include a robust lifecycle support strategy for the new system that considers not only the availability of the components, but also product warranties, on-site service and ongoing technical support.
4. Obsolescence. When deciding whether to upgrade or migrate to a
new system, there are two aspects of obsolescence to assess. In a migra-tion, it’s important to understand the history of the technologies supported by the company behind the product under consideration. Does this com-pany actively support the long-term lifecycles of products as they are typi-cally employed in a process opera-tion? Do upgrades have significant backwards compatibility? How often are upgrades typically released for this system and what is required for instal-lation? For upgrades, it’s important to understand what the future outlook is for the system under consideration. With the significant maintenance and security issues tied to process control systems, you should always consider your risk of system obsolescence and the associated costs incurred with such a scenario versus the costs of moving to a better-supported system. The good news is that, in the process industries, most vendors are very aware of the long-term use of their systems by end users and thus tend to support their systems for multiple decades rather a single decade, as is more common with office IT systems. As newer automation technologies become core components of process control systems, be sure to talk with your supplier about their support plan for those newer technologies.
Implementing a new distributed con-trol system is one of the biggest and most complicated projects in a pro-cess control engineer’s career. Doing one successfully requires everything from a well-defined project document to good grounding practices. Here are recommendations for best practices and some pitfalls to avoid.
1. Standardize. Use of standard wir-ing throughout the system will make it for easier for others to understand and troubleshoot. Use standard, off-the-shelf components for ease of stocking and reordering. If possible, have two sources for the products being used or purchase interchangeable brands. 2. Remember the basics. It’s the little things that can trip you up. Make sure you use proper grounding, proper grouping of signals and proper termi-nation of electrical signals. Make sure you understand the supplier’s ground-ing requirements for your DCS system. Grounding principles need to be clearly understood by all automation engineers, not just the electrical staff. International standards can be misin-terpreted. Instruments and the control system need to be grounded separate-ly. Double check the grounding before powering up any DCS system to avoid any short circuits, particularly during
factory acceptance or site acceptance testing (FAT/SAT).
3. Is communication complete? While most automation suppliers have different software versions for com-municating with the system, make sure they will transmit all the required information. Many systems only transmit the basic parameters, which means all diagnostic features will not be available. The introduction of the “Control in Field” concept, although not often used, has added some com-plications and needs to be thoroughly examined when implementing a DCS.
4. Structuring I/O. Since today’s electronics are available with high temperature specs and may be G3 compliant (conforming coating), the I/O structures should be moved to the field, reducing the rack room footprint and cabling cost. Communication links should be used over fiber optic, in a ring configuration to provide some level of redundancy, to interconnect the field I/O structures. Extended I/O terminal blocks (three to four termi-nals per channel) should also be used to allow field wiring to be connected directly, avoiding marshaling terminal strips with the related space,
addi-Eight Ideas for
Successful DCS Implementation
Successfully implementing a DCS project requires that all stakeholders
(operations, maintenance, project team, vendor, management, etc.) have
a clear definition of what they want from the system. In both upgrading
and installing new DCS systems, the best tip is to keep the end in mind.
Good up-front engineering pays dividends. Automation technology can
only assist us if we know what the needs are. Maintenance must know
what reports and information they really require to do their work.
Opera-tions must be completely sure how they operate and what is the best
way to do it. Don’t assume anything. Write everything down that’s
actu-ally required and all the things the technology can do. Be very specific.
In the end, the best DCS is the one that best satisfies all the important
requirements in the plant. Writing and signing this definition document
should be the first step in any project.
tional cost, installation cost and the possibility of poor connections. 5. Dual purpose. The purpose of DCS is twofold. Centralized human control and interface to the plant as well as a centralized location for MIS info to the management network. DCS control should not include auto tuning of control loops other than simple on/off or start/stop functions. These should be the function of a local dedicated controller. Use the DCS to update the tuning parameters.
6. Good links. Distributed control sys-tems are only as good as their commu-nications links. Choose a very solid and reliable link between processing units. 7. FAT is where it’s at. Make sure you do a comprehensive and detailed factory acceptance test-ing (FAT) test before cutover. FAT involves experienced operations people interacting with engineering to validate graphics and verify that instruments in the configuration exist and will remain in service.
8. Use single server. Base the selec-tion of a DCS system on its redundant capability. A single server system is preferred. Pay attention to the hard-ware license for client and server to avoid delays during a system or hard disk crash. Care must also be taken in selecting appropriate layered switches for communication. Make sure you properly configure trends and history data for future analysis.
13 Suggestions for
Control System Migrations
As anyone who has been involved in a control system migration will tell you, it’s never an easy process. Whether it’s an upgrade, expansion, stepwise migration or rip-and-re-place, the bigger and more complex the project, the more fraught with tension and risk. To help you get through the project with your sanity intact, Automation World readers share their recommendations and suggest pitfalls to avoid:
1. Determine strategy. Your migra-tion strategy will depend on which type of automation you’re dealing with: scripts, workflow tools, policy-based orchestration, configuration or control systems. The different activities that can be automated (provisioning, maintenance, proac-tive incident response, production execution, etc.) and the different degrees of automation (automating just a few actions, partial workflows or end-to-end) will determine your strategy in terms of resources, time scale, production stops, etc. 2. Virtualize first. Automation upgrades or migrations need to be scheduled properly in terms of sys-tem commission date to extend the warranty or for a vendor’s obsolete notice date. The best practice is to
conduct a virtualization of the new automation system. The future of automation will need virtualized infrastructure and platforms to deal with the IT spectrum, cyber security and better management capabili-ties. Virtualization has many benefits in terms of technology, investment, maintenance and lifecycle cost. 3. Take it one step at a time. Avoid changing the entire system or manufacturer if you are upgrading. Upgrading to the newer modules or systems of the same vendor provides a bit more reliability, since the basic architecture remains the same. 4. Don’t experiment. While innova-tion is important, there is a counter-argument for doing what you know will work. If rip-and-replace is pos-sible (and that means you have to stop the plant for several days, weeks, or months depending on the circum-stances) and you know that it works, that is the best choice. But if you can’t afford a shutdown, then go for a step-by-step migration. Make sure you work with an experienced vendor and proven technology.
5. Three critical migration issues. When doing a migration there are three points to think about: how to update software and whether you
have the right conversion tools; what you need to do to avoid system failure or risk for the migration step; what is the expected lifecycle of the new system.
6. Make no assumptions. Try to foresee every small step in a migra-tion implementamigra-tion. Don’t assume anything. Every implementation is done to achieve some objective of the operation. The needs could range from some reporting or alarm functions to an action initiated due to alarm. Always visit the site to understand the requirements and the nuances completely.
7. Changing horses adds some complexity. The difficulty of a process migration usually increases when you change DCS suppliers since different brands often don’t have similar functions. Factor that into your timeline and risk assess-ment when weighing whether to switch vendors.
8. Start with data needs. First you need to understand what data the user will require and how quickly the data is needed. That should be the starting point in developing your migration strategy. The second prior-ity is to determine the impact on the safety and productivity of the plant.
13 SUGGESTIONS FOR CONTROL SYSTEM MIGRATIONS
9. Focus on controllers.The best strategy is to first upgrade the con-trollers, then replace the I/O chassis piece-by-piece going forward. Some I/O changes could be driven by other projects, such as a motor control center(MCC) replacement.
10. Do your homework. Do some up-front analysis to avoid creating problems for yourself by not choosing the right migration path. For example, migrating from one generation of processor to another one may not be a wise choice. Reviewing the instruction sets and information available about
conversions and manufacturer recom-mendations will give you insight into the difficulty of the conversion. If you do your homework, you might choose a different processor to make the conversion easier.
11. Technology education. It is important to educate everyone on the new technology. Remember, it is easy to use “old” thinking instead of chang-ing practices to take advantage of the benefits of the new technology. 12. Decentralize. The architecture has to be critically reviewed and
trans-formed, keeping in view the improved performance of the local controllers. Your mantra should be to decentralize the controls as far as possible.
13. Aging equipment. Depending on the technology you have installed, when your equipment is more than 10 years old you will need to imple-ment a rip-and-replace. If you are just making some modifications you can upgrade or make an expansion only. Most of the problems that arise during a migration are with the field equip-ment you have installed and control room facilities.
When developing HMI screens, realize that you are attempting to capture the essence of the machine or pro-cess, not just posting key automation variables and control mechanisms. Operational feedback is vital for ef-ficient HMI screen layouts. Think of yourself as an artist, commissioned by manufacturing operations to cre-ate the HMI screens.
1. Less is more. It’s important to keep the HMI simple and with the operator in mind. It’s best when it’s self-explan-atory and easily understood. Also, try to make the pages similar and follow the same page layout throughout. Avoid making the display too techni-cal. It’s normal for engineers to try to give the customer everything, but with HMI, less really is more. 2. Right-size displays. Don’t try to save money by selecting an HMI display screen that’s too small. It’s also important not to cram too much information onto a screen. Size the display according to the amount of information that is most important for the operator to see. Always discuss requirements with the equipment’s operators well ahead of time, not just with their managers. Operators usually have different needs and the success of your system depends on their usage. 3. Design tips. A good design requires careful use of layout, color
and content. If you get it wrong, your operator misses an indication, you lose money, or worse, someone is injured. The ”bad” screen is less than satisfactory: The layout is poor, the plant representation isn’t logical and the screen layout makes it difficult to locate the data. Poor selection of colors, excessive use of capitals in a serif font and repetitive use of units with all data values makes this a really difficult screen to read—especially at a glance or from a distance. Avoid colors that could create problems for people with color blindness. Minimize the use
of colors to allow actual device state and alarms to stand out. For alarming, choose colors that contrast with the normal process view so the operator will notice the change.
4. Plant review forum. Hold a design review with a group of plant person-nel to discuss any status notifications, events, alerts and alarms that need to be programmed, both from the per-spective of an audio-visual action and an operations response. Step through the intended functional system, once as the designer, once as the user and
then invite at least two levels of users who will be interfacing with the HMI. Doing this prior to specifying equip-ment helps to identify the features that users will want in the HMI station. It also avoids surprises at point of commissioning.
5. Location, location, location. Real estate can be prime in a busy produc-tion area. Locate the HMI in a practi-cal place, out of heavy traffic areas but accessible. Be aware of near-future projects in the area. Guard the HMI location so others don’t park or configure something else on top of the station.
6. Back up work periodically. Back-ups are especially important before implementing upgrades or changes. Software such as Norton’s Ghost
Im-age can be invaluable to support and maintain HMI systems.
7. Visualize the process. HMI graph-ics should illustrate the production process in the plant to provide better visualization to the operators, giv-ing them a sense of the action that’s required. Use hardware that meets minimum requirements and keeps the number of failure points low and as-sures high availability of the system. 8. Only essential data. Make control and monitoring of the process simpler by selecting only the most essential information from the process data-base for the historian. This will reduce the load on the system and keep it from stalling or failing. Don’t forget the need for maintenance and make sure you schedule periodic backups.
9. Think about flow. It is essential to have a clear design approach to the HMI. Decide how the display blocks naturally flow and how sections need to be grouped together for the operator. Do not blindly follow P&I diagrams. The S88 functional hier-archy is a good place to start. Make paper-based designs to get a feel for screens, navigation and other requirements, and review with clients prior to designing and making elec-tronic screens.
10. Alarm strategy. Alarming needs to have a well-articulated strategy. Alarms must be used for conditions that require intervention and must have a clear corrective action associ-ated with each one. Anything else should not be an alarm.
Safety: The Lifecycle Approach
Production safety is generally thought of as a series of steps necessary to ensure safe interaction with industrial equipment. The process of identifying, agreeing upon and delineating those steps is where things tend to get complicated. That’s why international standards groups play such a signifi-cant role, as they set the guidelines for all of industry to follow.
For the process industries, IEC 61511 is probably the most widely used safety standard, as it applies to those indus-tries that base their safety systems upon instrumentation. The goal of safety-system design in IEC 61511 is for the process, whatever it may be, to go to a safe state whenever a process parameter exceeds preset limits. A New Way of Approaching Safety Understanding IEC 61511 means that you must know a thing or two about IEC 61508 — a functional safety stan-dard that provides the framework for building industry-specific functional standards. IEC 61511 was created from the guidelines established by IEC 61508.
The key point to understand about IEC 61508 is that it is designed to establish an engineering discipline that will generate safer designs and build safer processes. The uniform procedures built on these disciplines are contin-gent upon appropriate experts within
a company contributing to projects. In addition, the standard also makes it easy for outside auditors and govern-mental agencies to follow the process. IEC 61508 can seem confusing at first, because its underlying philosophy is new for safety standards. Older, more conventional safety standards stipu-lated specific rules and specifications for making processes safe. IEC 61508
and its derivative standards, such as IEC 61511, departed from this ap-proach by being more functional, or performance-based.
A principal aspect of this new ap-proach to safety standards is that it leverages two fundamental principles: safety lifecycles and probabilistic failure analysis. Unlike previous stan-dards that claimed to cover the entire lifecycle of a project, IEC 61508 and its
offshoots actually do—from project conception to maintenance to decom-missioning.
In essence, the standards specify safety lifecycle activities that need to be fol-lowed over the entire life of a produc-tion system. Safety lifecycle manage-ment provides a method or procedure that enables companies to specify, design, implement and maintain safety systems to achieve overall safety in a documented and verified manner. Four Phases of The Safety Lifecycle The IEC 61511 standard promulgated by the International Electrotechnical Commission specifies twelve steps in the safety lifecycle. These are seg-mented into four phases: analysis, realization, maintenance and ongoing functions.
Safety Lifecycle I: Analysis Phase
The analysis phase includes the initial planning, identification and specifica-tion of safety funcspecifica-tions required for the safe operation of a manufacturing process.
Specific activities include:
•
Perform hazard and risk analysis: Determine hazards and hazardous events, the sequence of events leading to a hazardous condition, the associated process risks, the requirements of risk reduction and the safety functions required.SAFETY: THE LIFECYCLE APPROACH
•
Allocate safety functions to pro-tection layers: Check the available layers of protection. Allocate safety functions to protection layers and safety systems.•
Specify requirements for safety systems: If tolerable risk is still out of limit, then specify the require-ments for each safety system and their safety integrity levels.Safety Lifecycle II: Realization Phase
The realization phase not only in-cludes design, installation and testing of safety systems, but also the design, development and installation of other effective risk reduction methods. Spe-cific activities include:
•
Design and engineer a safety system: Design system to meet the safety requirements.•
Design and develop other means of risk reduction: Means of protec-tion other than programmable safety systems include mechanical systems, process control systems and manual systems.•
Install, commission and validate the safety protections: Install and validate that the safety system meets the all safety requirements to the required safety integrity levels.Safety Lifecycle III: Maintenance Phase
The maintenance phase begins at the start-up of a process and continues until the safety system is decommissioned or redeployed. Specific activities include:
•
Operate and maintain: Ensure that the safety system functions are maintained during operation and maintenance.•
Modify and update: Make correc-tions, enhancements and adapta-tions to the safety system to ensure that the safety requirements are maintained.•
Decommissioning: Conduct review and obtain required authorization before decommissioning a safety system. Ensure that the required safety functions remain operational during decommissioning.Safety Lifecycle IV: Ongoing Functions
Certain functions are ongoing. Ex-amples include managing functional safety, planning and structuring the safety lifecycle, and performing pe-riodic safety system verification and safety audits over the whole lifecycle. Specific activities include:
•
Manage functional safety, safety assessment, and safety audit: Identify the management activities that are required to ensure that the functional safety objectives are met.•
Plan and structure safety lifecy-cle: Define safety lifecycle in terms of inputs, outputs and verification activities.•
Verify safety system: Demonstrate by review, analysis and/or testing that the required outputs satisfythe defined requirements for each phase of the safety lifecycle. Activities for Phases I to III are typically carried out consecutively, while Phase IV runs concurrently with the other phases. However, like all models, the safety lifecycle is an approximation.
Bottom Line:
A Requirements Definition Readers should note that the stan-dards define requirements for safety management, rather than system development. Not all safety lifecycle phases will be relevant to every ap-plication; management must define which requirements are applicable in each case. The standards do not prescribe exactly what should be done in any particular case, but guide management toward decisions and offer advice.
Control System Security Tips
Recognizing that the biggest security risk to your control system assets are the operators who interface with the system on a daily basis is the most im-portant step to successfully securing your systems. For a thorough analysis of your risks and setup of reliable control system security technologies and processes, consult an industrial control system security expert such as
scadahacker.com, tofinosecurity.com, or industrialdefender.com. Following are the ground level security steps that a batch process facility should implement at a bare minimum: 1. Assess your systems. Compile an accurate list of all the assets in your plant: make, model, and serial number. Where are your computers? Where are your PLCs? It’s difficult to secure some-thing when you don’t know it exists. This should be a high-level assessment in which you go through your plant and figure out what is high risk and what is low risk, which is determined by two key factors: how likely is a problem to occur? How serious is the problem? For example, if something happened to your chlorine tank, it would be really ugly. That chip pile, not so ugly. Get a feel for the signifi-cant risks. Where do you have to focus your effort? The answer is going to drive your decisions and your capital allocation.
2. Document your policies and procedures. No company operates in a vacuum. Each company will have a series of policies and procedures for things like safety and performance, re-liability, and change management. Lay those out and understand how they impact control systems and security, and then build on that to create a set of additional security requirements. 3. Start training. No one is going to follow policies unless they know about them and understand why they are necessary. All levels of employees that interact with the control system need to understand what an attack looks like and how to respond to one. You should end up with a matrix of training for the various levels of users; it doesn’t have to be onerous, but it has to be done.
4. Understand your traffic flows. You need a diagram that shows all the things that require intercommunica-tion. Smart companies will have a comprehensive diagram showing that the accounting department needs data out of this area, and maintenance needs data out of this area, and so on. 5. Remember that SCADA security is used to control access. Access should be segmented to specific net-work resources, hardware resources, and HMI. Effective security practices should prevent access to all layers by unwanted external connections. 6. Leverage safety reports. Those responsible for safety, when they do reports and analyses, have done a good deal of the work needed to understand the security risks. 7. Use separate networks. Though this step is becoming less and less practical, some still advocate that the process control network be kept separate from business networks, and also isolated from the Internet. For this approach, which may not be viable in the longer term, utilize operating system (OS) implemented security, with active directory “domain group security” as the preferred approach. 8. Security in the operator interface should be considered broadly. With advanced human-machine
CONTROL SYSTEM SECURITY TIPS
interface technologies, security can be implemented for individual attributes. HMI should be the only accessible program, with user-specific excep-tions, connected to the control operat-ing system at a dedicated user station. All other resources for that particular terminal should be restricted. 9. Use unique user accounts and passwords. All users should have unique user accounts and passwords to minimize the risk of unauthorized access.
10. Provide port security. With this approach, the Ethernet MAC address connected to the switch port allows only that MAC address to communi-cate on that port. If any other MAC
address tries to communicate through the port, port security will disable it. Most of the time, network administra-tors configure the switch to send an SNMP trap to their network
monitor-ing solution that the port’s disabled for security reasons. When using port security, you can prevent unwanted devices from accessing the network. 11. Administer antivirus protec-tion. Use an antivirus solution that is compatible with the installed SCADA software.
12. Open and facilitate commu-nications between IT and process control groups. Roles need to be defined and an understanding of what each group needs must be ac-complished so true collaboration can take place to begin and continue the process of enabling a fully functional control system with adequate security protection.
How to Avoid Mistakes
with Control System Remote Access
As more operations aspects are tied to Ethernet networks and, therefore, are open to Internet-based access, the potential for greater collaborative op-eration and a freer work environment increases. But so does the potential for security problems. Following are some basic tips and considerations for achieving secure and reliable remote access:
1. Map out your project from the start. When companies fail to map out their projects thoroughly from the start, they often find themselves saddled with applications and auto-mation products that don’t work co-hesively as a single system. Once you
start implementing various silos—be they applications or products—things get more complex. This is typical of problems that occur when automation products are implemented hastily, without doing proper research, plan-ning, or analyzing current and future goals, or without realizing that imple-menting remote access monitoring for a facility is just step one of many. 2. Anticipate network interactions. When people have installed devices on a proprietary network then try to use something different (e.g., Wi-Fi or another protocol), individual systems may conflict. Or they may just cancel each other out, so that there is no
communication whatsoever. More often you find yourself managing so many different applications, protocols, and systems that you have more work and headaches than you imagined possible. This issue can be avoided if you select a network that is open and allows everything to work together. 3. Understand users and roles. Understanding users and their roles can have a significant impact on how the remote access strategy evolves. In most control systems operations, the roles that may require remote access to control assets may include, but are not limited to:
•
System operators and engineers for local systems;•
System operators and engineers for remote systems;•
Vendors;•
System integrators;•
System support specialists and maintenance engineers;•
Field technicians;•
Business/supply chain partners;•
Reporting or regulatory entities; and•
Managed service providers. The roles of users that would require remote access to mission-critical opera-tions can be extensive and theassign-HOW TO AVOID MISTAKES WITH CONTROL SYSTEM REMOTE ACCESS
ment of specific access depending on those roles can be complicated. Map out and document all acceptable ac-cess policies and procedures related to allowable network access and coordi-nate this with industrial control system security experts. Any user access that goes beyond simple viewing of data and permits changes to system param-eters should be extremely limited. 4. Know your vulnerabilities. Begin-ning at the remote user and following the connection to the data or service, remote access can be compromised at any of the following points:
•
The user or system can be imper-sonated to fool the target system.•
The attacker can use captured or guessed credentials to impersonate the user.•
The attacker can intimidate or coerce the user to provide valid credentials, or to perform activities at the attacker’s demand.•
The user’s access device (laptop, PDA, etc.) can be attacked,com-promised, and used to access the control system network.
•
The target system can be imperson-ated by an attacker to fool the user and thus gain credentials or other information from the user system.•
Communication can be listened to by third parties anywhere along the communication chain.•
The communication can be inter-rupted or jammed.•
Communications can have data injected into them by an attacker.•
Communication can be hijacked after it has been initiated (does not rely on impersonation) or intercept-ed during initiation (impersonating both user and target, also known as a man-in-the-middle attack).•
Parts of a communication can be replayed to a target, even if the at-tacker cannot decipher the content (also known as a replay attack).•
The target communication soft-ware listening for requests can be attacked and potentially compro-mised.•
An attacker can impersonate a valid communications node and gain access to the underlying communi-cations medium.•
A denial-of-service attack can hap-pen to the authentication server (e.g., radius server or RAS).•
A denial-of-service attack can hap-pen to the outward communication device (e.g., an outside router for remote access). Four Tips for Dealing with Wireless
Latency and Bandwidth Issues
More and more, systems engineers are taking advantage of industrial wireless technologies to reduce the amount of cabling in their designs. There are some issues to be aware of, however, when replacing dedicated connec-tions with wireless links:
1. Need latency tolerance. Today’s wired Ethernet connections are full duplex. This means that each end device can both transmit and re-ceive at the same time. On the other hand, wireless technologies such as 802.11a/b/g/n are half duplex. This means that when any one device is transmitting, all other devices must wait. Make sure that your applica-tion is designed to be tolerant of the latency introduced due to the half duplex nature of wireless.
2. Control multicast traffic. When im-plementing wireless technology in fac-tory automation projects, be aware of any multicast traffic coming from PLCs or producer devices. Multicast traffic is handled differently than unicast traffic by wireless access points. Multiple devices can receive multicast traffic, while unicast is destined for only one
device. Wireless access points transmit multicast traffic at a minimal rate to en-sure that all listening clients will be able to receive the traffic. This results in low aggregate bandwidth over the wireless AP as it has to lower its transmit rate down from the maximum.
3. Low bandwidth requirements. Make sure that your application’s bandwidth requirements are low enough to be satisfied by the lower rates. Many designers overlook these points and experience problems when moving to wireless solutions. Being aware of the limitations of wireless technology can ensure that your
upfront design will work in a wireless deployment.
4. Don’t take shortcuts with wire-less. Consider the entire system design and the support lifecycle of the system before choosing technology and vendors. Time spent up front on site surveys, path loss calculations and fade margin will pay dividends when it comes time for installation. Design in fade margin. Wireless is very reliable when well designed, but if you don’t design in appropriate fade margin you’ll have problems in the future.
AUTOMATION PROJECT SURVIVAL GUIDE
How to Properly Select
and Vet a System Integrator
The process of finding a qualified system integrator for your automation project requires effort and attention to the details. Experience, expertise, staff capabilities and financial wherewithal are all crucial factors to consider in finding the right integrator partner. 1. Selection criteria. Search for a system integrator who has a long list of successful projects in the areas you are looking for. Check out any refer-ences they provide and find out how long they have been in the field. They should also have a broad range of products they have worked with and have enough staff to handle all the various areas of a project. People who have done a lot of motion control may not have the expertise to handle a complex SCADA project.
2. Be suspicious of over-promises. If during negotiations and setting requirements, a system integrator continues saying, “No problem. That’s easy. We can do all you want”... you can be sure that It will be a problem, it will not be so easy and It will be something that is more complicated than assumed. The integrator should prove that he understood your re-quirements, didn’t underestimate the project and that he has experience with similar projects. Be especially
careful if you get a much lower price than expected or than others have quoted.
3. Familiarity with standards. Find out what partners the integrator works with since no one can do it alone. It’s also important to see how an integrator manages a project and what their code library looks like. Do they follow S88 and S95 methodolo-gies? They don’t need to follow these to the letter, but if they don’t have a methodology and aren’t even aware of the standards, don’t even consider them.
4. Comfort factor. In addition to reli-ability and professional capabilities, choose an integrator you feel comfort-able with, who understands your pro-cess needs and who has experience in
the field. The integrator also needs to have a staff with expertise and domain knowledge in your business area. 5. Expertise. Focus on their knowl-edge, techniques and skills. Make sure they have full knowledge of system engineering, as well as sufficient experience to handle your project. A proven track record and references from the projects they have done are essential.
6. Current experience. Prior experi-ence in your discipline is key to the selection of your Integrator. Experi-ence keeps the integrator current on new technologies and new hardware and software. As a result of the recent recession, integrators are not as abundant as before, with many un-able to survive the economic turmoil.
Extensive planning is complete, timelines and schedules are determined,
budgets and ROI calculated and all the textbook preparations and
considerations have been met. What could go wrong? Plenty! Always
vet your system integrator. Get references, see a system designed and
implemented by them in use, visit their factory and, most important, run
credit checks and investigate their financial health. Nothing is more
de-structive than having an integrator run out of money before the project
has been completed.
Many integrators have reduced staff, minimized technology education op-portunities and made other cutbacks. Take the time to assess the strengths and weaknesses of any integrator you consider to ensure that they are capable of delivering the system that you require.
7. Stay involved. Has your system integrator done something similar before? Chances are the pool of tal-ent isn’t all that big. Can you allocate any resources to working with that
integrator on a day-to-day basis? You will have to take ownership of the system, so you will need to know how to modify it and maintain it or you will be tied into a system that might need unallocated cash to make changes. Get involved at the zero level in the planning, simulation, detailed layout, software handling techniques and maintenance requirements as much as you possibly can in order to get the biggest possible benefits and to learn in excruciating detail how it all goes together.
8. Take a long-term view. Select an integrator with experience in similar systems, preferably of the same make. Tie payments to project milestones. Make sure his services will be avail-able for upgrades and maintenance by signing a separate contract.
9. Problem-solvers. Choose an integrator who has experience in the tasks you need performed. They have probably already solved many of the problems you may face if you choose one whose experience is outside the necessary area of expertise.
10. Ask questions. Choosing a system integrator is the hardest and easily the most overlooked part of an automa-tion project. Ask quesautoma-tions about types of projects they’ve done, vertical preferences and size of projects. Have them include project details, such as were they on time and on or under budget, and what percentage of the time.
11. Experience has its limits. Be aware that most integrators have experience either in a vertical industry or with a certain type of project, such as PLC/HMI programming. Either way, they may lack the capabilities needed to do projects outside of that experi-ence. Many HMI/DCS vendors have a list of endorsed or recommended sys-tem integrators on their home page. This is a good place to start.
12. Smart isn’t enough. Choose an integrator as you would choose an employee. Spend time, talk to
refer-1.
One of the most important factors in selecting a system
integra-tor is his willingness to develop a good project proposal. Avoid any
integrator whose proposal is just one or two pages long.
2.
Automation projects must have good system requirements from
the customer, and the system integrator must list in his proposal
what requirements will be met and what will not.
3.
If the requirements and proposal terms are properly defined
from the beginning, the result will be a project with no or minimum
change orders.
4.
Some system integrators take advantage of a poorly written
requirements document from a customer and present a very generic
proposal, so the price might look attractive at the beginning. When
the project is awarded, then the customer has to face a series of
change orders because a requirement that might be obvious was not
listed in the proposal. The customer ends up paying far more money
for the project than originally estimated.
5.
Establishing a good project requirement list is not only an
essen-tial customer task, but also requires the cooperation of the system
integrator.
HOW TO PROPERLY SELECT AND VET A SYSTEM INTEGRATOR
ences and know that while every firm out there enlists very smart engineers, you don’t want them cutting their teeth on your project.
13. Professionalism counts. Make sure an integrator can confidently provide you with a project plan, with decision points, contingency plans and staffing that will meet your time-line and project goals.
14. Test the team. Verify the integra-tor’s capabilities by giving a test to the personnel who will perform the work on your project. Make sure those peo-ple are listed in the contract, including fallback or substitute candidates. 15. Do they have business skills? Look beyond technology expertise or project experience to consider an
integrator’s commercial qualifications: Are they CSIA certified? Do they have insurance? How many years have they been in business?
16. Are they open? Select an inte-grator that is open to your requests and ideas. Beware of someone that constantly pushes back. If you hear
the phrase “nobody does it like that” or “this is how everyone does it,” you might want to consider another integrator that is more open minded. You are paying that integrator to get what you want and need—not just what they are willing to build because it’s easy or they “always do it that way.” Yes, you hired them for their experi-ence and would like their suggestions, but don’t discount your own ideas just because this is your first time. Also allow for the ability to make some changes—especially if your approach is new and unconventional. Be open for changes and tweaks as you go if it makes the end result easier to use and more flexible. You need to stay involved throughout the whole process. Don’t pass up the learning opportunity!
