API Management Introduction and Principles
by Vijay Alagarasan, Principal Architect, Enterprise Architecture and Strategy of Asurion
Abstract: This article is focused on providing solutions for common problems faced by large enterprises
implementing SOA when they try to transform digitally as the market demands. It covers pain points commonly faced by such enterprises, and how API management product capabilities can be used to eliminate issues. In addition, the article also provides a set of principles to keep in mind when implementing API management for an enterprise.
Challenges faced by the Enterprises as they Transform Digitally
Today, enterprises that have adopted SOA in the past want their information assets and capabilities exposed without any boundaries as they transform digitally. Change at the speed of innovation drives enterprises to be flexible, adaptable, and accessible, with varying levels of security. Product development teams are aimed to get the new products into the market within a few weeks, and are prepared to fail fast and go back. On the other hand, service providers are trying to meet those new demands and timelines.
Large enterprises are desired to transform digitally and improve their agility in responding to market feedback. They want to get new features into the market within a week or so, just like startups. Enterprises want their minimal viable product (MVP) out before their competitors, and deliver the changes/features incrementally. This style of working demands small independent releases and favors A/B experiments in order to keep up with customer satisfaction.
While experimenting with new techniques, they must somehow preserve their established market share and revenue streams that still run on legacy applications where the core business rules and processes are bundled.
Most enterprises spent their dollars building SOA platforms in the last decade or so, and some of them are still in the migration process. Most of them bought expensive SOA suites, adopted one technology and built large SOAP web services. As their market expectations shift towards MVP, fail fast and faster response to customer feedback, first generation SOA principles are getting compromised at the service layer and the room for innovation is pushed out by a myopic focus on availability and stability.
It's not just application-to-application or business-to-business communication anymore. Today, it can be between thing-to-thing, i.e. machine-to-machine. To meet such demands, the services have to be mobile friendly, for example RESTful Web APIs. Large SOAP Web services are not mobile or machine friendly. Some of the common challenges and question asked are:
■ Contract inflexibility by forcing consumers to adapt to single contract, i.e. can you expose the same service as REST?
■ Security mechanism inflexibility, i.e. do we have multiple authentication choices? The same service is exposed to more than one type of consumer.
■ Data formatting inflexibility, i.e. can we expose the same service to the public and suppress PII?
■ Documentation gaps due to ungoverned service profiles, i.e. do you have documentation for your service? Consumers don't have time to talk to your expert.
■ Unable to govern consumers, i.e. do you know who is consuming what? Can we handle unknown consumers? We are planning to drive innovation from third party developers.
■ Trend analysis on consumer and provider behavior, i.e. is there a way to block a consumer?
■ Forcing the consumer to upgrade due to service provider upgrade, i.e. why should the consumer change if they are not really impacted? They don't have time.
Now product teams are all of a sudden expecting service operations and development teams to do this, and of course, they don't express in technical terms. If they have not yet asked for your enterprise, they will soon ask for it.
So, how do we address these? It's not the best idea to solve these concerns for every service individually, or by the technology by which they were developed, or by the environment in which they were hosted. None of these options will drive consistency. It will become more expensive and will end up in more overheads to service providers and operations. Solving the problem may add more weight than the problem itself.
API Management is the Remedy
So, how do we enable the enterprises to reach their goals? An API management solution, which resolves these pain points in a centralized fashion, represents the Big SOA services as a slim-down API that has real, meaningful agility results. A range of enterprise-level API management solutions are now available in the market, like CA Layer 7, Apigee Edge, SOA Software, Intel Mashrey, TIBCO API Exchange, Microsoft API Management, etc. The enabling technology should at least include capabilities around:
■ Appropriate user authorization access in securing API and service endpoints
■ Dynamic routing and transformation for legacy solutions, including sensitive information protection in motion and at rest
■ Mobile platform access to legacy enterprise queuing mechanisms
■ Consumer quota management of resource consumption so shared services can guarantee a quality of service to all consumers
■ Consumer/provider usage monitoring to aid in incident resolution and operational readiness
■ Consumer self-service for service integration and consumption to accelerate new consumption models ■ Service cataloging for API and service capability discovery to encourage service reuse
API Management: Conceptual View
Each API management solution available in the market achieves these capabilities differently. Each of them has different logical and physical product architecture.
In general, there are four components: ■ API Gateway
■ API Developer Portal ■ API Analytics Portal
The diagram below purely represents the conceptual view of API management.
API Gateway:
It's a logical and physical entity that sits between consumer and provider (backend services). Service providers are not visible to the consumers anymore; therefore all transactions must pass through the gateway services or proxy. In general, gateway exposes the backend services, such as Web APIs. However, it's not limited to Web APIs; they can also mediate SOAP, JMS, XMPP, AMQP, and WebSocket services. These capabilities will vary based on the API product, so the right product should be picked based on the enterprise needs.
API Developer Portal:
It's a self-servicing portal through which developers can register themselves to gain access to the Web APIs. Every API must be published to the portal. The primary goal of such a portal is to eliminate human dependency quickly onboard the known and unknown API consumers/developers. Of course, APIs should meet the
consumer requirements for whatever product that they are aiming to build.
Some of the key capabilities that are accomplished by the portal are API Key Management, Consumer/User Management, Subscription Management, API Discovery, API Lifecycle and API Documentation.
API Analytics:
Every gateway proxy or service must be hooked up with at least basic analytics. Most API products collect inbound and outbound metrics as soon as the API is published or enabled, which is essential to measure the API consumption trend. Analytics provide transparency around consumer behavior and the backend service behavior, and allows the API owner to proactively identify and allocate necessary capacity and resources based on the consumption trend.
API Development UI:
It's a user interface where the API services are developed, deployed, un-deployed and migrated. The API development UI, in general, comes with several built-in policies that can be configured to build services. Policies are building blocks for a gateway service. However, when there are no built-in policies available for a specific need, developers can also build custom policies.
API Types
APIs are required to be classified based on their visibility and scope. This allows for layout of a strategy, security, SLA, and to measure their performance based on their objective.
■ Private APIs: These APIs are not visible outside of an enterprise. They are available only to the internal applications. Consumers are known. In general, the objective of these APIs is to drive down the cost.
■ Public APIs: Public APIs are visible outside of an enterprise, i.e. they are publicly available for anyone to use. There are known and unknown consumers. In general, the objective for these APIs is monetizing the assets exposed to mobile, Telemetry collection, and drive innovation.
■ Partner APIs: These APIs are visible only to partners. Consumers are known. In general, the objective for these APIs is collaborative business process and operation. These APIs sometimes fall under the realm of Private API when they are shared.
Applied Principles for API Management
Name Description Rationale Implication
API Gateway is
lightweight API Gateway service includes the policies that are only related to: security, routing, caching, transformation and throttling.
Having complex policy logics in API Gateway services degrades overall solution performance and will increase maintenance cost, leads to scalability concerns.
API Gateway services should not include complex policy logics (e.g.: XPATH transformation rules, parsing the message body for routing), transformation rules, and business process orchestration.
API Gateway layer may validate the consumers, apply data representation transformation, route the requests between backend services, and secure the backend services.
Transport Headers and URL parameters are the only ones that should be used for routing, throttling, and security policies. API Gateway is
Performant API Gateway services are designed and configured so that the API layer will perform with exceptionally low latency.
API management should not add latency in order to keep up with the customer satisfaction and existing QoS agreement.
Eliminates the need for every backend service to implement caching and throttling to improve performance.
Operations should monitor API usage and consumption and proactively add more nodes as the business grows.
Measure and monitor the API gateway service latency and backend service latency to continuously improve. Include API Gateway as part of the performance testing, establish the API Gateway service benchmark, and validate the performance requirements. API Gateway
availability API gateway is adequately available to handle traffic spikes, works around temporary failures in the enterprise backend, and fails gracefully in the event of a backend outage. The goal is to keep up with the availability requirements as
required by the solution.
API Gateway should be highly available since it brokers all the transactions between the consumer and provider.
Mobile apps that leverage backend systems can lead to sudden growth in IT traffic that can result in crashes and consequent unavailability.
Availability and stability is key to establish consumer trust and adoption.
API proxies should include policies to throttle incoming requests.
Employ monitoring and alerts for the API Gateway and API Portal to ensure the availability of the gateway endpoints, backend service endpoints, gateway nodes, portal services, and portal nodes. API Gateway services are built to handle the backend failures gracefully as needed by the solution.
API Life cycle must be
Managed API lifecycle has to be carefully governed to provide guidance for creation, assure the quality, and evolve for the needs.
There is a need to manage the API changes gracefully for consumers to operate without any downtime. Assures that APIs are rationalized at definition and development, and managed as an asset that has a lifecycle.
Ensuring the APIs for its adherence towards architecture/design principles and QoS that it is required to meet.
Manage API catalog in API developer portal so that consumers can
API Methods are
accessed by authorized consumers
API services include policies to verify consumer
authorization. The goal is to protect APIs and from unauthorized access appropriately as defined by security requirements.
Managing the security in a centralized API management system reduces cost and improves security coverage. Protecting the informational assets from unauthorized users improves enterprise credibility with partners.
Consider the API scope (public or private API) and employ the required security needs as needed by enterprise security. Ability to block, blacklist and whitelist the API consumers.
API gateway should provide multiple options for consumers to authenticate and authorize, such as IAM systems, API key validation, mutual authentication, and service account-based access.
APIs are monitored for
utilization and behavior Collect metrics and to monitor APIs for usage, QoS compliance, exceptions,
performance, latency, and much more, as needed. The goal is to be transparent with the utilization and behavior of the APIs and their associated backend services.
Gain realtime visibility into API performance and trend analysis by tracking metrics overtime.
Align the API
consumption with the consumer's value proposition.
Transparency around API consumption and how they are consumed.
Every API gateway service must be hooked up with basic built-in analytics such as usage, exceptions, performance, and latency.
APIs are Published with
Documentation APIs are easy to understand and utilize. API developer portal offer interactive documentation and communication tools to make it easy to create and manage the applications built on APIs.
Improves agility, as the developers build and test APIs without human dependency. Favors the atmosphere for building new
solutions or applications by composing existing APIs faster and more cost effectively.
Create and manage technical and interactive documentation for all the APIs and their methods.
Self-servicing tools and processes are defined to perform user management, consumer
management, key generation, and API provisioning.
Manage subscription plans for the consuming applications and users. API roadmap is
managed and published to all API consumers.
Conclusion
API management found its space in larger enterprises to keep them moving towards their digital transformation strategy. It adds a buffer to the transformation process so that the organization can migrate gracefully.
Vijay Alagarasan is a Principal Architect in the Enterprise Architecture and Strategy of Asurion, a leading global technology support and protection company (www.asurion.com).
As a domain architect, his focus areas are services, API management and integration
architecture horizontally across Asurion. His contribution includes defining applied principles, reference architecture, reference implementation, patterns and best practices for services, and API management and integration. He works closely with other domain and solution architects to take the Asurion services for future needs.
Vijay Alagarasan specializes in Enterprise Architecture, Service Oriented Architecture, Microservices, API Management and Service Governance.
He has also held the roles of Principal Software Engineer, Tech Lead, and Software Developer. He has more than 10 years of IT experience, with a Bachelor of Engineering degree in Computer Science and Engineering from the University of Madras in India.
He has worked hands-on in various TIBCO Products, Layer 7, Web Technologies, Java and RDBMS products. His interests include API management, Service Oriented Architecture, Microservices, Internet of things, organic agriculture, environmental improvements, food and travel.
Contributions
■ API Management Introduction and Principles
