What is a Container? Cloud and SOA Converge in API Management (Container Architecture Series Part 2)

This is the second in a series of posts on container-centric integration architecture. The first post provided a brief background and definition of the Container architecture pattern. This post explores how Service Oriented Architecture (SOA) and Cloud initiatives converge in API Management, and how Platforms provide the Containerization infrastructure necessary for API Management.

Today we are seeing an increasing emphasis on Services and Composite Applications as the unit of product management in the Cloud. Indeed, API Management can be seen as the logical convergence of Cloud and SOA paradigms. Where SOA traditionally emphasized agility within the enterprise, API Management focuses on agility across the ecosystem of information, suppliers and consumers. With SOA, the emphasis was on re-use of the business domain contract. API Management addresses extensibility at all layers of the solution stack. Platforms take responsibility for non-business interfaces and deliver them via Containerization to business layer Service developers who can focus on the domain logic.

Most enterprise applications have historically had a single enterprise owner responsible for design, configuration and business operation of the Application. Each application team has its own release cycle and controls more or less all of its own dependencies subject to Enterprise Architecture policies and central IT operations provisioning. In this model the unit of design and delivery is at the Application level. Larger enterprises may have independent IT operations to operate and manage the production environment, but the operations contract has always been between individual application development teams as the supplier of business logic and the central IT organization, rather than a peer-to-peer ecosystem of business API’s shared between organizations. 

As long as the number of interdependencies between projects remains low and the universe of contributors and stakeholders stays narrow, the Application delivery model works fine.

The relevant principle here is that organizational structure impacts solution architecture. This is natural because the vectors of interest of stakeholders define the decision making context within the Software Development Life Cycle (SDLC). Applications built in isolation will reflect the assumptions and design limitations implicit in the environment in which they were created. A narrow organizational scope will therefore influence requirements decisions. Non-functional requirements for extensibility are understandably hard to justify in a small and shallow marketplace. 

SOA and Cloud change all of this.

As the pace of development accelerates, enterprises see an increasing emphasis placed on agility. SOA is about creating composite solutions through assembly of reusable Services.  Each line of business provides its own portfolio of services to other teams. Likewise each organization consumes many services. Each of service has its own product lifecycle. Because Services are smaller than Applications, they have a much faster product lifecycle. By encapsulating re-usable business functionality as a more modular Service rather than a monolithic Application, the enterprise increases agility by decreasing cycle time for the minimal unit of business capability. 

In response, development and operations teams have crafted DevOps strategies to rapidly deploy new capability. It is not just that DevOps and Continuous Delivery have revolutionized the SDLC; the unit of release has changed as well. In these agile environments, the unit of business delivery is increasingly the Service rather than the Application. In large organizations the Application is still the unit of product management and funding lifecycles, but the Service module is a much more agile model for Cloud and SOA ecosystems. As a result, evolution takes place more rapidly.

This has implications beyond the enterprise boundary. More modular composite applications can be quickly constructed from modules that are contributed by a broader ecosystem of stakeholders, not just those within a single enterprise. From this perspective, SOA design patterns enable more rapid innovation by establishing social conventions for design that allow a faster innovation cycle from a broader population. As a result, evolution takes place more broadly.

SOA initiatives often emphasize standards-based interoperability. Interoperability is necessary but not sufficient for successful adoption of SOA. Standardization, modularity, flexibility, and extensibility are necessary in practice. These can be provided via Containerized Platforms. 

SOA drives containerization because the resulting composite business solutions cross organizational and domain boundaries. Coarse grained Service API’s are shared between organizations via Service Contracts. But these come at a cost; the same granularity that provides the increased agility also increases the interactions between organizations and the complexity of the enterprise as a whole. The added complexity is not a negative since it is essential complexity for the modern enterprise; but it must be managed via SOA Governance and API Management. 

While Service Orientation can be incorporated into technology oriented services such as Security, the primary focus of SOA contracts is on the business domain. Increased integration both within and across the enterprise boundary have implicit dependencies on these cross-cutting technology services. Data must be secured both in transit and at rest, audit and non-repudiation must be provided in a non-invasive manner, capacity must be provisioned elastically to respond to changing demand. All of these cross-cutting concerns require collaboration across the enterprise boundary; they impact service consumers as well as service providers. In order to scale, these non-functional requirements can be delegated to Platforms that realize them through Containers.

What distinguishes a Platform from an Application is the open architecture exposed by deeper API’s that are consumed by the Services running on the platform. In order for the Services to be composed into solutions, the Platform API must be loosely coupled to the Services. This is the function of the Container.  In addition to loose coupling, the Container provides Separation of Concerns between the Service development team and the Operations team. Lightweight containers make the delivery of Services much more modular, enabling among other things the operations team to deliver the fine-grained control necessary for elasticity. In an open ecosystem, a Service Container can also encapsulate the collaborative contract for the community of service providers and consumers, increasing flexibility and re-use. API contracts also support infrastructure features such as authentication and authorization which become pluggable parts of the Platform. An extensible and open infrastructure allows further innovation while insulating business domain developers from the IT concerns.

Platform architectures can be applied within the enterprise or across enterprises. As Cloud delivery models have become popular, Platforms have emerged as a means of accelerating adoption of Cloud offerings.  Google, Facebook, Microsoft, Amazon, and SalesForce are all examples of Platforms.  But there are other, older examples of Platform architectures. In many ways B2B portals are the archetype Platform, offering an ecosystem of extensible, layered API’s upon which higher level services can be delivered across enterprise boundaries.

Of course Cloud and SOA are intimately related. After all, Cloud comes in three flavors, Infrastructure, Platform, or Software as a Service. So Service orientation is implicit in the Cloud paradigm.  What Cloud adds to Service Orientation is a maturity benchmark for delivery and operations of services. Whether they are top-level business services or the supporting infrastructure, Cloud requires self-service on-demand, measurability, visibility, and elasticity of resources. Whereas an enterprise can deliver SOA design patterns, the agility achieved in practice will be constrained if operations cannot step up to the Cloud-delivery model. Containerized Platforms provide both the efficiency necessary for Cloud maturity as well as the extensible modularity required for successful SOA adoption in the larger ecosystem.

The next post will explore how Talend applies Apache OSGI technology to provide a Service Container for enterprise integration as part of the Talend Unified Platform.

 

Related Resources

5 Ways to Become A Data Integration Hero

Products Mentioned

Talend Data Integration

Share

Comments

fred57
Would be great to see an "API Management" platform in Talend, do you recommend one market tool or is it planned to add it to the Talend Suite ?
Data Profiling Services
very good article. Thanks! to sharing post.

Leave a comment

Comments

Comment: 
very good article. Thanks! to sharing post.
Comment: 
Would be great to see an "API Management" platform in Talend, do you recommend one market tool or is it planned to add it to the Talend Suite ?

Add new comment

More information?
Image CAPTCHA
More information?