Selecting EDAF

When I left off at the end of May I was discussing EDAF and how we used it to implement our services. What follows is a description of EDAF and how we use it.

In order to implement Activity (some of them), Entity, and Infrastructure Services we've taken the approach of using the guidance created by the Microsoft Patterns and Practices Group (PAG). The collection of Microsoft best practices in this area comes in the form of the Enterprise Development Reference Architecture (EDRA) formerly known as "Shadowfax". This .NET code base is designed to provide architectural guidance to organizations moving to service orientation by including the following:

• An extensible application framework called the Enterprise Development Application Framework (EDAF)
• Four QuickStart applications to familiarization with EDRA
• An application template and supporting documentation used to build new services
• The Global Bank Reference Implementation (GBRI) that uses EDRA to implement an online banking scenario
• A good base of documentation on which to work

All of the information for EDRA including its releases can be found on its GotDotNet workspace at http://workspaces.gotdotnet.com/shadowfx. In addition, more information can be found on the Channel 9 EDRA Wicki at http://channel9.msdn.com/wiki/default.aspx/Channel9.EDRAWiki.

For the CI-SOA (Compassion International SOA) the final released version, 1.1, is being used. In addition the CI-SOA will integrate several tools included with EDRA in the future.

• UI support for Visual Studio
• A service agent
• A configuration editing tool

At a high level, the EDAF has four primary goals:

• Separating the service interface from the service implementation
• Separating the business logic from the cross-cutting concerns mentioned previously
• Separating the business logic from the underlying transport so that different transports can be used to access a service
• Develop a stable service interface to ensure resiliency when services are deployed

In evaluating whether or not the EDAF would be used for the CI-SOA the project, the other options included:

• Implementing Entity and Infrastructure services as simple .ASMX. This is the approach Microsoft recommends in order to make it as simple as possible to move to the new Indigo service framework when it is released (for example See David Chappell’s overview of Indigo at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnlong/html/introindigov1-0.asp) in the spring of 2006. Specifically, the guidance offered was in opposition to implementing services today using Enterprise Services or .NET Remoting. Using straightforward .ASMX, however, does not provide the infrastructure for duplicate message handling (idempotent messaging), message logging, performance monitoring, access through multiple interfaces, and other cross-cutting concerns. Since these are standards in our Services Platform that the project team will implement, and since it was not practical to wait for Indigo in order to implement since the first deliverable in August 2005, it was decided that the EDAF provided the core framework needed to build services. The thought is that the separation of service interface from service implementation that the EDAF provides will allow the business logic of Compassion services to be re-factored into Indigo services when it is deemed practical in the future.
• Implementing Entity and Infrastructure services as BizTalk orchestrations. The other option that was considered was implementing all services as BizTalk Orchestrations and exposing those services using the BizTalk orchestration web publishing wizard as illustrated in the following diagram.

The advantages of doing so include:

• The fact that some of the cross-cutting concerns such as logging and performance monitoring are already implemented within BizTalk Server
• There is expertise in the industry regarding BizTalk and how to build orchestrations along with examples and schemas
• It provides a one-stop approach to upgrading
• Future interoperability with technologies such as Indigo will be packaged into upgrades of BizTalk server.

The disadvantages of doing so include:

• A complete dependency on BizTalk for all service development which reduces flexibility
• The need to build façade type orchestrations that may become complex in order to expose multiple operations in a single web service
• The ability to perform maintenance and upgrade pieces of the architecture is reduced
• The need to build out other cross-cutting concerns within BizTalk including duplicate message handling, and authorization
• The need to roll out a set of services in a Partner Country or Field Office requires that BizTalk server be deployed and maintained in that environment. Services built using EDAF would require a smaller footprint and investment in terms of licensing. At present, however, there is no small-footprint option for deploying process services. We assume that if those require orchestrations BizTalk would be installed in the field.

In final analysis, the decision was made to take advantage of the strengths of BizTalk which include orchestrating long running business processes and not attempt to shoe-horn the implementation of more straightforward request-response style services into it. The only caveat to the above is that Activity Services that require transactional coordination among Entity Services or the ability to retry operations in the event of a data store being unavailable will be implemented as BizTalk orchestrations. The reason is that transactional coordination across service invocations is not implemented in the standard platform shipped by Microsoft (for example through support of WS-AtomicTransaction) nor is it a part of the EDAF (the EDAF supports compensating business actions and atomic transactions within a service invocation but not across service invocations). It will be a part of Indigo, however, and so Activity Services should be able to be implemented using Indigo in the future.


In order to satisfy these goals the EDAF implements two primary patterns.

Interception Pattern
The EDAF abstracts how the service is invoked (its service implementation) from the core logic of the service (its service implementation) as shown below. This allows services to be invoked in a variety of scenarios, for example by web sites, Windows applications, Smart Device applications, and even by a BizTalk orchestration that places the message in a queue. This architecture also allows organizations to customize their deployment across several severs although typically the service interface will use the Web Services interface transport and the service implementation the InProc dispatching transport.

The space between the client and the service interface and the service interface and service implementation includes the notion of transports and adapters. The transports abstract the invocation of the service (interface transports) and its implementation (dispatching transports) while the adapters are responsible for receiving the message and initiating message processing.

The interface transports that ship with EDAF include Web Services, .NET Remoting, and Message Queuing. The dispatching transports include Web Services, Message Queueing, DCOM, and in-process (InProc). Each transport, with the exception of InProc needs a host and so for web services the host is IIS/ASP.NET. For .NET Remoting and MSMQ the EDAF provides both a Windows Service called TransportService.exe and a console application called TransportConsole.exe.

Organizations can also implement their own interface transport by creating a class that implements the ITransportListenerController interface and hosting it the provided hosts or an executable. This interface includes simple Start and Stop methods that are called by the EDAF to begin and to quit listening for requests respectively. For example, one could create an FTP Transport by hosting a component in a Windows Service that "listens" for requests by polling the folder used by an FTP site.

Each of the interface transports has an associated adapter that is responsible for extracting the request message from the transport and creating the EDAF Context object which is a kind of wrapper object that contains all of the information needed to process the request. The Context is then passed into the service interface. Likewise, each dispatching transport has an adapter that retrieves the Context object and passes it into the service implementation. Each adapter is customized for its transport, for example, the Web Services interface transport’s adapter is implemented as a SOAP extension.

This pattern is an integral part of EDAF and will be used when implementing services in order to separate the service interfaces from the service implementations. By doing so the service implementations can be invoked from a variety of service interfaces.

Pipeline Pattern
One of the major goals of the EDAF is to separate cross-cutting concerns such as logging, security, transactional behavior, and event notification from the business logic. To achieve this goal both the service interface and the service implementation include the concept of pipelines. A pipeline consists of a chain of handlers that each implement one of the cross-cutting functionalities as shown below. This is an implementation of the well-known Chain of Responsibility pattern.

These pipelines are executed by the adapters and passed the Context object discussed in the previous section. When all of the handlers have been executed the pipeline finally executes a target object. The target object is specific to what it will execute, for example, whether the target is to pass the Context object to the dispatching transport or whether to a business action.

The service interface pipeline shown in the diagram is transport-specific and typically focuses on boundary concerns such as authentication, request monitoring, validating the request message, and handling duplicate request messages by returning a cached result (idempotence). As a result, the service interface pipeline might be configured to include the AuthenticationInfrastructure and AuthorizationInfrastructure handlers to perform client authentication and authorization using the Windows identity and either Active Directory or NTLM groups.

The pipelines and the handlers that are included are configured in the EDAF configuration file making these easy to adjust and turn on and off as needed.

The service implementation handlers on the other hand usually include service specific behaviors such as raising business events using WMI, logging requests, implementing transactions, or transforming messages. The following table shows the handlers that ship with the EDAF.

Handler
ClientTrace                  
Returns a trace of the execution of the request with the response

AuthorizationDatabase        
Performs authorization of the request using a SQL database

DuplicateHandling            
Implements once-only processing of a message

AppInstrumentationExecution  
Monitors the time taken to execute the request from this point forward

AuthenticationIdentity       
Trusts the current credentials passed in with the request

AuthenticationInfrastructure 
Ensures that the identity is authenticated using Windows

AuthorizationInfrastructure  
Authorizes a request using Windows groups

AppInstrumentation           
Updates performance monitor counters

MessageTransformation        
Uses an XSLT to transform a message

PasswordAuthorizationHandler
Performs user name and password authentication

PublishBusinessEvent         
Publishes a business event

SignedMessageAuthentication  
Uses a public key to verify the signature of an XML message

SyntacticValidation          
Transforms the message into a strongly-typed message

ExecutionTimeout             
Monitors the execution of the request beyond this handler and times it

Transaction                  
Starts a new COM+ transaction

Handlers also provide an extensibility point where the project team will insert custom processing in the pipeline. The EDAF includes three handlers interfaces that include IAtomicHandler for basic process, IStatefulAroundHandler for handlers that need to execute in the same stack frame, and IStatelessAroundHandler for handlers that execute both before and after the target is executed.

This pattern is an integral part of EDAF and will be used to add handlers that participate in the processing of service requests and responses. The cross-cutting concerns mentioned above are implemented with handlers.

In the next post I'll take a look at how we customized EDAF to fit our Services Platform.

Real SOA!

I know it's been a long time since I published an update on our SOA implementation here at Compassion International. However, it's been a busy summer (and I've been doing alot of baseball writing as well). I'm pleased to report though that on August 30 we rolled out our first deliverable based on this architecture. This deliverable helps to automate the updating of sponsor information entered on our web site and to kick certain requests to a human workflow process for resolution based on a series of business rules.

The solution consists of our external web site Compassion.com, a web service with a series of web methods implemented using Microsoft's EDAF (about which more to follow) implemented in C#, MSMQ, three BizTalk Server 2004 orchestrations and two workflow processes implemented in the Ultimus BMP Suite 7.1.

Since it went into production we've recorded over 35,000 service calls and the system is humming along quite well. Currently, we're in the process of making a few tweaks in order to provide a 1.1 release before moving on to our next deliverable.

During this time I'll take the opportunity to continue the discussion of our Services Platform.