Encryption Handler

As promised awhile back we'll take a look at the EDAF handler we created in order to encrypt and decrypt data within our messages.

To do so we built the handler in an assembly called Compassion.Services.EDAF.Handlers.Security.dll. This is an AtomicHandler (it doesn't hold state between invocations) that can be configured either in the before or after pipeline of service action calls in order to encrypt or decrypt messages using various key providers.

For example, the configuration of the handler in the ProcessConstituentRequest façade pipeline in order to encrypt messages before they are saved in the ConstituentRequest and ServiceManager databases.

<before>
 <handler handlerName="MessageEncryptor">
  <ci:loggerSettings localValue="US" 
    keyDir="C:\CompassionServices\Config\      
     Keys\CIPublicOnlyKey.xml" 
    operation="Encrypt"  
    message="Request"  
    provider=
      "Compassion.Services.Security.Common.KeyFileProvider,
    Compassion.Services.Security.Common, 
    Version=1.0.0.0, Culture=neutral, 
    PublicKeyToken=7674363010ccf400" />
 </handler>
</before>

As you can see the handler is configured in the before pipeline and is set to encrypt the Request message using the KeyFileProvider. The handler uses reflection to load the key provider dynamically and pass in constructor arguments such as the file path and asymmetric algorithm. At this time the handler hard codes the instantiation of the RSACryptoServiceProvider since it is the only AsymmetricAlgorithm implemented in v1.1 of the .NET Framework.

In order to encrypt data on the way out of a service call (for example for an operation that selects payment data the handler would be configured like so:

<after>
 <handler handlerName="MessageEncryptor">
  <ci:loggerSettings localValue="US" 
   keyDir=
     "C:\CompassionServices\Config\Keys      
      CIPublicOnlyKey.xml" 
   operation="Encrypt"  
   message="Response"  
   provider=
    "Compassion.Services.Security.Common.KeyFileProvider,
   Compassion.Services.Security.Common, 
   Version=1.0.0.0, Culture=neutral, 
   PublicKeyToken=7674363010ccf400" />
 </handler>
</after>

One of the interesting things about the implementation of this handler is that the ObjectEncryptor deals with objects whereas EDAF pipelines pass XmlDocument instances to their handlers via the Execute method. As a result, the handler is responsible for pulling the type information out of the EDAF configuration file and using it deserialize the type and pass it to the encryptor.

In the previous post on this topic a reader asked why we didn't use WS-Security in order to encrypt the message. That's a great question and the answer is that if we had used WS-Security we could certainly have encypted the message body but this would not have allowed us to view the message when it was logged in our infrastructure. In other words, we want visibility into the all parts of the message except those explicitly marked as needing protection.

ESBs and Microsoft

Day four at TechEd and spent this morning in a session on "Developing the Next Generation ESB on the Microsoft Platform" which was a case study of sorts of a project donw at Kaiser Permanente.

They (three speakers shared the podium) started the talk with a discussion of just what is an Enterprise Service Bus? A 2006 survey found the following characteristics as most important:

Message Broker

Message Transformation Service

Message Validation

Message-Oriented Middleware

The Microsoft speaker then noted that given this list Microsoft will not come out with an "ESB product" because they feel their products are already a superset of the tools required with BizTalk, VS, MSMQ, SQL Server, Host Integration Server, and MOM, along with the guidance they are putting out.

What was interesting about the talk, highlighted by Brian Loesgen, was that for this particular solution that can be used to literally connect thousands of applications, they created a Core ESB Engine that features an envelope (schema) that specified at the meta data level the kind of message entering the ESB as well as what needed to be done with it. From there they dynamically performed both mapping and runtime endpoint resolution (routing). In other words the architecture supported adding new maps and new endpoints without recompiling or redeploying any part of the ESB. In addition they developed a message oriented approach to exception handling whereby developers publish exception messages and handlers (for example SharePoint) subscribe to them. This allows for highly targeted process-specific handlers to be created for custom exception recovery operations.

The final speaker from Kaiser highlighted the client and server integration modules (CIM and SIM) that they use to provide abstraction for clients and servers with regards to policy, location, management, security, etc. The CIM is analgous to our Service Agent Framework while the SIM is used for interoperability as well since they support both SOAP invocation of services as well as the use of BizTalk adapters and other custom mechanisms. He also briefly showed their management console they can use to monitor services and develop SLAs.

Now obviously an organization could build an ESB in this way and the architecture seems more than solid. The problem, and why an organization might be driven to adopt another vendor, is that this solution was developed with the help both of Microsoft and a systems integrator at I'm sure a significant cost in terms of time and money. For smaller organizations or those that can't invest the time required to develop such a solution going to these lengths is not really an option.

Services on CF

Attended a session on the .NET Compact Framework 2.0 this afternoon. I haven't looked at CF since I came to Compassion over a year ago and so I was quite disappointed to discover that the System.Configuration namespace is not implemented in the new version. For us this is problematic since our Service Agent Framework relies on IConfigurationSectionHandler in order to read information about the service agent types, mock messages, and logging that the framework does on the client.

As a result, if we want to expose our services to mobile clients we'll need to write a Mobile Service Agent Framework that relies on custom XML configuration files as described in the book I wrote with Jon Box back in 2003. We can't simply use a web reference proxy since we also rely on custom SOAP headers that the proxy does not create.

Pragmatic Architecture

"Pragmatism urges us to look to the consequences of what we do, which the discipline of architecture infused with an idealistic focus on intentions frequently resists"

Attended a pretty entertaining presentation titled "Pragmatic Architecture" by Ted Neward and Cathi Geri. Because the room was heavily populated with architects they had this humorous definition of just what an architect is:

An architect is someone who...

Defines architecture

Gets paid more than real developers do

Focuses on issues that have nothing to do with real-world problems

Thinks in terms of code not clouds

Lost the respect of developer friends & teammates

Speaks with big words and power point slides

Has upper management fooled

But more seriously they went on to talk about the roles or personas that architects take on which include Infraastructure, Enterprise, and Solution (responsible for the design of one or more applications or services within an organization and which the remainder of their talk focused on) and identified by Simon Guest.

They then laid out a taxonomy solution architects can use as they think through laying out solutions...

Communication/Distribution - Patterns of communication such as request/response and the technologies used to move data.

Presentation/Interaction - Style (graphical, console, prgrammatic), implementation, stovepipe vs. composite/mash-up (one front end, many back ends) vs. jewel (many front ends, one backend), and perspectives (user, admin).

State Management - Lifetime, "shape" of the state (relational vs. object vs. hierarchical), location (where, access, and transparency)

Processing - Implementation, concurrent/parallel, transactional

Resource Management - Locator/registry, discovery (plug-and-play, peer-to-peer), injected, a priori knowledge

Evolving to Patterns

Day three of TechEd 2006 and this morning I attended a session on "Evolving to Patterns" by James Newkirk. What I found most interesting was his use of the "broken windows" paradigm and its application it to software development (others have done so as well). For those not familiar, the idea came from an article written by James Q. Wilson called "Broken Windows" that appeared in the March 1982 issue of Atlantic Monthly. In a nutshell, the idea is that human beings tend not to respect that appear to be in a state of disrepair leading to a vicious cycle of further deterioration. The example used by Wilson is that of an apartment building with one broken window that acts as a signal to vandals to break even more. The same idea can be applied to litter and even the accumulation of junk or dishes in the sink in your own house. The lesson is to fix problems when they are small.

As applied to software development the notion of fixing your broken windows can be distilled into three basic ideas.

Remove Duplication. Duplicated code is broken code because it encourages sloppiness and thought processes where developers don't look for ways to inject abstraction.

Simplify Your Code. Code that is difficult to understand from a structural perspective is broken code because developers would rather write something themselves than have to figure out how to refactor existing code to get the job done.

Clarify your Code's Intent. A corollary to the above is that the intent of the code needs to be clear. Code's whose intent is not clear through the judicious use of comments and naming conventions is broken because it leads to misusing the code or once again the tendency to put that code in a virtual black box that is not to be touched which leads to duplication.

And of course evolving your code to use patterns helps us to design code that is ready to handle the changes that will inevitably come.

Moving to Agile

Well, day two at TechEd 2006 in Boston and just came out of a talk on implementing agile methodologies in your organization. What I found interesting is that many of the best practices the speakers (from the patterns and practices group) discussed were ones that our project team are currently implementing. As a result, it was a good check for us in seeing what we have yet to tackle both from a team and organizational perspective. Briefly, here are a few of the practices mentioned and what we've done:

Building a team space. While our team members still have cubes, 90% of their work is done in a conference room dedicated to the project. In my view, that has increased the speed of communication and helps us to resolve issues more quickly. It also has the effect of allowing everyone to be more or less on the same page with how things are implemented in areas that they perhaps are not currently working in.

15 minute rule. Essentially this rule says that if you are blocked for 15 minutes, give up and go ask. The team space definitely encourages this behavior although we haven't talked about it specifically. The corollary to this is that when someone asks for help you can't say no. In our organization this doesn't tend to be a problem anyway but surfacing the need to keep the communication open is key.

Real-time code review, aka Pair Programming. We don’t pair program today but we do have some overlapping of assignments with two core BizTalk developers and two services devs. In some cases they work together on a feature and of course I assist and help talk through the implementation of code. We should perhaps be more intentional about this.

Give team time to gel. One of the things that has helped us be successful is that our team had a two month time period in which we implemented a reference architecture before turning our attention to actual production code. As a result, we were able to find our roles before the pressure ramped up. While not intentional, to me, this turned out to be a very helpful by product of the process.

Get a coach or mentor. None of the team had had much exposure to agile methods before and could have used a bit of coaching before the project commenced. That said, we're not really officially doing agile within Compassion and so this probably wasn't an option. But if we had to do it again, a bit more knowledge would have been useful.

Encourage questions and discussion. The team space really makes this possible. It tends to keep the environment open and questions are often raised within the room to no one in particular. To me, that's a sign that people are comfortable with each other and no one is too intimidated to ask.

Stack rank work by importance to customer. While our customers have been very involved in the project, they have only stack ranked work related to hot fixes once various releases have gone into production. Our organization has some way to go it seems before customers are entrusted to do this. The corollary to this is reevaluating that stack frequently. This also needs some work.

Developer estimates are not to be trusted. The speakers were adamant that developer estimates of more than three days are probably essentially guesses. Unless you can break down a feature into very small tasks, then you're not really estimating. Our developer estimates are a bit larger than that and so we probably have some work to do.

Clarify Customer Requirements. We have used use case documents and a feature matrix in order communicate the requirements. Unfortunately, the documents are not always as clear as they could be largely because the conceptual model used in the development of the use cases was not familiar to the developers or the users. Had we to do it again, the use cases would be written in the language of the users and not in the language of the logical model. The team would then be responsible for translating that language into a new model used to implement the software. The speakers also note that automated tools should be evaluated that allow the requirements to be executable.

Customer (or their proxy) has to be available. One of the strengths of our project is that the business users and stake holders almost all reside in the same building and we have had for the most part a business analyst that does a great job of communicating. That allows the team to surface an issue and get feedback very quickly. The average turnaround time is something like four hours.

Implement the 3 strikes rule. In a nutshell this means you don't refactor code until you have three consumers for it. This has been a balancing act for our team since most of what we're building is being done for the express purpose of reuse across future projects. However, the features within the services we build we evaluate and implement based only on the certainty that they will be reused. In other words, our strategy has been to implement the infrastructure that will allow us to refactor when necessary but not to over-engineer in the near term. Examples include the business rules infrastructure we've implemented as well as the exception reporting and suppression techniques we've used. Interestingly, they noted that when you need to refactor, one technique to allow developers to integrate it into the project is to position it as "making room" for a feature before implementation. Their point is that the term "refactoring" implies rework which signals a red flag among some.

Insist on clarity of code. As with most agilists, their emphasis was on code readability for maintenance and refactoring. We insist on NDoc comments and code standards in terms of naming conventions etc. to assist with this but should probably be even more diligent about this.

Reward individuals for team behavior. Our team celebrates success after releases and there is very little in the way of individual rewards. The goal is to get the team to own the entire pie and not just their little piece of it. In that regard, something we do need to work on is the concept of developers working on features and not owning them. We tend to segment our devs into three buckets and each bucket's owns a set of features. While to some extent it is very difficult for everyone to know everything, we should do a better job of specifically asking devs to make changes in code that they have not had a hand in developing. This would increase team ownership.

Everyone is entitled to participate. In our team room we include not only devs but also testers, the project manager, infrastructure support, and technical writing. Including everyone certainly helps with communication issues but the speakers were actually suggesting that these other disciplines take part in real-time code reviews and design sessions. Each person would then contribute at their level. We certainly have not done this explicitly but again, having everyone in the room provides them accessibility to those discussions. At the end of the project we also do lessons learned off site sessions where we collect feedback to help us improve the process going forward.

Compassion International Back on Television

For those who aren't aware of what Compassion does, there will a TV program airing soon. You can find a list of stations and times here. The following is the press release sent out a few days ago.

From a technical perspective we're interested in seeing how well our service oriented infrastructure handles the additional anticipated load so stay tuned.


COLORADO SPRINGS, CO, May 30, 2006 – Beginning June 1, Compassion International is breaking back into the television scene. Compassion will air a 30-minute television special in major markets throughout the United States. Hosted by contemporary Christian music artist Rebecca St. James, the special will focus on Compassion’s child sponsorship program.

In the 1980s, Compassion was a pioneer among Christian ministries in using television programming, but rising costs eventually led to discontinuing this venue. Today, with hundreds of television channels available, this venue has again become a cost-effective way to share the message of children in poverty.

“This television special creates a wonderful opportunity to share the needs of children in poverty around the world with many people throughout the United States who may never have heard Compassion’s message before,” said Rick Davis, Compassion’s Director of Marketing. “Our hope is that by telling Compassion’s story through a new venue we will help thousands of people to make a difference in the life of a child in need.”

The television special is co-hosted by Tim Glenn, Compassion’s U.S. Advocacy Director, and also features nationally renowned author and speaker Tony Campolo and Hollywood actors Victoria Jackson and Cary Elwes. All participants in the special sponsor children of their own through Compassion’s child sponsorship program.

A complete listing of airtimes can be found on Compassion’s Web site. Visit www.compassion.com and click on the link to the television program.

Compassion International is one of the world’s largest Christian child development organizations, working with more than 65 denominations and thousands of indigenous church partners in Africa, Asia, Central and South America and the Caribbean. Since 1952, Compassion has touched the lives of more than one million children.

It has been recognized for its financial integrity with top ratings and recommendations by several of the nation’s leading not-for-profit “watchdog” organizations. Recently Compassion was recognized by Charity Navigator for its fifth consecutive four-star rating, putting it in the top one percent of 5,000 charities.

For information about sponsoring a child, contact Compassion online at www.compassion.com or by calling (800) 336-7676, Monday through Friday, 8 a.m. to 5 p.m., MST.