Sweet!

Today we deployed the second release of our SOA project code-named "Sweet". We have four deliverables remaining and each is named after a type of pickle and include "Dill", "Half-Sour", "Kosher", and "Blue." It's a long story that came out of the requirements gathering process and too much caffeine at an offsite meeting.

This release introduced a new EDAF based service built on the .NET Framework v1.1 to handle payment information and an augmentation of our service that manages constituents. We also introduced a new Ultimus 7.1 workflow to handle reconciling constituents and one updated and four new BizTalk 2004 orchestrations. Overall we deployed code to five application (two web servers, BizTalk, Ultimus, and our EDAF services machines) and four SQL Server database servers.

From a schedule standpoint the work was done primarily by three in house developers and three consultants over a span of a little over two months and with the assistance of a QA manager. In that time the team modified or wrote 49 assemblies, 26 stored procedures, four ASP.NET and one ASP site, and one Ultimus workflow - some very nice work contributed across the board.

Probably the biggest deployment challenge in this release involved introducing new BizTalk orchestrations while needing to keep existing production instances running. We had one core orchestration in our first release which was processing each request (which take on average 48 hours to complete with human interaction in Ultimus). In the Sweet release that orchestration became merely a component of a larger process as we incorporated the processing of new transaction types. So we had to create a new version of that orchestration that returned a response document among other things.

So when we deployed we had over 100 orchestrations we suspended before deploying the new assemblies. Once we deployed we had to add a binding policy in the GAC to one of our schema assemblies that had changed and then we were able to resume the existing orchestrations (as an aside I had originally thought that deploying a new schema assembly would be problematic based on other things I had read but it turned out to be quite painless). As users in Ultimus completed their work and notified BizTalk via an HTTP receive, the orchestrations completed under the old version. New requests that were received via our EDAF service facade were then processed in the new version of the orchestration. Eventually, we'll be able to shut down the first version of the orchestration once all the existing Ultimus incidents have been resolved.

We also learned that with the addition of new orchestrations and a new service we need to automate the installation of all components using msi packages. We used BizTalk scripts which worked well but ensuring that the GAC was updated correctly on our services machines along with configuration files made the deployment to five application and four database servers a two and half hour process that included a couple missed configuration settings.

Architecturally the biggest challenge was in putting in place an "uber" orchestration that acts as the controller for all requests that are processed through our infrastructure to go along with a tracking database. This design will allow us to plug in new transaction types as we automate subsequent business processes.

Other interesting aspects included building a web services operation that attempts to match sponsors to unvalidated information submitted to the process (a so called "Professed" request as opposed to a "Validated" one where the user has provided credentials) and called from an orchestration responsible for "reconciling" sponsor data, and (through some slick .NET coding) allowing the Ultimus user interface to act as a consumer of our services using our Service Agent framework in order to retrieve and create sponsors.

Overall since our first release in September the service-oriented infrastructure has handled over 400,000 service requests and has been remarkably stable. I've been impressed with the EDAF infrastructure on which the services are based and BizTalk has easily handled the load we've thrown at it.