Sponsoring Children and the SOA

Last Friday morning the third release (the first two were at the end of August and middle of December) of our SOA-related project, code named "Dill" went into production.

For me, the most exciting aspect of the release is that we are beginning to automate the linking of sponsors with children they wish to sponsor by selecting the child on the web. Our system runs a variety of verification rules on both the child and the sponsor (surfaced through operations of a "Need Management Service" where a child is logically defined as a "Need"). If the service operations flag issues that need to be resolved (for example, the child's curernt photo is not available or their biography hasn't been updated) we then send the request to Ultimus where a member of Compassion's Commitment Team can resolve it.

Thus far the system is running smoothly and the team has not seen any systems exceptions (which we define as exceptions thrown by services that are unhandled and therefore end up in a System Exception queue in Ultimus and can be retried once the issue has been corrected).

Today, the team is meeting off-site to discuss lessons learned from this release and review the use case documentation for later releases.

The two biggest issues we had with this release were that...

1) the release encompassed two different business areas which meant the development of BizTalk orchestrations and service operations for both tracks simultaneously. As a result we had two BizTalk developers and two service developers working on their separate tracks. However, we have one Ultimus subject matter expert (SME) and one database development SME who were stretched pretty thin. Not to mention me on the architecture side of the house trying to juggle the technical and business requirements for both sides (which I didn't always do very well). As a result, during the user acceptance testing we logged 52 issues: 21 were coding errors, 1 was a configuration error, 10 were related to missing business requirements, and 20 were missing development requirements (meaning that the business process was understood but all the details were not communicated to developers, mostly my responsibility)

2) we changed a few of the core components, for example our controller orchestration that manages the long running business transaction and fixed issues with correlation which meant that either we deploy all of our components side by side or re-process the requests. We chose the latter since this afforded us the opportunity not to have to retry the system exceptions that were already in the system, clean out all the running orchestrations in BizTalk and incidents in Ultimus, and take advantage of new functionality related to schema changes.

I wrote a little tool that extracted the core documents from our production environment. When we deployed we had 836 current requests in the system and so we used the tool to export these documents (in production we have tracking turned on for this document type).

The core of the code for this tool uses WMI to retrieve the documents and save them to a directory. The UI is very simple and asks for the name of an orchestration for which to extract messages and a path where the documents are saved.

int status = 0;
string statusSQL = string.Empty;
string ServiceName = txtOrchestration.Text;
int i = 0;

switch (cbStatus.SelectedItem.ToString())
{
  case "All":
    break;
  case "Active":
    status = 2;
    break;
  case "Ready To Run":
    status = 1;
    break;
  case "Suspended Resumable":
    status = 4;
    break;
  case "Dehydrated":
    status = 8;
    break;
  case "Completed With Discarded Messages":
    status = 16;
    break;
  case "Suspended Non-Resumable":
    status = 32;
    break;
}

if (status != 0)
{
  statusSQL = " and ServiceStatus = "
   + status.ToString();
}

ManagementObjectSearcher services =
  new ManagementObjectSearcher(
  "\\root\\MicrosoftBizTalkServer",
  "select * from MSBTS_ServiceInstance where ServiceName='"
  + ServiceName + "'" + statusSQL);

int docs = Convert.ToInt32(txtMax.Text);
bool done = false;
   
foreach(ManagementObject o in services.Get())
{
  ManagementObjectSearcher mos =
   new ManagementObjectSearcher(
   "root\\MicrosoftBizTalkServer",
   "select * from MSBTS_MessageInstance
    where ServiceInstanceID='"
   + o.GetPropertyValue("InstanceID").ToString()
   + "'");

   foreach(ManagementObject mo in mos.Get())
   {
     i++;
     mo.InvokeMethod("SaveToFile",
      new object[]{txtPath.Text});
     sbPanel1.Text = "Extracting message "
      + i.ToString();
     if (i == docs)
     {
       done = true;
       break;
     }
   }

   if (done ==true)
 break;
}

sbPanel1.Text = "Done. Extracted " + i.ToString()
  + " messages.";

When the document is extracted BizTalk saves both a .OUT file which contains the XML document and an .XML file which contains the context properties which include our message and message stream ids used in our custom SOAP headers.

We also then have a tool which uses our Service Agent Framework to resubmit these documents. The core of the

private void RunIt(FileInfo f)
{
  try
  {
    label1.Text = "Processing " + f.FullName;
    this.Refresh();

    XmlDocument payload = new XmlDocument();
    payload.Load(f.FullName);

    XmlNodeList nodes = payload.GetElementsByTagName(
     "ConstituentProcessRequest",
     "http://schemas.compassion.com/constituent/
       processrequest/2005-10-01");
    XmlDocument xdoc = new XmlDocument();
    xdoc.LoadXml(nodes[0].OuterXml );

    // add the updateable attribute to the profiles
    AddUpdateable(xdoc);
    AddPRType(xdoc);

    XmlDocument resp = ServiceAgentProxy.ExecuteOperation(
     "Compassion.Services.Agent.
     Constituent.ConstituentServiceAgent",
     "ProcessConstituentRequest",xdoc,Guid.NewGuid().ToString(),
     Guid.NewGuid().ToString(),Guid.NewGuid().ToString(),
     "performance","urn:CPR Load Utility",String.Empty );
  }
  catch (Exception ex)
  {
    listBox1.Items.Add(f.FullName + " " + ex.Message );
    System.Windows.Forms.MessageBox.Show(ex.Message);
  }
}

The call to the ExecuteOperation method of the ServiceAgentProxy object submits the document to the service facade (the ProcessConstituentRequest operation of the Constituent Management Service) then then uses MSMQT to submit the document to BizTalk.

The other interesting thing that this tool does is to modify the documents where appropriate in order to support some of the minor schema changes we made. As a result we wrote couple helper methods that add new attributes to the document where needed.

We'll then use this tool to submit the 836 requests to the infrastructure over the next few days.