pr0g33k

 collapse all
  1. Building a Windows Service to Host Workflow Foundation 4 Activities

    The past few months have been very busy for me. The public school district for which I work purchased a new student information management system and they need to integrate their existing software packages with the new system. Each software package comes with different requirements for integration. Some are in-house and located on local servers while others are remote. They all perform imports and exports via text files, though, and communicate via protocols like SNMP, HTTP WebDAV, FTP, and SFTP. Some of the file processing can be event-driven since they have locally-installed applications and I can monitor their directories while the remote applications have to be polled. And some of the imports/exports are expected at specific times during the day while others are needed on time intervals. Oh yeah, these integrations can and likely will change sporadically so they need to be flexible and easy to maintain and the solution needs to simple enough that a group of junior developers can manage it if needed. Needless to say, this has kept me busy for the past several months.

    WF 4.5 to the Rescue!

    Choosing Workflow Foundation for a task like this was a no-brainer. I needed the ability to pick up files from one location, perform modifications (in some instances), change file names, transport them to other locations, and archive the files in between. These step-by-step processes are exactly what Workflow Foundation was made to do. And the WF designer in Visual Studio meant that pretty much anyone could drag and drop the WF activities onto the workflows and make simple adjustments when needed.

    Building the workflow activities for the individual tasks was pretty simple and I'll have some future posts about building the activities. What was not simple, though, was figuring out how to host the workflows. Pretty much everything I found online regarding Workflow Foundation hosting was about AppFabric. I was really excited by what AppFabric promised to deliver - a slick management interface, easier persistence and tracking configuration, and simple deployments. AppFabric is awesome - really awesome - but only if you're doing WCF-centric development. The first workflow I built relied upon a file system event to trigger the workflow (a file creation event monitored by a FileSystemWatcher-based WF activity). No matter what I tried, I couldn't get AppFabric to run the workflow. It turns out that AppFabric really only wants to run workflows when triggered by a WCF call picked up by a RecieveRequest activity. That AppFabric is presented as an IIS component should have been my first clue but I was hoping that I could just run my workflows without any regard to Internet-based traffic and still take advantage of all the other hotness in AppFabric. Unfortunately, no such luck...

    Kickin' It Old-School with a Windows Service

    I settled on developing a relatively simple Windows Service to host the workflow activities. Now, because many of the workflows needed to run uninterrupted, I didn't want to hard-code the creation of each workflow into the service. Instead, I wanted the workflows to be created and destroyed dynamically without having to restart or reinstall the service everytime a workflow was added, deleted, or changed. Fortunately, workflows can be coded in XAML and dynamically loaded from those XAML definitions. So, what I built is a Windows Service that uses a FileSystemWatcher to monitor a directory where I can add and delete XAML workflow definitions and the service manages the lifetime of each workflow. I do need to point out that the workflows you'll be adding to this service are workflow activities (XAML) and not WCF Workflow Services (XAMLX).

    Here's the code for the service:

    public partial class Service : ServiceBase
    {
        private FileSystemWatcher _FileSystemWatcher;
        private Dictionary<String, WorkflowApplication> _WorkflowApplications;
    
        public Service()
        {
            InitializeComponent();
        }
    
        protected override void OnStart(String[] args)
        {
            AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(OnUnhandledException);
    
            String path = ConfigurationManager.AppSettings["WorkflowsPath"];
    
            if (!String.IsNullOrEmpty(path))
            {
                _WorkflowApplications = new Dictionary<String, WorkflowApplication>();
                String filter = "*.xaml";
    
                if (!String.IsNullOrEmpty(Path.GetExtension(path)))
                    path = path.Replace(Path.GetFileName(path), String.Empty);
    
                if (!path.EndsWith("\\"))
                    path = String.Format("{0}\\", path);
    
                foreach (String file in Directory.GetFiles(path, filter))
                    CreateWorkflowApplication(file);
    
                _FileSystemWatcher = new FileSystemWatcher(path, filter);
                _FileSystemWatcher.NotifyFilter = NotifyFilters.FileName;
                _FileSystemWatcher.Created += new FileSystemEventHandler(OnCreated);
                _FileSystemWatcher.Deleted += new FileSystemEventHandler(OnDeleted);
                _FileSystemWatcher.Renamed += new RenamedEventHandler(OnRenamed);
                _FileSystemWatcher.EnableRaisingEvents = true;
            }
        }
    
        protected override void OnStop()
        {
            _FileSystemWatcher.EnableRaisingEvents = false;
            _FileSystemWatcher.Dispose();
    
            foreach (var key in _WorkflowApplications.Keys)
            {
                _WorkflowApplications[key].Unload();
            }
        }
    
        private void OnCreated(Object sender, FileSystemEventArgs e)
        {
            Thread.Sleep(1000);
            CreateWorkflowApplication(e.FullPath);
        }
    
        private void OnDeleted(Object sender, FileSystemEventArgs e)
        {
            DeleteWorkflowApplication(e.FullPath);
        }
    
        private void OnRenamed(Object sender, RenamedEventArgs e)
        {
            DeleteWorkflowApplication(e.OldFullPath);
            CreateWorkflowApplication(e.FullPath);
        }
    
        private void OnUnhandledException(Object sender, UnhandledExceptionEventArgs e)
        {
            LogError(e.ExceptionObject as Exception);
        }
    
        private void CreateWorkflowApplication(String fullPath)
        {
            if (!_WorkflowApplications.ContainsKey(fullPath))
            {
                try
                {
                    ActivityXamlServicesSettings activityXamlServicesSettings = new ActivityXamlServicesSettings { CompileExpressions = true };
                    DynamicActivity dynamicActivity = ActivityXamlServices.Load(fullPath, activityXamlServicesSettings) as DynamicActivity;
                    WorkflowApplication workflowApplication = new WorkflowApplication(dynamicActivity);
                    workflowApplication.InstanceStore = new XmlInstanceStore(fullPath);
                    workflowApplication.PersistableIdle = (e) => { return PersistableIdleAction.Persist; };
                    workflowApplication.OnUnhandledException = (x) =>
                    {
                        RemoveWorkflowApplication(fullPath);
                        return UnhandledExceptionAction.Abort;
                    };
                    workflowApplication.Completed = (c) =>
                    {
                        if (c.CompletionState == ActivityInstanceState.Closed)
                            RemoveWorkflowApplication(fullPath);
                    };
    
                    _WorkflowApplications.Add(fullPath, workflowApplication);
                    workflowApplication.Run();
                }
                catch (Exception ex)
                {
                    LogError(ex);
                    RemoveWorkflowApplication(fullPath);
                }
            }
            else
            {
                DeleteWorkflowApplication(fullPath);
                CreateWorkflowApplication(fullPath);
            }
        }
    
        private void DeleteWorkflowApplication(String fullPath)
        {
            try
            {
                if (_WorkflowApplications.ContainsKey(fullPath))
                    _WorkflowApplications[fullPath].Terminate("Workflow Activity removed.");
    
                RemoveWorkflowApplication(fullPath);
            }
            catch (Exception ex)
            {
                LogError(ex);
                RemoveWorkflowApplication(fullPath);
            }
        }
    
        private void RemoveWorkflowApplication(String fullPath)
        {
            if (_WorkflowApplications.ContainsKey(fullPath))
            {
                _WorkflowApplications[fullPath] = null;
                _WorkflowApplications.Remove(fullPath);
            }
        }
    
        private void LogError(Exception exception)
        {
            if (exception != null)
            {
                String path = ConfigurationManager.AppSettings["WorkflowsPath"];
    
                if (!String.IsNullOrEmpty(path))
                {
                    String errorPath = Path.Combine(path, "Errors");
    
                    try
                    {
                        if (!Directory.Exists(errorPath))
                            Directory.CreateDirectory(errorPath);
    
                        using (StreamWriter file = File.CreateText(Path.Combine(errorPath, String.Format("{0}.txt", DateTime.Now.ToString("yyyyMMdd_HH-mm-ss")))))
                        {
                            file.WriteLine("--------------------EXCEPTION--------------------");
                            file.WriteLine(exception.Message);
                            file.WriteLine();
                            file.WriteLine("-------------------STACK TRACE-------------------");
                            file.WriteLine(exception.StackTrace);
    
                            if (exception.InnerException != null)
                            {
                                file.WriteLine();
                                file.WriteLine("-----------------INNER EXCEPTION-----------------");
                                file.WriteLine(exception.InnerException.Message);
                                file.WriteLine();
                                file.WriteLine("-------------------STACK TRACE-------------------");
                                file.WriteLine(exception.InnerException.StackTrace);
                            }
    
                            file.Close();
                        }
                    }
                    catch (Exception)
                    {
                        throw;
                    }
                }
            }
        }
    }

    There are a couple of things to take note of here. I'm only filtering the notifications using NotifyFilters.FileName and responding to the Created, Deleted, and Renamed events. The FileSystemWatcher fires off several notifications - often duplicated - due to the way the operating system manages file-related events. To avoid the duplications and make the code a bit simpler, I limited the number of filters and events I'm monitoring. Because of this, changing a file by overwriting it will not get picked up by the FileSystemWatcher so if you want to re-deploy a XAML file, you will need to first delete it and then copy in the updated file.

    Another thing to consider is the persistence of the workflows. The workflows I had to run are not long-running so persistence is not really required (at least not yet). Workflows are pretty closely tied to persistence, however. For example, calling Unload() on a workflow will attempt to use its persistence provider to persist its state. By default, WF uses SQL Server as a persistence store but you can disable persistence altogether, create a "dummy" InstanceStore that does nothing, or create a custom InstanceStore that saves the state in whatever format you want. I didn't have access to a SQL Server instance so I debated whether or not to enable persistence at all. I ended up creating an InstanceStore that serializes the workflow to XML so that I could at least have something to monitor the ongoing states of the workflows. Here is the XmlInstanceStore:

    public class XmlInstanceStore : InstanceStore
    {
        private readonly Guid _InstanceOwnerId;
        private readonly String _Path;
    
        public XmlInstanceStore(String path)
            : this(path, Guid.NewGuid())
        {
        }
    
        public XmlInstanceStore(String path, Guid instanceOwnerId)
        {
            _Path = path;
            _InstanceOwnerId = instanceOwnerId;
        }
    
        protected override Boolean TryCommand(InstancePersistenceContext context, InstancePersistenceCommand command, TimeSpan timeout)
        {
            return EndTryCommand(BeginTryCommand(context, command, timeout, null, null));
        }
    
        protected override IAsyncResult BeginTryCommand(InstancePersistenceContext context, InstancePersistenceCommand command, TimeSpan timeout, AsyncCallback callback, Object state)
        {
            EnsurePersistenceDirectoryExists();
    
            if (command is CreateWorkflowOwnerCommand)
            {
                context.BindInstanceOwner(_InstanceOwnerId, Guid.NewGuid());
            }
            else if (command is SaveWorkflowCommand)
            {
                SaveWorkflowCommand saveCommand = (SaveWorkflowCommand)command;
                Save(saveCommand.InstanceData);
            }
            else if (command is LoadWorkflowCommand)
            {
                try
                {
                    using (FileStream inputStream = new FileStream(FilePath, FileMode.Open))
                    {
                        context.LoadedInstance(InstanceState.Initialized, LoadInstanceDataFromFile(inputStream), null, null, null);
                    }
                }
                catch (Exception ex)
                {
                    throw ex;
                }
            }
    
            return new CompletedAsyncResult<Boolean>(true, callback, state);
        }
    
        protected override Boolean EndTryCommand(IAsyncResult result)
        {
            return CompletedAsyncResult<Boolean>.End(result);
        }
    
        private IDictionary<XName, InstanceValue> LoadInstanceDataFromFile(Stream inputStream)
        {
            IDictionary<XName, InstanceValue> data = new Dictionary<XName, InstanceValue>();
            NetDataContractSerializer netDataContractSerializer = new NetDataContractSerializer();
            XmlDocument xmlDocument = new XmlDocument();
    
            using (XmlReader xmlReader = XmlReader.Create(inputStream))
            {
                xmlDocument.Load(xmlReader);
            }
    
            XmlNodeList instanceElements = xmlDocument.GetElementsByTagName("InstanceValue");
    
            foreach (XmlElement instanceElement in instanceElements)
            {
                XmlElement keyElement = (XmlElement)instanceElement.SelectSingleNode("descendant::key");
                XName key = (XName)DeserializeObject(netDataContractSerializer, keyElement);
    
                XmlElement valueElement = (XmlElement)instanceElement.SelectSingleNode("descendant::value");
                Object value = DeserializeObject(netDataContractSerializer, valueElement);
                InstanceValue instanceValue = new InstanceValue(value);
    
                data.Add(key, instanceValue);
            }
    
            return data;
        }
    
        private Object DeserializeObject(NetDataContractSerializer serializer, XmlElement element)
        {
            Object deserialized = null;
    
            using (MemoryStream memoryStream = new MemoryStream())
            {
                using (XmlDictionaryWriter xmlDictionaryWriter = XmlDictionaryWriter.CreateTextWriter(memoryStream))
                {
                    element.WriteContentTo(xmlDictionaryWriter);
                    xmlDictionaryWriter.Flush();
                    memoryStream.Position = 0;
                    deserialized = serializer.Deserialize(memoryStream);
                }
            }
    
            return deserialized;
        }
    
        private void Save(IDictionary<XName, InstanceValue> instanceData)
        {
            XmlDocument xmlDocument = new XmlDocument();
            xmlDocument.LoadXml("<InstanceValues/>");
    
            foreach (KeyValuePair<XName, InstanceValue> data in instanceData)
            {
                XmlElement instanceValue = xmlDocument.CreateElement("InstanceValue");
                XmlElement key = SerializeObject("key", data.Key, xmlDocument);
                instanceValue.AppendChild(key);
    
                XmlElement value = SerializeObject("value", data.Value.Value, xmlDocument);
                instanceValue.AppendChild(value);
    
                xmlDocument.DocumentElement.AppendChild(instanceValue);
            }
    
            xmlDocument.Save(FilePath);
        }
    
        private XmlElement SerializeObject(String elementName, Object o, XmlDocument doc)
        {
            NetDataContractSerializer netDataContractSerializer = new NetDataContractSerializer();
            XmlElement xmlElement = doc.CreateElement(elementName);
    
            using (MemoryStream memoryStream = new MemoryStream())
            {
                netDataContractSerializer.Serialize(memoryStream, o);
                memoryStream.Position = 0;
                StreamReader rdr = new StreamReader(memoryStream);
                xmlElement.InnerXml = rdr.ReadToEnd();
            }
    
            return xmlElement;
        }
    
        private String FilePath
        {
            get
            {
                String fileName = Path.GetFileNameWithoutExtension(_Path);
    
                if (String.IsNullOrEmpty(fileName))
                    fileName = _InstanceOwnerId.ToString();
    
                return Path.Combine(Path.GetDirectoryName(_Path), String.Format("{0}.xml", fileName));
            }
        }
    
        private void EnsurePersistenceDirectoryExists()
        {
            try
            {
                if (!Directory.Exists(Path.GetDirectoryName(_Path)))
                    Directory.CreateDirectory(Path.GetDirectoryName(_Path));
            }
            catch (Exception)
            {
                throw;
            }
        }
    }

    I also created a NullInstanceStore in case I wanted to configure some workflows to persist and others to do nothing.

    public class NullInstanceStore : InstanceStore
    {
        protected override Boolean TryCommand(InstancePersistenceContext context, InstancePersistenceCommand command, TimeSpan timeout)
        {
            return EndTryCommand(BeginTryCommand(context, command, timeout, null, null));
        }
    
        protected override IAsyncResult BeginTryCommand(InstancePersistenceContext context, InstancePersistenceCommand command, TimeSpan timeout, AsyncCallback callback, Object state)
        {
            return new CompletedAsyncResult<Boolean>(true, callback, state);
        }
    
        protected override Boolean EndTryCommand(IAsyncResult result)
        {
            return CompletedAsyncResult<Boolean>.End(result);
        }
    }

    (You can find the AsyncResult and CompletedAsyncResult as well as the rest of the code in the accompanying project.)

    Wrapping Up

    Like I said, the service is relatively simple. I've been running it in a production environment for the past several months without any issues. The custom activities have been really easy to build and the built-in activities like StateMachine and Parallel have been a joy with which to work. Workflow Foundation has been a lot of fun, honestly.

    One last thing... If you don't want to use InstallShield Limited Edition to create an installer for the service, Microsoft released their trusty Visual Studio Installer as a Visual Studio Extension. You can find it here.

    In the next few blog posts, I'll be talking about the CodeActivity and NativeActivity activities I've built to handle all of the file brokering in the system I built for my employer. Enjoy!

    Update!

    A few people have asked me for the source code to the entire project - including the custom activities I built to create my workflows. You can download it here (4.9 MB).