I am currently working on a horizontally scaled architecture due to the massive workload we are required to process each day. So far my team has been able to automate the creation and configuration of a LF environment (we call it a stack) using a bit of configuration management (ansible) and the various LF SDKs. So far everything is working great and our playbooks can easily create and configure the Repositories, QF Sessions, Import Agent, etc... However, the one application that isn't so easy to automate/script is Workflow.
Since Workflow Server maintains all of it's configuration, workflows, starting rules and conditions in a database with a unique id, it is hard for us to mirror workflows across multiple stacks. Our goal is to achieve a sort of... "multi-tenate" environment where any issues in our process will only affect a portion of our business. In order for this to work, we must be able to maintain an near identical configuration across the stacks.
I started off by building a Master WF server where all changes will be made. Then using the WF SDK I have extracted the MonitoredRepositories, WorkflowObjects, Rules and settings and push them into each of the other stacks. While this seems like it may work... it is incredibly difficult to maintain and I feel like I am re-inventing the wheel. Also, I am not aware of all the business rules inside WF itself to be confident that all the necessary settings are being transferred to these servers.
I am curious if there is a better way to approach this... or if I should continue going this direction?