How to run large number of sub-workflows? Start subworkflow (SQL) action configuration. How to run thousands of instances of WEBCON BPS workflow?
Problem: large number of sub-workflows
Using WEBCON BPS, you may need to register thousands of sub-workflows simultaneously. Unfortunately, start such a number of workflow instances without encountering errors is impossible. Below you will find an example of the WEBCON BPS process configuration that will allow you to run any number of sub-workflows.
Solution
Suppose we have two workflows in our process. The first of them (“Task starter” workflow) has an item list form field, on the basis of which sub-workflows (“Task” workflow) are started.
So, how do we should start bunch of 2000 sub-workflows? If we gonna try to register them at once, form will be loading a while and we gonna get an error message.
To avoid getting errors and long path transition loading we may add an additional system step. In this step instead of registering all tasks at once, they will be registered in smaller packages (for example 250 instances per action). We also need to add workflow control step, which will verify that all tasks have been created. Workflow may looks like this:
On the “Start sub-workflow” path there will be an action that starts the sub-workflows based on the values in the item list form field on workflow instance. But not all of them, only a certain amount. In our case it is 250, although this number may vary. It depends on many variables such as database performance, type of database server hard disks, or the number of operations (actions) performed during sub-workflow starting.
Our “Task” workflow consists of only two steps. It is for illustrative purposes only and is not fully functional:
WEBCON BPS: Configuration
On the “Sub-workflow registration” step there is an timeout configured like below. One minute after entering step Path transition action will be called. There are 2 actions on this path:
- Start sub-workflows – starting another 250 elements, from item list on the workflow instance that have not yet been started
- Count registered – counting number of registered sub-workflows
After passing, the “Start sub-workflows” path, the workflow instance will be directed to the workflow control step. The path selection condition is set below:
Starting sub-workflows as packages: How it works?
Instead of registering all sub-workflows at once, they are registered in smaller packages – so there is no risk of error. The moment workflow instance enter the system step, after a minute, the system will automatically go the transition path and register some of the “Task” workflow instances. If all sub-workflows are registered, the main “Task starter” workflow will end.
Summary
The method described above allows you to handle a large number of sub-workflows. However, it is not without flaws:
- Pros
- Allow to register any number of sub-workflows
- User do not have to wait long time on transition path
- Should not have big impact on overall WEBCON BPS platform performance
- Cons
- But it may took a while because of 1 minute waiting after each package
- User will not see an error if there will be error on sub-workflow registration and whole flow may stuck in “Sub-workflow registration” step
Featured image: Photo by Mika Baumeister on Unsplash