Database Reference
In-Depth Information
Figure 2.8
Model operationalize phase
While scoping the effort involved in conducting a pilot project, consider running
the model in a production environment for a discrete set of products or a single
line of business, which tests the model in a live setting. This allows the team to
learn from the deployment and make any needed adjustments before launching
the model across the enterprise. Be aware that this phase can bring in a new set of
team members—usually the engineers responsible for the production environment
who have a new set of issues and concerns beyond those of the core project team.
This technical group needs to ensure that running the model fits smoothly into the
production environment and that the model can be integrated into related business
processes.
Part of the operationalizing phase includes creating a mechanism for performing
ongoing monitoring of model accuracy and, if accuracy degrades, finding ways
to retrain the model. If feasible, design alerts for when the model is operating
“out-of-bounds.” This includes situations when the inputs are beyond the range
that the model was trained on, which may cause the outputs of the model to be
inaccurate or invalid. If this begins to happen regularly, the model needs to be
retrained on new data.
