Introduction
A typical packaged software lifecycle, from the client-organization perspective, is packaged software selection followed by implementation, installation, training, and maintenance (that includes upgrades). Traditional software maintenance has been acknowledged by many researchers as the longest and most costly phase in the software lifecycle. This fact is no exception in the ERP packaged software maintenance context (Moore, 2005; Whiting, 2006).
According to Ng, Gable, & Chan (2002, pg. 100) ERP maintenance is defined as “post-implementation activities related to the packaged application software undertaken by the client-organization from the time the system goes live (i.e., successfully implemented and transported to the production environment), until it is retired from an organization’s production system, to keep the system running; adapt to a changed environment in order to operate well; provide helps to the system users in using the system; realize benefits from the system (best business processes, enhanced system integration, cost reduction); and keep the system a supported-version and meet the vendor’s requirements for standard code. These activities include: implementing internal change-requests (initiated by an ERP-using organization’s system users and IT-staff); responding or handling user-support requests (initiated by an ERP-using organization’s system users); upgrading to new versions/releases (introduced by the vendor); and performing patches (support provided by the vendor).”
In order to achieve the abovementioned maintenance objectives of keeping the ERP system running, adapting the system to a new operating environment, and ensuring the system up to the vendor’s requirement for standard code; and realizing benefits such as competitive advantages from the system, the IT department staff has to collect some metrics or relevant data on patches and modifications done to the ERP system so that they can know or can tell the status and the performance of their maintenance activities. The authors in Fenton (1991), Fenton & Pfleeger (1997), and Florac (1992), agree that software maintenance data are useful for planning, assessment, tracking, and predictions on software maintenance. Although, there is a lot of literature on ERP, we find almost no literature on ERP maintenance metrics.
Thus, this text is meant to provide some fundamental metrics on ERP patches and modifications which could be useful for ERP maintenance management in order to answer questions on the state of their ERP system, their patch implementation costs, and the ongoing maintenance costs for their previous modification or custom development.
Background: Metric
The IEEE Standard Glossary of Software Engineering Terminology (1990) defines a metric as a “quantitative measure of the degree to which a system, component, or process possesses a given attribute.” Based on the definition, this text interprets a metric as being derived from data, and as quantifiable, meaningful, and used for strategic, tactical and/or operational purposes. Data (or data item), in turn, is defined as a quantitative indication of the extent, amount, dimension, capacity, size, or characteristic of particular attributes of a task or activity in a process. It can be collected using forms (e.g., change request form, change report, software engineering report), interviews (with the users, testers, programmers, analysts, managers), and via computerized systems (e.g., the in-built change management system in ERP, change request database). A goal/question/metric (GQM) paradigm is a systematic way of collecting predefined data, with intended goal(s), and the associated sets of predetermined questions, in order to derive the anticipated measurable metrics. Basili and Weiss (1984) advocate this methodology for collecting valid data. In GQM, which is also known as the top-down approach, the timing (in terms of the software life cycle or activity), interviewees, and reasons for collection are all predetermined.
The literature reports that successful use of measurement/metric program avoids recurring errors (Ebert, Dumke, Bundschuh, & Schmietendorf, 2005), improves software maintenance processes at Burrough Corporation (Rombach & Ulery, 1989), and Motorola (Smith, 1993), and improves product quality at AT&T (Fenton & Pfleeger, 1997).
On the flip side, a known metric – together with the context for its interpretation – can determine what data might be collected (Rombach & Ulery, 1989). There are three main purposes of metrics: assessment, prediction, and control.
Table 1. Main characteristics and purpose of three main metric categories
| Metric category | Main characteristic | Purpose | Example of ERP maintenance metric |
| Assessment | Informative (about person, process, object); may be used in decision making or for controlling purposes | Retain knowledge | Programmer ID, problem description, description of changes, issues of consideration, patch ID |
| Prediction or decision-making | Conclusive; most likely derived from the assessment and control metrics; usually describes what should be done | Make decision, planning and estimation | Estimated maintenance time, quotation for a maintenance request, action to be taken, maintenance request type, projected availability |
| Control | Indicative (indicating that something needs to be done); most likely used to pinpoint that a particular decision needs to be made; usually requires data to be collected over a period of time; usually has some attached baseline value | Monitor
performance, track progress, identify problem |
Problem status, approved by, accepted by, maintenance request ID, time of problem occurrence, resolution impact |
Table 2. Application of software metrics in practice
| Purpose | Case name | Metrics used | Goal |
| Assessment | Hewlett-Packard (Wood, 2003). | Event chronology, problem symptoms, diagnostic information, release version information | Problem analysis and resolution |
| Decision-making | NASA’s Mission Operations Directorate (Stark, Durst, & Vowell, 1994) | Previous project delivery rate | Estimate a test schedule |
| Hewlett-Packard to (Grady, 1994). | Defect trend | Determine time to release a product | |
| Control and monitoring | NASA’s Mission Operations Directorate (Stark, et al., 1994) | Earned value management technique: project cost and schedule | Monitor project cost and schedule performance |
| Defect density | Track quality in subsystem, efficiency in testing, and backlog of fault | ||
| Hewlett-Packard (Grady, 1994) | Code size and time | Monitor project progress | |
| Bull’s Arizona (Weller, 1994) | Effort, resources, product size, estimated completion date and defect detected | Manage project and improve project planning | |
| Siemens (Paulish & Carleton, 1994) | Defect rate (i.e. number of defect/ product size) | Measure performance | |
| Product size and effort (i.e. product size/effort) | Project productivity | ||
| Productivity gain, error detection rate, and reduction in time to market | Measure software process improvement |
Table 1 shows the main characteristics and objectives of these three metric categories. In addition to this, their application in practice is provided in Table 2.
APPLYING METRICS IN THE ERP MAINTENANCE (AND UPGRADE) CONTEXT: AN IMPLICATION FOR MANAGEMENT
The following discussion focuses on the practical application of the data items or software metrics. As stated earlier, metrics are derived from the data items collected from the activities that produce them. The metrics covered in this section are: (1) the state of the current system relative to the vendor’s expectation, (2) the number of modifications or custom developments likely to be affected by each patch implementation, (3) patch implementation costs, (4) effort needed to reapply a previous modification or custom development, and (5) ongoing maintenance costs for modification requests. As shown later, these metrics are developed by manipulating (summation, division, multiplication, etc.) simple data items that are directly collected. Maintenance managers can use these metrics to manage maintenance activities for tactical and operational management decision-making. Table 3 provides the notations used in the abovementioned five metrics.
In addressing the manager question of: How up-to-date is my ERP system compared to my vendor’s standard code?
With the aim of achieving economies-of-scale, many organizations “batch” the patches instead of implementing them as they arrive. Thus, in order to measure how up-to-date or state-of-the-art an installed ERP system is relative to the vendor’s patch introduction for a given version, one needs to know how many patches have already been implemented, that is:
In addressing the manager question of: On average, how many modifications are affected by a patch implementation and what does it cost to implement a patch?
Table 3. Notations for metrics
| Context | Notation | Metrics description | How it is measured? |
| User
organization |
k | Probability that a modification or custom development will be affected by future patch implementation | Done by studying the ‘description of changes’, ‘related changes’ and ‘issues of consideration’ related to that modification |
| L | Labor rate | [a direct data item] | |
| Previous modification | M | Number of modifications and/or custom developments | Summing up all the previous modifications based on previous record on it |
| ti | Effort required to conduct impact analysis for a modification | [a direct data item] | |
| tr | Effort required to reapply a modification | [a direct data item] | |
| T
a&t |
Effort required to apply and conduct complete testing for a modification | [a direct data item] | |
| Patch | N | Number of patch projects | Summing up all the previous patch projects based on previous record on it |
| pi | Number of patches in the i-th project | Each project may have different number of patched implemented at a time. This is usually depending on economy-of-scales and/or urgency of some patches | |
| tp, | Average effort required to implement a patch in the i-th project | Dividing the total effort in the i’-th project by the total number of patches in the i-th project | |
| rp | Patch introduction rate | Number of patches introduced by the vendor per year | |
| ra | Patch implementation/application rate | Number of patches implemented by the client-organization per year |
The IEEE 1219 – Standard for Software Maintenance (1998) suggests that cost and benefit analysis is required in order to determine the feasibility of a maintenance request, and to quantify the long-term or ongoing cost of a maintenance request. Patch implementation effort is believed to increase, to some degree, depending on the number of modifications already made to an ERP system (Ng, 2001). Patch implementation involves replacing some portion of the existing customized ERP code with the vendor’s standard code. It can overwrite existing modifications or custom developments. However, not all modifications or custom developments are necessarily affected by each patch implementation. It is observed in a study by Ng (2001) based on an upgrade experience that only certain portion of them are affected. (Note that although a patch implementation is a small scale of an upgrade implementation, both of them could have overwritten effects on modifications done on existing system.) Therefore, the number of previous modifications or custom developments probably affected by each patch implementation, m, is:
As a result,
Patch implementation costs = average number of patches per project X average effort per patch X labor rate + effort required to reapply previous modifications X labor rate, or:
In addressing the manager question of: How much efforts is needed to reapply previous modification, and what are the ongoing maintenance costs for a modification request?
As modifications or custom developments may be overwritten during patch implementation, extra effort is needed in order to check whether such overwriting has occurred. If the affected modifications and custom developments are still needed, they may have to be reapplied to ensure that they can operate as they did before the patch(es) were implemented. Although activities such as analysis, design, and coding do not need to be repeated, complete testing (involving integration, validation, and system testing) for the previous modifications is required. These additional efforts generate the so-called ongoing maintenance costs for modification request. Hence, the major effort to reapply a single previous modification or custom development, TR, is:
The greater the number of patch projects that must be carried out in an existing system, the greater the likelihood of ongoing maintenance costs associated with a modification in that system. Assuming that all modifications and custom developments will be affected by the patch and that the vendor will incorporate these modifications and custom developments into the new version. they are no longer needed after the upgrade. A simple formula for ongoing maintenance costs for a modification is as follows:
Ongoing maintenance costs = Number of patch project X effort to reapply the modification X labor rate
Thus, with these metrics, maintenance manager will be in a better position to charge the ongoing maintenance cost of the modification to the respective user department. Alternatively, this information can be used to convince the system user department to delay or forego the maintenance, based on a more accurate assessment of total perceived benefits and estimated costs.
future trends
Many organizations start to recognize the importance of collecting ERP system’s performance metrics such as inventory level, operational costs, schedule compliance, and on-time delivery (Jutras, 2007). Like other software systems, ERP system is not a panacea for all. ERP is built to be generic in order to meet a wide range of clients needs. Thus, while some clients try to change their business processes, some change the standard code in order to incorporate their unique business processes where they perceive critical for competitive advantages. Also, although ERP software provides a comprehensive set of business applications that can serve different departments under a single system, most companies still maintain their idiosyncrasy legacy systems and some buy best-of-breed packaged software from different vendors. These are because of unwillingness or resistance to replace a system that is still working, and the packaged software does not meet their requirements or is simply not good enough. Besides, there could also be owing to economy slowdown, wanting to avoid vendor lock-in, waiting for the Web-application (erpwire.com, 2006b) and mobile-application (erpwire. com, 2006a) to mature, and so forth. However, multi-system leads to integration problem. As a consequence, time and effort need to be invested in order to make disparate systems talk to each other. Some perceive that the answer to this is the Web and mobile application. But, as far as Web application is concerned, there are a number of issues yet to consider and address, that is, cost, benefit, network bandwidth and infrastructure, threats and risks, security, and ethics. Also, developing mobile ERP application can be very complicated due to non-standardization in mobile devices (e.g., various sizes and shapes), and small screen display; it’s not simply the broadband or bandwidth boost issue. Moreover, in light of growing customer base of the ERP system, it is valuable to investigate how ERP maintenance and upgrade could be better managed. How patches implementation can be better queued and implemented to achieve economy-of-scale? What and how process model will best describe activities in ERP maintenance and upgrade?
KEY TERMs
ERP Modification: A type of maintenance request which results in changes being made to the existing ERP (standard) code and custom objects being created.
ERP Software Maintenance Data (or Data Item):
ERP software maintenance quantitative indication of the extent, amount, dimension, capacity, size, or characteristic of particular attributes of a task or activity in a process.
ERP Software Maintenance Metric: ERP software maintenance measure that is derived from data, and is quantifiable, meaningful and used for strategic, tactical, and/or operational purposes.
ERP Vendor’s Standard Code: Program code that is without any modification or custom development at all.
ERP Vendor’s Supported-Version: An ERP version that is still supported by the ERP vendor. This means that the vendor will provide patches for bug fixes and minor enhancements for this version.
Ongoing Maintenance Costs: Additional costs, besides the initial implementation costs, incurred as a result of up-keeping some software code or objects created in previous maintenance request.
Patch Implementation Costs: Costs incurred in implementing (usually) a batch of patches provided by the ERP vendor. As custom code or previous modifications may be overwritten while implementing the patches, usually these costs also include the costs of reapplying previous modifications.
