Public Sector Case Study on the Benefits of IS/IT

INTRODUCTION

While organisations continue to invest heavily in IS/IT, research studies and practitioner surveys report contradictory findings on the effect of the expenditures on organisational productivity (Grover et al., 1998). Therefore, it is not surprising to see that the term “productivity paradox” is gaining increasing notoriety as several studies point toward fairly static productivity and rising IS/IT expenditure. Despite large investments in IS/IT over many years, it has been difficult to determine where the IS/IT benefits have actually occurred, if indeed there have been any (Willcocks & Lester, 1997). The main objective of this article is to generate a deeper understanding of issues surrounding the current practices and norms in managing IS/IT benefits and investments evaluation.

RESEARCH APPROACH

A case study was conducted to investigate the practices of IS/IT investment evaluation and benefits realisation in large Australian organisations. Semi-structured interviews were used to gain a deeper understanding of issues.

CASE DESCRIPTION

Seven interviews were conducted with participants from a Western Australian state government department and two major outsourcing contractors. The questions asked related to the formal benefits realisation methodology used, major outsourcing contracts, contractual relationship between department and contractors, and IS/IT investment evaluation methodology or technique deployed. Other data collected included contract documents, planning documents and minutes of relevant meetings. Around 80 pages of transcripts were coded and analysed. The analysis was conducted in a cyclical manner and followed guidelines for interpretive research set out by Klein and Myers (1999).


CASE STUDY RESULTS

A number of issues arose from the analysis of this text data and the key issues are presented next.

Issue 1: Lack of formal IS/IT investment evaluation methodology

Most of the participants claimed that some sort of formal methodology or process was put in place for evaluating these contracts. However, closer examination revealed that what was described did not constitute a formal IS/IT investment evaluation methodology. Participants wrongly considered various contract control mechanisms as a formal IS/IT investment evaluation methodology.

Issue 2: A formal IS/IT benefits realisation methodology was used

Every participant was aware that a formal IS/IT benefits realisation methodology was being used for outsourcing contracts and projects. A benefits realisation approach was used as an end-to-end process to assist in: (1) providing a rigorous process to select the right projects; (2) placing responsibility and accountability at the appropriate level; (3) driving process re-engineering through changes in the organisation; (4) ensuring benefits are realised; and (5) ensuring agreed re-investment of time savings applied as expected.

Issue 3: Lack of understanding of IS/IT investment evaluation methodology

The confusion indicated in Issue 1 about what constitutes a formal IS/IT investment evaluation methodology demonstrated a lack of understanding of such methodologies. This may be due to the fact that the department was unable to introduce a formal IS/IT investment evaluation methodology because it was required to follow the state government’s outsourcing guidelines (MOPC, 2000; SSC, 1999b).

Issue 4: Existence of an informal IS/IT investment evaluation process

Despite the fact that no formal IS/IT investment evaluation methodology or process was used, contract control and evaluation mechanisms specified within the SLAs or government guidelines do represent an informal IS/IT investment evaluation process. Although these informal mechanisms may not totally replace a formal methodology (e.g., Kaplan and Norton’s (1992) Balanced Scorecard), they were able to assist in evaluating the performance of the outsourcing contracts. These mechanisms were largely based on the standard state government contract process and purchasing guidelines (SSC, 1999a, 1999b).

Issue 5: Good understanding of benefits realisation practices

A benefits realisation methodology, called benefits realisation approach, was introduced before outsourcing IS/IT functions because there was a concern that IS/IT investments did not deliver value (which would jeopardize future government funding). To ensure that the IS/IT investments deliver the promised value and benefits as well as bring the focus back to the department’s main business, a large internal change program was required.

Issue 6: Focus on quantitative IS/IT investment evaluation measures

Without employing more qualitative measures (e.g., relationship, culture and leadership) and a formal IS/IT investment evaluation methodology or process, the use of quantitative or accounting-based measures alone did not assist in full evaluation and monitoring of the performance.

Issue 7: Different motivations for outsourcing

Several reasons were put forward as the main motivation for IS/IT outsourcing. Only two of the four contractor representatives cited access to the required technical expertise as one of the department’s reasons to outsource and two either did not know or did not respond to the question. However, all of the department’s participants mentioned access to required technical expertise as a major reason to outsource some IS/IT functions. Therefore, the department’s motivation for outsourcing was somewhat different from the contractors.

Issue 8: Success of the contracts perceived differently by stakeholders

Customer satisfaction, achieving the contractor’s projected revenue, bringing value/benefits to the organisation, and meeting the SLA provisions were mentioned. Other criteria mentioned included technical competence to deliver what is required, risk factors, contractors’ experience in a relevant area, and business continuity of the contractors. Both representatives from the first contractor mentioned achieving the projected revenue for themselves and satisfying customers as their only criteria for determining the success of their outsourcing contracts with the department. This may indicate that the first contractor’s aim is to maximise the profit while maintaining a certain level of customer satisfaction. However, participants from the department seemed to have used different criteria for determining the success of the outsourcing contracts. Bringing value/benefits to the organisation, meeting the SLA provisions, and pricing/cost were mentioned by three out of four participants.

Issue 9: Better control over the IS/IT skill shortage within the department

Access to the required technical expertise was the most often cited reason for outsourcing. The department did not have the required IS/IT expertise to implement a major internal change program and outsource some of its IS/IT functions. In order to obtain the required technical expertise and skills from outside, the department had to transfer some of its IS/IT staff to the first contractor. According to most of the participants, those who went across to the first contractor were quite happy about the whole process. However, the department did not lose the capacity to manage and assess its own IS/IT needs after relinquishing the control of some of its IS/IT functions and staff, as has happened in many outsourcing organisations previously (Currie & Willcocks, 1998).

Issue 10: Embedded contract mentality

Staff of the department seemed to have a “contract mentality,” as the operation of the contracts was based on the specifications set out in the SLAs within the outsourcing contracts. Several participants clearly indicated that there was a pre-agreed set of evaluation and control mechanisms in the SLAs within the outsourcing contracts such as metrics, monthly reports, reviews, and regular meetings. Moreover, half thought these contract control mechanisms were all part of the IS/IT investment evaluation methodology.

Issue 11: Lack of user involvement/ participation in contract development

There appeared to be an organisational memory gap where units within the department possessed knowledge of different parts of the IS/IT systems development cycle. However, the knowledge did not seem to be shared by all units because different units participated in different stages. The department’s whole outsourcing process may have been more successful if the participants were involved in both original tendering and outsourcing contracts negotiation as well as benefits realisation process.

Issue 12: Conflict between motivations and success criteria for outsourcing

It seemed that the participants had different expectations regarding the outsourcing as a whole and the outsourcing contracts. Alternatively, they may have felt that the outsourcing contracts had already brought in the required technical expertise and should not be used as a criterion for determining contract success

Issue 13: General lack of commitment by contractors

It was easy to see that department and contractors had different agendas in mind, despite the fact that the contract was a partnership-type arrangement. The first contractor’s criteria for determining success of outsourcing contracts seemed to be maximisation of profit/ revenue while keeping the customers satisfied. The contractors’ lack of commitment could also be demonstrated by the fact that they either did not know why the department outsourced, or could not agree on a single reason for outsourcing. However, the department agreed that access to required technical expertise was a reason for outsourcing.

Issue 14: Restrictive government outsourcing contract guidelines

All state departments have to follow the contract tendering and outsourcing guidelines set out by the State Supply Commission (SSC, 1999a, 1999b). This was one reason why the department failed to adopt a formal IS/IT investment evaluation methodology. The state government guidelines specify numerous principles, steps, and procedures for state departments to follow when dealing with external contractors. However, they do not determine what formal IS/IT investment evaluation methodology a department should use.

Issue 15: Benefits realisation methodology provides rigorous processes

As mentioned in Issue 1, the benefits realisation approach provided the department with rigorous processes for achieving benefits delivery. These processes did not exist prior to its introduction. Instead, some crude methods had been used in an attempt to realise benefits within the organisation. These methods included budget cuts, projects statements presentations, and activity surveys. The introduction of the benefits realisation approach provided the department with rigorous processes.

Issue 16: Ability to manage the outsourcing contracts without external influence and assistance

To acquire technical expertise and skills, the department had to outsource some of its IS/IT functions and transferred some of its IS/IT staff to the first contractor. However, unlike other outsourcing organisations, the department appeared able to manage its outsourcing contracts internally without external influence or assistance. This may be because the department had transferred as few IS/IT staff to the first contractor as possible and, simultaneously, benefited from the second contractor’s IS/IT technical expertise obtained under the contract.

CONCLUSION

This case study was conducted in a large government department with a mix of insourced and outsourced IS/IT activities. While the department appears to operate without any major problem, the mostly negative issues shown previously indicate weaknesses in the way it deals with the level of formality and integration in applying the methodologies. The problems mentioned in Issues 6 -14 were mostly caused by the lack of attention to IS/IT investment evaluation (Issues 1, 3, and 4). For example, if formal IS/IT investment evaluation was adopted by the department, more qualitative measures may been used to evaluate the outsourcing contracts (Issue 6).
So why didn’t the department formally evaluate its IS/ IT investments? One possible explanation was that the restrictive nature of the state government’s outsourcing contract guidelines (Issue 14) made it difficult to implement a formal IS/IT investment evaluation methodology. Another explanation was that none of the IS/IT staff was familiar with the formal IS/IT investment evaluation process and hence possessed an “embedded contract mentality” (Issue 10) by simply following conditions set out within the SLAs. Seddon et al. (2001) suggest that under some circumstances, cost of formal IS/IT evaluations may seem likely to exceed benefits. However, several positive issues (Issues 9, 15, and 16) from this case study indicate that the use of a benefits realisation methodology enabled greater control and better management of outsourcing contracts.
Despite large investments in IS/IT over many years, it has been difficult for organisations to determine where benefits have occurred, if indeed there have been any. IS/ IT investment evaluation practice remains a hotly debated topic in the IS literature. Little published work has been conducted in Australia and there is still a lot to be learned in the area of processes and practices of IS/IT investment evaluation and benefits management. We hope that more studies of the practice of IS/IT investment evaluation will benefit other researchers in this field and the business community as a whole. Through the research program introduced in this article it is hoped that better approaches may be developed for Australian organisations.

KEY TERMS

Benefits Management: A managed and controlled process of checking, implementing and adjusting expected results and continuously adjusting the path leading from investments to expected business benefits.
Benefits Realisation: See Benefits Management.
IS/IT Benefits Realisation Methodologies: Approaches that are used to ensure that benefits expected in the IS/IT investments by organisations are realised or delivered.
IS/IT Investment Evaluation: This is the weighing up process to rationally assess the value of any acquisition of software or hardware that is expected to improve business value of an organisation’s information systems.
IS/IT Investment Evaluation Methodologies: Approaches that are used to evaluate organisations’ IS/IT investments.
IS/IT Outsourcing: The process of transferring IS/IT assets, staff, and management responsibility for delivery of services from internal IS/IT functions to external contractors.
Productivity Paradox: Despite large investments in IS/IT over many years, there have been conflicting reports as to whether or not the IS/IT benefits have actually occurred.

Next post:

Previous post: