Improving Information Management Software System Deployment Practices

Age : 45 Joined : 08 Mar 2008 Posts : 1850

Responding to a request by the United States Army Program Executive Office for Standard Army Management Information Systems (PEO STAMIS), the Logistics Management Institute (LMI) assisted in a study to improve the deployment of software-intensive information management systems1. At the time of the research, PEO STAMIS was responsible for nearly 40 retail information system products for the U.S. Army and other services.

The products spanned a wide range of functionality and geographical distribution both inside and outside the continental United States. Examples of the kinds of systems for which PEO STAMIS was responsible included the Standard Installation/Division Personnel System, Unit Level Logistics System, and the Joint Computer-Aided Acquisition and Logistics System.

The following are two major research outcomes of interest to the general software community:

Development of a deployment process model applicable to both government and commercial practice.
Identification of a set of at least very good (if not best) deployment practices applicable to both military information systems and the commercial realm.

This article describes the deployment process model and provides the best practices. The full model is available at the CrossTalk Web site www.stsc.hill.af.mil/crosstalk/2003/06/forbes_model.html. We obtained the basic data primarily via structured interviews of PEO STAMIS Program Management Offices, PEO STAMIS customers, and a number of commercial organizations outside of the defense industry. The interview questions that led to the validation of the model constructs and the definition of the best practices are also available at the www.stsc.hill.af.mil/crosstalk/2003/06/forbes_interview_questions.html.

Background

The PEO STAMIS vision was to be the warfighter's choice for leading edge, integrated, global information solutions across the operational spectrum. The perception of the PEO and its customers was that processes within PEO STAMIS for deploying hardware, software, and training - and sustaining them once deployed - were interfering with the realization of the PEO STAMIS vision. Each product manager used product-specific processes for hardware and software fielding, system training, and sustainment. The end result was often customer dissatisfaction - much of which was attributable to inconsistent and sometimes ineffective deployment practices.

The objective of the LMI study was to give the PEO STAMIS several potential strategies that would improve fielding and training, sustainment, customer satisfaction, and reduce life-cycle costs. The LMI was asked to perform five tasks to fulfill its part in the study:

Survey selected projects to assess current deployment practices.
Selectively survey user communities to determine their satisfaction with the current practices for transitioning systems to operational use.
Selectively survey commercial organizations to determine their approach to deploying systems.
Determine commercial practices that improve usability and reduce time and resources during deployment.
Determine a set of best practices for deployment and improving usability.

The survey of commercial practices in use by successful companies was considered to be an important element, since customer satisfaction is recognized as essential to their continuing success.

Methodology

To provide a consistent framework against which to compare the various PEO STAMIS offices' and commercial firms' practices, we created a three-stage (initial, intermediate, and advanced) deployment process model. It comprised 16 general areas (such as computer hardware, software, architecture, and training), nine of which are further broken down into sub-areas, comprising a total of 32 categories. Many, but not all, areas have sub-areas. Three levels of ability (initial, intermediate, and advanced) further characterized each area and sub-area. A listing of areas and sub-areas is provided in Table 1. We obtained information using interview questionnaires keyed to the deployment process model.

In our deployment process model, the initial level is characterized by ad hoc practices. Very little planning is done, and situations are addressed as they arise. Also, little or no consideration is given to identifying potential risks and implementing practices to avoid them. The intermediate level is characterized by some degree of planning, although a number of activities are still addressed informally. Organizations functioning at this level in a specific area are getting by, but their actions are not as effective as they probably could or should be. Detailed planning and risk avoidance characterize the advanced level. An advanced-level organization is focused on functioning as effectively as it can.

We initially populated the deployment process model based on our understanding of what the functional levels would be for each of the areas or sub-areas. We set forth our perception of what practices would exist for each level of the model. We then used the interviews with commercial participants to validate this model. The complete process model, together with the typical practices at each level, is too large to include in this article; however, Table 2 illustrates what a sub-area looks like. The complete model can be viewed at the CrossTalk Web site www.stsc.hill.af.mil/crosstalk/2003/06/forbes_model.html.

Participating companies included a biotech firm, a systems integrator, an oil company, and a major producer of commercial software products. For commercial firms, our interview technique was to ask the respondent one question for each area or sub-area. For each area and each level within the process model, we asked the respondent to describe practices followed by a typical organization within their industry. Our intent was not to make the questions specific to the company, but to have the respondents characterize what they believe the model would look like based on common practices within their segment of the industry. This was done for two reasons:

We wanted to validate our model constructs.
We wanted to ensure the respondent would answer all the questions and not opt out because a response could or would reveal proprietary practices.
Results

The results indicated that the deployment model the LMI constructed reasonably represented the various levels of maturity - or capability - indicated by our model. Some changes were made to the model during the interview process. The bulk of these changes resulted from additional characterizations of the levels proposed by the interviewees.

The interview process identified 16 commercial best practices, all of which were applicable to PEO STAMIS. However their relative merit was not clear. As an aid in implementation, and in conjunction with the PEO STAMIS staff, we evaluated the best practices against the factors contained in each of the seven major areas of the deployment model and then integrated the results to create a prioritized list of best practices. (Evaluation was based on the multi-attribute utility method, i.e., we assigned scores to indicate the importance of each practice to each factor.) We also confirmed that eight of these practices already existed within the PEO. However they were not widely shared or replicated from one product to another.

It should also be noted that although all best practices are in use by commercial firms no one firm used them all. In some cases, a given best practice was identified as being used by some of the respondent companies, while others indicated a desire that their organizations use a similar practice. Consequently, one of the results of the study was to develop a list of best deployment practices useful in industry. Table 3 lists these best practices.

Interviews of PEO STAMIS product offices demonstrated that no product was totally situated in the initial, intermediate, or advanced stage - all had a mixture of characteristics from more than one phase. Although there may have been some unintended inflation in our results, intermediate and advanced stages dominated. Of particular note, product managers had created internal best practices well tailored to the PEO STAMIS environment. The problem was not lack of best practices inside or outside the PEO, but the lack of sharing and replication of best practices across product offices.

Recommendations

The LMI recommended that PEO STAMIS implement a process improvement effort that emphasized replication of best practices. We identified and evaluated five potential strategies. We also recommended implementation of a collaborative process improvement strategy that had at its foundation the best practices identified by this study and the deployment process model created by this study. The deployment process model can provide a uniform groundwork for product manager self assessment, an essential element of improvement.

A collaborative strategy meant its execution included product managers, PEO STAMIS headquarters staff, developers, and users. The product managers were closest to the customers and in the best position to understand real-world problems that needed to be solved. The headquarters, on the other hand, was in the best position to see across products and facilitate replication of best practices. Developers had the best information on functionality embedded in applications. Involvement of user representatives was essential so users could understand what was being attempted, how it was being approached, and how it was expected to effectively address their needs.

We recognized that the PEO could not attempt to fix everything at once; organizations can absorb only so much change at one time, and not all changes are equally beneficial. The working group the LMI supported identified four initiatives that appeared to lend themselves to early implementation and momentum building. These recommendations included best practices that were very comparable to some of the best practices used by commercial organizations:

Use of one particular product as a pilot vehicle to develop a template for replication of best practices. This product was early in its life cycle and was controlled by one of the PEO STAMIS directorates, minimizing the lateral coordination that would be needed.
Replication of the Systems Extension and Acceptance Team (SEAT) fielding practice for other products. This was a PEO STAMIS best practice. The SEAT concept is essentially a team that is responsible for planning and implementing the deployment of systems, but was only used for a limited number of systems. Users specifically recommended expanding the use of the SEAT methodology.
Expanded use of an existing test laboratory for retail-level systems. This is a practice consistent with the use of test beds by commercial organizations to ensure the compatibility of system interfaces. Broader use of this laboratory would facilitate a common approach to testing.
Replication of the three-tier help-desk architecture vision of the Global Combat Support System-Army across additional products. This architecture had the preferred modern features of an excellent help-desk capability.

Deployment of operational or production information systems is a process that many organizations do not always perform well, whether or not we are talking about governmental or commercial organizations. Hopefully, such organizations can benefit from the results of this study. We believe the deployment process model and the interview guides - because they were intentionally constructed to span government and commercial practices rather than those specifically within the sphere of PEO STAMIS - can be valuable to enterprises other than PEO STAMIS. The same is true of the best practices we identified: One result of this study was the determination of a set of good deployment practices that have been applied not only to military information systems, but also to the commercial world.