System Integration and Re-engineering Using XML/Web Services
Dr. Michael J. Hu
Police Information Technology Organization (PITO)
New King’s Beam House, 22 Upper Ground
London SE1 9QY
Telephone: +44 208 358 5402
Email: michael.hu@pito.pnn.police.uk
ABSTRACT
Use of latest technologies, such as the Internet, to enhance the government services to the public in the United Kingdom has been underpinned by the British government’s emphasis of several key e-initiatives, including e-policing and integration of criminal justice IT (CJIT) systems. This paper presents an overview of several on-going and planned system integration and re-engineering programmes in the Police Service and CJIT in the UK using XML & Web Services, as well as key findings and lessons learnt from undertaking these programmes.
General Terms
Algorithms, Management, Performance, Reliability, Experimentation, Security, Standardization.
Keywords
XML, Web Services, system integration, system re-engineering.
A wide range of IT systems have been developed during the last decades within the fifty-two police forces in the United Kingdom. Many of these systems are currently being upgraded with added functionality, while new systems are continuously being developed and rolled-out. Integrating such a wide range of new and legacy systems in the Police Service, and linking these systems to those in the CJIT communities (including the British Crown Court, the Crown Prosecution Service, National Probation Service, etc.) and other government agencies (e.g., the Home Office), has been undertaken under the guidelines of an national Information System Strategy for the Police Service published in 2002 [1]. Further effort has been made to provide European Union-wide or even global police intelligence & investigation, and counter-crime capabilities. A number of system integration and re-engineering models have been deployed at the national and regional/local levels. This has resulted in a number of different integration strategies and migration paths towards the XML/Web Services based enterprise information architecture that are discussed in this paper.
In this paper, we review each of these system re-engineering models, and their convergence to XML/Web Services based enterprise information architecture for the Police Force in the UK. In Section 2, we first outline the enterprise information architecture, as a foundation for integrating and modernizing the Police IT systems in the United Kingdom. In Section 3, we discuss each key re-engineering model in more detail, as well as the potential options for migrating it to the enterprise architecture. We also present key issues and lessons learnt in implementing these models, as well as our comments and recommendations with regard to the Web Services Architecture that has been proposed recently by the Web Services Architecture Working Group.
It is envisioned that the XML/Web Services based enterprise information architecture for the Police Service constitutes a three-tier architecture, as it is shown in Figure 1.
Figure 1 Enterprise information architecture for the Police Service
XML has been widely adopted internationally as the lingua franca for data interchange between computer systems; its impact on current and future development in all public and private sectors world-wide is enormous. In the UK, XML has been adopted as the basis for data standard and data interchange in all government ministries and agencies, through the e-Government Interchange Framework (e-GIF) [2]. Web Services, as it is defined by the W3C, is a software system identified by a URI whose public interfaces and bindings are defined and described using XML. As a result, its definition can be discovered by other software systems (as service requestors). These systems may then interact with the Web Services in a manner prescribed by its definition, using XML based messages conveyed by internet protocols [3][4][6].
The XML/Web Services based enterprise architecture provides a loosely coupled (and therefore more flexible) yet robust environment, that complies with the Information System Strategy for the Police Service. With such architecture, service requestors do not have to worry about the location of information assets and enterprise services requested, nor the operating systems/ programming languages/component models used to create or access these enterprise services. It is, therefore, ideal for integrating, modernizing and re-engineering Police IT systems.
While a number of large-scale system integration and modernization programmes have been undertaken or are being planned in the Police Service and in CJIT in the UK, most of them are currently based on the re-engineering models outlined in Section 1. The forward plan is to converge these different models and strategies using the XML/Web Services based enterprise architecture shown in Figure 1. In this section, we discuss two key elements as pre-requisites for such convergence: 1) A standard data structure or data model, and 2) An enterprise-wide component structure or catalogue. Thereafter, we discuss these different re-engineering models in more detail, focusing on their deployment and the key issues that we have encountered.
In our experience, two key enablers to convergence are: 1) A shared data structure or data model across the entire enterprise; and 2) A unified component structure shared by enterprise services.
Corporate Data Model (CorDM)
Establishing a standard data model as a baseline for legacy systems, new applications, and emerging enterprise information solutions, is the first step towards the XML/Web Services based enterprise architecture. The data model provides the vocabulary (e.g., data types etc.), based on which the XML schema for inter-service messaging can be developed. This makes it possible for the enterprise service providers to announce their services using the Web Services Description Language (WSDL) [4].
Secondly, a standard data model provides the standardized reference point for system integration and re-engineering. In other words, it provides a data-layer convergence point for legacy systems and new enterprise information solutions for the Police Service, while the former are undergoing the modernization and data migration process.
PITO has been leading the definition and implementation of a Corporate Data Model (CorDM) that provides common data standards for the information used in the Police Service and CJIT community in the UK [5]. One of the key inputs has been the UK government's the e-Government Interoperability Framework (e-GIF) published by the Office of e-Envoy [2]. The CorDM also formulates the basis for common message schema used for inter-service data exchange and transmission using Simple Object Access Protocol (SOAP) [3][4].
One of the key lessons learnt in a pilot project in the Metropolitan Police Service (MPS) is that adopting CorDM for service publication by legacy systems, and using it as a base for message schema for SOAP messaging, is absolutely key to the success of integrating these legacy systems, and moving towards the XML/Web Services based enterprise architecture [10]. Another lesson learnt in the project was that mapping the legacy data format(s) to the standard message schema was a lengthy and extremely difficult task. As the result, in the earlier stages of the project, publication of MPS existing services was not implemented in full, and mapping the legacy data formats to the standard message schema was not possible though originally planned, due to time constraints and technical difficulties. This resulted in a partial implementation of MPS’ Web Services, in which SOAP messages were exchanged between one legacy system (as service requestor) and another (as service provider) in the format of system-to-system messaging, or XML-RPC (remote process call). As the message format is proprietary between these two systems, and binding is constrained to two specific port (types), it drastically limited the potential of the system architecture.
Corporate Component Catalogue for the Police Service
To fulfill the vision of providing independent and distributed enterprise services available over the Internet, which are usable on any platform from any development language, and are reusable/sharable among different services, a standard component structure or catalogue is needed.
In the UK, we are at the inception and planning stage of developing a Corporate Component Catalogue for the Police Service. In preparation, several local or regional forces and the Police National Computer (PNC) have been piloting and testing some components that are developed either in house, or by third parties [10].
3.2 CONVERGENCE PATHS
Convergence of different system integration and re-engineering models that are currently deployed in the Police Service in the UK, to the XML/Web Services based enterprise architecture, is considered an important step towards the Information System Strategy for the Police Service. Different migration strategies may be adopted by each individual police force as part of its system re-engineering and modernization process, taking into consideration its existing system architecture, legacy application portfolio and on-going service development.
3.2.1 From system-to-system messaging to Web Services
Transforming system-to-system based messaging architecture to the XML/Web Services based enterprise architecture constitutes the challenge that many police forces face in the UK. The key is to upgrade the messaging component from the traditional XML RPC to the more loosely coupled SOAP messaging, by standardizing the message format(s) and introducing a more flexible service invocation process using WSDL and SOAP. Key steps in such a transformation process can be summarized as follows:
It is noted that the features included in the basic and extended Web Services Architecture published by the Web Services Architecture Working Group are not adequate for our implementation. This will be discussed in more detail in Section 4.
3.2.2 Web Services: registration, publication, and discovery
The UK Police National Computer (PNC) is currently using the request-and-response broker based architecture to provide a wide range of centralized services to its users and business partners. Its convergence to the XML/Web Services based enterprise architecture is currently being defined and tested within the PNC Modernization Programme. Key activities within this programme are discussed below:
The first challenge is to set up and provide a standard PNC registry, using the Universal Description, Discovery and Integration (UDDI) standard [3][4][6][7], and to publish PNC’s services and interfaces. As it is shown in Figure 2, the PNC UDDI Registry includes the entities/information stored in PNC Data Servers and a range of distributed enterprise services that PNC offers to its users and partners, as well as those offered by its partners. The PNC UDDI Registry provides the standard UDDI service functions including:
Figure 2 Police National Computer (PNC)
There has been some discussion in the PNC Modernization Programme about redefining the Police National Computer as the Enterprise Nerve Centre (ENC) for the Police Service and the CJIT communities and as the main gateway of these communities to access other e-government initiatives in the UK. We are also currently reviewing PNC’s strategic position in the context of XML/Web Services based enterprise architecture for the Police Service, in which centralized, federated, and distributed enterprise services co-exist with one another.
Figure 4 Secured UDDI Registries
Furthermore, these UDDI registries (including PNC UDDI Registry, regional/local UDDI registries, etc.) will become part of Internal Enterprise Application Integration UDDI, a secured UDDI Service Cloud, sitting inside the secured/confidential environment of Police National Network (PNN) and Criminal Justice Exchange (CJX), as it is shown in Figure 4.
The PNC UDDI Registry can also play a key role in the system/service re-engineering process in PNC, when: 1) Some legacy data or systems may be relocated from the centralized PNC mainframe to some regional data centres, or even to local forces; 2) Some legacy data or systems may be replaced by upgraded enterprise services; 3) New enterprise services are introduced and rolled out, either in PNC, or somewhere else, but registered in the PNC UDDI Registry.
Figure 5 System re-engineering and management using PNC UDDI Registry
Figure 5 shows a conceptual view of system re-engineering scenarios using PNC UDDI Registry, migrating PNC legacy systems [(1) in Figure 5] to some interim solutions (i.e. upgraded or wrapped version of these systems, as shown in (2), and finally to the new enterprise information solutions (3). During such a migration process, it is only necessary to update the information stored in the PNC UDDI Registry. This simplifies security and increases system reliability as it prevents conflicting changes at different operator nodes.
3.2.3 From data gateway to XML/Web Services based enterprise architecture
Figure 6 Centralized and regional data centres
Migrating the data gateways/data centres to the XML/Web Services based enterprise architecture requires separation of the above features and functionality, and replacing and upgrading the service using UDDI, WSDL and SOAP messaging (Figure 7), as described in more detail as follows:
Figure 7 Enterprise services for the Police Service
The Web Services Architecture Working Group (WSA WG) has published the Web Services reference architecture, which includes the basic architecture and extended architecture [3][4][6]. The Web Services Architecture places into relationship various components and technologies that comprise a Web Services "stack" or functional implementation. While valid implementations may include subsets or parts of the stack, they must at least provide the components defined in the basic architecture. Components and technologies that extend the basic architecture are represented within the extended architecture.
The basic Web Services Architecture defines an interaction between software agents as an exchange of messages between service requesters and service providers. It includes Web Services technologies capable of:
- Exchanging messages
- Describing Web services
- Publishing and discovering Web Service descriptions
The extended Web Services architecture incorporates additional features and functionality by extending the technologies and components defined within the basic Web Services Architecture. A partial list of these features includes:
- Asynchronous messaging
- Attachment
- Caching
- Messaging exchange pattern (MEP)
- Reliable message
- Message authentication
- Message confidentiality
- Message integrity
- Message routing
- Management messages
Some of these features, such as asynchronous SOAP messaging have been implemented in the various projects in the police forces such as the Metropolitan Police Service. Reliable SOAP messaging was implemented in our pilot projects with acknowledgement from the service providers, and using caching at the messaging components. Other advanced features, such as several messaging exchange patterns, are planned to be implemented in later stages of these projects.
A number of issues have been identified in our projects, which we believe have not been fully addressed by WSA WG. We summarize these issues as follows, as comments to the Web Services Architecture published by WSA WG [3], and as recommendations to the Web Services software developers/ manufacturers:
Messaging Exchange Pattern (MEP) An MEP is a specialized form of feature that describes a generalized pattern of message exchange between two services [3][4][8][9]. We found that the MEPs published by WSA WG are not comprehensive enough to cover some of the messaging patterns in our implementations. For instance, issues emerged in one of our projects that some or all of the following MEPs would be needed for the advanced features designed in the system:
We believe that some of these, called the composite MEPs, could be constructed by using a number of primitive MEPs or other composite MEPs. It is suggested that WSA WG look into the issues. If it is allowed to construct a MEP from other MEPs, rules should be specified as part of the Web Services Architecture.
Message authentication, confidentiality, security, and request/access control There exists a comprehensive security and confidentiality marking scheme covering all the enterprise services/networks in the Police Service, as well as all the entities and information stored in the Police IT systems. We believe that SOAP does not currently provide adequate message authentication and confidentiality features. We also identified that the security issue of Web Services needs to be addressed. Furthermore, request/access control should be enforced by the service provider before responding to a service request, ensuring that the confidential services and information would only be sent to the adequately "vetted" service requestors. Such an issue becomes even more challenging if other MEPs such as cascading messaging are permitted in the enterprise architecture.
Private UDDI registries We foresee two types of UDDI deployment scenarios in the Police Service and the CJIT community: 1) Internal Enterprise Application Integration UDDI - a secured UDDI Service Cloud inside the Police intranet [i.e., the Police National Network (PNN) and Criminal Justice Exchange (CJX)], which allows applications in different police forces, PNC, and other "vetted" partners to publish and find services. 2) Portal UDDI - the UDDI registry inside the PNN/CJX, but it allows external users to find operations on the registry that are accessible to the public [7].
Populating and updating UDDI registries As the enterprise services for the police forces could be highly time-critical, populating and updating the UDDI registries automatically needs to be implemented, by either "push" or "pull" deployment strategies.
In this paper, we present the XML/Web Services based enterprise architecture for the Police Service in the United Kingdom. We also discuss the main system integration and re-engineering models that have been deployed in the UK, as well as their convergence paths by different regional and local forces. Many of our programmes are still on-going, in the Police National Computer (PNC) Modernization Programme and in several pioneering police forces including the Metropolitan Police Service. We have summarized our experiences and forward plan in the paper, as well as the key issues yet to be addressed, hopefully as lessons to be learnt by others, and as useful comments to the WSA WG and software developers.
6. REFERENCES