Distributed Systems Engineering

Recording on-the-fly global states of distributed executions is an important paradigm when one is interested in analysing, testing, or verifying properties associated with these executions. Since Chandy and Lamport`s (1985) seminal paper on this topic, this problem is called the snapshot problem. Unfortunately, the lack of both a globally shared memory and a global clock in a distributed system, added to the fact that transfer delays in these systems are finite but unpredictable, makes this problem non-trivial. This paper first discusses issues which have to be addressed to compute distributed snapshots in a consistent way. Then several algorithms which determine on-the-fly such snapshots are presented for several types of networks (according to the properties of their communication channels, namely, FIFO, non-FIFO, and causal delivery).

The increasing deployment of workstations and high performance endsystems in addition to the operation of mainframe computers leads to a situation where many companies can no longer afford for their expensive workstations to run idle for long hours during the night or with little load during daytime. Distributed queuing systems and batch systems (DQSs) provide an efficient basis to make use of these unexploited resources and allow corporations to replace expensive supercomputers with clustered workstations running DQSs. To employ these innovative DQSs on a large scale, the management policies for scheduling jobs, configuring queues, etc must be integrated in the overall management process for the IT infrastructure. For this purpose, the concepts of application management and management policies are introduced and discussed. The definition, automatic transformation, and implementation of policies on management platforms to effectively manage DQSs will show that policy-based application management is already possible using the existing management functionality found in today's systems.

Distributed Systems Engineering has proved a valuable resource for those involved in the applied aspects of distributed and networked systems engineering. However, even during the life of the journal, what was once a niche area of research has undergone tremendous development, both technically and academically. The emphasis of the subject has shifted to such an extent that the positioning of the journal is now inappropriate. This then is the final issue of Distributed Systems Engineering, as the journal ceases publication at the end of 1999.

The publication of Distributed Systems Engineering also represented a major step forward for three of the UK's leading learned societies, in pursuing a common interest in distributed engineering. Distributed Systems Engineering has been instrumental in bringing these organizations together on a number of other issues of mutual interest.

We thank all those who have submitted papers, whether published or not, during the lifetime of our journal. To our Editorial Advisory Board, Guest Editors and the many referees who have supported this endeavour we owe a special debt of gratitude. And of course we are grateful to the readers of Distributed Systems Engineering for their support and for many kind comments during the past six years. To all of you we say: your many and various contributions have helped to make the journal an important source of information for those involved in the practical engineering aspects of distributed and networked systems.

The publishers would like to thank the Editors, Morris Sloman and David Hutchison, and the members of the Editorial Board, whose commitment to the journal has resulted in the publication of consistently high quality research, including a number of commissioned special issues in areas of topical interest.

As a service to the distributed systems community, the full electronic archive of the journal will be maintained, with free availability to all, at http://www.iop.org/Journals/ds

David HutchisonMorris SlomanThe British Computer SocietyThe Institution of Electrical EngineersInstitute of Physics Publishing

An analysis of the distributed shared memory (DSM) work carried out by other researchers shows that it has been able to improve the performance of applications, at the expense of ease of programming and use. Many implementations require application programmers to write code to explicitly associate shared variables with synchronization variables or to label the variables according to their access patterns. Programmers are required to explicitly initialize parallel applications and, in particular, to create DSM parallel processes on a number of workstations in the cluster of workstations. The aim of this research has been to improve the ease of programming and use of a DSM system while not compromising its performance. RHODOS' DSM allows programmers to write shared memory code exploiting their sequential programming skills without the need to learn the DSM concepts. The placement of DSM within the operating system allows the DSM environment to be automatically initialized and transparent. The results of running two applications demonstrate that our DSM, despite paying attention to ease of programming and use, achieves high performance.

The use of new computing paradigms is intended to ease the design of complex systems. However, the non-functional aspects of a system, including performance, reliability and scalability, remain significant issues. It is hard to detect and correct many scalability problems through system testing alone - especially when the problems are rooted in the higher levels of the system design. Late corrections to the system can have serious implications for the clarity of the design and code.

We have analysed the design of a system of multiple near-identical, `reactive' agents for scalability. We believe that the approach taken is readily applicable to many object oriented systems, and may form the basis of a rigorous design methodology. It is a simple, yet scientific extension to current design techniques using message sequence charts, enabling design options to be compared quantitatively rather than qualitatively. Our experience suggests that such analysis should be used to consider the effect of artificial intelligence, to ensure that autonomous behaviour has an overall beneficial effect for system performance.

In this paper we present an approach to support interoperation between autonomous database systems. In particular, we concentrate on distributed information discovery and access for systems with a large number of databases. We avoid the need for integrated global schemas or centralized structures containing information on the available data and its location. We instead provide an architecture that supports data distribution, autonomy and heterogeneity. The architecture also supports system evolution by the addition and removal of databases. A distributed information discovery algorithm is provided to perform data requests, database location and data access. A feature of our approach is to distribute the information about database contents using simple hierarchical information structures composed of special terms. A prototype has been developed to demonstrate and evaluate the approach. A hospital case study is used to illustrate its feasibility and applicability.

We rely more and more on computers. For example, the Internet reshapes the way we do business. A `computer outage' can cost a company a substantial amount of money. Not only with respect to the business lost during an outage, but also with respect to the negative publicity the company receives. This is especially true for Internet companies. After recent computer outages of Internet companies, we have seen a drastic fall of the shares of the affected companies.

There are multiple causes for computer outages. Although computer hardware becomes more reliable, hardware related outages remain an important issue. For example, some of the recent computer outages of companies were caused by failed memory and system boards, and even by crashed disks - a failure type which can easily be masked using disk mirroring. Transient hardware failures might also look like software failures and, hence, might be incorrectly classified as such. However, many outages are software related. Faulty system software, middleware, and application software can crash a system.

Dependable computing systems are systems we can rely on. Dependable systems are, by definition, reliable, available, safe and secure [3]. This special section focuses on issues related to dependable distributed systems. Distributed systems have the potential to be more dependable than a single computer because the probability that all computers in a distributed system fail is smaller than the probability that a single computer fails. However, if a distributed system is not built well, it is potentially less dependable than a single computer since the probability that at least one computer in a distributed system fails is higher than the probability that one computer fails. For example, if the crash of any computer in a distributed system can bring the complete system to a halt, the system is less dependable than a single-computer system.

Building dependable distributed systems is an extremely difficult task. There is no silver bullet solution. Instead one has to apply a variety of engineering techniques [2]: fault-avoidance (minimize the occurrence of faults, e.g. by using a proper design process), fault-removal (remove faults before they occur, e.g. by testing), fault-evasion (predict faults by monitoring and reconfigure the system before failures occur), and fault-tolerance (mask and/or contain failures).

Building a system from scratch is an expensive and time consuming effort. To reduce the cost of building dependable distributed systems, one would choose to use commercial off-the-shelf (COTS) components whenever possible. The usage of COTS components has several potential advantages beyond minimizing costs. For example, through the widespread usage of a COTS component, design failures might be detected and fixed before the component is used in a dependable system. Custom-designed components have to mature without the widespread in-field testing of COTS components. COTS components have various potential disadvantages when used in dependable systems. For example, minimizing the time to market might lead to the release of components with inherent design faults (e.g. use of `shortcuts' that only work most of the time). In addition, the components might be more complex than needed and, hence, potentially have more design faults than simpler components. However, given economic constraints and the ability to cope with some of the problems using fault-evasion and fault-tolerance, only for a small percentage of systems can one justify not using COTS components.

Distributed systems built from current COTS components are asynchronous systems in the sense that there exists no a priori known bound on the transmission delay of messages or the execution time of processes. When designing a distributed algorithm, one would like to make sure (e.g. by testing or verification) that it is correct, i.e. satisfies its specification. Many distributed algorithms make use of consensus (eventually all non-crashed processes have to agree on a value), leader election (a crashed leader is eventually replaced by a new leader, but at any time there is at most one leader) or a group membership detection service (a crashed process is eventually suspected to have crashed but only crashed processes are suspected).

From a theoretical point of view, the service specifications given for such services are not implementable in asynchronous systems. In particular, for each implementation one can derive a counter example in which the service violates its specification. From a practical point of view, the consensus, the leader election, and the membership detection problem are solvable in asynchronous distributed systems. In this special section, Raynal and Tronel show how to bridge this difference by showing how to implement the group membership detection problem with a negligible probability [1] to fail in an asynchronous system.

The group membership detection problem is specified by a liveness condition (L) and a safety property (S): (L) if a process p crashes, then eventually every non-crashed process q has to suspect that p has crashed; and (S) if a process q suspects p, then p has indeed crashed. One can show that either (L) or (S) is implementable, but one cannot implement both (L) and (S) at the same time in an asynchronous system. In practice, one only needs to implement (L) and (S) such that the probability that (L) or (S) is violated becomes negligible. Raynal and Tronel propose and analyse a protocol that implements (L) with certainty and that can be tuned such that the probability that (S) is violated becomes negligible.

Designing and implementing distributed fault-tolerant protocols for asynchronous systems is a difficult but not an impossible task. A fault-tolerant protocol has to detect and mask certain failure classes, e.g. crash failures and message omission failures. There is a trade-off between the performance of a fault-tolerant protocol and the failure classes the protocol can tolerate. One wants to tolerate as many failure classes as needed to satisfy the stochastic requirements of the protocol [1] while still maintaining a sufficient performance.

Since clients of a protocol have different requirements with respect to the performance/fault-tolerance trade-off, one would like to be able to customize protocols such that one can select an appropriate performance/fault-tolerance trade-off. In this special section Hiltunen et al describe how one can compose protocols from micro-protocols in their Cactus system. They show how a group RPC system can be tailored to the needs of a client. In particular, they show how considering additional failure classes affects the performance of a group RPC system.

References

[1] Cristian F 1991 Understanding fault-tolerant distributed systems Communications of ACM34 (2) 56-78

[2] Heimerdinger W L and Weinstock C B 1992 A conceptual framework for system fault tolerance Technical Report 92-TR-33, CMU/SEI

[3] Laprie J C (ed) 1992 Dependability: Basic Concepts and Terminology (Vienna: Springer)