P3 – a homeostatic environment for persistent and distributed computation on POSIX clusters.

Speaker: Mark ONeill

24th May 2006 , 2pm , 701 Claremont Tower

Abstract

P3 is portable cluster computing environment which has been designed to provide a homeostatic environment for coarse grained parallel computations (e.g. databases). P3 borrows many ideas from biological systems. For example, it the P3 environment permits processes to migrate within a clustered environment to those nodes capable of offering them optimal support (in terms of disk, processing speed, specialist devices etc). P3 also supports direct peer-to-peer communication with running processes. An authenticated user or process can request a terminal-like interface to a running process on the fly and then make status enquiries, change parameters or plug new code into the (running) process using this interface. P3 also support shared heaps. These are persistent objects which may be concurrently mapped into the address spaces processes which may be running on different hosts. Lastly, P3 supports and encourages associating processes with names rather than PIDs: this can greatly simplify process management within a network/cluster environment.

The P3 system is implemented in ANSI-C and is POSIX compliant. This means that it is easy to port P3 to almost any modern variant of UNIX or UNIX-like operating system. Currently, ports exist for Linux, FreeBSD, Solaris, Darwin and Cygwin. To date, only Linux and FreeBSD, the primary development environments, are capable of supporting the P3 environment in its entirety. The P3 system facilitates the development of complex multi-process and multi-host computations by emulating colonies of homeostatically regulated organisms. P3 has been used to write applications in the areas of computational neuroscience, neuroinformatics and digital photogrammetry. P3 is an especially useful tool in the implementation of long running persistent distributed computations by providing standardised interfaces and environmental support for:

 Direct communication with running processes.  Attaching (and detaching) dynamic functionality.  Mapping shared objects (shared heaps) into the address spaces of multiple processes.  Delivering network-wide signals within a distributed computing environment.  Process checkpointing and migration (via either the MOSIX-2 or CHPOX packages)  Access authentication to resources on a per-process basis.  Process compute context migration (via the MOSIX-2 environment).  Grid capable (via MOSIX-2).

The central philosophy of P3 is each process is treated as a simple digital lifeform, responsible for maintaining its own environment. The P3 libraries provide routines which allow processes to protect valuable resources such as files areas of shared memory, and their own thread of control from accidental deletion or change and/or termination. For example, P3 processes protect their files by relinking them in response to unexpected deletion events. In P3 almost anything may be protected by a homeostat. In addition to homeostats, the P3 system also employs redundancy. For example important P3 applications may be multithreaded, so if a current process providing critical functionality is damaged or incapacitated, its function can taken over by other processes.

P3 is written in ANSI-C, is POSIX compliant and should run under any POSIX compliant UNIX or UNIX clone with minimal changes. P3 also runs in the Microsoft-NT environment (using the Cygwin libraries) and with coLinux parasites. At the present time P3 has been ported to the following systems:

 BSD 4.4 (FreeBSD)  BSD 4.4 (Darwin)  OSF1 V5.1 (TRU-64 UNIX)  Solaris 5.7 (SVR4)  Linux (including parasitic Linuxes e.g. UML or coLinux)  Cygwin