The First Experience of Class Evolution Support in PJama
M. Dmitriev
Published in the Advances in Persistent Object Systems - Proceedings of the Eighth International Workshop on Persistent Object Systems (POS-8) and The Third International Workshop on Persistence and Java (PJAVA-3) , R.Morrison, M.J.Jordan and M.P.Atkinson (Eds.), Morgan Kaufmann, August 30 - September 4, 1998, Tiburon, California.
Abstract The design and implementation of the initial release of class evolution technology for the PJama persistent system is described. This technology supports modifications to individual classes and class hierarchy, plus user-defined conversion of data contained in class instances when a modification to the class leads to a change of instance format. PJama evolution technology was evaluated on an application of considerable size and proved to be very helpful for an application developer.
Download: PostScript PDF
Published in the Proceedings of the 1st ECOOP Workshop on Object-Oriented Databases, In Association with 13th European Conference on Object-Oriented Programming, Lisbon, Portugal, June 14 - 18, 1999.
Abstract The persistent object conversion facilities available in the evolution technology for the PJama persistent programming language are described. They include default and custom conversion, and within the latter - bulk and fully controlled conversion. Where the programmer needs to specify transformations different from the default, they are coded in standard Java. During conversion the ``old'' object graph remains unchanged, that is, substitute objects are not directly reachable from it, which is crucial for comprehensible semantics of evolution code. The authors believe that the present set of facilities is complete, i.e. it is enough to convert any persistent store instead of rebuilding it from scratch. However, the question of whether this can be generally proven is raised.
Download: PostScript PDF
Published in the Proceedings of the ECOOP 2000 Symposium on Objects and Databases, In Association with 14th European Conference on Object-Oriented Programming, Springer Verlag, LNCS 1944, Cannes, France, June 12 - 16, 2000.
Abstract With the merging of two developing technologies, namely Sphere persistent store and Java Implementation based on the Solaris Research VM, to provide a new version of an orthogonally persistent Java - PJama1, the developers are given an excellent window of opportunity to provide evolution support as a standard, scalable and recoverable mechanism. This paper presents the issues that arise during the design of the store level foundation for the evolution technology, our present design solutions and their implementation. The goals of scalability and recoverability were met at a price of minor semantical constraints on the programmer-defined conversion code. We are now working on removing these remaining constraints.
Download: PostScript PDF
Technical Report TR-2000-57, Department of Computing Science, University of Glasgow, Glasgow G12 8QQ, Scotland.
Abstract This report details the design and implementation of the
high-level part of the evolution techology for PJama persistent
platform. This technology supports an arbitrary set of
changes to class definitions and allows to convert persistent
instances of these classes to make them conform to the new class
versions. The novel approach called persistent build, that
combines evolution with ``smart'' recompilation of .java
sources for classes, is presented. This approach guarantees the
safety of changes applied to the persistent store, in the sense
that no source incompatible changes, as defined in the
Java Language Specification, can be introduced. We also present
a rich set of object conversion facilities implemented in PJama,
that includes default and custom conversion, and
within the latter - bulk and fully controlled modes.
The important internal details of conversion support are explained.
Download: PostScript PDF
This report effectively summarizes the issues initially presented in the first two papers on this page. It also gives more consistent and detailed picture, providing the most exhaustive coverage of the VM-level support for evolution in PJama.
To be published in the Proceedings of the Ninth International Workshop on Persistent Object Systems: Design, Implementation and Use (POS-9), Springer Verlag, Lilihammer, Norway, September 6 - 8, 2000.
Abstract PJama is the latest version of an orthogonally persistent platform for Java. It depends on a new persistent object store, Sphere, and provides facilities for class evolution. This evolution technology supports an arbitrary set of changes to the classes, which may have arbitrarily large populations of persistent objects. We verify that the changes are safe. When there are format changes, we also convert all of the instances, while leaving their identities unchanged. We aspire to both very large persistent object stores and freedom for developers to specify arbitrary conversion methods in Java to convey information from old to new formats.
Evolution operations must be safe and the evolution cost should be approximately linear in the number of objects that must be reformatted. In order that these conversion methods can be written easily, we continue to present the pre-evolution state consistently to Java executions throughout an evolution. At the completion of applying all of these transformations, we must switch the store state to present only the post-evolution state, with object identity preserved. We present an algorithm that meets these requirements for eager, total conversion.
This paper focuses on the mechanisms built into Sphere to support safe, atomic and scalable evolution. We report our experiences in using this technology and include a preliminary set of performance measurements.
Download: PostScript PDF
PhD Thesis, Department of Computing Science, University of Glasgow, Glasgow G12 8QQ, Scotland.
Abstract
There is a growing class of applications implemented in object-oriented languages that are large and complex, that exploit object persistence, and need to run uninterrupted for long periods of time. Development and maintenance of such applications can present challenges in the following interrelated areas: consistent and scalable evolution of persistent data and code, optimal build management, and runtime changes to applications.
The research presented in this thesis addresses the above issues. Since Java is becoming increasingly popular platform for implementing large and long-lived applications, it was chosen for experiments.
The first part of the research was undertaken in the context of the PJama system, an orthogonally persistent platform for Java. A technology that supports persistent class and object evolution for this platform was designed, built and evaluated. This technology integrates build management, persistent class evolution, and support for several forms of eager conversion of persistent objects.
Research in build management for Java has resulted in the creation of a generally applicable, compiler-independent smart recompilation technology, which can be re-used in a Java IDE, or as a standalone Java-specific utility similar to make.
The technology for eager object conversion that we developed allows the developers to perform arbitrarily complex changes to persistent objects and their collections. A high level of developer's control over the conversion process was achieved in part due to introduction of a mechanism for dynamic renaming of old class versions. This mechanism was implemented using minor non-standard extensions to the Java language. However, we also demonstrate how to achieve nearly the same results without modifying the language specification. In this form, we believe, our technology can be largely re-used with practically any persistent object solution for Java.
The second part of this research was undertaken using as an implementation platform the HotSpot Java Virtual Machine (JVM), which is currently Sun's main production JVM. A technology was developed that allows the engineers to redefine classes on-the-fly in the running VM. Our main focus was on the runtime evolution of server-type applications, though we also address modification of applications running in the debugger. Unlike the only other similar system for Java known to us, our technology supports redefinition of classes that have methods currently active. Several policies for handling such methods have been proposed, one of them is currently operational, another one is in the experimental stage. We also propose to re-use the runtime evolution technology for dynamic fine-grain profiling of applications.
Download: PostScript PDF
Presented at the Workshop on Engineering Complex Object-Oriented Systems for Evolution, held in Association with OOPSLA 2001 International Conference, Tampa Bay, Florida, USA, October 14-18, 2001.
Abstract There is a class of important computer applications that must run without interruption and yet must be changed from time to time to fix bugs or upgrade functionality. In this paper, we present the initial runtime evolution framework which we have developed for the HotSpot Java Virtual Machine, that allows us to change running applications on-the-fly, without interruption. We describe our staged implementation plan, where stages correspond to increasing levels of implementation complexity, yet on each stage a reasonably complete set of facilities is provided. The first stage - support for changes to method bodies only - has already been implemented and is to be included in the forthcoming Java 2 Platform release. We discuss multiple policies for dealing with active methods of running applications, present our thoughts on instance conversion implementation, and suggest that runtime evolution technology can be used for dynamic fine-grain profiling of applications.
Download: PostScript PDF
Presented at the Workshop on Unanticipated Software Evolution, held at ECOOP 2002 International Conference, Malaga, Spain, June 10-14, 2002.
Abstract A functionality to support dynamic class evolution in running applications has been recently implemented in the Sun's Java HotSpot Virtual Machine. This functionality was initially considered useful for "fix-and-continue" feature of debuggers and for updating running server programs. Despite looking potentially attractive, the second option so far have not gained enough attention of major customers of the Java platform, the main reason being that dynamic application fixing or upgrading looks "too risky". However, it appears that evolution technology in the form of dynamic bytecode instrumentation may become extermely useful in the areas of application profiling and monitoring. A JVM supporting dynamic bytecode instrumentation would allow developers to profile dynamically selected parts of the target application, turn emission of various kinds of profiling events on and off and thus dramatically reduce the overhead presently associated with instrumentation-based profiling. However, a special API different from the one currently implemented for class evolution, is required to support fast and scalable dynamic bytecode instrumentation.
Download: PostScript PDF
To be published in the Proceedings of the OOPSLA 2002 International Conference, Seattle, USA, November 4-8, 2002.
Abstract
Keeping the code of a Java application consistent (code is consistent if all of
the project classes can be recompiled together without errors) prevents late linking
errors, and thus may significantly improve development turnaround time. In this paper
we describe a make technology for the Java programming language, that is based on smart
dependency checking, guarantees consistency of the project code, and at the same
time reduces the number of source code recompilations to the minimum. After project code
consistency is initially assured by complete recompilation, the information extracted
from the binary classes is stored in a so-called project database. Whenever the source
code for some class C is changed, its recompiled binary is compared
to the old version of C preserved in the project database. As a result, we
find a minimum subset of classes that depend on C and may be affected by the
particular change made to it. These are recompiled in turn, and absence of compilation
errors at this phase guarantees the consistency of the new project code. To determine
which dependent classes to recompile, we categorize all source incompatible changes, and
for each category establish a criterion for finding the smallest possible subset of
dependent classes.
Download: PostScript PDF