C-JDBC Developer Guide
The image “http://c-jdbc.objectweb.org/images/c-jdbc_logo.gif” cannot be displayed, because it contains errors.
Version 1.2 Date 04/11/05	Authors (alphabetical order): Emmanuel Cecchet (emmanuel.cecchet@emicnetworks.com) Steen Jansdal (steen@jansdal.dk) Julie Marguerite (Julie.Marguerite@inrialpes.fr) Nicolas Modrzyk (Nicolas.Modrzyk@inrialpes.fr) Mathieu Peltier (Mathieu.Peltier@inrialpes.fr) Marc Wick (mwick@dplanet.ch)

Copyright © 2002-2005 French National Institute For Research In Computer Science And Control (INRIA)

Java, and all Java-based trademarks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Table of Contents

1. Getting the Source Code	3

1.1. Source distribution	3

1.2. CVS Access	3

2. Checking out, compiling and debugging C-JDBC with eclipse	4

2.1. Checking out	4

2.2. Setting up Eclipse Preferences for C-JDBC	7

2.3. Compiling C-JDBC in Eclipse	8

2.4. Debugging the C-JDBC controller in Eclipse	11

3. Contributing code to C-JDBC	22

3.1. What and how to contribute?	22

3.2. Coding style	22

3.3. C-JDBC release step by step	22

4. Architecture	24

4.1. Design Overview	24

4.2. Recovery Log	24

4.2.1. Planned features	24

4.2.2. Portability	25

4.2.3. Performance	25

4.3. C-JDBC and JMX 1.2	26

4.3.1. Security	26

4.3.2. Configuration of authorization to VirtualDatabase	27

4.3.3. Tight coupling versus loose coupling	28

4.3.4. Implementation	29

4.3.5. X500 Principal	29

4.3.6. Description Hooks	29

5. Regression Testing	31

5.1. Overview	31

5.2. Scenario Package Description	31

5.3. Scenario Output	33

Getting the Source Code

Source distribution

The C-JDBC source distribution can be downloaded from the C-JDBC's Web site. It includes the source code of :

C-JDBC driver

C-JDBC controller

C-JDBC console

as well as the test suite and all tools.

The following three formats are available (where x.y is the C-JDBC release number):

c-jdbc-x.y-src-installer.jar: Java™ graphical installer (powered by IzPack).

c-jdbc-x.y-src.tar.gz: source distribution for the Unix platforms users.

c-jdbc-x.y-src.zip: source distribution for the Windows platforms users.

The easiest way to install the source distribution is to use the Java graphical installer. If you want to use the other distribution formats, just uncompress the downloaded file in the directory of your choice.

	Note	
	Nightly builds releases are also available for download. These releases are regenerated every day at 23:45 PM CEST.

CVS Access

The whole C-JDBC codebase can be checked out through anonymous access using any CVS (Concurrent Versions System) client. The CVS server to use is cvs.forge.objectweb.org (in pserver mode). The C-JDBC repository is named /cvsroot/c-jdbc. Unix users can use for example the following commands:

bash> cvs -d:pserver:anonymous@cvs.forge.objectweb.org:/cvsroot/c-jdbc login

bash> cvs -z3 -d:pserver:anonymous@cvs.forge.objectweb.org:/cvsroot/c-jdbc co c-jdbc

When prompted for a password, just hit enter.

The available modules are:

c-jdbc: used to get the C-JDBC codebase.

web: used to get the C-JDBC Web site.

.: used to get the whole C-JDBC repository.

	Note	
	The latest snapshot of the CVS repository is also available for download. The files to download are: c-jdbc-current-cvs.tar.gz or c-jdbc-current-cvs.zip. These are automatically regenerated each night at 00:00 AM CEST.

Checking out, compiling and debugging C-JDBC with eclipse

This section shows you how to setup C-JDBC relevant Eclipse settings, download C-JDBC by checking it out of the CVS repository, compile it and setup a debugging environment, allowing you get an insight into how C-JDBC is working by setting breakpoints and single step through the code. This section shows you how it's done in Windows. Under Linux the process is almost the same.

Checking out

We are going to use the raidb1 demo and using the SQuirreL SQL Client to access our C-JDBC controller.

Start eclipse and select the menu ”File” -> ”Import...”

Select ”Checkout Projects from CVS” and press ”Next >”. On the next screen enter the following data and press ”Next >”

On the next screen click on the radiobutton ”Use an existing module” and wait until the list of existing modules are retrieved. Select ”c-jdbc” and press ”Next >”

You should end up with something like this:

We have now downloaded c-jdbc and we are now ready to compile it.

Setting up Eclipse Preferences for C-JDBC

The following changes will be very useful in order to follow the C-JDBC style of coding within Eclipse. The below steps set up the correct preferences for the Code Formatter, Code Templates, and Code Style. The should be performed once you have checked out C-JDBC via the above steps.

Go to Window->Preferences->Java->Code Style->Code Formatter->Import and select the appropriate CodeFormatter config file from the'eclipse-settings' directory within C-JDBC that corresponds to your version of Eclipse.

Next, go to Window->Preferences->Java->Code Style->Code Templates->Import and select the appropriate CodeTemplates config file from the 'eclipse-settings' directory within C-JDBC

Finally download the Checkstyle builder from http://sourceforge.net/projects/eclipse-cs and decompress it. Exit from Eclipse and copy the com.atlassw.tools.eclipse.checkstyle_<checkstyle-version> folder to your Eclipse plug-in folder. Restart Eclipse. Go to Windows->Preferences->Checkstyle->Import Plugin Configuration and select the appropriate CheckstylePlugin config file from the 'eclipse-settings' directory within C-JDBC

Compiling C-JDBC in Eclipse

Locate the ”build.xml” file under ”c-jdbc” and right click on the file. On the popup menu select ”Run” -> ”Ant Build...”

Click on the ”Targets” tab, make sure the ”dist-bin” target is selected and press ”Run”.

Hopefully you will see a ”BUILD SUCCESSFUL” message or the CVS is broken.

Debugging the C-JDBC controller in Eclipse

Since we are going to debug the c-jdbc controller, we can't use the ”demo-raidb1.bat” script without modifications.

We want to start the backend databases but not the controller, so in the ”demo-raidb1.bat” script the lines where the controller is started are removed (or remarked out):

The lines in demo-raidb1.bat:

echo Starting Controller with Raidb1 Configuration

start \B controller.bat -f ..\config\controller\controller-raidb1.xml

are remarked out:

rem echo Starting Controller with Raidb1 Configuration

rem start \B controller.bat -f ..\config\controller\controller-raidb1.xml

Start the backend databases with:

cd C:\eclipse\workspace\c-jdbc\c-jdbc\build\dist\bin\demo

set CJDBC_HOME=c:\eclipse\workspace\c-jdbc\c-jdbc\build\dist\bin

demo-raidb1

Three instances of the HSQLDB server are started in the same window, and you should be able to see 3 lines saying HSQLDB server 1.8.0 is online.

In the package explorer locate the ”org.objectweb.cjdbc.controller.core” class in the package explorer and right click. Select ”Debug” -> ”Debug...”.

On the ”Create, manage and run configurations” screen, click on the classpath tab and select ”User Entries”.

Click on the ”Advanced...” button, select ”Add folders” and press OK.

In the folder selection screen select ”controller” as here:

You have now added the ”\c-jdbc\c-jdbc\build\dist\bin\config\controller” folder to the classpath. Add the following folders too:

”\c-jdbc\c-jdbc\build\dist\bin\config\language”

”\c-jdbc\c-jdbc\build\dist\bin\config\virtualdata”

”\c-jdbc\c-jdbc\build\dist\bin\xml”

”\c-jdbc\c-jdbc\build\dist\bin\config”

”\c-jdbc\c-jdbc\build\dist\bin\drivers”

Add ”\c-jdbc\c-jdbc\build\dist\bin\3rdparty\hsqldb\lib\hsqldb.jar” file to the classpath too.

On the ”Arguments” tab add the following to the ”Program arguments” and ”VM arguments”.

We are now ready to debug the controller. Try to set a breakpoint on the programs entry point and start debugging the program, which should break at your breakpoint.

Before we continue set a breakpoint at ”org.objectweb.cjdbc.controller.core.ControllerServerThread” just after the line:

clientSocket = serverSocket.accept();

Start the SquirreL SQL program and connect to C-JDBC:

When we press OK in the connection box, eclipse should break and we are now able to single step through C-JDBC's handling of a client connection.

When you press OK eclipse should break at the specified breakpoint:

Contributing code to C-JDBC

What and how to contribute?

All contributions to C-JDBC code are welcome. You need to be registered on the ObjectWeb Forge in order to get CVS write access (CVS is open in read to everybody). You can check the task list as well as the bugs and feature requests on the C-JDBC Forge site. We keep track of all developments through the Forge facilities. Don't hesitate to share your plans and ideas on the c-jdbc@objectweb.org mailing list. We open CVS write access upon request.

Coding style

C-JDBC development team uses Eclipse, a popular open source IDE for Java. We encourage developers to use Eclipse and checkout C-JDBC Eclipse settings from the C-JDBC CVS repository. These settings include code formatting, code style checking and code generation templates. This way all developers use the same coding conventions. All Eclipse settings are located in the /eclipse-settings directory of the CVS repository.

C-JDBC uses the Sun's Java standard code conventions with little modifications (for example, braces in statements, ...). For more details, please see the source code itself. Code formatting settings can be imported from the /eclipse-settings/codeFormatter.eclipseXXX.xml file where XXX represents the Eclipse version number.

The CVS repository also contains configuration files for the Checkstyle Eclipse plugin (2.1 and 3.x versions). This plugin allows you to check code conventions and Javadoc automatically in Eclipse and provides the user with many facilities to correct errors. These files are named /eclipse-settings/checkstyle.plugin.config.x.y in the CVS repository and can be imported to modify automatically the Checkstyle preferences (open “Window” menu, then choose “Preferences” and click to “Checkstyle”).

The CVS repository contains also standard Eclipse project files that can be used to manage a c-jdbc project (.project and .classpath).

C-JDBC release step by step

Here are the steps to follow when releasing a new version of C-JDBC:

Copy xml/c-jdbc.dtd to web/html/dtds/c-jdbc-VERSION.dtd (example: web/html/dtds/c-jdbc-1.2.1.dtd).

Copy xml/c-jdbc-controller.dtd to web/html/dtds/c-jdbc-controller-VERSION.dtd (example: web/html/dtds/c-jdbc-controller-1.2.1.dtd).

Update the version number in build.properties.

Add the release name, date and CVS tag in HISTORY.txt.

Update web/xml/root/history.xml with information from HISTORY.txt.

Add a news in web/xml/root/index.xml.

Update C-JDBC current version number in web/xml/root/index.xml.

Make sure that generate.doc=true in build.xml.

Build a distribution using ant dist.

Commit all files in CVS repository.

Add a tag in CVS according to what is described in HISTORY.txt.

Upload the documentation, Javadoc and Doxygen files to the ObjectWeb Forge site using Webdav on https://c-jdbc.forge.objectweb.org:8002 or using scp build/doc/* forge.objectweb.org:/var/lib/gforge/chroot/home/groups/c-jdbc/htdocs.

Add a new binary, source and documentation release on the Forge site at https://forge.objectweb.org/project/admin/editpackages.php?group_id=42

Copy/paste the HISTORY.txt in the release notes for binary, source and documentation releases.

Upload and add the corresponding files and edit the file types for each of them (.tgz, .pdf, ...).

Finally send a email release notice for each release (step 4 on Forge page).

In the Forge project admin page, press 'Push Project Web Site from Development to Production Now'

Post a news on C-JDBC's Forge front page and click on the checkbox to publish the news to www.objectweb.org.

Announce the new release on the C-JDBC mailing list.

Announce the new release on freshmeat.net.

Architecture

Design Overview

This section has to be documented. Meanwhile, you can find a description of C-JDBC design in the documentation section of the web site. The main articles are:

The research report titled 'RAIDb: Redundant Array of Inexpensive Databases' explains the RAIDb concepts and how they are implemented by C-JDBC. This document also includes a performance evaluation section.

The technical report titled 'C-JDBC Horizontal Scalability Design' described how multiple controller failover is implemented and discusses the various design options we considered.

Recovery Log

Planned features

Add a timestamp to the transactions stored in the recovery log. With the timestamp it will be possible to replay the transactions up to a given point in time. This servers as a logical backup in case someone accidentally deletes or updates some rows in a database and wants to undo an already committed transaction.

(Requires additional table for transactions or denormalization)

Start and stop the recovery log without having to restart controller. For performance reasons an application might want to temporarily stop the recovery log without having to restart the controller.

Portability

Table and column names used by the recovery log have to be configurable to avoid naming conflicts with different database vendors. The create table statements for the tables may be provided by the application in form of templates, as the default sql92 create table statement is not optimized for performance. Oracle installations for example want to specify the tablespace the recovery log should be using.

current_timestamp,current_date,current_time etc :

Time relevant database functions such as current_timestamp are difficult to deal with for c-jdbc. The problem is seriously aggravated by the recovery log as the time difference between execution of the request is no longer defined by network latency and clock difference. Most applications could live with timestamp differences in the millisecond range. With the recovery log, however, the time differences may be hours or even days.

workaround : applications shouldn't use current_timestamp and the like. C-jdbc may offer ways of dealing with it, but always with a performance penalty.

Note : the problem is related to inserts and updates not selects.

current_timestamp in inserts and updates : c-jdbc should parse the request and replace the functions with the time value on the fly.

default columns on the database : c-jdbc has to read the column default from the database metadata. Inserts have to be parsed and for columns with time-defaults an explicit value has to be inserted.

Performance

To avoid the recovery log from becoming a performance bottleneck, we have to consider several points :

Insert

As the recovery log transactions are independent of the 'business transactions' of the applications we can use batch commits for the inserts and commit a couple of rows in one database transaction. The optimal number of rows for a commit depends on the database vendor, the application and the hardware (memory, disk io,...).

The following parameters have to be made configurable :

- number of rows inserted per commit

- maximal bytes inserted per commit

- maximal timeout to wait for further rows before committing

Delete

The recovery log table has to be cleaned up periodically. Deleting huge number of rows in a transactional database is an expensive operation and may often fail (rollback segment errors on oracle for example). If the application is doing the cleanup it can use database vendor specific features to do this efficiently (such as partitioned tables in oracle).

However, c-jdbc has to provide a portable way of cleanup. It is suggested to use rotating tables for the recovery log to be able to drop tables when they are no longer needed.

Replay

Performance of the replay could also be dramatically improved with wider transaction boundaries then in the original transactions. At anycase, however, the original transactions must not be splitted and have to be run in the same transaction.

At the other hand it might even make sens to artificially lower the throuput of the replay as it means a considerable load on the controller (or whichever node is feeding the replay).

Remarks :

The c-jdbc recovery log is comparable to the oracle archiver.

C-JDBC and JMX 1.2

C-jdbc is trying to use standard jmx features (jmx 1.2) wherever possible, installations with java 1.5 can do without mx4j as long as they don't use any addiational features such as the html adaptor.

Security

Security is relying on the two java standard extensions jaas and jsse both of which have become part of j2se with java 1.4. For java 1.3 the extensions can be downloaded from sun and put into the jre/lib/ext directory of the java installation.

jaas :	java Authentication and Authorization Service (subjects, delegation)

http://java.sun.com/products/jaas/

jsse :	Java Secure Socket Extension (ssl)

http://java.sun.com/products/jsse

Configuration of security options for jmx :

<RmiConnector port ="">

<PasswordAuthenticator username="" password="password"/>

<SSL keyStore="server.keystore"

keyStorePassword="password"

keyStoreKeyPassword="password"

isClientAuthNeeded="true"

trustStore="server.keystore"

trustStorePassword="password" />

</RmiConnector>

Authorization of each method call via jmx is done with a decorated MBeanServer to make sure all connections are authorized. (Using interception with an MBeanServerForwarder is done between the connector server and the mbean server and has to be added to every connector/adaptor. This might lead to security leaks if it gets forgotten for one connection type. A decorated server at the other hand is more difficult to implement, but cannot be circumvented.)

Configuration of authorization to VirtualDatabase

Authorization for a virtual database is configured in the controller xml by defining all authorized users with xml elements.

The user element corresponds to a org.objectweb.cjdbc.commons.jmx.authentication.VirtualDatabaseUser whereas the X509 element is instanciated as javax.security.auth.x500.X500Principal both of which implement the Principal interface.

<!ELEMENT USER EMPTY>

<!ATTLIST USER

username	CDATA #IMPLIED

password	CDATA #IMPLIED

>

<!ELEMENT X509 EMPTY>

<!ATTLIST X509

CN	CDATA #IMPLIED

OU	CDATA #IMPLIED

O	CDATA #IMPLIED

L	CDATA #IMPLIED

ST	CDATA #IMPLIED

C	CDATA #IMPLIED

Alias	CDATA #IMPLIED

>

Open issue : How does the mx4j html adaptor give access to the certificate chain for the client ssl session ?

Note : configuration of the authorized users in xml does not scale for large number of users. Plugin options to use the corporate infrastructure for authorization may be provided later.

Authentication is done on the connection level to access the controller. Every connector/adaptor might have different kind of authentication mechanisms. The connection can be authenticated by a JMXAuthenticator and/or with pki (ssl option requireClientAuthentication=true).

Most actions are protected by further authentication mechanisms (virtual database admin, backend), which require a separate username/password for every method call or the use of subject delegation.

(Optional : specifiy classname of JMXAuthenticator in config, users might provide their own implementation of a JMXAuthenticator)

Tight coupling versus loose coupling

Jmx provides generic access to beans (kind of loose coupling) which doens't need the bean interfaces on the classpath of the jmx client. On the other hand the use of proxies enables the compiler to check the validity of the method calls at compile time.

Example : The same method call generic and with the use of a proxy

Generic :

server.invoke(db, "viewAllBackendNames", new Object[]{

"admin", "c-jdbc"},

new String[]{"java.lang.String", "java.lang.String"}));

Proxy :

proxy.viewAllBackendNames()

The proxy approach seems preferable for static clients such as the cjdbc console and application specific jmx clients. However, many jmx clients will not be written for cjdbc only and are thus required to use generic access. Authentication of the proxy method call is done with subject delegation - the proxy is created with the use of a subjet. All method calls on the proxy are done on behalf of the specified subject.

Cjdbc should therefore offer both ways of access.

(Open points : is the username/password passing for every generic method request required ? Or is there a standard way to deal with generic principals and subject delegation similar to the proxy approach ? )

Implementation

The MBean Server (with its decorators) is started while starting the controller using a MBeanServerManager (Singleton). References to the MBean Server don't have to be passed arround and can be got from the MBeanServerManager at any time to register new MBeans.

Connectors and adaptors are managed by the MBeanServerManager, which keeps track of all active connectors and adaptors as well as the respective connections. It is keeping references to all connections to avoid them from being garbage collected.

If you want to add a new connector or adaptor you configure your connector/adaptor and register it with the MBeanServerManager. This way it is possible to dynamically add and remove (reconfigure) connectors/adaptors of any kind.

The System property javax.management.builder.initial is set to org.objectweb.cjdbc.controller.jmx.MBeanServerBuilder in the main method of the controller to enforce usage of the decorated authentication.

X500 Principal

The jmx specification does not define access to the certificate chain of ssl connections.

(see http://archives.java.sun.com/cgi-bin/wa?A2=ind0405&L=jmx-forum&D=0&P=4820)

We have three possibilites

Decorate our own SSLServerSocketFactory: keep the certificates in a static hashtable with the connectoinid as key, get the certificates for a request with the connectionid, remove certificates with an ssl listener once they are deactivated.

jmxmp

jndi (jrei)

Description Hooks

The mx4j.MBeanDescriptionAdapter is replaced by javax.management.StandardMBean.

Packages

clients and server :

org.objectweb.cjdbc.common.jmx.mbeans : mbean interfaces

org.objectweb.cjdbc.common.jmx.notifications

org.objectweb.cjdbc.common.jmx.authentication : subjects, credentials

clients, server and driver

org.objectweb.cjdbc.common.net : SocketFactory and helper classes (ssl)

server (controller) :

org.objectweb.cjdbc.controller.jmx

classes :

- PasswordAuthenticator implements JMXAuthenticator

- AbstractDatabaseUser implements Principal

- subclasses admin, backend

- MBeanServerManager

- AuthenticationMBeanServer, AuthenticationMBeanServerBuilder

(possibly VirtualDatabaseAdminAuthenticator, BackendAuthenticator)

- RmiConnector

- HttpAdaptor

- (InvocationTracker implements MBeanServerForwarder : logging all requests on a connection)

Jars

cjdbc-driver.jar

cjdbc-controller.jar

cjdbc-console.jar

cjdbc-jmxclient.jar

Regression Testing

Overview

Start
Scenario are used for regression testing of C-JDBC. The scenario can be started from the main build.xml file, with the following commands:
Start all the scenario (takes a huge amount of time!)
ant scenario
Or start a single scenari with the following command:
ant scenari -Dscenari=<scenarioname>
For example, to start the BasicHorizontalScalabilityScenario type:
ant scenari -Dscenari=horizontal.BasicHorizontalScalabilityScenario
All the classes that can be tested have the Scenario key word appended to their name of class, just like Test for Junit test. Scenario are in fact extended Junit tests anyway.

What is a Scenario
A scenario is based on a template, found in the template package, that takes care of starting and stopping the different hypersonic processes along with the controller, as well as loading the base virtual database to test.
All the different templates that take care of loading and unloading are extended to actually implemented the different scenario.

How to write new scenario
To add a scenario, just extends the template you need, or if it's not there, add a template in the templates package and extend it.

Scenario Package Description

Basic
This contains the basic scenario, that are more validating the templates framework than testing anything.
This will test Starting and stopping the controller, starting and stopping the different hypersonic processes.

Config
While using scenario, we used a little configuration from the root as C-JDBC as possible.
The classpath used is different than for other junit test. You can see in the main ant build.xml file:
<path id="classpath-scenario">
That depends on :
<path id="classpath-test-common">
This includes:
- All the required libraries
- The newly compiled classes and jar files
- Language configuration
- Dtd files from the xml dir at the root of C-JDBC.
Note that NO other files from the C-JDBC root config directory is used.
Config files for scenario includes:
- controller dir, with configuration files for the controller.
(This are not used, though ...)
- database, that contains the data for the hypersonic servers
- image (used for testing blobs)
- virtualdatabase, the different configuration of virtual databases.
- jgroups.xml (maybe we should used the one in config?)
- log4j.properties, is the same as in config but is setting all loggers to DEBUG ouput.

Horizontal
Horizontal scenario are testing multiple controllers configuration. They mostly depends on Horizontal templates. See scenario status but scenario with re-enabling a faulted controller is failing, because of Jgroups.

Raidb1
This is the most commly used configuration and so many of the scenario can be found here.
- driver, test the stability of the driver
- loadbalancer, test the different loadbalancers functionalities
- recovery, aims at validating backup and recovery with octopus
- startup, test validity of some configuration files

Raidb2
Test raidb2 specific functionalities.

Templates
The templates take care of setting the scenario up, and to cleanly finish after a scenario, whether it was successful or not.
Each template should therefore be careful of stopping any controller and any hypersonic server it might have spawned.

Tools
Tools classes are not test or scenario, but they encapsulate the component to manage for the templates in nice interfaces.

Scenario Output

The results of the different test can always be found in:

<cjdbc>/build/test/report/html/index.html

11/28

2004-11-24T17:40:20