DBCP (Database Connection Pool) Configurations

JNDI Datasource HOW-TO

print-friendly version

Table of Contents

Introduction Database Connection Pool (DBCP) Configurations Tyrex Connection Pool Non DBCP Solutions Oracle 8i with OCI client Common Problems

Introduction

JNDI Datasource configuration is covered extensively in the JNDI-Resources-HOWTO however, feedback from tomcat-user has shown that specifics for individual configurations can be rather tricky. Here then are some example configurations that have been posted to tomcat-user for popular databases and some general tips for db useage. You should be aware that since these notes are derived from configuration and/or feedback posted to tomcat-user YMMV :-). Please let us know if you have any other tested configurations that you feel may be of use to the wider audience, or if you feel we can improve this section in anyway.

Database Connection Pool (DBCP) Configurations

DBCP provides support for JDBC 2.0. On systems using a 1.4 JVM DBCP will support JDBC 3.0. Please let us know if you have used DBCP and its JDBC 3.0 features with a 1.4 JVM. See the DBCP Javadocs BasicDataSource class for a complete list of configuration parameters. Installation DBCP uses the Jakarta-Commons Database Connection Pool. It relies on number of Jakarta-Commons componenets: Jakarta-Commons DBCP 1.0 Jakarta-Commons Collections 2.0 Jakarta-Commons Pool 1.0 These jar files along with your the jar file for your JDBC driver should be installed in $CATALINA_HOME/common/lib. NOTE:Third Party drivers should be in jarfiles, not zipfiles. Tomcat only adds $CATALINA_HOME/common/lib/*.jar to the classpath. NOTE: Do not install these jarfiles in your /WEB-INF/lib, or $JAVA_HOME/jre/lib/ext, or anywhere else. You will experience problems if you install them anyplace other than $CATALINA_HOME/common/lib. Preventing dB connection pool leaks A database connection pool creates and manages a pool of connections to a database. Recycling and reusing already existing connections to a dB is more efficient than opening a new connection. There is one problem with connection pooling. A web application has to explicetely close ResultSet's, Statement's, and Connection's. Failure of a web application to close these resources can result in them never being available again for reuse, a db connection pool "leak". This can eventually result in your web application db connections failing if there are no more available connections. There is a solution to this problem. The Jakarta-Commons DBCP can be configured to track and recover these abandoned dB connections. Not only can it recover them, but also generate a stack trace for the code which opened these resources and never closed them. To configure a DBCP DataSource so that abandoned dB connections are removed and recycled add the following paramater to the ResourceParams configuration for your DBCP DataSource Resource: <parameter> <name>removeAbandoned</name> <value>true</value> </parameter> When available db connections run low DBCP will recover and recyle any abandoned dB connections it finds. The default is false. Use the removeAbandonedTimeout parameter to set the number of seconds a dB connection has been idle before it is considered abandoned. <parameter> <name>removeAbandonedTimeout</name> <value>60</value> </parameter> The default timeout for removing abandoned connections is 300 seconds. The logAbandoned parameter can be set to true if you want DBCP to log a stack trace of the code which abandoned the dB connection resources. <parameter> <name>logAbandoned</name> <value>true</value> </parameter> The default is false. MySQL DBCP Example 0. Introduction Versions of MySQL and the mm.mysql JDBC driver when have been reported to work: MySQL 3.23.47, MySQL 3.23.47 using InnoDB, MySQL 4.0.1alpha mm.mysql 2.0.14 (JDBC Driver) Please let us know if you have tested the new MySQL mm.mysql 3.0 driver. 1. MySQL configuration Ensure that you follow these instructions as variations can cause problems. Create a new test user, a new database and a single test table. Your MySQL user must have a password assigned. The driver will fail if you try to connect with an empty password. mysql> GRANT ALL PRIVILEGES ON *.* TO javauser@localhost -> IDENTIFIED BY 'javadude' WITH GRANT OPTION; mysql> create database javatest; mysql> use javatest; mysql> create table testdata ( -> id int not null auto_increment primary key, -> foo varchar(25), -> bar int); Note: the above user should be removed once testing is complete! Next insert some test data into the testdata table. mysql> insert into testdata values(null, 'hello', 12345); Query OK, 1 row affected (0.00 sec) mysql> select * from testdata; +----+-------+-------+ | ID | FOO | BAR | +----+-------+-------+ | 1 | hello | 12345 | +----+-------+-------+ 1 row in set (0.00 sec) mysql> 2. server.xml configuration Configure the JNDI DataSource in Tomcat by adding a declaration for your resource to $CATALINA_HOME/conf/server.xml. Add this in between the </Context> tag of the examples context and the </Host> tag closing the localhost definition. <Context path="/DBTest" docBase="DBTest" debug="5" reloadable="true" crossContext="true"> <Logger className="org.apache.catalina.logger.FileLogger" prefix="localhost_DBTest_log." suffix=".txt" timestamp="true"/> <Resource name="jdbc/TestDB" auth="Container" type="javax.sql.DataSource"/> <ResourceParams name="jdbc/TestDB"> <parameter> <name>factory</name> <value>org.apache.commons.dbcp.BasicDataSourceFactory</value> </parameter> <!-- Maximum number of dB connections in pool. Make sure you configure your mysqld max_connections large enough to handle all of your db connections. Set to 0 for no limit. --> <parameter> <name>maxActive</name> <value>100</value> </parameter> <!-- Maximum number of idle dB connections to retain in pool. Set to 0 for no limit. --> <parameter> <name>maxIdle</name> <value>30</value> </parameter> <!-- Maximum time to wait for a dB connection to become available in ms, in this example 10 seconds. An Exception is thrown if this timeout is exceeded. Set to -1 to wait indefinitely. --> <parameter> <name>maxWait</name> <value>10000</value> </parameter> <!-- MySQL dB username and password for dB connections --> <parameter> <name>username</name> <value>javauser</value> </parameter> <parameter> <name>password</name> <value>javadude</value> </parameter> <!-- Class name for mm.mysql JDBC driver --> <parameter> <name>driverClassName</name> <value>org.gjt.mm.mysql.Driver</value> </parameter> <!-- The JDBC connection url for connecting to your MySQL dB. The autoReconnect=true argument to the url makes sure that the mm.mysql JDBC Driver will automatically reconnect if mysqld closed the connection. mysqld by default closes idle connections after 8 hours. --> <parameter> <name>url</name> <value>jdbc:mysql://localhost:3306/javatest?autoReconnect=true</value> </parameter> </ResourceParams> </Context> 3. web.xml configuration Now create a WEB-INF/web.xml for this test application. <?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd"> <web-app> <description>MySQL Test App</description> <resource-ref> <description>DB Connection</description> <res-ref-name>jdbc/TestDB</res-ref-name> <res-type>javax.sql.DataSource</res-type> <res-auth>Container</res-auth> </resource-ref> </web-app> 4. Test code Now create a simple test.jsp for use later. <html> <head> <title>DB Test</title> </head> <body> <% foo.DBTest tst = new foo.DBTest(); tst.init(); %> <h2>Results</h2> Foo <%= tst.getFoo() %><br/> Bar <%= tst.getBar() %> </body> </html> And create a Java class to actually use your new Datasource and connection pool. Note: this code isn't anywhere near production ready - it's only supposed to be used as a simple test :-) package foo; import javax.naming.*; import javax.sql.*; import java.sql.*; public class DBTest { String foo = "Not Connected"; int bar = -1; public void init() { try{ Context ctx = new InitialContext(); if(ctx == null ) throw new Exception("Boom - No Context"); DataSource ds = (DataSource)ctx.lookup( "java:comp/env/jdbc/TestDB"); if (ds != null) { Connection conn = ds.getConnection(); if(conn != null) { foo = "Got Connection "+conn.toString(); Statement stmt = conn.createStatement(); ResultSet rst = stmt.executeQuery( "select id, foo, bar from testdata"); if(rst.next()) { foo=rst.getString(2); bar=rst.getInt(3); } conn.close(); } } }catch(Exception e) { e.printStackTrace(); } } public String getFoo() { return foo; } public int getBar() { return bar;} } Finally deploy your web app into $CATALINA_HOME/webapps either as a warfile called DBTest.war or into a sub-directory called DBTest Once deployed, point a browser at http://localhost:8080/DBTest/test.jsp to view the fruits of your hard work. Oracle 8i, 9i & 10g 0. Introduction Oracle requires minimal changes from the MySQL configuration except for the usual gotchas :-) Drivers for older Oracle versions may be distributed as *.zip files rather than *.jar files. Tomcat will only use *.jar files installed in $CATALINA_HOME/common/lib. Therefore classes111.zip or classes12.zip will need to be renamed with a .jar extension. Since jarfiles are zipfiles, there is no need to unzip and jar these files - a simple rename will suffice. Some early versions of Tomcat 4.0 when used with JDK 1.4 will not load classes12.zip unless you unzip the file, remove the javax.sql.* class heirarchy and rejar. For Oracle 9i onwards you should use oracle.jdbc.OracleDriver rather than oracle.jdbc.driver.OracleDriver as Oracle have stated that oracle.jdbc.driver.OracleDriver is deprecated and support for this driver class will be discontinued in the next major release. 1. server.xml configuration In a similar manner to the mysql config above, you will need to define your Datasource in your server.xml file. Here we define a Datasource called myoracle using the thin driver to connect as user scott, password tiger to the sid called mysid. (Note: with the thin driver this sid is not the same as the tnsname). The schema used will be the default schema for the user scott. Use of the OCI driver should simply involve a changing thin to oci in the URL string. <Resource name="jdbc/myoracle" auth="Container" type="javax.sql.DataSource"/> <ResourceParams name="jdbc/myoracle"> <parameter> <name>factory</name> <value>org.apache.commons.dbcp.BasicDataSourceFactory</value> </parameter> <parameter> <name>driverClassName</name> <value>oracle.jdbc.OracleDriver</value> </parameter> <parameter> <name>url</name> <value>jdbc:oracle:thin:@127.0.0.1:1521:mysid</value> </parameter> <parameter> <name>username</name> <value>scott</value> </parameter> <parameter> <name>password</name> <value>tiger</value> </parameter> <parameter> <name>maxActive</name> <value>20</value> </parameter> <parameter> <name>maxIdle</name> <value>10</value> </parameter> <parameter> <name>maxWait</name> <value>-1</value> </parameter> </ResourceParams> 2. web.xml configuration You should ensure that you respect the elemeent ordering defined by the DTD when you create you applications web.xml file. <resource-ref> <description>Oracle Datasource example</description> <res-ref-name>jdbc/myoracle</res-ref-name> <res-type>javax.sql.DataSource</res-type> <res-auth>Container</res-auth> </resource-ref> 3. Code example You can use the same example application as above (asuming you create the required DB instance, tables etc.) replacing the Datasource code with something like Context initContext = new InitialContext(); Context envContext = (Context)initContext.lookup("java:/comp/env"); DataSource ds = (DataSource)envContext.lookup("jdbc/myoracle"); Connection conn = ds.getConnection(); //etc. PostgreSQL 0. Introduction PostgreSQL is configured in a similar manner to Oracle. Again, highlighting the differences. These notes are untested as yet and we would appreciate feedback. 1. server.xml configuration <Resource name="jdbc/postgres" auth="Container" type="javax.sql.DataSource"/> <ResourceParams name="jdbc/postgres"> <parameter> <name>factory</name> <value>org.apache.commons.dbcp.BasicDataSourceFactory</value> </parameter> <parameter> <name>driverClassName</name> <value>org.postgresql.Driver</value> </parameter> <parameter> <name>url</name> <value>jdbc:postgresql://127.0.0.1:5432/mydb</value> </parameter> <parameter> <name>username</name> <value>myuser</value> </parameter> <parameter> <name>password</name> <value>mypasswd</value> </parameter> <parameter> <name>maxActive</name> <value>20</value> </parameter> <parameter> <name>maxIdle</name> <value>10</value> </parameter> <parameter> <name>maxWait</name> <value>-1</value> </parameter> </ResourceParams> 2. web.xml configuration <resource-ref> <description>postgreSQL Datasource example</description> <res-ref-name>jdbc/mydb</res-ref-name> <res-type>javax.sql.DataSource</res-type> <res-auth>Container</res-auth> </resource-ref>

Tyrex Connection Pool

Introduction Tomcat 4.1 provides transaction management and resource configuration support through the use of Tyrex 1.0. This allows the user to obtain JTA/JCA resources from the JNDI namespace, as well as the standard javax.transaction.UserTransaction. Installing Required JARs In order for a web application to use Tyrex, the webapp and Tomcat need to have access to the Tyrex jar, as well as the jars it requires. Here is a list of the required jars, and where to obtain them: The following jars are included with Tyrex binary distribution, available at http://tyrex.exolab.org. tyrex-1.0.jar ots-jts_1.0.jar jta_1.0.1.jar xerces-J_1.4.0.jar The following two jars are required as well: Castor XML jar (http://castor.exolab.org/, version 0.92 or higher is reccommended) Log4J (http://jakarta.apache.org/log4j/, version 1.0.4 or higher is reccommended) All six of these jar files need to be placed on $TOMCAT_HOME/common/lib so that both Tomcat and your web application will see them. Configuring Tyrex The Tyrex documentation (http://tyrex.exolab.org) provides complete details on how to properly configure Tyrex. As an example, we will use the following Tyrex configuration, specified in Tyrex's domain configuration XML file: <domain> <name>myDomain</name> <resources> <dataSource> <name>myDatasource</name> <jar>/home/david/mm.mysql-2.0.14-bin.jar</jar> <class>org.gjt.mm.mysql.jdbc2.optional.MysqlXaDataSource</class> <config> <user>david</user> <password>secret</password> <serverName>localhost</serverName> <port>3306</port> <database>daviddb</database> </config> </dataSource> </resources> </domain> A few things to note: You need to specify the full pathname of the JAR file (for relative paths, Tyrex looks in the current working directory, this usually isn't what you want). You can also specify a URL. Any elements nested inside the <config></config>elements are passed as parameters to the datasource class, using standard setter methods. More configuration options are available, as well as a better description of how to setup Tyrex, at http://tyrex.exolab.org/configuration.html This XML config file needs to be placed where Tomcat's classloader can find it using getResource(). This means that the WEB-INF/classes directory under your webapp is a very good choice. Configuring Tomcat Now that your Tyrex XML config file is in place and ready, you must enlist the Tyrex resources in the JNDI namespace. This is done through Tomcat's server.xml file. Two important parameters must be specified: the name of the domain config file (tyrexDomainConfig), and the name of the Tyrex domain that is to be used (tyrexDomainName). These need to be setup as Environment parameters, like so: <Environment name="tyrexDomainConfig" type="java.lang.String" value="domain-config.xml"/> <Environment name="tyrexDomainName" type="java.lang.String" value="myDomain"/> Now, you must configure the resource (under the <Context> element of your webapp): <Resource name="my-datasource" auth="Container" type="tyrex.resource.Resource"/> <ResourceParams name="my-datasource"> <parameter> <name>name</name> <value>myDataSource</value> </parameter> </ResourceParams> A couple of things to point out: The type of resource should always be tyrex.resource.Resource, regardless of how you have Tyrex configured. Only one ResourceParam parameter is needed, name -- the value should be set to the name of resource specified in the Tyrex config file. Note the difference between a Tomcat/JNDI resource and a Tyrex resource (it can be confusing at first glance!) Coding Your Application Making use of your Tyrex resource should now be relatively simple. To obtain your datasource, simply use JNDI: InitialContext initCtx = new InitialContext(); DataSource ds = (DataSource) initCtx.lookup("java:comp/env/my-datasource"); Connection conn = ds.getConnection(); ...and so on. Tyrex also provides a javax.transaction.UserTransaction, obtainable through JNDI at the standard location (java:comp/UserTransaction).

Non DBCP Solutions

These solutions either utilise a single connection to the database (not recommended for anything other than testing!) or some other pooling technology.

Oracle 8i with OCI client

Introduction Whilst not strictly addressing the creation of a JNDI DataSource using the OCI client, these notes can be combined with the Oracle and DBCP solution above. In order to use OCI driver, you should have an Oracle client installed. You should have installed Oracle8i(8.1.7) client from cd, and download the suitable JDBC/OCI driver(Oracle8i 8.1.7.1 JDBC/OCI Driver) from otn.oracle.com. After renaming classes12.zip file to classes12.jar for Tomcat, copy it into $CATALINA_HOME/common/lib. You may also have to remove the javax.sql.* classes from this file depending upon the version of Tomcat and JDK you are using. Putting it all together Ensure that you have the ocijdbc8.dll or .so in your $PATH or LD_LIBRARY_PATH (possibly in $ORAHOME/bin) and also confirm that the native library can be loaded by a simple test program using System.loadLibrary("ocijdbc8"); You should next create a simple test servlet or jsp that has these critical lines: DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver()); conn = DriverManager.getConnection("jdbc:oracle:oci8:@database","username","password"); where database is of the form host:port:SID Now if you try to access the URL of your test servlet/jsp and what you get is a ServletException with a root cause of java.lang.UnsatisfiedLinkError:get_env_handle. First, the UnsatisfiedLinkError indicates that you have a mismatch between your JDBC classes file and your Oracle client version. The giveaway here is the message stating that a needed library file cannot be found. For example, you may be using a classes12.zip file from Oracle Version 8.1.6 with a Version 8.1.5 Oracle client. The classeXXXs.zip file and Oracle client software versions must match. A $PATH, LD_LIBRARY_PATH problem. It has been reported that ignoring the driver you have downloded from otn and using the classes12.zip file from the directory $ORAHOME/jdbc/lib will also work. Next you may experience the error ORA-06401 NETCMN: invalid driver designator The Oracle documentation says : "Cause: The login (connect) string contains an invalid driver designator. Action: Correct the string and re-submit." Change the database connect string (of the form host:port:SID) with this one: (description=(address=(host=myhost)(protocol=tcp)(port=1521))(connect_data=(sid=orcl))) Ed. Hmm, I don't think this is really needed if you sort out your TNSNames - but I'm not an Oracle DBA :-)

Common Problems

Here are some common problems encountered with a web application which uses a database and tips for how to solve them. Intermittent dB Connection Failures Tomcat runs within a JVM. The JVM periodically performs garbage collection (GC) to remove java objects which are no longer being used. When the JVM performs GC execution of code within Tomcat freezes. If the maximum time configured for establishment of a dB connection is less than the amount of time garbage collection took you can get a db conneciton failure. To collect data on how long garbage collection is taking add the -verbose:gc argument to your CATALINA_OPTS environment variable when starting Tomcat. When verbose gc is enabled your $CATALINA_BASE/logs/catalina.out log file will include data for every garbage collection including how long it took. When your JVM is tuned correctly 99% of the time a GC will take less than one second. The remainder will only take a few seconds. Rarely, if ever should a GC take more than 10 seconds. Make sure that the db connection timeout is set to 10-15 seconds. For the DBCP you set this using the parameter maxWait. Random Connection Closed Exceptions These can occur when one request gets a db connection from the connection pool and closes it twice. When using a connection pool, closing the connection just returns it to the pool for reuse by another request, it doesn't close the connection. And Tomcat uses multiple threads to handle concurrent requests. Here is an example of the sequence of events which could cause this error in Tomcat: Request 1 running in Thread 1 gets a db connection. Request 1 closes the db connection. The JVM switches the running thread to Thread 2 Request 2 running in Thread 2 gets a db connection (the same db connection just closed by Request 1). The JVM switches the running thread back to Thread 1 Request 1 closes the db connection a second time in a finally block. The JVM switches the running thread back to Thread 2 Request 2 Thread 2 tries to use the db connection but fails because Request 1 closed it. Here is an example of properly written code to use a db connection obtained from a connection pool: Connection conn = null; Statement stmt = null; // Or PreparedStatement if needed ResultSet rs = null; try { conn = ... get connection from connection pool ... stmt = conn.createStatement("select ..."); rs = stmt.executeQuery(); ... iterate through the result set ... rs.close(); rs = null; stmt.close(); stmt = null; conn.close(); // Return to connection pool conn = null; // Make sure we don't close it twice } catch (SQLException e) { ... deal with errors ... } finally { // Always make sure result sets and statements are closed, // and the connection is returned to the pool if (rs != null) { try { rs.close(); } catch (SQLException e) { ; } rs = null; } if (stmt != null) { try { stmt.close(); } catch (SQLException e) { ; } stmt = null; } if (conn != null) { try { conn.close(); } catch (SQLException e) { ; } conn = null; } } REFERENCE:http://tomcat.apache.org/tomcat-4.1-doc/jndi-datasource-examples-howto.html#Database%20Connection%20Pool%20(DBCP)%20Configurations