Persisting Dynamic Jobs with Quartz and Spring
Quartz Scheduler is a richly featured, open source job scheduling library that can be integrated within virtually any Java application - from the smallest stand-alone application to the largest e-commerce system.
Welcome to the second post in the Dynamic Scheduling with Quartz series. In the previous post I talked about scheduling jobs dynamically and storing then in memory with RAMJobStore. Although this approach is performant, the jobs do not survive application crash or a restart.
In this post, I will show you how to create peristent jobs that are stored in a Relational Database. If I get enough requests, I will do another post that stores the jobs in a Non-Relational Database.
To store jobs in a relational database with quartz the
JDBCJobStore is used. In the next section I will
delve deeper into the JDBCJobStore.
JobStore’s are responsible for keeping track of all the “work data” that you give to the scheduler:
calendars, etc. Selecting the appropriate JobStore for your Quartz scheduler instance is an important step. Luckily,
the choice should be a very easy one once you understand the differences between them.
In this post I will talk about the
JDBCJobStore and persistence in a relational database using
Liquibase for database
JDBCJobStore keeps all of its data in a database via JDBC. Because of this it is a bit more complicated to configure
RAMJobStore, and it also is not as fast. However, the performance draw-back is not terribly bad, especially if
you build the database tables with indexes on the primary keys. On fairly modern set of machines with a decent LAN
(between the scheduler and database) the time to retrieve and update a firing trigger will typically be less than 10 milliseconds.
JDBCJobStore works with nearly any database, it has been used widely with
DB2. In this post we will be using
HSQLDB as the persistence backend. This
can be adapted to fit any relational database of your choosing.
To use JDBCJobStore, you must first create a set of database tables for Quartz to use. You can find table-creation SQL scripts in the “org/quartz/impl/jdbcjobstore” directory of the Quartz distribution. If there is not already a script for your database type, just look at one of the existing ones, and modify it in any way necessary for your DB. One thing to note is that in these scripts, all the the tables start with the prefix “
QRTZ_” (such as the tables “
QRTZ_TRIGGERS”, and “
QRTZ_JOB_DETAIL”). This prefix
can actually be anything you’d like, as long as you inform JDBCJobStore what the prefix is
(in your Quartz properties). Using different prefixes may be useful for creating multiple sets of tables, for multiple
scheduler instances, within the same database.
There are two seperate JDBCJobStore classes that you can select between, depending on the transactional behaviour you need:-
JobStoreTX - manages all transactions itself by calling
rollback()) on the database connection after every action (such as the addition of a job).
JobStoreTXis appropriate if you are using Quartz in a stand-alone application, or within a servlet container if the application is not using JTA transactions. The JobStoreTX is selected by setting the ‘
org.quartz.jobStore.class’ property in the
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
JobStoreCMT - relies upon transactions being managed by the application which is using Quartz. A JTA transaction must be in progress before attempt to schedule (or unschedule) jobs/triggers. This allows the “work” of scheduling to participate in a global transaction.
JobStoreCMTactually requires the use of two datasources
- one that has it’s connection’s transactions managed by the application server (via JTA) and
- one datasource that has connections that do not participate in global (JTA) transactions.
JobStoreCMT is appropriate when applications are using JTA transactions (such as via EJB Session Beans) to perform their work.
The JobStore is selected by setting the ‘
org.quartz.jobStore.class’ property in the
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreCMT
Once you have decided on the type of
JDBCJobStore to use based on your transactional needs, you need to select a
DriverDelegate for the JobStore to use. The DriverDelegate is responsible for doing any JDBC work that may be needed
for your specific database.
StdJDBCDelegate is a delegate that uses “vanilla” JDBC code (and SQL statements) to do
its work. If there isn’t another delegate made specifically for your database, try using this delegate. Other
delegates can be found in the “
org.quartz.impl.jdbcjobstore” package, or in its sub-packages. Other delegates
DB2v6Delegate (for DB2 version 6 and earlier),
HSQLDBDelegate (for HSQLDB),
MSSQLDelegate (for Microsoft
PostgreSQLDelegate (for PostgreSQL),
WeblogicDelegate (for using JDBC drivers made by Weblogic),
OracleDelegate (for using Oracle), and others.
Once you’ve selected your delegate, set its class name as the delegate for JDBCJobStore to use:
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
The contents of the archive should be similar to the directory structure below:
. |__src/ | |__main/ | | |__java/ | | | |__com/ | | | | |__juliuskrah/ | | | | | |__quartz/ | | | | | | |__Application.java | | | | | | |__job/ | | | | | | | |__EmailJob.java | | | | | | |__model/ | | | | | | | |__JobDescriptor.java | | | | | | | |__TriggerDescriptor.java | | | | | | |__service/ | | | | | | | |__AsyncMailSender.java | | | | | | | |__EmailService.java | | | | | | |__web/ | | | | | | | |__rest/ | | | | | | | | |__EmailResource.java | | | |__org/ | | | | |__springframework/ | | | | | |__boot/ | | | | | | |__autoconfigure/ | | | | | | | |__quartz/ | | | | | | | | |__AutowireCapableBeanJobFactory.java | | |__resources/ | | | |__db/ | | | | |__changelog/ | | | | | |__db.changelog-master.yaml | | | |__application.yaml | | | |__quartz.properties |__pom.xml
To follow alongside this guide by implementing the code, you should have the following set up:
Getting the project
Extract the contents somewhere on your development system and let us begin.
Preparing Liquibase Migration Scripts
We will setup our liquibase migration script:
And tell Spring to run the migration on application initialization:
We will create a bean of type
SchedulerFactoryBean where we will setup a JDBCJobStore:
In the snippet above we are setting the
Datasource. When we do this, we are actually setting a JobStoreCMT of
LocalDataSourceJobStore - Subclass of Quartz’s JobStoreCMT class that delegates to a Spring-managed DataSource instead of using a Quartz-managed connection pool. This JobStore will be used if SchedulerFactoryBean’s “dataSource” property is set.
Supports both transactional and non-transactional DataSource access. With a non-XA DataSource and local Spring transactions, a single DataSource argument is sufficient.
Operations performed by this JobStore will properly participate in any kind of Spring-managed transaction, as it uses Spring’s DataSourceUtils connection handling methods that are aware of a current transaction.
Note that all Quartz Scheduler operations that affect the persistent job store should usually be performed within active transactions, as they assume to get proper locks etc.
We need a
quartz.properties on the classpath to set the
org.quartz.impl.jdbcjobstore.DriverDelegate for HSQLDB:
We are all set. Now you can schedule dynamic jobs and these jobs will be saved in the database and persisted across application restarts.
In the following sections I will show you how to optimize threads and jobs.
Using Spring Task Execution Abstraction
Long-running jobs prevent others from running (if all threads in the ThreadPool are busy).
If you feel the need to call Thread.sleep() on the worker thread executing the Job, it is typically a sign that the job is not ready to do the rest of its work because it needs to wait for some condition (such as the availability of a data record) to become true.
A better solution is to release the worker thread (exit the job) and allow other jobs to execute on that thread. The job can reschedule itself, or other jobs before it exits.
We will setup up a service class that runs
async methods using
Spring TaskExecutor abstraction:
To understand the
async annotation take a look at this article.
For Spring to process all
Async annotations, we need to update the configuration class to
Quartz by default uses a
org.quartz.simpl.SimpleThreadPool for creating scheduling and executing jobs. We will
override this behaviour and ask Quartz to share in the thread-pool created by Spring:
The above delegates all jobs to the
org.springframework.scheduling.quartz.LocalTaskExecutorThreadPool when executing
AsyncMailSender in the
That’s all folks.
In this post we learned how to schedule quartz jobs dynamically using
JDBCJobStore. We also covered how do use
Async annotation to make our jobs execute faster. In the next post we will cover
Error Handling in a REST service.
You can find the source to this guide in the github repository. Until the next post, keep doing cool things .