Grails application gets stuck on startup because DATABASECHANGELOG table is locked

Question

I have a Grails application hosted on Heroku that uses the database migration plugin. From time to time when the application is restarted it get stucked on startup without any helping messages:

2012-11-12T11:21:06+00:00 app[web.1]: Configuring Spring Security Core ...
2012-11-12T11:21:06+00:00 app[web.1]: ... finished configuring Spring Security Core
2012-11-12T11:21:06+00:00 app[web.1]:
2012-11-12T11:21:22+00:00 heroku[web.1]: Stopping process with SIGKILL
2012-11-12T11:21:22+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2012-11-12T11:21:24+00:00 heroku[web.1]: Process exited with status 137
2012-11-12T11:21:34+00:00 heroku[router]: Error H20 (App boot timeout) -> GET abc.herokuapp.com/ dyno= queue= wait=75000ms service= status=503 bytes=
2012-11-12T11:22:34+00:00 heroku[router]: Error H20 (App boot timeout) -> GET abc.herokuapp.com/ dyno= queue= wait=75000ms service= status=503 bytes=
2012-11-12T11:25:10+00:00 heroku[router]: Error H10 (App crashed) -> GET abc.herokuapp.com/ dyno= queue= wait= service= status=503 bytes=

After some investigation, I discovered that this was happening because there was a record on the DATABASECHANGELOGLOCK table with the column LOCKED=TRUE. When I manually set the LOCKED value to FALSE and restart the application, the startup process completes without problems.

My questions are:

1. How can I find what's leaving the database changelogs locked and how can I avoid it.
2. If it's not possible to avoid the lock, there is any way to clear the lock on the application startup process before it gets stucked?

Answer 1

This issue occurred in my projects when the application had been killed during the bootstrap process and the database migration could not complete its db transaction.

What liquibase (the underlying library used by database-migration) does is, it creates the lock entry and then starts a database transaction to work through all change sets that need to be applied. If, by any chance, the db transaction does not commit successfully, the lock entry will stay in DATABASECHANGELOGLOCK .

The same question has already been asked some time ago in the liquibase forum .

The DATABASECHANGELOG table keeps record of the successfully applied db change sets, you can use dbm-db-doc or the DbDocController to visualise this information (or you could access the db table with SQL directly, of course).