从 Spring Boot 连接到 Heroku Postgres

Question

I'm looking for the simplest , cleanest way of connecting to Heroku Postgres in a Spring Boot app using JPA/Hibernate.

I don't see a good, complete example for this combo in either Heroku or Spring Boot documentation, so I'd like to document this on Stack Overflow.

I'm trying to go with something like this:

@Configuration   
public class DataSourceConfig {

    Logger log = LoggerFactory.getLogger(getClass());

    @Bean
    @Profile("postgres")
    public DataSource postgresDataSource() {        
        String databaseUrl = System.getenv("DATABASE_URL")
        log.info("Initializing PostgreSQL database: {}", databaseUrl);

        URI dbUri;
        try {
            dbUri = new URI(databaseUrl);
        }
        catch (URISyntaxException e) {
            log.error(String.format("Invalid DATABASE_URL: %s", databaseUrl), e);
            return null;
        }

        String username = dbUri.getUserInfo().split(":")[0];
        String password = dbUri.getUserInfo().split(":")[1];
        String dbUrl = "jdbc:postgresql://" + dbUri.getHost() + ':' 
            + dbUri.getPort() + dbUri.getPath();

        // fully-qualified class name to distuinguish from javax.sql.DataSource 
        org.apache.tomcat.jdbc.pool.DataSource dataSource 
            = new org.apache.tomcat.jdbc.pool.DataSource();
        dataSource.setUrl(dbUrl);
        dataSource.setUsername(username);
        dataSource.setPassword(password);
        return dataSource;
    }

}

I'm using Profiles , which seems a good match for what I want: on Heroku SPRING_PROFILES_ACTIVE is set to postgres , while in local development spring.profiles.active is h2 to use a H2 in-memory database (whose config omitted here). This approach seems to work fine.

In application-postgres.properties ( profile-specific properties ):

spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.driverClassName=org.postgresql.Driver

DataSource from Tomcat seemed like a good option since the default dependencies include it, and because Spring Boot reference guide says :

We prefer the Tomcat pooling DataSource for its performance and concurrency, so if that is available we always choose it.

(I'm also seeing BasicDataSource from Commons DBCP being used with Spring Boot . But to me this does not seem like the cleanest choice as the default dependencies do not include Commons DBCP. And in general I'm wondering if Apache Commons could really , in 2015, be the recommended way to connect to Postgres... Also Heroku documentation offers " BasicDataSource in Spring" for this kind of scenario; I assume this refers to Commons DBCP, since I don't see such class in Spring itself.)

Dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>       
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>9.4-1205-jdbc42</version>
</dependency>

Current status : failing with "Not loading a JDBC driver as driverClassName property is null":

eConfig$$EnhancerBySpringCGLIB$$463388c1 : Initializing PostgreSQL database: postgres:[...]
j.LocalContainerEntityManagerFactoryBean : Building JPA container EntityManagerFactory for persistence unit 'default'
org.hibernate.cfg.Environment            : HHH000206: hibernate.properties not found
[...]
o.a.tomcat.jdbc.pool.PooledConnection    : Not loading a JDBC driver as driverClassName property is null.    
o.a.tomcat.jdbc.pool.PooledConnection    : Not loading a JDBC driver as driverClassName property is null.
[...]
org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQLDialect

In logs I see that my postgresDataSource is called just fine, and that PostgreSQLDialect is in use (without this it was failing with "Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set").

My specific questions

1. Well, how to get this working? I am setting spring.datasource.driverClassName , so why "Not loading a JDBC driver as driverClassName property is null"?
2. Is the use of Tomcat's DataSource fine or would you recommend something else?
3. Is it mandatory to define postgresql dependency as above with a specific version ? (I was getting "no suitable driver found" error without this.)
4. Is there a simpler way to do all this (while sticking to Java code and/or properties; no XML please)?

Answer 1

Simplest cleanest way for Spring Boot 2.x with Heroku & Postgres

I read all answers, but didn´t find what Jonik was looking for:

I'm looking for the simplest, cleanest way of connecting to Heroku Postgres in a Spring Boot app using JPA/Hibernate

The development process most people want to use with Spring Boot & Heroku includes a local H2 in-memory database for testing & fast development cycles - and the Heroku Postgres database for staging and production on Heroku.

• First thing is - you don´t need to use Spring profiles for that!
• Second: You don´t need to write/change any code!

Let´s have a look on what we have to do step by step. I have a example project in place that provides a fully working Heroku deployment and configuration for Postgres - only for the sake of completeness, if you want to test it yourself: github.com/jonashackt/spring-boot-vuejs .

The pom.xml

We need the following depencencies:

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>

    <!-- In-Memory database used for local development & testing -->
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
    </dependency>

    <!-- Switch back from Spring Boot 2.x standard HikariCP to Tomcat JDBC,
    configured later in Heroku (see https://stackoverflow.com/a/49970142/4964553) -->
    <dependency>
        <groupId>org.apache.tomcat</groupId>
        <artifactId>tomcat-jdbc</artifactId>
    </dependency>

    <!-- PostgreSQL used in Staging and Production environment, e.g. on Heroku -->
    <dependency>
        <groupId>org.postgresql</groupId>
        <artifactId>postgresql</artifactId>
        <version>42.2.2</version>
    </dependency>

One tricky thing here is the usage of tomcat-jdbc , but we´ll cover that in a second.

Configure Environment Variables on Heroku

In Heroku Environment Variables are named Config Vars . You heard right, all we have to do is to configure Enviroment Variables! We just need the correct ones. Therefore head over to https://data.heroku.com/ (I assume there´s already a Postgres database configured for your Heroku app, which is the default behavior).

Now click on your application´s corresponding Datastore and switch over to the Settings tab. Then click on View Credentials... , which should look something similar like this:

Now open a new browser tab and go to your Heroku application´s Settings tab also. Click on Reveal Config Vars and create the following Environment Variables:

• SPRING_DATASOURCE_URL = jdbc :postgres ql :// YourPostgresHerokuHostNameHere :5432/ YourPostgresHerokuDatabaseNameHere (mind the leading jdbc: and the ql addition to postgres !)
• SPRING_DATASOURCE_USERNAME = YourPostgresHerokuUserNameHere
• SPRING_DATASOURCE_PASSWORD = YourPostgresHerokuPasswordHere
• SPRING_DATASOURCE_DRIVER-CLASS-NAME = org.postgresql.Driver (this isn´t always needed since Spring Boot can deduce it for most databases from the url , just for completeness here)
• SPRING_JPA_DATABASE-PLATFORM = org.hibernate.dialect.PostgreSQLDialect
• SPRING_DATASOURCE_TYPE = org.apache.tomcat.jdbc.pool.DataSource
• SPRING_JPA_HIBERNATE_DDL-AUTO = update (this will automatically create your tables according to your JPA entities, which is really great - since you don´t need to hurdle with CREATE SQL statements or DDL files)

In Heroku this should look like this:

Now that´s all you have to do! Your Heroku app is restarted every time you change a Config Variable - so your App should now run H2 locally, and should be ready connected with PostgreSQL when deployed on Heroku.

Just if you´re asking: Why do we configure Tomcat JDBC instead of Hikari

As you might noticed, we added the tomcat-jdbc dependency to our pom.xml and configured SPRING_DATASOURCE_TYPE=org.apache.tomcat.jdbc.pool.DataSource as a Environment variable. There´s only a slight hint in the docs about this saying

You can bypass that algorithm completely and specify the connection pool to use by setting the spring.datasource.type property. This is especially important if you run your application in a Tomcat container, ...

There are several reasons I switched back to Tomcat pooling DataSource instead of using the Spring Boot 2.x standard HikariCP. As I already explained here , if you don´t specifiy spring.datasource.url , Spring will try to autowire the embedded im-memory H2 database instead of our PostgreSQL one. And the problem with Hikari is, that it only supports spring.datasource.jdbc-url .

Second, if I try to use the Heroku configuration as shown for Hikari (so leaving out SPRING_DATASOURCE_TYPE and changing SPRING_DATASOURCE_URL to SPRING_DATASOURCE_JDBC-URL ) I run into the following Exception:

Caused by: java.lang.RuntimeException: Driver org.postgresql.Driver claims to not accept jdbcUrl, jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE

So I didn´t get Spring Boot 2.x working on Heroku & Postgres with HikariCP, but with Tomcat JDBC - and I also don´t want to brake my development process containing a local H2 database described upfront. Remember: We were looking for the simplest, cleanest way of connecting to Heroku Postgres in a Spring Boot app using JPA/Hibernate!

Answer 2

Simplest Spring Boot / Heroku / Hibernate Configuration

Apart from DATABASE_URL , which is always there, Heroku creates 3 environment variables at Runtime. They are:

JDBC_DATABASE_URL
JDBC_DATABASE_USERNAME
JDBC_DATABASE_PASSWORD

As you may be aware, Spring Boot will automatically configure your database if it finds spring.datasource.* properties in your application.properties file. Here is an example of my application.properties

spring.datasource.url=${JDBC_DATABASE_URL}
spring.datasource.username=${JDBC_DATABASE_USERNAME}
spring.datasource.password=${JDBC_DATABASE_PASSWORD}
spring.jpa.show-sql=false
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=update

Hibernate / Postgres Dependencies

In my case I'm using Hibernate (bundled in spring-boot-starter-jpa with PostgreSQL, so I needed the right dependencies in my build.gradle :

dependencies {
    compile("org.springframework.boot:spring-boot-starter-data-jpa")
    compile('org.postgresql:postgresql:9.4.1212')
}

Answer 3

To get the database connection working (in a stable manner) two things were missing in the setup I described in the question:

• As jny pointed out , I needed to set JDBC driver explicitly :
  • dataSource.setDriverClassName("org.postgresql.Driver");
  • (The reason for this is that I'm defining a custom datasource, overriding Spring's default, causing my spring.datasource.driverClassName property to have no effect. And to my understanding, due to the dynamic nature of Heroku's DATABASE_URL , I need custom datasource to make it work.)
• After this the connection worked, but it wasn't stable; I started getting org.postgresql.util.PSQLException: This connection has been closed. after the app had been running for a while. A somewhat surprising solution (based on this answer ) was to enable certain tests such as testOnBorrow on the Tomcat DataSource:
  • dataSource.setTestOnBorrow(true); dataSource.setTestWhileIdle(true); dataSource.setTestOnReturn(true); dataSource.setValidationQuery("SELECT 1");

So, the fixed version of my DataSourceConfig:

@Configuration
public class DataSourceConfig {

    Logger log = LoggerFactory.getLogger(getClass());

    @Bean
    @Profile("postgres")
    public DataSource postgresDataSource() {
        String databaseUrl = System.getenv("DATABASE_URL")
        log.info("Initializing PostgreSQL database: {}", databaseUrl);

        URI dbUri;
        try {
            dbUri = new URI(databaseUrl);
        }
        catch (URISyntaxException e) {
            log.error(String.format("Invalid DATABASE_URL: %s", databaseUrl), e);
            return null;
        }

        String username = dbUri.getUserInfo().split(":")[0];
        String password = dbUri.getUserInfo().split(":")[1];
        String dbUrl = "jdbc:postgresql://" + dbUri.getHost() + ':' 
                       + dbUri.getPort() + dbUri.getPath();

        org.apache.tomcat.jdbc.pool.DataSource dataSource 
            = new org.apache.tomcat.jdbc.pool.DataSource();
        dataSource.setDriverClassName("org.postgresql.Driver");
        dataSource.setUrl(dbUrl);
        dataSource.setUsername(username);
        dataSource.setPassword(password);
        dataSource.setTestOnBorrow(true);
        dataSource.setTestWhileIdle(true);
        dataSource.setTestOnReturn(true);
        dataSource.setValidationQuery("SELECT 1");
        return dataSource;
    }

}

With only this in application-postgres.properties :

spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect

Now, both of the problems I had may be specific to the DataSource from Tomcat ( org.apache.tomcat.jdbc.pool ). Apparently BasicDataSource (Commons DBCP) has more sensible defaults. But as mentiond in the question, I rather used something that comes with Spring Boot by default, especially as it's strongly endorsed in the reference guide.

I'm open to competing / simpler / better solutions, so feel free to post, especially if you can address the doubts 2–4 at the end of the question!

Using JDBC_DATABASE_* variables instead

Update: Note that using JDBC_DATABASE_* is much simpler than the above, as pointed out in this answer . For a long time I was under the impression that DATABASE_URL should be preferred, but nowadays I'm not so sure anymore.

Answer 4

Try using JDBC_DATABASE_URL as your spring.datasource.url instead of parsing up DATABASE_URL.

Parsing up DATABASE_URL is recommended, but if you can't get it to work, the JDBC_DATABASE_URL should be fine.

Answer 5

This is the top answer for googling Postgres problems with the sample Java Application that Heroku provides.

These are the steps that I did to get it to work (Win 7).

1.) The production Server application.properties file will contain the System environments (make sure this file has been committed)

spring.datasource.url=${JDBC_DATABASE_URL}
spring.datasource.username=${JDBC_DATABASE_USERNAME}
spring.datasource.password=${JDBC_DATABASE_PASSWORD}

2.) Now do git update-index --assume-unchanged .\\src\\main\\resources\\application.properties

3.) Change the local application.properties to be hardcoded. You can see the raw values by running heroku run env

spring.datasource.url=jdbc://..
spring.datasource.username=XYZ
spring.datasource.password=ABC

This is what I had to get the local copy of my application to work. If anyone found a better way please do share!

Answer 6

@Configuration
@Component
public class HerokuConfigCloud {

private static final Logger logger = 
LoggerFactory.getLogger(HerokuConfigCloud .class);

@Bean()
//@Primary this annotation to be used if more than one DB Config was used.  In that case,
// using @Primary would give precedence to a the particular "primary" config class
@Profile("heroku")
public DataSource dataSource(
        @Value("${spring.datasource.driverClassName}") final String driverClass,
        @Value("${spring.datasource.url}") final String jdbcUrl,
        @Value("${spring.datasource.username}") final String username,
        @Value("${spring.datasource.password}") final String password
        ) throws URISyntaxException {


    return DataSourceBuilder
            .create()
            .username(username)
            .password(password)
            .url(url)
            .driverClassName(driverClass)
            .build();
    }
}

Answer 7

I built a library to make this easy: https://github.com/vic-cw/heroku-postgres-helper

This is all the more helpful if you need to access the database both in your build script and in your application logic. See why here .

Usage:

build.gradle:

// If using connection string in build script:
buildscript {
    repositories {
        maven { url 'https://jitpack.io' }
    }
    dependencies {
        classpath 'com.github.vic-cw:heroku-postgres-helper:0.1.0'
    }
}
import com.github.viccw.herokupostgreshelper.HerokuPostgresHelper;

// Use connection string in build script:
flyway {
    url = HerokuPostgresHelper.getDatabaseJdbcConnectionString()
    driver = 'org.postgresql.Driver'
}

// If using connection string inside application logic:
repositories {
    maven { url 'https://jitpack.io' }
}

dependencies {
    compile group: 'com.github.vic-cw', name: 'heroku-postgres-helper', version: '0.1.0'
}

Java application code:

import com.github.viccw.herokupostgreshelper.HerokuPostgresHelper;

...

String databaseConnectionString = HerokuPostgresHelper.getDatabaseJdbcConnectionString();