Using foreign key constraints in Clojure with clojure.java.jdbc

Question

I'm working on a wiki program and using SQLite as the database. I want to create a many-to-many relationship between wiki pages and tags describing those pages. I'm using clojure.java.jdbc to handle the database operations. I would like to enforce foreign key constraints in the page-to-tags cross-reference table. I looked at the information about foreign keys on the SQLite site ( https://www.sqlite.org/foreignkeys.html ) and believe something like this is what I want;

(def the-db-name "the.db")
(def the-db {:classname   "org.sqlite.JDBC"
             :subprotocol "sqlite"
             :subname     the-db-name})

(defn create-some-tables
  "Create some tables and a cross-reference table with foreign key constraints."
  []
  (try (jdbc/db-do-commands
         the-db false
         ["PRAGMA foreign_keys = ON;"
          (jdbc/create-table-ddl :pages
                                 [[:page_id :integer :primary :key]
                                  ;...
                                  [:page_content :text]])
          (jdbc/create-table-ddl :tags
                                 [[:tag_id :integer :primary :key]
                                  [:tag_name :text "NOT NULL"]])
          (jdbc/create-table-ddl :tags_x_pages
                                 [[:x_ref_id :integer :primary :key]
                                  [:tag_id :integer]
                                  [:page_id :integer]
                                  ["FOREIGN KEY(tag_id) REFERENCES tags(tag_id)"]
                                  ["FOREIGN KEY(page_id) REFERENCES pages(page_id)"]])])

       (catch Exception e (println e))))

But attempting to turn the pragma on has no effect.

Just trying to turn the pragma on and check for effect:

(println "Check before:" (jdbc/query the-db ["PRAGMA foreign_keys;"]))
; Transactions on or off makes no difference.
(println "Result of execute!:" (jdbc/execute! the-db
                                              ["PRAGMA foreign_keys = ON;"]))
(println "Check after:" (jdbc/query the-db ["PRAGMA foreign_keys;"]))

;=> Check before: ({:foreign_keys 0})
;=> Result of execute!: [0]
;=> Check after: ({:foreign_keys 0})

The results indicate that the library (org.xerial/sqlite-jdbc "3.21.0.1") was compiled to support foreign keys since there were no errors, but trying to set the pragma has no effect.

I found this in the JIRA for the clojure JDBC back in 2012. The described changes have been implemented since then, but the code still has no effect.

Finally found this answer to a Stackoverflow question that pointed to this post back in 2011. That allowed me to cobble together something that did seem to set the pragma. The code below depends on creating a specially configured Connection .

(ns example
  (:require [clojure.java.jdbc :as jdbc])
  (:import (java.sql Connection DriverManager)
           (org.sqlite SQLiteConfig)))

(def the-db-name "the.db")
(def the-db {:classname   "org.sqlite.JDBC"
             :subprotocol "sqlite"
             :subname     the-db-name})

(defn ^Connection get-connection
  "Return a connection to a SQLite database that
  enforces foreign key constraints."
  [db]
  (Class/forName (:classname db))
  (let [config (SQLiteConfig.)]
    (.enforceForeignKeys config true)
    (let [connection (DriverManager/getConnection
                       (str "jdbc:sqlite:" (:subname db))
                       (.toProperties config))]
      connection)))

(defn exec-foreign-keys-pragma-statement
  [db]
  (let [con ^Connection (get-connection db)
        statement (.createStatement con)]
    (println "exec-foreign-keys-pragma-statement:"
             (.execute statement "PRAGMA foreign_keys;"))))

Based on the above, I rewrote the table creation code above as:

(defn create-some-tables
  "Create some tables and a cross-reference table with foreign key constraints."
  []
  (when-let [conn (get-connection the-db)]
    (try
      (jdbc/with-db-connection
        [conn the-db]
        ; Creating the tables with the foreign key constraints works.
        (try (jdbc/db-do-commands
               the-db false
               [(jdbc/create-table-ddl :pages
                                       [[:page_id :integer :primary :key]
                                        [:page_content :text]])
                (jdbc/create-table-ddl :tags
                                       [[:tag_id :integer :primary :key]
                                        [:tag_name :text "NOT NULL"]])
                (jdbc/create-table-ddl :tags_x_pages
                                       [[:x_ref_id :integer :primary :key]
                                        [:tag_id :integer]
                                        [:page_id :integer]
                                        ["FOREIGN KEY(tag_id) REFERENCES tags(tag_id)"]
                                        ["FOREIGN KEY(page_id) REFERENCES pages(page_id)"]])])

             ; This still doesn't work.
             (println "After table creation:"
                      (jdbc/query the-db "PRAGMA foreign_keys;"))

             (catch Exception e (println e))))

      ; This returns the expected results.
      (when-let [statement (.createStatement conn)]
        (try
          (println "After creating some tables: PRAGMA foreign_keys =>"
                   (.execute statement "PRAGMA foreign_keys;"))
          (catch Exception e (println e))
          (finally (when statement
                     (.close statement)))))
      (catch Exception e (println e))
      (finally (when conn
                 (.close conn))))))

The tables are created as expected. Some of the clojure.java.jdbc functions still don't seem to work as desired though. (See the jdbc/query call in the middle of the listing.) Getting things to always work as expected seems very "manual" having to fall back on java interop. And it seems like every interaction with the database requires using the specially configured Connection returned by the get-connection function.

Is there a better way to enforce foreign key constraints in SQLite in Clojure?

Answer 1

I've not played with SqlLite, but would recommend you test with either

• H2: Pure java, can run in memory for tests ( http://www.h2database.com )
• Postgres: Needs install, but is the gold standard for SQL compliance ( https://www.postgresql.org )

Also, when debugging it may be easier to use pure SQL strings (see http://clojure-doc.org/articles/ecosystem/java_jdbc/using_sql.html ):

(j/execute! db-spec
            ["update fruit set cost = ( 2 * grade ) where grade > ?" 50.0])

Using pure SQL strings (especially when debugging) can avoid many misunderstandings/pitfalls with JDBC. Also, keep in mind that you may discover a bug in either the Clojure JDBC libs or the DB itself.

Answer 2

I'm not sure SQLite does support the features you described above. If you really want to keep your data being consisted with strict constraints, use PostgeSQL database. I know that working with SQLite seems easier especially when you've just started the project, but believe me, using Postgres really worth it.

Here is an example of post and tags declaration using Postgres that takes lots of details into account:

create table post(
  id serial primary key,
  title text not null,
  body text not null
);

create table tags(
  id serial primary key,
  text text not null unique
);

create table post_tags(
  id serial primary key,
  post_id integer not null references posts(id),
  tag_id integer not null references tags(id),
  unique(post_id, tag_id)
);

Here, the tags table cannot contain two equal tags. That's important to keep only unique tag strings to prevent the table from growing.

The bridge table that links a post with tags has a special constraint to prevent a case when a specific tag is linked to a post several times. Say, if a post has "python" and "clojure" tags attached, you won't be able to add "python" one more time.

Finally, each reference clause when declaring a table creates a special constraint that prevents you from referencing an id that does not exist in a target table.

Installing Postgres and setting it up might be a bit difficult, but nowadays there are such one-click applications like Postgres App that are quite easy to use even if you are not familiar with them.

Answer 3

With the advent of next.jdbc you can now do that like so:

(ns dev
  (:require [next.jdbc :as jdbc]
            [next.jdbc.sql :as sql]))

(with-open [conn (jdbc/get-connection {:dbtype "sqlite" :dbname "test.db"})]
  (println (sql/query conn ["PRAGMA foreign_keys"]))
  (jdbc/execute! conn ["PRAGMA foreign_keys = ON"])
  ; jdbc/execute whatever you like here...
  (println (sql/query conn ["PRAGMA foreign_keys"])))

This outputs

[{:foreign_keys 0}]
[{:foreign_keys 1}]