Skip to content

Commit

Permalink
Fixes #123 by adding type-hinting functions
Browse files Browse the repository at this point in the history
  • Loading branch information
seancorfield committed Jun 24, 2020
1 parent 85734ab commit 0379230
Show file tree
Hide file tree
Showing 9 changed files with 110 additions and 6 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ Only accretive/fixative changes will be made from now on.

## Stable Builds

* 2020-06-24 -- 1.0.next
* Address #123 by adding `next.jdbc.types` namespace, full of auto-generated `as-xxx` functions, one for each of the `java.sql.Types` values.

* 2020-06-22 -- 1.0.476
* Extend default options behavior to `next.jdbc.sql` functions.

Expand Down
4 changes: 3 additions & 1 deletion doc/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,12 +398,14 @@ If you are using [Component](https://github.com/stuartsierra/component), a conne

By default, `next.jdbc` relies on the JDBC driver to handle all data type conversions when reading from a result set (to produce Clojure values from SQL values) or setting parameters (to produce SQL values from Clojure values). Sometimes that means that you will get back a database-specific Java object that would need to be manually converted to a Clojure data structure, or that certain database column types require you to manually construct the appropriate database-specific Java object to pass into a SQL operation. You can usually automate those conversions using either the [`ReadableColumn` protocol](/doc/result-set-builders.md#readablecolumn) (for converting database-specific types to Clojure values) or the [`SettableParameter` protocol](/doc/prepared-statements.md#prepared-statement-parameters) (for converting Clojure values to database-specific types).

In particular, PostgreSQL does not seem to perform a conversion from `java.util.Date` to a SQL data type automatically. You must `require` the [`next.jdbc.date-time` namespace](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time) to enable that conversion.
In particular, PostgreSQL does not seem to perform a conversion from `java.util.Date` to a SQL data type automatically. You can `require` the [`next.jdbc.date-time` namespace](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time) to enable that conversion.

If you are working with Java Time, some JDBC drivers will automatically convert `java.time.Instant` (and `java.time.LocalDate` and `java.time.LocalDateTime`) to a SQL data type automatically, but others will not. Requiring `next.jdbc.date-time` will enable those automatic conversions for all databases.

> Note: `next.jdbc.date-time` also provides functions you can call to enable automatic conversion of SQL date/timestamp types to Clojure data types when reading result sets. If you need specific conversions beyond that to happen automatically, consider extending the `ReadableColumn` protocol, mentioned above.
The `next.jdbc.types` namespace provides over three dozen convenience functions for "type hinting" values so that the JDBC driver might automatically handle some conversions that the default parameter setting function does not. Each function is named for the corresponding SQL type, prefixed by `as-`: `as-bigint`, `as-other`, `as-real`, etc. An example of where this helps is when dealing with PostgreSQL enumerated types: the default behavior, when passed a string that should correspond to an enumerated type, is to throw an exception that `column "..." is of type ... but expression is of type character varying`. You can wrap such strings with `(as-other "...")` which tells PostgreSQL to treat this as `java.sql.Types/OTHER` when setting the parameter.

## Processing Database Metadata

JDBC provides several features that let you introspect the database to obtain lists of tables, views, and so on. `next.jdbc` does not provide any specific functions for this but you can easily get this metadata from a `java.sql.Connection` and turn it into Clojure data as follows:
Expand Down
2 changes: 2 additions & 0 deletions doc/prepared-statements.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ You can also extend this protocol via metadata so you can do it on a per-object
(with-meta obj {'next.jdbc.prepare/set-parameter (fn [v ps i]...)})
```

The `next.jdbc.types` namespace provides functions to wrap values with per-object implementations of `set-parameter` for every standard `java.sql.Types` value. Each is named `as-xxx` corresponding to `java.sql.Types/XXX`.

The converse, converting database-specific types to Clojure values is handled by the `ReadableColumn` protocol, discussed in the previous section ([Result Set Builders](/doc/result-set-builders.md#readablecolumn)).

As noted above, `next.jdbc.prepare/set-parameters` is available for you to call on any existing `PreparedStatement` to set or update the parameters that will be used when the statement is executed:
Expand Down
23 changes: 22 additions & 1 deletion doc/tips-and-tricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,27 @@ If you have a query where you want to select where a column is `IN` a sequence o

What does this mean for your use of `next.jdbc`? In `plan`, `execute!`, and `execute-one!`, you can use `col = ANY(?)` in the SQL string and a single primitive array parameter, such as `(int-array [1 2 3 4])`. That means that in `next.jdbc.sql`'s functions that take a where clause (`find-by-keys`, `update!`, and `delete!`) you can specify `["col = ANY(?)" (int-array data)]` for what would be a `col IN (?,?,?,,,?)` where clause for other databases and require multiple values.

PostgreSQL has a SQL extension for defining enumerated types and the default `set-parameter` implementation will not work for those. You can use `next.jdbc.types/as-other` to wrap string values in a way that the JDBC driver will convert them to enumerated type values:

```sql
CREATE TYPE language AS ENUM('en','fr','de');

CREATE TABLE person (
...
speaks language NOT NULL,
...
);
```

```clojure
(require '[next.jdbc.sql :as sql]
'[next.jdbc.types :refer [as-other]])

(sql/insert! ds :person {:speaks (as-other "fr")})
```

That call produces a vector `["fr"]` with metadata that implements `set-parameter` such that `.setObject()` is called with `java.sql.Types/OTHER` which allows PostgreSQL to "convert" the string `"fr"` to the corresponding `language` enumerated type value.

### Streaming Result Sets

You can get PostgreSQL to stream very large result sets (when you are reducing over `plan`) by setting the following options:
Expand Down Expand Up @@ -150,7 +171,7 @@ create table example(
;; => #:example{:tags ["tag1" "tag2"]}
```

Note: PostgreSQL JDBC driver supports only 7 primitive array types, but not such as `UUID[]` -
> Note: PostgreSQL JDBC driver supports only 7 primitive array types, but not array types like `UUID[]` -
[PostgreSQL™ Extensions to the JDBC API](https://jdbc.postgresql.org/documentation/head/arrays.html).

### Working with Date and Time
Expand Down
6 changes: 5 additions & 1 deletion src/next/jdbc/prepare.clj
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,11 @@
See also https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time
for implementations of `SettableParameter` that provide automatic
conversion of Java Time objects to SQL data types."
conversion of Java Time objects to SQL data types.
See also https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.types
for `as-xxx` functions that provide per-instance implementations of
`SettableParameter` for each of the standard `java.sql.Types` values."
(:require [clojure.java.data :as j]
[next.jdbc.protocols :as p])
(:import (java.sql Connection
Expand Down
39 changes: 39 additions & 0 deletions src/next/jdbc/types.clj
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
;; copyright (c) 2018-2020 Sean Corfield, all rights reserved

(ns next.jdbc.types
"Provides convenience functions for wrapping values you pass into SQL
operations that have per-instance implementations of `SettableParameter`
so that `.setObject()` is called with the appropriate `java.sql.Types` value."
(:require [clojure.string :as str]
[next.jdbc.prepare :as prep])
(:import (java.lang.reflect Field Modifier)
(java.sql PreparedStatement)))

(set! *warn-on-reflection* true)

(defmacro ^:private all-types
[]
(let [names
(into []
(comp (filter #(Modifier/isStatic (.getModifiers ^Field %)))
(map #(.getName ^Field %)))
(.getDeclaredFields java.sql.Types))]
`(do
~@(for [n names]
(let [as-n (symbol (str "as-"
(-> n
(str/lower-case)
(str/replace "_" "-"))))]
`(defn ~as-n
~(str "Wrap a Clojure value in a vector with metadata to implement `set-parameter`
so that `.setObject()` is called with the `java.sql.Types/" n "` SQL type.")
[~'obj]
(with-meta [~'obj]
{'next.jdbc.prepare/set-parameter
(fn [[v#] ^PreparedStatement s# ^long i#]
(.setObject s# i# v# ~(symbol "java.sql.Types" n)))})))))))

(all-types)

(comment
(macroexpand '(all-types)))
13 changes: 10 additions & 3 deletions test/next/jdbc/sql_test.clj
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
[next.jdbc.sql :as sql]
[next.jdbc.test-fixtures
:refer [with-test-db ds column default-options
derby? jtds? maria? mssql? mysql? postgres? sqlite?]]))
derby? jtds? maria? mssql? mysql? postgres? sqlite?]]
[next.jdbc.types :refer [as-other as-real as-varchar]]))

(set! *warn-on-reflection* true)

Expand Down Expand Up @@ -79,8 +80,9 @@
:else :FRUIT/ID)]
(testing "single insert/delete"
(is (== 5 (new-key (sql/insert! (ds) :fruit
{:name "Kiwi" :appearance "green & fuzzy"
:cost 100 :grade 99.9}))))
{:name (as-varchar "Kiwi")
:appearance "green & fuzzy"
:cost 100 :grade (as-real 99.9)}))))
(is (= 5 (count (sql/query (ds) ["select * from fruit"]))))
(is (= {:next.jdbc/update-count 1}
(sql/delete! (ds) :fruit {:id 5})))
Expand Down Expand Up @@ -156,3 +158,8 @@
(when (postgres?)
(let [data (sql/find-by-keys (ds) :fruit ["id = any(?)" (int-array [1 2 3 4])])]
(is (= 4 (count data))))))

(deftest enum-pg
(when (postgres?)
(let [r (sql/insert! (ds) :lang_test {:lang (as-other "fr")})]
(is (= {:lang_test/lang "fr"} r)))))
12 changes: 12 additions & 0 deletions test/next/jdbc/test_fixtures.clj
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,18 @@
(try
(do-commands con [(str "DROP TABLE " fruit)])
(catch Exception _))
(when (postgres?)
(try
(do-commands con ["DROP TABLE LANG_TEST"])
(catch Exception _))
(try
(do-commands con ["DROP TYPE LANGUAGE"])
(catch Exception _))
(do-commands con ["CREATE TYPE LANGUAGE AS ENUM('en','fr','de')"])
(do-commands con ["
CREATE TABLE LANG_TEST (
LANG LANGUAGE NOT NULL
)"]))
(do-commands con [(str "
CREATE TABLE " fruit " (
ID INTEGER " auto-inc-pk ",
Expand Down
14 changes: 14 additions & 0 deletions test/next/jdbc/types_test.clj
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
;; copyright (c) 2020 Sean Corfield, all rights reserved

(ns next.jdbc.types-test
"Some tests for the type-assist functions."
(:require [clojure.test :refer [deftest is testing]]
[next.jdbc.types :refer [as-varchar]]))

(set! *warn-on-reflection* true)

(deftest as-varchar-test
(let [v (as-varchar "Hello")]
(is (= ["Hello"] v))
(is (contains? (meta v) 'next.jdbc.prepare/set-parameter))
(is (fn? (get (meta v) 'next.jdbc.prepare/set-parameter)))))

0 comments on commit 0379230

Please sign in to comment.