From 1dc08f38a8ee3a94f72350ac8db5ae8130ce434b Mon Sep 17 00:00:00 2001 From: Doug Martin Date: Thu, 5 Mar 2015 18:07:06 -0600 Subject: [PATCH] Update README --- README.md | 488 +++++++++++++++++++++++++++++++++++++++- adapters.go | 10 +- exec.go => crud_exec.go | 21 +- database.go | 40 ++-- dataset.go | 3 +- dataset_actions.go | 23 +- dataset_delete.go | 1 + dataset_insert.go | 1 + dataset_update.go | 1 + goqu.go | 273 +++++++++++++++++++++- 10 files changed, 813 insertions(+), 48 deletions(-) rename exec.go => crud_exec.go (92%) diff --git a/README.md b/README.md index 76b4e536..5ba8d066 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,489 @@ +``` + __ _ ___ __ _ _ _ + / _` |/ _ \ / _` | | | | +| (_| | (_) | (_| | |_| | + \__, |\___/ \__, |\__,_| + |___/ |_| +``` [![wercker status](https://app.wercker.com/status/7eec67205c1ce1cc96ef81664f21256b/s "wercker status")](https://app.wercker.com/project/bykey/7eec67205c1ce1cc96ef81664f21256b) [![GoDoc](https://godoc.org/github.com/doug-martin/goqu?status.png)](http://godoc.org/github.com/doug-martin/goqu) -[![GoCover](http://gocover.io/_badge/github.com/doug-martin/goqu)](http://gocover.io/github.com/doug-martin/goqu) \ No newline at end of file +[![GoCover](http://gocover.io/_badge/github.com/doug-martin/goqu)](http://gocover.io/github.com/doug-martin/goqu) + +`goqu` is an expressive SQL builder + +This library was built with the following goals: + +* Make the generation of SQL easy and enjoyable +* Provide a DSL that accounts for the common SQL expressions, NOT every nuance for each database. +* Allow users to use SQL when desired +* Provide a simple query API for scanning rows +* Allow the user to use the native sql.Db methods when desired + +## Features + +`goqu` comes with many features but here are a few of the more notable ones + +* Query Builder +* Parameter interpolation (e.g `SELECT * FROM "items" WHERE "id" = ?` -> `SELECT * FROM "items" WHERE "id" = 1`) +* Built from the ground up with adapters in mind +* Insert, Multi Insert, Update, and Delete support +* Scanning of rows to struct[s] or primitive value[s] + +While goqu may support the scanning of rows into structs it is not intended to be used as an ORM if you are looking for common ORM features like associations, +or hooks I would recommend looking at some of the great ORM libraries such as: + +* https://github.com/jinzhu/gorm +* https://github.com/eaigner/hood + +## Basics + +In order to start using goqu with your database you need to load an adapter. We have included some adapters by default. + +1. Postgres - github.com/doug-martin/goqu/adapters/postgres +2. MySQL - github.com/doug-martin/goqu/adapters/mysql + +Adapters in goqu work the same way as a driver with the database in that they register themselves with goqu once loaded. + +```go +import ( + "database/sql" + "github.com/doug-martin/goqu" + _ "github.com/doug-martin/goqu/adapters/postgres" + _ "github.com/lib/pq" +) +``` +Notice that we imported the adapter and driver for side effect only. + +Once you have your adapter and driver loaded you can create a goqu.Database instance + +```go +pgDb, err := sql.Open("postgres", "user=postgres dbname=goqupostgres sslmode=disable ") +if err != nil { + panic(err.Error()) +} +db := goqu.New("postgres", pgDb) +``` +Once you have your goqu.Database you can build your SQL and it will be formatted appropriately for the provided dialect. + +```go +sql, _ := db.From("user").Where(goqu.I("password").IsNotNull()).Sql() +fmt.Println(sql) + +sql, args, _ := db.From("user").Where(goqu.I("id").Eq(10)).ToSql(true) +fmt.Println(sql) +``` +Output +```sql +SELECT * FROM "user" WHERE "password" IS NOT NULL +SELECT * FROM "user" WHERE "id" = $1 +``` + +## Expressions + +goqu provides a DSL for generating the SQL however the Dataset only provides the the different clause methods (e.g. Where, From, Select), most of these clause methods accept Expressions(with a few exceptions) which are the building blocks for your SQL statement, you can think of them as fragments of SQL. + +The entry points for expressions are: + +* [`I()`](https://godoc.org/github.com/doug-martin/goqu#example-I) - An Identifier represents a schema, table, or column or any combination +```go +goqu.I("my_schema.table.col") +goqu.I("table.col") +goqu.I("col") +``` +If you look at the IdentiferExpression it implements many of your common sql operations that you would perform. +```go + goqu.I("col").Eq(10) + goqu.I("col").In([]int64{1,2,3,4}) + goqu.I("col").Like(regexp.MustCompile("^(a|b)") + goqu.I("col").IsNull() +``` +Please see the exmaples for [`I()`](https://godoc.org/github.com/doug-martin/goqu#example-I) to see more in depth examples + +* [`L()`](https://godoc.org/github.com/doug-martin/goqu#example-L) - An SQL literal. You may find yourself in a situation where an IdentifierExpression cannot expression an SQL fragment that your database supports. In that case you can use a LiteralExpression +```go +goqu.L(`"col"::TEXT = ""other_col"::text`) +``` +You can also use placeholders in your literal. When using the LiteralExpressions placeholders are normalized to the ? character and will be transformed to the correct placeholder for your adapter (e.g. ? - $1 in postgres) +```go +goqu.L("col IN (?, ?, ?)", "a", "b", "c") +``` +Putting it together +```go + db.From("test").Where( + goqu.I("col").Eq(10), + goqu.L(`"json"::TEXT = "other_json"::TEXT`), + ) +``` +```sql +SELECT * FROM "test" WHERE (("col" = 10) AND "json"::TEXT = "other_json"::TEXT) +``` +Both the Identifier and Literal expressions will be ANDed together by default. +You may however want to have your expressions ORed together you can use the [`Or()`](https://godoc.org/github.com/doug-martin/goqu#example-Or) function to create an ExpressionList +```go + db.From("test").Where( + Or( + goqu.I("col").Eq(10), + goqu.L(`"col"::TEXT = "other_col"::TEXT`), + ), + ) +``` +```sql +SELECT * FROM "test" WHERE (("col" = 10) OR "col"::TEXT = "other_col"::TEXT) +``` +You can also use Or and the And function in tandem which will give you control not only over how the Expressions are joined together, but also how they are grouped +```go + db.From("test").Where( + Or( + I("a").Gt(10), + And( + I("b").Eq(100), + I("c").Neq("test"), + ), + ), + ) +``` +```sql +SELECT * FROM "test" WHERE (("a" > 10) OR (("b" = 100) AND ("c" != 'test'))) +``` + +### Complex Example + +```go +db.From("test"). + Select(goqu.COUNT("*")). + InnerJoin(goqu.I("test2"), goqu.On(goqu.I("test.fkey").Eq(goqu.I("test2.id")))). + LeftJoin(goqu.I("test3"), goqu.On(goqu.I("test2.fkey").Eq(goqu.I("test3.id")))). + Where( + goqu.I("test.name").Like(regexp.MustCompile("^(a|b)")), + goqu.I("test2.amount").IsNotNull(), + goqu.Or( + goqu.I("test3.id").IsNull(), + goqu.I("test3.status").In("passed", "active", "registered"), + ), + ). + Order(goqu.I("test.created").Desc().NullsLast()). + GroupBy(goqu.I("test.user_id")). + Having(goqu.AVG("test3.age").Gt(10)) +``` + +Would generate the following SQL + +```sql +SELECT COUNT(*) +FROM "test" + INNER JOIN "test2" ON ("test"."fkey" = "test2"."id") + LEFT JOIN "test3" ON ("test2"."fkey" = "test3"."id") +WHERE ( + ("test"."name" ~ '^(a|b)') AND + ("test2"."amount" IS NOT NULL) AND + ( + ("test3"."id" IS NULL) OR + ("test3"."status" IN ('passed', 'active', 'registered')) + ) +) +GROUP BY "test"."user_id" +HAVING (AVG("test3"."age") > 10) +ORDER BY "test"."created" DESC NULLS LAST +``` + +### Querying + +goqu also has basic query support through the use of either the Database or the Dataset. + +### Dataset + +* [`ScanStructs`](http://godoc.org/github.com/doug-martin/goqu#Dataset.ScanStructs) - scans rows into a slice of structs +```go +var users []User +if err := db.From("user").ScanStructs(&users){ + fmt.Println(err.Error()) + return +} +fmt.Printf("\n%+v", users) +``` + +* [`ScanStruct`](http://godoc.org/github.com/doug-martin/goqu#Dataset.ScanStruct) - scans a row into a slice a struct, returns false if a row wasnt found +```go +var user User +found, err := db.From("user").ScanStruct(&user) +if err != nil{ + fmt.Println(err.Error()) + return +} +if !found{ + fmt.Println("No user found") +}else{ + fmt.Printf("\nFound user: %+v", user) +} +``` + +* [`ScanVals`](http://godoc.org/github.com/doug-martin/goqu#Dataset.ScanVals) - scans a rows of 1 column into a slice of primitive values +```go +var ids []int64 +if err := db.From("user").Select("id").ScanVals(&ids){ + fmt.Println(err.Error()) + return +} +fmt.Printf("\n%+v", ids) +``` + +* [`ScanVal`](http://godoc.org/github.com/doug-martin/goqu#Dataset.ScanVal) - scans a row of 1 column into a primitive value, returns false if a row wasnt found. **Note** when using the dataset a `LIMIT` of 1 is automatically applied. +```go +var id int64 +found, err := db.From("user").Select("id").ScanVal(&id) +if err != nil{ + fmt.Println(err.Error()) + return +} +if !found{ + fmt.Println("No id found") +}else{ + fmt.Printf("\nFound id: %d", id) +} +``` + +* [`Count`](http://godoc.org/github.com/doug-martin/goqu#Dataset.Count) - Returns the count for the current query +```go +count, err := db.From("user").Count() +if err != nil{ + fmt.Println(err.Error()) + return +} +fmt.Printf("\nCount:= %d", count) +``` + +* [`Pluck`](http://godoc.org/github.com/doug-martin/goqu#Dataset.Pluck) - Selects a single column and stores the results into a slice of primitive values +```go +var ids []int64 +if err := db.From("user").Pluck(&ids, "id"); err != nil{ + fmt.Println(err.Error()) + return +} +fmt.Printf("\nIds := %+v", ids) +``` + +* [`Insert`](http://godoc.org/github.com/doug-martin/goqu#Dataset.Insert) - Creates an `INSERT` statement and returns a [`CrudExec`](http://godoc.org/github.com/doug-martin/goqu#CrudExec) to execute the statement +```go +insert := db.From("user").Insert(goqu.Record{"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}) +if _, err := insert.Exec(); err != nil{ + fmt.Println(err.Error()) + return +} +``` +Insert will also handle multi inserts if supported by the database +```go +users := []goqu.Record{ + {"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Sally", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Jimmy", "last_name":"Yukon", "created": time.Now()}, +} +if _, err := db.From("user").Insert(users).Exec(); err != nil{ + fmt.Println(err.Error()) + return +} +``` +If your database supports the `RETURN` clause you can also use the different Scan methods to get results +```go +var ids []int64 +users := []goqu.Record{ + {"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Sally", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Jimmy", "last_name":"Yukon", "created": time.Now()}, +} +if err := db.From("user").Returning(goqu.I("id")).Insert(users).ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return +} +``` + +* [`Update`](http://godoc.org/github.com/doug-martin/goqu#Dataset.Update) - Creates an `UPDATE` statement and returns an[`CrudExec`](http://godoc.org/github.com/doug-martin/goqu#CrudExec) to execute the statement +```go +update := db.From("user"). + Where(goqu.I("status").Eq("inactive")). + Update(goqu.Record{"password": nil, "updated": time.Now()}) +if _, err := update.Exec(); err != nil{ + fmt.Println(err.Error()) + return +} +`````` +If your database supports the `RETURN` clause you can also use the different Scan methods to get results +```go +var ids []int64 +update := db.From("user"). + Where(goqu.I("status").Eq("inactive")). + Returning(goqu.I("id")). + Update(goqu.Record{"password": nil, "updated": time.Now()}) +if err := update.ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return +} +``` +* [`Delete`](http://godoc.org/github.com/doug-martin/goqu#Dataset.Delete) - Creates an `DELETE` statement and returns a [`CrudExec`](http://godoc.org/github.com/doug-martin/goqu#CrudExec) to execute the statement +```go +delete := db.From("invoice"). + Where(goqu.I("status").Eq("paid")). + Delete() +if _, err := delete.Exec(); err != nil{ + fmt.Println(err.Error()) + return +} +`````` +If your database supports the `RETURN` clause you can also use the different Scan methods to get results +```go +var ids []int64 +delete := db.From("invoice"). + Where(goqu.I("status").Eq("paid")). + Returning(goqu.I("id")). + Delete() +if err := delete.ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return +} +``` + +### Database + +The Database also allows you to execute queries but expects raw SQL to execute. The supported methods are + +* [`Exec`](http://godoc.org/github.com/doug-martin/goqu#Database.Exec) +* [`Prepare`](http://godoc.org/github.com/doug-martin/goqu#Database.Prepare) +* [`Query`](http://godoc.org/github.com/doug-martin/goqu#Database.Query) +* [`QueryRow`](http://godoc.org/github.com/doug-martin/goqu#Database.QueryRow) +* [`ScanStructs`](http://godoc.org/github.com/doug-martin/goqu#Database.ScanStructs) +* [`ScanStruct`](http://godoc.org/github.com/doug-martin/goqu#Database.ScanStruct) +* [`ScanVals`](http://godoc.org/github.com/doug-martin/goqu#Database.ScanVals) +* [`ScanVal`](http://godoc.org/github.com/doug-martin/goqu#Database.ScanVal) +* [`Begin`](http://godoc.org/github.com/doug-martin/goqu#Database.Begin) + +## Transactions + +`goqu` has builtin support for transactions to make the use of the Datasets and querying seamless + +```go +tx, err := db.Begin() +if err != nil{ + return err +} +//use tx.From to get a dataset that will execute within this transaction +update := tx.From("user").Where(goqu.I("password").IsNull()).Update(goqu.Record{"status": "inactive"}) +if _, err = update.Exec(); err != nil{ + if rErr := tx.Rollback(); rErr != nil{ + return rErr + } + return err +} +if err = tx.Commit(); err != nil{ + return err +} +return +``` + +The [`TxDatabase`](http://godoc.org/github.com/doug-martin/goqu/#TxDatabase) also has all methods that the [`Database`](http://godoc.org/github.com/doug-martin/goqu/#Database) has along with + +* [`Commit`](http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Commit) +* [`Rollback`](http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Rollback) +* [`Wrap`](http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Wrap) + +### Wrap + +The [`TxDatabase.Wrap`](http://godoc.org/github.com/doug-martin/goqu/#TxDatabase.Wrap) is a convience method for automatically handling `COMMIT` and `ROLLBACK` + +```go +tx, err := db.Begin() +if err != nil{ + return err +} +err = tx.Wrap(func() error{ + update := tx.From("user").Where(goqu.I("password").IsNull()).Update(goqu.Record{"status": "inactive"}) + if _, err = update.Exec(); err != nil{ + return err + } + return nil +}) +//err will be the original error from the update statement, unless there was an error executing ROLLBACK +if err != nil{ + return err +} +``` + +## Logging + +To enable trace logging of SQL statments use the [`Database.Logger`](http://godoc.org/github.com/doug-martin/goqu/#Database.Logger) method to set your logger. + +**NOTE** The logger must implement the [`Logger`](http://godoc.org/github.com/doug-martin/goqu/#Logger) interface + +**NOTE** If you start a transaction using a database your set a logger on the transaction will inherit that logger automatically + + +## Adapters + +Adapters in goqu are the foundation of building the correct SQL for each DB dialect. + +When creating your adapters you must register your adapter with [`RegisterAdapter`](http://godoc.org/github.com/doug-martin/goqu/#RegisterAdapter). This method requires 2 arguments. + +1. Dialect - The dialect for your adapter. +2. DatasetAdapterFactory - This is a factory function that will return a new goqu.Adapter used to create the the dialect specific SQL. + +Between most dialects there is a large portion of shared syntax, for this reason we have a DefaultAdapter that can be used as a base for any new Dialect specific adapter. +In fact for most use cases you will not have to override any methods but instead just override the default values as documented for DefaultAdapter. + +For example the code for the postgres adapter is fairly short. +```go +package postgres + +import ( + "github.com/doug-martin/goqu" +) + +//postgres requires a $ placeholder for prepared statements +const placeholder_rune = '$' + +func newDatasetAdapter(ds *goqu.Dataset) goqu.Adapter { + ret := goqu.NewDefaultAdapter(ds).(*goqu.DefaultAdapter) + + //override the settings required + ret.PlaceHolderRune = placeholder_rune + //postgres requires a paceholder number (e.g. $1) + ret.IncludePlaceholderNum = true + return ret +} + +func init() { + //register our adapter with goqu + goqu.RegisterAdapter("postgres", newDatasetAdapter) +} +``` + +If you are looking to write your own adapter take a look at the postgres or mysql adapter located at https://github.com/doug-martin/goqu/tree/master/adapters. + +## Contributions + +I am always welcoming contributions of any type. Please open an issue or create a PR if you find an issue with any of the following. + +* An issue with Documentation +* You found the documentation lacking in some way + +If you have an issue with the package please include the following + +* The dialect you are using +* A description of the problem +* A short example of how to reproduce (if applicable) + +Without those basics it can be difficult to reproduce your issue locally. You may be asked for more information but that is a good starting point. + +### New Features + +New features and/or enhancements are great and I encourage you to either submit a PR or create an issue. In both cases include the following as the need/requirement may not be readily apparent. + +1. The use case +2. A short example + +If you are issuing a PR also also include the following + +1. Tests - otherwise the PR will not be merged +2. Documentation - otherwise the PR will not be merged +3. Examples - [If applicable] see example_test.go for examples + +If you find an issue you want to work on please comment on it letting other people know you are looking at it and I will assign the issue to you. + +If want to work on an issue but dont know where to start just leave a comment and I'll be more than happy to point you in the right direction. diff --git a/adapters.go b/adapters.go index 4526a506..87270c3f 100644 --- a/adapters.go +++ b/adapters.go @@ -193,8 +193,8 @@ var ds_adapters = make(map[string]func(dataset *Dataset) Adapter) //Registers an adapter. // -//dialect: The dialect this adapter is for -//factory: a function that can be called to create a new Adapter for the dialect. +// dialect: The dialect this adapter is for +// factory: a function that can be called to create a new Adapter for the dialect. func RegisterAdapter(dialect string, factory func(ds *Dataset) Adapter) { dialect = strings.ToLower(dialect) ds_adapters[dialect] = factory @@ -202,7 +202,7 @@ func RegisterAdapter(dialect string, factory func(ds *Dataset) Adapter) { //Returns true if the dialect has an adapter registered // -//dialect: The dialect to test +// dialect: The dialect to test func HasAdapter(dialect string) bool { dialect = strings.ToLower(dialect) _, ok := ds_adapters[dialect] @@ -219,8 +219,8 @@ func removeAdapter(dialect string) { //Creates the appropriate adapter for the given dialect. // -//dialect: the dialect to create an adapter for -//dataset: The dataset to be used by the adapter +// dialect: the dialect to create an adapter for +// dataset: The dataset to be used by the adapter func NewAdapter(dialect string, dataset *Dataset) Adapter { dialect = strings.ToLower(dialect) if adapterGen, ok := ds_adapters[dialect]; ok { diff --git a/exec.go b/crud_exec.go similarity index 92% rename from exec.go rename to crud_exec.go index edf8a924..f474ecb9 100644 --- a/exec.go +++ b/crud_exec.go @@ -15,7 +15,7 @@ type ( GoType reflect.Type } columnMap map[string]columnData - Exec struct { + CrudExec struct { database database Sql string Args []interface{} @@ -26,11 +26,11 @@ type ( var struct_map_cache = make(map[interface{}]columnMap) -func newExec(database database, err error, sql string, args ...interface{}) *Exec { - return &Exec{database: database, err: err, Sql: sql, Args: args} +func newCrudExec(database database, err error, sql string, args ...interface{}) *CrudExec { + return &CrudExec{database: database, err: err, Sql: sql, Args: args} } -func (me Exec) Exec() (sql.Result, error) { +func (me CrudExec) Exec() (sql.Result, error) { if me.err != nil { return nil, me.err } @@ -44,8 +44,9 @@ func (me Exec) Exec() (sql.Result, error) { // } // //use your structs // +// //i: A pointer to a slice of structs. -func (me Exec) ScanStructs(i interface{}) error { +func (me CrudExec) ScanStructs(i interface{}) error { if me.err != nil { return me.err } @@ -71,7 +72,7 @@ func (me Exec) ScanStructs(i interface{}) error { // } // //i: A pointer to a struct -func (me Exec) ScanStruct(i interface{}) (bool, error) { +func (me CrudExec) ScanStruct(i interface{}) (bool, error) { if me.err != nil { return false, me.err } @@ -92,7 +93,7 @@ func (me Exec) ScanStruct(i interface{}) (bool, error) { // } // //i: Takes a pointer to a slice of primitive values. -func (me Exec) ScanVals(i interface{}) error { +func (me CrudExec) ScanVals(i interface{}) error { if me.err != nil { return me.err } @@ -138,8 +139,8 @@ func (me Exec) ScanVals(i interface{}) error { // fmt.Println("NOT FOUND") // } // -//i: Takes a pointer to a primitive value. -func (me Exec) ScanVal(i interface{}) (bool, error) { +// i: Takes a pointer to a primitive value. +func (me CrudExec) ScanVal(i interface{}) (bool, error) { if me.err != nil { return false, me.err } @@ -165,7 +166,7 @@ func (me Exec) ScanVal(i interface{}) (bool, error) { return count != 0, nil } -func (me Exec) scan(i interface{}, query string, args ...interface{}) (bool, error) { +func (me CrudExec) scan(i interface{}, query string, args ...interface{}) (bool, error) { var ( found bool results []Record diff --git a/database.go b/database.go index 88491290..262b90de 100644 --- a/database.go +++ b/database.go @@ -27,6 +27,7 @@ type ( //This is the common entry point into goqu. // //dialect: This is the adapter dialect, you should see your database adapter for the string to use. Built in adpaters can be found at https://github.com/doug-martin/goqu/tree/master/adapters +// //db: A sql.Db to use for querying the database // import ( // "database/sql" @@ -102,6 +103,7 @@ func (me *Database) Trace(op, sql string, args ...interface{}) { //Uses the db to Execute the query with arguments and return the sql.Result // //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) Exec(query string, args ...interface{}) (sql.Result, error) { me.Trace("EXEC", query, args...) @@ -138,7 +140,7 @@ func (me *Database) Prepare(query string) (*sql.Stmt, error) { return me.Db.Prepare(query) } -//Can be used to prepare a query. +//Used to query for multiple rows. // //You can use this in tandem with a dataset by doing the following. // sql, err := db.From("items").Where(goqu.I("id").Gt(10)).Sql() @@ -158,13 +160,14 @@ func (me *Database) Prepare(query string) (*sql.Stmt, error) { // } // //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) Query(query string, args ...interface{}) (*sql.Rows, error) { me.Trace("QUERY", query, args...) return me.Db.Query(query, args...) } -//Can be used to prepare a query. +//Used to query for a single row. // //You can use this in tandem with a dataset by doing the following. // sql, err := db.From("items").Where(goqu.I("id").Gt(10)).Limit(1).Sql() @@ -178,49 +181,58 @@ func (me *Database) Query(query string, args ...interface{}) (*sql.Rows, error) // //scan your row // //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) QueryRow(query string, args ...interface{}) *sql.Row { me.Trace("QUERY ROW", query, args...) return me.Db.QueryRow(query, args...) } -//Queries the database using the supplied query, and args and uses Exec#ScanStructs to scan the results into a slice of structs +//Queries the database using the supplied query, and args and uses CrudExec.ScanStructs to scan the results into a slice of structs // //i: A pointer to a slice of structs +// //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) ScanStructs(i interface{}, query string, args ...interface{}) error { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanStructs(i) } -//Queries the database using the supplied query, and args and uses Exec#ScanStruct to scan the results into a struct +//Queries the database using the supplied query, and args and uses CrudExec.ScanStruct to scan the results into a struct // //i: A pointer to a struct +// //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) ScanStruct(i interface{}, query string, args ...interface{}) (bool, error) { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanStruct(i) } -//Queries the database using the supplied query, and args and uses Exec#ScanVals to scan the results into a slice of primitive values +//Queries the database using the supplied query, and args and uses CrudExec.ScanVals to scan the results into a slice of primitive values // //i: A pointer to a slice of primitive values +// //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) ScanVals(i interface{}, query string, args ...interface{}) error { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanVals(i) } -//Queries the database using the supplied query, and args and uses Exec#ScanVal to scan the results into a primitive value +//Queries the database using the supplied query, and args and uses CrudExec.ScanVal to scan the results into a primitive value // //i: A pointer to a primitive value +// //query: The SQL to execute +// //args...: for any placeholder parameters in the query func (me *Database) ScanVal(i interface{}, query string, args ...interface{}) (bool, error) { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanVal(i) } @@ -287,25 +299,25 @@ func (me *TxDatabase) QueryRow(query string, args ...interface{}) *sql.Row { //See Database#ScanStructs func (me *TxDatabase) ScanStructs(i interface{}, query string, args ...interface{}) error { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanStructs(i) } //See Database#ScanStruct func (me *TxDatabase) ScanStruct(i interface{}, query string, args ...interface{}) (bool, error) { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanStruct(i) } //See Database#ScanVals func (me *TxDatabase) ScanVals(i interface{}, query string, args ...interface{}) error { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanVals(i) } //See Database#ScanVal func (me *TxDatabase) ScanVal(i interface{}, query string, args ...interface{}) (bool, error) { - exec := newExec(me, nil, query, args...) + exec := newCrudExec(me, nil, query, args...) return exec.ScanVal(i) } diff --git a/dataset.go b/dataset.go index d2d0a854..a4059e56 100644 --- a/dataset.go +++ b/dataset.go @@ -66,7 +66,7 @@ type ( // * Count() - Returns a count of rows // * Pluck(i interface{}, col string) - Retrives a columns from rows and scans the resules into a slice of primitive values. // - //Update, Delete, and Insert return an Exec struct which can be used to scan values or just execute the statment. You might + //Update, Delete, and Insert return an CrudExec struct which can be used to scan values or just execute the statment. You might //use the scan methods if the database supports return values. For example // UPDATE "items" SET updated = NOW RETURNING "items".* //Could be executed with ScanStructs. @@ -161,6 +161,7 @@ func (me *Dataset) hasSources() bool { // * Expressions // //buf: The SqlBuilder to write the generated SQL to +// //val: The value to serialize // //Errors: diff --git a/dataset_actions.go b/dataset_actions.go index d948feb6..27e615b1 100644 --- a/dataset_actions.go +++ b/dataset_actions.go @@ -5,7 +5,7 @@ package goqu //i: A pointer to a slice of structs func (me *Dataset) ScanStructs(i interface{}) error { sql, err := me.Sql() - return newExec(me.database, err, sql).ScanStructs(i) + return newCrudExec(me.database, err, sql).ScanStructs(i) } //Generates the SELECT sql for this dataset and uses Exec#ScanStruct to scan the result into a slice of structs @@ -13,7 +13,7 @@ func (me *Dataset) ScanStructs(i interface{}) error { //i: A pointer to a structs func (me *Dataset) ScanStruct(i interface{}) (bool, error) { sql, err := me.Limit(1).Sql() - return newExec(me.database, err, sql).ScanStruct(i) + return newCrudExec(me.database, err, sql).ScanStruct(i) } //Generates the SELECT sql for this dataset and uses Exec#ScanVals to scan the results into a slice of primitive values @@ -21,15 +21,15 @@ func (me *Dataset) ScanStruct(i interface{}) (bool, error) { //i: A pointer to a slice of primitive values func (me *Dataset) ScanVals(i interface{}) error { sql, err := me.Sql() - return newExec(me.database, err, sql).ScanVals(i) + return newCrudExec(me.database, err, sql).ScanVals(i) } //Generates the SELECT sql for this dataset and uses Exec#ScanVal to scan the result into a primitive value // //i: A pointer to a primitive value func (me *Dataset) ScanVal(i interface{}) (bool, error) { - sql, err := me.Sql() - return newExec(me.database, err, sql).ScanVal(i) + sql, err := me.Limit(1).Sql() + return newCrudExec(me.database, err, sql).ScanVal(i) } //Generates the SELECT COUNT(*) sql for this dataset and uses Exec#ScanVal to scan the result into an int64. @@ -42,6 +42,7 @@ func (me *Dataset) Count() (int64, error) { //Generates the SELECT sql only selecting the passed in column and uses Exec#ScanVals to scan the result into a slice of primitive values. // //i: A slice of primitive values +// //col: The column to select when generative the SQL func (me *Dataset) Pluck(i interface{}, col string) error { return me.Select(col).ScanVals(i) @@ -51,23 +52,23 @@ func (me *Dataset) Pluck(i interface{}, col string) error { // db.From("test").Update(Record{"name":"Bob", update: time.Now()}).Exec() // //See Dataset#UpdateSql for arguments -func (me *Dataset) Update(i interface{}) *Exec { +func (me *Dataset) Update(i interface{}) *CrudExec { sql, err := me.UpdateSql(i) - return newExec(me.database, err, sql) + return newCrudExec(me.database, err, sql) } //Generates the UPDATE sql, and returns an Exec struct with the sql set to the INSERT statement // db.From("test").Insert(Record{"name":"Bob").Exec() // //See Dataset#InsertSql for arguments -func (me *Dataset) Insert(i ...interface{}) *Exec { +func (me *Dataset) Insert(i ...interface{}) *CrudExec { sql, err := me.InsertSql(i...) - return newExec(me.database, err, sql) + return newCrudExec(me.database, err, sql) } //Generates the DELETE sql, and returns an Exec struct with the sql set to the DELETE statement // db.From("test").Where(I("id").Gt(10)).Exec() -func (me *Dataset) Delete() *Exec { +func (me *Dataset) Delete() *CrudExec { sql, err := me.DeleteSql() - return newExec(me.database, err, sql) + return newCrudExec(me.database, err, sql) } diff --git a/dataset_delete.go b/dataset_delete.go index 0d6c18c6..2fe96cbc 100644 --- a/dataset_delete.go +++ b/dataset_delete.go @@ -88,6 +88,7 @@ func (me *Dataset) TruncateWithOptsSql(opts TruncateOptions) (string, error) { //Generates a TRUNCATE statement. // //isPrepared: Set to true to true to ensure values are NOT interpolated. See examples. +// //opts: Options to use when generating the TRUNCATE statement // //Errors: diff --git a/dataset_insert.go b/dataset_insert.go index 43be9e4c..5c7fba9f 100644 --- a/dataset_insert.go +++ b/dataset_insert.go @@ -33,6 +33,7 @@ func (me *Dataset) InsertSql(rows ...interface{}) (string, error) { // } // //isPrepared: Set to true to true to ensure values are NOT interpolated +// //rows: variable number arguments of either map[string]interface, Record, struct, or a single slice argument of the accepted types. // //Errors: diff --git a/dataset_update.go b/dataset_update.go index 3acb1693..838ebf8b 100644 --- a/dataset_update.go +++ b/dataset_update.go @@ -38,6 +38,7 @@ func (me *Dataset) canUpdateField(field reflect.StructField) bool { // } // //isPrepared: set to true to generate an sql statement with placeholders for primitive values +// //update: can either be a a map[string]interface{}, Record or a struct // //Errors: diff --git a/goqu.go b/goqu.go index 491f7909..d97adc59 100644 --- a/goqu.go +++ b/goqu.go @@ -1,5 +1,12 @@ /* -goqu is an expressive SQL builder +goqu is an expressive SQL builder. + + __ _ ___ __ _ _ _ + / _` |/ _ \ / _` | | | | + | (_| | (_) | (_| | |_| | + \__, |\___/ \__, |\__,_| + |___/ |_| + goqu was built with the following goals: @@ -11,7 +18,7 @@ goqu was built with the following goals: Features -goqu was comes with many features but here are a few of the more notable ones +goqu comes with many features but here are a few of the more notable ones 1. Query Builder 2. Parameter interpolation (e.g SELECT * FROM "items" WHERE "id" = ? -> SELECT * FROM "items" WHERE "id" = 1) @@ -20,7 +27,9 @@ goqu was comes with many features but here are a few of the more notable ones 5. Scanning of rows to struct[s] or primitive value[s] While goqu may support the scanning of rows into structs it is not intended to be used as an ORM if you are looking for common ORM features like associations, -or hooks I would recommend looking at some of the great ORM libraries such as https://github.com/jinzhu/gorm +or hooks I would recommend looking at some of the great ORM packages such as: + * https://github.com/jinzhu/gorm + * https://github.com/eaigner/hood Basics @@ -48,10 +57,10 @@ Once you have your adapter and driver loaded you can create a goqu.Database inst db := goqu.New("postgres", pgDb) Once you have your goqu.Database you can build your SQL and it will be formatted appropiately for the provided dialect. - sql, _ := db.From("user").Where(gq.I("password").IsNotNull()).Sql() + sql, _ := db.From("user").Where(goqu.I("password").IsNotNull()).Sql() fmt.Println(sql) //SELECT * FROM "user" WHERE "password" IS NOT NULL - sql, args, _ := db.From("user").Where(gq.I("id").Eq(10)).ToSql(true) + sql, args, _ := db.From("user").Where(goqu.I("id").Eq(10)).ToSql(true) fmt.Println(sql) //SELECT * FROM "user" WHERE "id" = $1 Expressions @@ -99,12 +108,239 @@ You can also use Or and the And function in tandem which will give you control n ), ) //SELECT * FROM "test" WHERE (("a" > 10) OR (("b" = 100) AND ("c" != 'test'))) +Complex Example + + db.From("test"). + Select(goqu.COUNT("*")). + InnerJoin(goqu.I("test2"), goqu.On(goqu.I("test.fkey").Eq(goqu.I("test2.id")))). + LeftJoin(goqu.I("test3"), goqu.On(goqu.I("test2.fkey").Eq(goqu.I("test3.id")))). + Where( + goqu.I("test.name").Like(regexp.MustCompile("(a|b)\\w+")), + goqu.I("test2.amount").IsNotNull(), + goqu.Or( + goqu.I("test3.id").IsNull(), + goqu.I("test3.status").In("passed", "active", "registered"), + ), + ). + Order(goqu.I("test.created").Desc().NullsLast()). + GroupBy(goqu.I("test.user_id")). + Having(goqu.AVG("test3.age").Gt(10)) + + //SELECT COUNT(*) + //FROM "test" + // INNER JOIN "test2" ON ("test"."fkey" = "test2"."id") + // LEFT JOIN "test3" ON ("test2"."fkey" = "test3"."id") + //WHERE ( + // ("test"."name" ~ '(a|b)\w+') AND + // ("test2"."amount" IS NOT NULL) AND + // ( + // ("test3"."id" IS NULL) OR + // ("test3"."status" IN ('passed', 'active', 'registered')) + // ) + //) + //GROUP BY "test"."user_id" + //HAVING (AVG("test3"."age") > 10) + //ORDER BY "test"."created" DESC NULLS LAST + +Querying + +goqu also has basic query support through the use of either the Database or the Dataset. + +Dataset + +* ScanStructs - scans rows into a slice of structs + var users []User + if err := db.From("user").ScanStructs(&users){ + fmt.Println(err.Error()) + return + } + fmt.Printf("\n%+v", users) + +* ScanStruct - scans a row into a slice a struct, returns false if a row wasnt found + var user User + found, err := db.From("user").ScanStruct(&user) + if err != nil{ + fmt.Println(err.Error()) + return + } + if !found{ + fmt.Println("No user found") + }else{ + fmt.Printf("\nFound user: %+v", user) + } + +* ScanVals - scans a rows of 1 column into a slice of primitive values + var ids []int64 + if err := db.From("user").Select("id").ScanVals(&ids){ + fmt.Println(err.Error()) + return + } + fmt.Printf("\n%+v", ids) + +* ScanVal - scans a row of 1 column into a primitive value, returns false if a row wasnt found. **Note** when using the dataset a `LIMIT` of 1 is automatically applied. + var id int64 + found, err := db.From("user").Select("id").ScanVal(&id) + if err != nil{ + fmt.Println(err.Error()) + return + } + if !found{ + fmt.Println("No id found") + }else{ + fmt.Printf("\nFound id: %d", id) + } + +* Count - Returns the count for the current query + count, err := db.From("user").Count() + if err != nil{ + fmt.Println(err.Error()) + return + } + fmt.Printf("\nCount:= %d", count) + +* Pluck - Selects a single column and stores the results into a slice of primitive values + var ids []int64 + if err := db.From("user").Pluck(&ids, "id"); err != nil{ + fmt.Println(err.Error()) + return + } + fmt.Printf("\nIds := %+v", ids) + +* Insert - Creates an `INSERT` statement and returns a CrudExec to execute the statement + insert := db.From("user").Insert(goqu.Record{"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}) + if _, err := insert.Exec(); err != nil{ + fmt.Println(err.Error()) + return + } +Insert will also handle multi inserts if supported by the database + users := []goqu.Record{ + {"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Sally", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Jimmy", "last_name":"Yukon", "created": time.Now()}, + } + if _, err := db.From("user").Insert(users).Exec(); err != nil{ + fmt.Println(err.Error()) + return + } +If your database supports the `RETURN` clause you can also use the different Scan methods to get results + var ids []int64 + users := []goqu.Record{ + {"first_name": "Bob", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Sally", "last_name":"Yukon", "created": time.Now()}, + {"first_name": "Jimmy", "last_name":"Yukon", "created": time.Now()}, + } + if err := db.From("user").Returning(goqu.I("id")).Insert(users).ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return + } + +* Update - Creates an `UPDATE` statement and returns a CrudExec to execute the statement + update := db.From("user"). + Where(goqu.I("status").Eq("inactive")). + Update(goqu.Record{"password": nil, "updated": time.Now()}) + if _, err := update.Exec(); err != nil{ + fmt.Println(err.Error()) + return + } +If your database supports the `RETURN` clause you can also use the different Scan methods to get results + var ids []int64 + update := db.From("user"). + Where(goqu.I("status").Eq("inactive")). + Returning(goqu.I("id")). + Update(goqu.Record{"password": nil, "updated": time.Now()}) + if err := update.ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return + } + +* Delete - Creates an `DELETE` statement and returns a CrudExec to execute the statement + delete := db.From("invoice"). + Where(goqu.I("status").Eq("paid")). + Delete() + if _, err := delete.Exec(); err != nil{ + fmt.Println(err.Error()) + return + } +If your database supports the `RETURN` clause you can also use the different Scan methods to get results + var ids []int64 + delete := db.From("invoice"). + Where(goqu.I("status").Eq("paid")). + Returning(goqu.I("id")). + Delete() + if err := delete.ScanVals(&ids); err != nil{ + fmt.Println(err.Error()) + return + } + +Database + +The Database also allows you to execute queries but expects raw SQL to execute. The supported methods are + * Exec - http://godoc.org/github.com/doug-martin/goqu#Database.Exec + * Prepare - http://godoc.org/github.com/doug-martin/goqu#Database.Prepare + * Query - http://godoc.org/github.com/doug-martin/goqu#Database.Query + * QueryRow - http://godoc.org/github.com/doug-martin/goqu#Database.QueryRow + * ScanStructs - http://godoc.org/github.com/doug-martin/goqu#Database.ScanStructs + * ScanStruct - http://godoc.org/github.com/doug-martin/goqu#Database.ScanStruct + * ScanVals - http://godoc.org/github.com/doug-martin/goqu#Database.ScanVals + * ScanVal - http://godoc.org/github.com/doug-martin/goqu#Database.ScanVal + * Begin - http://godoc.org/github.com/doug-martin/goqu#Database.Begin + + +Transactions + +goqu has builtin support for transactions to make the use of the Datasets and querying seamless + tx, err := db.Begin() + if err != nil{ + return err + } + //use tx.From to get a dataset that will execute within this transaction + update := tx.From("user").Where(goqu.I("password").IsNull()).Update(goqu.Record{"status": "inactive"}) + if _, err = update.Exec(); err != nil{ + if rErr := tx.Rollback(); rErr != nil{ + return rErr + } + return err + } + if err = tx.Commit(); err != nil{ + return err + } + return + +The TxDatabase also has all methods that the Database has along with + * Commit - http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Commit + * Rollback - http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Rollback + * Wrap - http://godoc.org/github.com/doug-martin/goqu#TxDatabase.Wrap + +Wrap + +The TxDatabase.Wrap is a convience method for automatically handling `COMMIT` and `ROLLBACK` + tx, err := db.Begin() + if err != nil{ + return err + } + err = tx.Wrap(func() error{ + update := tx.From("user").Where(goqu.I("password").IsNull()).Update(goqu.Record{"status": "inactive"}) + if _, err = update.Exec(); err != nil{ + return err + } + return nil + }) + //err will be the original error from the update statement, unless there was an error executing ROLLBACK + if err != nil{ + return err + } + +Logging + +To enable trace logging of SQL statments use the Database.Logger method to set your logger. + NOTE The logger must implement the [`Logger`](http://godoc.org/github.com/doug-martin/goqu/#Logger) interface + NOTE If you start a transaction using a database your set a logger on the transaction will inherit that logger automatically + Adapters Adapters in goqu are the foundation of building the correct SQL for each DB dialect. When creating your adapters you must register your adapter with goqu.RegisterAdapter. This method requires 2 arguments. - 1. Dialect - The dialect for your adapter. 2. DatasetAdapterFactory - This is a factory function that will return a new goqu.Adapter used to create the the dialect specific SQL. @@ -139,5 +375,30 @@ For example the code for the postgres adapter is fairly short. } If you are looking to write your own adapter take a look at the postgres or mysql adapter located at https://github.com/doug-martin/goqu/tree/master/adapters. + +Contributions + +I am always welcoming contributions of any type. Please open an issue or create a PR if you find an issue with any of the following. + * An issue with Documentation + * You found the documentation lacking in some way +If you have an issue with the package please include the following + * The dialect you are using + * A description of the problem + * A short example of how to reproduce (if applicable) +Without those basics it can be difficult to reproduce your issue locally. You may be asked for more information but that is a good starting point. + +New Features + +New features and/or enhancements are great and I encourage you to either submit a PR or create an issue. In both cases include the following as the need/requirement may not be readily apparent. + 1. The use case + 2. A short example +If you are issuing a PR also also include the following + 1. Tests - otherwise the PR will not be merged + 2. Documentation - otherwise the PR will not be merged + 3. Examples - [If applicable] see example_test.go for examples +If you find an issue you want to work on please comment on it letting other people know you are looking at it and I will assign the issue to you. + +If want to work on an issue but dont know where to start just leave a comment and I'll be more than happy to point you in the right direction. + */ package goqu