SNOW-1337069 latest missing pool notes + refactoring the docs

snowflakedb · May 13, 2024 · 15dae8f · 15dae8f
1 parent df87816
commit 15dae8f
Show file tree

Hide file tree

Showing 11 changed files with 992 additions and 960 deletions.
diff --git a/README.md b/README.md
diff --git a/doc/CodeCoverage.md b/doc/CodeCoverage.md
@@ -0,0 +1,72 @@
+## Getting the code coverage
+
+1. Go to .NET project directory
+
+2. Clean the directory
+
+```
+dotnet clean snowflake-connector-net.sln && dotnet nuget locals all --clear
+```
+
+3. Create parameters.json containing connection info for AWS, AZURE, or GCP account and place inside the Snowflake.Data.Tests folder
+
+4. Build the project for .NET6
+
+```
+dotnet build snowflake-connector-net.sln /p:DebugType=Full
+```
+
+5. Run dotnet-cover on the .NET6 build
+
+```
+dotnet-coverage collect "dotnet test --framework net6.0 --no-build -l console;verbosity=normal" --output net6.0_AWS_coverage.xml --output-format cobertura --settings coverage.config
+```
+
+6. Build the project for .NET Framework
+
+```
+msbuild snowflake-connector-net.sln -p:Configuration=Release
+```
+
+7. Run dotnet-cover on the .NET Framework build
+
+```
+dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;verbosity=normal" --output net472_AWS_coverage.xml --output-format cobertura --settings coverage.config
+```
+
+<br />
+Repeat steps 3, 5, and 7 for the other cloud providers. <br />
+Note: no need to rebuild the connector again. <br /><br />
+
+For Azure:<br />
+
+3. Create parameters.json containing connection info for AZURE account and place inside the Snowflake.Data.Tests folder
+
+4. Run dotnet-cover on the .NET6 build
+
+```
+dotnet-coverage collect "dotnet test --framework net6.0 --no-build -l console;verbosity=normal" --output net6.0_AZURE_coverage.xml --output-format cobertura --settings coverage.config
+```
+
+7. Run dotnet-cover on the .NET Framework build
+
+```
+dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;verbosity=normal" --output net472_AZURE_coverage.xml --output-format cobertura --settings coverage.config
+```
+
+<br />
+For GCP:<br />
+
+3. Create parameters.json containing connection info for GCP account and place inside the Snowflake.Data.Tests folder
+
+4. Run dotnet-cover on the .NET6 build
+
+```
+dotnet-coverage collect "dotnet test --framework net6.0 --no-build -l console;verbosity=normal" --output net6.0_GCP_coverage.xml --output-format cobertura --settings coverage.config
+```
+
+7. Run dotnet-cover on the .NET Framework build
+
+```
+dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;verbosity=normal" --output net472_GCP_coverage.xml --output-format cobertura --settings coverage.config
+```
diff --git a/doc/Connecting.md b/doc/Connecting.md
diff --git a/FeaturePooling.md → doc/ConnectionPooling.md b/FeaturePooling.md → doc/ConnectionPooling.md
@@ -110,14 +110,57 @@ Assert.AreEqual(2, poolSize);
 
 #### Changed Session Behavior
 
-When an application does a change to the connection using one of SQL commands: `use schema`, `use database`, `use warehouse`, `use role` then such
-an affected connection is marked internally as no longer matching with the pool it originated from.
+When an application does a change to the connection using one of SQL commands:
+* `use schema`, `create schema`
+* `use database`, `create database`
+* `use warehouse`, `create warehouse`
+* `use role`, `create role`
+
+then such an affected connection is marked internally as no longer matching with the pool it originated from.
 When parameter ChangedSession is set to `OriginalPool` it allows the connection to be pooled.
 Parameter ChangedSession set to `Destroy` (default) ensures that the connection is not pooled and after Close is called the connection will be removed.
 The pool will recreate necessary connections according to the minimal pool size.
 
+1) ChangedSession = Destroy
+
+In this mode application may safely alter session properties: schema, database, warehouse, role. Connection not matching
+with the connection string will not get pooled.
+
 ```cs
+var connectionString = ConnectionString + ";ChangedSession=Destroy";
+var connection = new SnowflakeDbConnection(connectionString);
+
+connection.Open();
+var randomSchemaName = Guid.NewGuid();
+connection.CreateCommand($"create schema \"{randomSchemaName}\").ExecuteNonQuery(); // schema gets changed
+// application is running commands on a schema with random name
+connection.Close(); // connection does not return to the original pool and gets destroyed; pool will reconstruct the pool
+                    // with new connections accordingly to the MinPoolSize
+
+var connection2 = new SnowflakeDbConnection(connectionString);
+connection2.Open();
+// operations here will be performed against schema indicated in the ConnectionString
+```
+
+2) ChangedSession = OriginalPool
+
+When application reuses connections affected by the above commands it might get to a point when using a connection
+it gets errors since tables, procedures, stages do not exists cause the operations are executed using wrong
+database, schema, user or role. This mode is purely for backward compatibility but is not recommended to be used.
+
+```cs
+var connectionString = ConnectionString + ";ChangedSession=OriginalPool;MinPoolSize=1;MaxPoolSize=1";
+var connection = new SnowflakeDbConnection(connectionString);
+
+connection.Open();
+var randomSchemaName = Guid.NewGuid();
+connection.CreateCommand($"create schema \"{randomSchemaName}\").ExecuteNonQuery(); // schema gets changed
+// application is running commands on a schema with random name
+connection.Close(); // connection returns to the original pool but it's schema will no longer match with initial value
 
+var connection2 = new SnowflakeDbConnection(connectionString);
+connection2.Open();
+// operations here will be performed against schema: randomSchemaName
 ```
 
 #### Pool Size Exceeded Timeout
@@ -268,62 +311,3 @@ There is also a way to clear all the pools initiated by an application.
 ```cs
 SnowflakeDbConnectionPool.ClearAllPools();
 ```
-
-### Single Connection Pool
-
-DEPRECATED VERSION
-
-Instead of creating a connection each time your client application needs to access Snowflake, you can define a cache of Snowflake connections that can be reused as needed.
-Connection pooling usually reduces the lag time to make a connection. However, it can slow down client failover to an alternative DNS when a DNS problem occurs.
-
-The Snowflake .NET driver provides the following functions for managing connection pools.
-
-| Function                                        | Description                                                                                             |
-|-------------------------------------------------|---------------------------------------------------------------------------------------------------------|
-| SnowflakeDbConnectionPool.ClearAllPools()       | Removes all connections from the connection pool.                                                       |
-| SnowflakeDbConnection.SetMaxPoolSize(n)         | Sets the maximum number of connections for the connection pool, where _n_ is the number of connections. |
-| SnowflakeDBConnection.SetTimeout(n)             | Sets the number of seconds to keep an unresponsive connection in the connection pool.                   |
-| SnowflakeDbConnectionPool.GetCurrentPoolSize()  | Returns the number of connections currently in the connection pool.                                     |
-| SnowflakeDbConnectionPool.SetPooling()          | Determines whether to enable (`true`) or disable (`false`) connection pooling. Default: `true`.         |
-
-The following sample demonstrates how to monitor the size of a connection pool as connections are added and dropped from the pool.
-
-```cs
-public void TestConnectionPoolClean()
-{
-  SnowflakeDbConnectionPool.ClearAllPools();
-  SnowflakeDbConnectionPool.SetMaxPoolSize(2);
-  var conn1 = new SnowflakeDbConnection();
-  conn1.ConnectionString = ConnectionString;
-  conn1.Open();
-  Assert.AreEqual(ConnectionState.Open, conn1.State);
-
-  var conn2 = new SnowflakeDbConnection();
-  conn2.ConnectionString = ConnectionString + " retryCount=1";
-  conn2.Open();
-  Assert.AreEqual(ConnectionState.Open, conn2.State);
-  Assert.AreEqual(0, SnowflakeDbConnectionPool.GetCurrentPoolSize());
-  conn1.Close();
-  conn2.Close();
-  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
-  var conn3 = new SnowflakeDbConnection();
-  conn3.ConnectionString = ConnectionString + "  retryCount=2";
-  conn3.Open();
-  Assert.AreEqual(ConnectionState.Open, conn3.State);
-
-  var conn4 = new SnowflakeDbConnection();
-  conn4.ConnectionString = ConnectionString + "  retryCount=3";
-  conn4.Open();
-  Assert.AreEqual(ConnectionState.Open, conn4.State);
-
-  conn3.Close();
-  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
-  conn4.Close();
-  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
-
-  Assert.AreEqual(ConnectionState.Closed, conn1.State);
-  Assert.AreEqual(ConnectionState.Closed, conn2.State);
-  Assert.AreEqual(ConnectionState.Closed, conn3.State);
-  Assert.AreEqual(ConnectionState.Closed, conn4.State);
-}
-```
diff --git a/doc/ConnectionPoolingDeprecated.md b/doc/ConnectionPoolingDeprecated.md
@@ -0,0 +1,60 @@
+## Using Connection Pools
+
+### Single Connection Pool (DEPRECATED)
+
+DEPRECATED VERSION
+
+Instead of creating a connection each time your client application needs to access Snowflake, you can define a cache of Snowflake connections that can be reused as needed.
+Connection pooling usually reduces the lag time to make a connection. However, it can slow down client failover to an alternative DNS when a DNS problem occurs.
+
+The Snowflake .NET driver provides the following functions for managing connection pools.
+
+| Function                                        | Description                                                                                             |
+|-------------------------------------------------|---------------------------------------------------------------------------------------------------------|
+| SnowflakeDbConnectionPool.ClearAllPools()       | Removes all connections from the connection pool.                                                       |
+| SnowflakeDbConnection.SetMaxPoolSize(n)         | Sets the maximum number of connections for the connection pool, where _n_ is the number of connections. |
+| SnowflakeDBConnection.SetTimeout(n)             | Sets the number of seconds to keep an unresponsive connection in the connection pool.                   |
+| SnowflakeDbConnectionPool.GetCurrentPoolSize()  | Returns the number of connections currently in the connection pool.                                     |
+| SnowflakeDbConnectionPool.SetPooling()          | Determines whether to enable (`true`) or disable (`false`) connection pooling. Default: `true`.         |
+
+The following sample demonstrates how to monitor the size of a connection pool as connections are added and dropped from the pool.
+
+```cs
+public void TestConnectionPoolClean()
+{
+  SnowflakeDbConnectionPool.ClearAllPools();
+  SnowflakeDbConnectionPool.SetMaxPoolSize(2);
+  var conn1 = new SnowflakeDbConnection();
+  conn1.ConnectionString = ConnectionString;
+  conn1.Open();
+  Assert.AreEqual(ConnectionState.Open, conn1.State);
+
+  var conn2 = new SnowflakeDbConnection();
+  conn2.ConnectionString = ConnectionString + " retryCount=1";
+  conn2.Open();
+  Assert.AreEqual(ConnectionState.Open, conn2.State);
+  Assert.AreEqual(0, SnowflakeDbConnectionPool.GetCurrentPoolSize());
+  conn1.Close();
+  conn2.Close();
+  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
+  var conn3 = new SnowflakeDbConnection();
+  conn3.ConnectionString = ConnectionString + "  retryCount=2";
+  conn3.Open();
+  Assert.AreEqual(ConnectionState.Open, conn3.State);
+
+  var conn4 = new SnowflakeDbConnection();
+  conn4.ConnectionString = ConnectionString + "  retryCount=3";
+  conn4.Open();
+  Assert.AreEqual(ConnectionState.Open, conn4.State);
+
+  conn3.Close();
+  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
+  conn4.Close();
+  Assert.AreEqual(2, SnowflakeDbConnectionPool.GetCurrentPoolSize());
+
+  Assert.AreEqual(ConnectionState.Closed, conn1.State);
+  Assert.AreEqual(ConnectionState.Closed, conn2.State);
+  Assert.AreEqual(ConnectionState.Closed, conn3.State);
+  Assert.AreEqual(ConnectionState.Closed, conn4.State);
+}
+```
diff --git a/doc/DataTypes.md b/doc/DataTypes.md
@@ -0,0 +1,41 @@
+## Data Types and Formats
+
+## Mapping .NET and Snowflake Data Types
+
+The .NET driver supports the following mappings from .NET to Snowflake data types.
+
+| .NET Framekwork Data Type | Data Type in Snowflake |
+| ------------------------- | ---------------------- |
+| `int`, `long`             | `NUMBER(38, 0)`        |
+| `decimal`                 | `NUMBER(38, <scale>)`  |
+| `double`                  | `REAL`                 |
+| `string`                  | `TEXT`                 |
+| `bool`                    | `BOOLEAN`              |
+| `byte`                    | `BINARY`               |
+| `datetime`                | `DATE`                 |
+
+## Arrow data format
+
+The .NET connector, starting with v2.1.3, supports the [Arrow data format](https://arrow.apache.org/)
+as a [preview](https://docs.snowflake.com/en/release-notes/preview-features) feature for data transfers
+between Snowflake and a .NET client. The Arrow data format avoids extra
+conversions between binary and textual representations of the data. The Arrow
+data format can improve performance and reduce memory consumption in clients.
+
+The data format is controlled by the
+DOTNET_QUERY_RESULT_FORMAT parameter. To use Arrow format, execute:
+
+```snowflake
+-- at the session level
+ALTER SESSION SET DOTNET_QUERY_RESULT_FORMAT = ARROW;
+-- or at the user level
+ALTER USER SET DOTNET_QUERY_RESULT_FORMAT = ARROW;
+-- or at the account level
+ALTER ACCOUNT SET DOTNET_QUERY_RESULT_FORMAT = ARROW;
+```
+
+The valid values for the parameter are:
+
+- ARROW
+- JSON (default)
+
diff --git a/doc/Disconnecting.md b/doc/Disconnecting.md
@@ -0,0 +1,21 @@
+## Close the Connection
+
+To close the connection, call the `Close` method of `SnowflakeDbConnection`.
+
+If you want to avoid blocking threads while the connection is closing, call the `CloseAsync` method instead, passing in a
+`CancellationToken`. This method was introduced in the v2.0.4 release.
+
+Note that because this method is not available in the generic `IDbConnection` interface, you must cast the object as
+`SnowflakeDbConnection` before calling the method. For example:
+
+```cs
+CancellationTokenSource cancellationTokenSource  = new CancellationTokenSource();
+// Close the connection
+((SnowflakeDbConnection)conn).CloseAsync(cancellationTokenSource.Token);
+```
+
+## Evict the Connection
+
+For the open connection, call the `PreventPooling()` to mark the connection to be removed on close instead being still pooled.
+The busy sessions counter will be decreased when the connection is closed.
+