feat: Bigtable authorized views requests on the Data plane (#1509)

* Create the new tabular api service file * feat: Bigtable authorized views - move the code over to the TabularApiSurface class (#1463) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Only include Table mocks that are necessary * Remove TODO * Rename TabularApiService to TabularApiSurface * Change all imports to tabular-api-surface * surface. not service --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views (#1464) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Trying out the FilterInformation class * Add a DataUtils module for the shared row function * Outsource functionality of filter to helper * debugging * Adjust proxyquire to include mocks for moved fn * Remove imports * Outsource code to a createRulesUtil function * Change proxyquire to include createRulesUtil * Move increment over to the utils folder * Move the functions into a static class for mocks * Remove mockCreateRules and shorten mock * Stub out FakeRowDataUtil * Remove unused dependencies * Add the functions to work with checkAndMutate and readWriteModifyRow * Fix regressions from the merge * Add documentation for the class * Document the new methods of table * Change the interface of the rowUtils * Pull the generateProperties code out avoid duplica * Move duplicate code out into a getProperties object * Remove console.log * Add method for creating views * Object for making grpc calls for authorized views * Remove import * Update the documentation for the Table * More specific type * Add documentation for each of the functions * Remove TODO * Add headers * Add documentation for new class * Remove imports * Reintroduce before * Remove unused import --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Create a view on the instance, not on the table (#1476) * Move view creation from the table class to instanc * feat: Create a view on the instance, not on the table * Add table name parameter to the documentation * Remove Table * feat: Create the unit tests for Authorized Views ensuring that requests are consistent (#1501) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Add a few @params and @returns * Add a comment about making requests for auth views * Added more comments - auth views vs table Also added a comment for calling increment * feat: Bigtable authorized views integration tests (#1504) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Create the before hook for auth views * Note to self on auth views - TODO * Add an insert statement * Make a test for getRows * Add the test for the getRows function. * Finish ‘should fail when writing to a row not in v * Add another mutate test, preserve table after test Add after hook to make sure table values stay the same. Add test for modifying from different column * Remove TODO * refactor the error message in the test * Add mutate, insert and sampleRowKeys tests * Fix sampleRowKeys test to require a fix upstream * Create samplerowkeys and createreadstream tests * Surround all tests in a try block This allows for better error reporting * Fixed the column filter so it works * Reduce verbosity in tests In many cases, 3 lines can be reduced to one variable. This is crucial for making the integration tests shorter. * Replace verbose data with data variable * Add an insert for multiple rows * Fix the check and mutate test to ignore corrupt d * Add the readModifyWriteRow tests * Add awaits * wrap columnFamily and columnIdInView in brackets * Eliminate the TODO * Remove only * Add key back to filter config option --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Kevin Kim <[email protected]>
googleapis · Nov 7, 2024 · da373b5 · da373b5
1 parent c4e2356
commit da373b5
Show file tree

Hide file tree

Showing 13 changed files with 2,614 additions and 1,006 deletions.
diff --git a/src/authorized-view.ts b/src/authorized-view.ts
@@ -0,0 +1,260 @@
+// Copyright 2024 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import {TabularApiSurface} from './tabular-api-surface';
+import {CallOptions} from 'google-gax';
+import {
+  CreateRulesCallback,
+  CreateRulesResponse,
+  FilterCallback,
+  FilterConfig,
+  FilterResponse,
+  IncrementCallback,
+  IncrementResponse,
+  Rule,
+} from './row';
+import {RowDataUtils, RowProperties} from './row-data-utils';
+import {RawFilter} from './filter';
+import {Family} from './chunktransformer';
+import {Instance} from './instance';
+import {promisifyAll} from '@google-cloud/promisify';
+
+interface FilterInformation {
+  filter: RawFilter;
+  rowId: string;
+}
+
+interface CreateRulesInformation {
+  rules: Rule | Rule[];
+  rowId: string;
+}
+
+interface IncrementInformation {
+  column: string;
+  rowId: string;
+}
+
+/**
+ * The AuthorizedView class is a class that is available to the user that
+ * contains methods the user can call to work with authorized views.
+ *
+ * @class
+ * @param {Table} table The table that the authorized view exists on.
+ * @param {string} id Unique identifier of the authorized view.
+ *
+ */
+export class AuthorizedView extends TabularApiSurface {
+  private readonly rowData: {[id: string]: {[index: string]: Family}};
+
+  constructor(instance: Instance, tableName: string, viewName: string) {
+    super(instance, tableName, viewName);
+    this.rowData = {};
+  }
+
+  createRules(
+    createRulesInfo: CreateRulesInformation,
+    options?: CallOptions
+  ): Promise<CreateRulesResponse>;
+  createRules(
+    createRulesInfo: CreateRulesInformation,
+    options: CallOptions,
+    callback: CreateRulesCallback
+  ): void;
+  createRules(
+    createRulesInfo: CreateRulesInformation,
+    callback: CreateRulesCallback
+  ): void;
+  /**
+   * Update a row with rules specifying how the row's contents are to be
+   * transformed into writes. Rules are applied in order, meaning that earlier
+   * rules will affect the results of later ones.
+   *
+   * @throws {error} If no rules are provided.
+   *
+   * @param {CreateRulesInformation} createRulesInfo The rules to apply to a row
+   *    along with the row id of the row to update.
+   * @param {object} [gaxOptions] Request configuration options, outlined here:
+   *     https://googleapis.github.io/gax-nodejs/CallSettings.html.
+   * @param {function} callback The callback function.
+   * @param {?error} callback.err An error returned while making this
+   *     request.
+   * @param {object} callback.apiResponse The full API response.
+   *
+   * @example <caption>include:samples/api-reference-doc-snippets/row.js</caption>
+   * region_tag:bigtable_api_create_rules
+   */
+  createRules(
+    createRulesInfo: CreateRulesInformation,
+    optionsOrCallback?: CallOptions | CreateRulesCallback,
+    cb?: CreateRulesCallback
+  ): void | Promise<CreateRulesResponse> {
+    this.initializeRow(createRulesInfo.rowId);
+    RowDataUtils.createRulesUtil(
+      createRulesInfo.rules,
+      this.generateProperties(createRulesInfo.rowId),
+      optionsOrCallback,
+      cb
+    );
+  }
+
+  /**
+   * Mutates a row atomically based on the output of a filter. Depending on
+   * whether or not any results are yielded, either the `onMatch` or `onNoMatch`
+   * callback will be executed.
+   *
+   * @param {FilterInformation} filter Filter to be applied to the contents of
+   * the row along with the row id of the affected row.
+   * @param {object} config Configuration object.
+   * @param {?object[]} config.onMatch A list of entries to be ran if a match is
+   *     found.
+   * @param {object[]} [config.onNoMatch] A list of entries to be ran if no
+   *     matches are found.
+   * @param {object} [config.gaxOptions] Request configuration options, outlined
+   *     here: https://googleapis.github.io/gax-nodejs/global.html#CallOptions.
+   * @param {function} callback The callback function.
+   * @param {?error} callback.err An error returned while making this
+   *     request.
+   * @param {boolean} callback.matched Whether a match was found or not.
+   *
+   * @example <caption>include:samples/api-reference-doc-snippets/row.js</caption>
+   * region_tag:bigtable_api_row_filter
+   */
+  filter(
+    filterInfo: FilterInformation,
+    config?: FilterConfig
+  ): Promise<FilterResponse>;
+  filter(
+    filterInfo: FilterInformation,
+    config: FilterConfig,
+    callback: FilterCallback
+  ): void;
+  filter(filterInfo: FilterInformation, callback: FilterCallback): void;
+  filter(
+    filterInfo: FilterInformation,
+    configOrCallback?: FilterConfig | FilterCallback,
+    cb?: FilterCallback
+  ): void | Promise<FilterResponse> {
+    this.initializeRow(filterInfo.rowId);
+    RowDataUtils.filterUtil(
+      filterInfo.filter,
+      this.generateProperties(filterInfo.rowId),
+      configOrCallback,
+      cb
+    );
+  }
+
+  /**
+   * Generates request properties necessary for making an rpc call for an
+   * authorized view.
+   *
+   * @param {string} id The row id to generate the properties for.
+   * @private
+   */
+  private generateProperties(id: string): RowProperties {
+    return {
+      requestData: {
+        data: this.rowData[id],
+        id,
+        table: this,
+        bigtable: this.bigtable,
+      },
+      reqOpts: {
+        authorizedViewName: this.name + '/authorizedViews/' + this.viewName,
+      },
+    };
+  }
+
+  increment(
+    columnInfo: IncrementInformation,
+    value?: number
+  ): Promise<IncrementResponse>;
+  increment(
+    columnInfo: IncrementInformation,
+    value: number,
+    options?: CallOptions
+  ): Promise<IncrementResponse>;
+  increment(
+    columnInfo: IncrementInformation,
+    options?: CallOptions
+  ): Promise<IncrementResponse>;
+  increment(
+    columnInfo: IncrementInformation,
+    value: number,
+    options: CallOptions,
+    callback: IncrementCallback
+  ): void;
+  increment(
+    columnInfo: IncrementInformation,
+    value: number,
+    callback: IncrementCallback
+  ): void;
+  increment(
+    columnInfo: IncrementInformation,
+    options: CallOptions,
+    callback: IncrementCallback
+  ): void;
+  increment(
+    columnInfo: IncrementInformation,
+    callback: IncrementCallback
+  ): void;
+  /**
+   * Increment a specific column within the row. If the column does not
+   * exist, it is automatically initialized to 0 before being incremented.
+   *
+   * @param {IncrementInformation} columnInfo The column we are incrementing a
+   * value in along with the row id of the affected row.
+   * @param {number} [value] The amount to increment by, defaults to 1.
+   * @param {object} [gaxOptions] Request configuration options, outlined here:
+   *     https://googleapis.github.io/gax-nodejs/CallSettings.html.
+   * @param {function} callback The callback function.
+   * @param {?error} callback.err An error returned while making this
+   *     request.
+   * @param {number} callback.value The updated value of the column.
+   * @param {object} callback.apiResponse The full API response.
+   *
+   * @example <caption>include:samples/api-reference-doc-snippets/row.js</caption>
+   * region_tag:bigtable_api_row_increment
+   */
+  increment(
+    columnInfo: IncrementInformation,
+    valueOrOptionsOrCallback?: number | CallOptions | IncrementCallback,
+    optionsOrCallback?: CallOptions | IncrementCallback,
+    cb?: IncrementCallback
+  ): void | Promise<IncrementResponse> {
+    this.initializeRow(columnInfo.rowId);
+    RowDataUtils.incrementUtils(
+      columnInfo.column,
+      this.generateProperties(columnInfo.rowId),
+      valueOrOptionsOrCallback,
+      optionsOrCallback,
+      cb
+    );
+  }
+
+  /**
+   * Sets the row data for a particular row to an empty object
+   *
+   * @param {string} id An string with the key of the row to initialize.
+   * @private
+   */
+  private initializeRow(id: string) {
+    if (!this.rowData[id]) {
+      this.rowData[id] = {};
+    }
+  }
+}
+
+promisifyAll(AuthorizedView, {
+  exclude: ['initializeRow', 'generateProperties'],
+});
diff --git a/src/chunktransformer.ts b/src/chunktransformer.ts
@@ -34,7 +34,7 @@ export interface Data {
   chunks: Chunk[];
   lastScannedRowKey?: Buffer;
 }
-interface Family {
+export interface Family {
   [qualifier: string]: Qualifier[];
 }
 export interface Qualifier {

diff --git a/src/instance.ts b/src/instance.ts
@@ -68,6 +68,7 @@ import {Bigtable} from '.';
 import {google} from '../protos/protos';
 import {Backup, RestoreTableCallback, RestoreTableResponse} from './backup';
 import {ClusterUtils} from './utils/cluster';
+import {AuthorizedView} from './authorized-view';
 
 export interface ClusterInfo extends BasicClusterConfig {
   id: string;
@@ -1475,6 +1476,16 @@ Please use the format 'my-instance' or '${bigtable.projectName}/instances/my-ins
       }
     );
   }
+
+  /**
+   * Gets an Authorized View object for making authorized view grpc calls.
+   *
+   * @param {string} tableName The name for the Table
+   * @param {string} viewName The name for the Authorized view
+   */
+  view(tableName: string, viewName: string): AuthorizedView {
+    return new AuthorizedView(this, tableName, viewName);
+  }
 }
 
 /*! Developer Documentation
@@ -1490,6 +1501,7 @@ promisifyAll(Instance, {
     'getBackupsStream',
     'getTablesStream',
     'getAppProfilesStream',
+    'view',
   ],
 });