Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(deps): Remove @google-cloud/promisify dependency in favor of native implementation #1967

Open
danielbankhead opened this issue May 25, 2022 · 1 comment
Labels
api: storage Issues related to the googleapis/nodejs-storage API. next major: breaking change this is a change that we should wait to bundle into the next major version priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@danielbankhead
Copy link
Contributor

Currently we utilize @google-cloud/promisify in numerous files to create functions and methods that handle both callback and Promises APIs. The problem with this approach is that it's difficult to determine when something is promisified on the type-level and it makes debugging/understanding the codebase a bit more cumbersome (it can come off as 'magic' for newcomers).

Source:

nodejs-storage/src/file.ts

Lines 4121 to 4130 in a9c4c18

promisifyAll(File, {
exclude: [
'publicUrl',
'request',
'save',
'setEncryptionKey',
'shouldRetryBasedOnPreconditionAndIdempotencyStrat',
'getBufferFromReadable',
],
});

Example (difficult to understand how a Promise is returned):

nodejs-storage/src/file.ts

Lines 1532 to 1676 in a9c4c18

createResumableUpload(
options?: CreateResumableUploadOptions
): Promise<CreateResumableUploadResponse>;
createResumableUpload(
options: CreateResumableUploadOptions,
callback: CreateResumableUploadCallback
): void;
createResumableUpload(callback: CreateResumableUploadCallback): void;
/**
* @callback CreateResumableUploadCallback
* @param {?Error} err Request error, if any.
* @param {string} uri The resumable upload's unique session URI.
*/
/**
* @typedef {array} CreateResumableUploadResponse
* @property {string} 0 The resumable upload's unique session URI.
*/
/**
* @typedef {object} CreateResumableUploadOptions
* @property {string} [configPath] A full JSON file path to use with
* `gcs-resumable-upload`. This maps to the {@link https://github.com/yeoman/configstore/tree/0df1ec950d952b1f0dfb39ce22af8e505dffc71a#configpath| configstore option by the same name}.
* @property {object} [metadata] Metadata to set on the file.
* @property {number} [offset] The starting byte of the upload stream for resuming an interrupted upload.
* @property {string} [origin] Origin header to set for the upload.
* @property {string} [predefinedAcl] Apply a predefined set of access
* controls to this object.
*
* Acceptable values are:
* - **`authenticatedRead`** - Object owner gets `OWNER` access, and
* `allAuthenticatedUsers` get `READER` access.
*
* - **`bucketOwnerFullControl`** - Object owner gets `OWNER` access, and
* project team owners get `OWNER` access.
*
* - **`bucketOwnerRead`** - Object owner gets `OWNER` access, and project
* team owners get `READER` access.
*
* - **`private`** - Object owner gets `OWNER` access.
*
* - **`projectPrivate`** - Object owner gets `OWNER` access, and project
* team members get access according to their roles.
*
* - **`publicRead`** - Object owner gets `OWNER` access, and `allUsers`
* get `READER` access.
* @property {boolean} [private] Make the uploaded file private. (Alias for
* `options.predefinedAcl = 'private'`)
* @property {boolean} [public] Make the uploaded file public. (Alias for
* `options.predefinedAcl = 'publicRead'`)
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {string} [chunkSize] Create a separate request per chunk. Should
* be a multiple of 256 KiB (2^18).
* {@link https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload| We recommend using at least 8 MiB for the chunk size.}
*/
/**
* Create a unique resumable upload session URI. This is the first step when
* performing a resumable upload.
*
* See the {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload| Resumable upload guide}
* for more on how the entire process works.
*
* <h4>Note</h4>
*
* If you are just looking to perform a resumable upload without worrying
* about any of the details, see {@link File#createWriteStream}. Resumable
* uploads are performed by default.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload| Resumable upload guide}
*
* @param {CreateResumableUploadOptions} [options] Configuration options.
* @param {CreateResumableUploadCallback} [callback] Callback function.
* @returns {Promise<CreateResumableUploadResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
* file.createResumableUpload(function(err, uri) {
* if (!err) {
* // `uri` can be used to PUT data to.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* file.createResumableUpload().then(function(data) {
* const uri = data[0];
* });
* ```
*/
createResumableUpload(
optionsOrCallback?:
| CreateResumableUploadOptions
| CreateResumableUploadCallback,
callback?: CreateResumableUploadCallback
): void | Promise<CreateResumableUploadResponse> {
const options =
typeof optionsOrCallback === 'object' ? optionsOrCallback : {};
callback =
typeof optionsOrCallback === 'function' ? optionsOrCallback : callback;
const retryOptions = this.storage.retryOptions;
if (
(options?.preconditionOpts?.ifGenerationMatch === undefined &&
this.instancePreconditionOpts?.ifGenerationMatch === undefined &&
this.storage.retryOptions.idempotencyStrategy ===
IdempotencyStrategy.RetryConditional) ||
this.storage.retryOptions.idempotencyStrategy ===
IdempotencyStrategy.RetryNever
) {
retryOptions.autoRetry = false;
}
resumableUpload.createURI(
{
authClient: this.storage.authClient,
apiEndpoint: this.storage.apiEndpoint,
bucket: this.bucket.name,
configPath: options.configPath,
customRequestOptions: this.getRequestInterceptors().reduce(
(reqOpts, interceptorFn) => interceptorFn(reqOpts),
{}
),
file: this.name,
generation: this.generation,
key: this.encryptionKey,
kmsKeyName: this.kmsKeyName,
metadata: options.metadata,
offset: options.offset,
origin: options.origin,
predefinedAcl: options.predefinedAcl,
private: options.private,
public: options.public,
userProject: options.userProject || this.userProject,
retryOptions: retryOptions,
params: options?.preconditionOpts || this.instancePreconditionOpts,
},
callback!
);
this.storage.retryOptions.autoRetry = this.instanceRetryValue;
}

Going forward, we should handle the logic directly in the methods - in a seamless way when possible.
Idea: Perhaps have an internal/private Promise-only (async/await) version of methods, and have the public methods internally handling all difference between callback and Promise (e.g. most logic would be in the internalized-async/await function).

@danielbankhead danielbankhead added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p3 Desirable enhancement or fix. May not be included in next release. labels May 25, 2022
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/nodejs-storage API. label May 25, 2022
@danielbankhead
Copy link
Contributor Author

danielbankhead commented Jun 7, 2023

This utility may help:

export function normalize<T = {}, U = Function>(
optionsOrCallback?: T | U,
cb?: U
) {
const options = (
typeof optionsOrCallback === 'object' ? optionsOrCallback : {}
) as T;
const callback = (
typeof optionsOrCallback === 'function' ? optionsOrCallback : cb
)! as U;
return {options, callback};
}

@danielduhh danielduhh added the next major: breaking change this is a change that we should wait to bundle into the next major version label Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/nodejs-storage API. next major: breaking change this is a change that we should wait to bundle into the next major version priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants