Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harden saved object deserialization #94842

Merged
merged 9 commits into from
Apr 15, 2021

Conversation

jportner
Copy link
Contributor

This is a follow-on to #94616.

Saved object deserialization is not currently consistent -- the SavedObjectsSerializer class's isRawSavedObject() method is stricter than the rawToSavedObject() method. In other words, a raw object that isRawSavedObject() would return false for, may still be successfully deserialized by rawToSavedObject().

The simplest solution is to add a check in rawToSavedObject() to first check the outcome of rawToSavedObject(), and throw an error if that returned false.

@jportner jportner added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.13.0 labels Mar 17, 2021
Copy link
Contributor Author

@jportner jportner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to the stricter namespace prefix validation, the isRawSavedObject() method is also stricter about the presence of the [type] field (e.g., the object's attributes).

Seeing as the current check doesn't actually check if the [type] field's value is valid (it just checks to see if it is present), I thought it might make also make sense to loosen that condition for isRawSavedObject(). Something like this:

Click to see diff
diff --git a/src/core/server/saved_objects/serialization/serializer.ts b/src/core/server/saved_objects/serialization/serializer.ts
index 6e8e6344745..922a86968c5 100644
--- a/src/core/server/saved_objects/serialization/serializer.ts
+++ b/src/core/server/saved_objects/serialization/serializer.ts
@@ -28,85 +28,85 @@ export class SavedObjectsSerializer {
 
   /**
    * @internal
    */
   constructor(registry: ISavedObjectTypeRegistry) {
     this.registry = registry;
   }
   /**
    * Determines whether or not the raw document can be converted to a saved object.
    *
    * @param {SavedObjectsRawDoc} doc - The raw ES document to be tested
    * @param {SavedObjectsRawDocParseOptions} options - Options for parsing the raw document.
    */
   public isRawSavedObject(doc: SavedObjectsRawDoc, options: SavedObjectsRawDocParseOptions = {}) {
     const { namespaceTreatment = 'strict' } = options;
     const { _id, _source } = doc;
     const { type, namespace } = _source;
     if (!type) {
       return false;
     }
     const { idMatchesPrefix } = this.parseIdPrefix(namespace, type, _id, namespaceTreatment);
-    return idMatchesPrefix && _source.hasOwnProperty(type);
+    return idMatchesPrefix;
   }
 
   /**
    * Converts a document from the format that is stored in elasticsearch to the saved object client format.
    *
    * @param {SavedObjectsRawDoc} doc - The raw ES document to be converted to saved object format.
    * @param {SavedObjectsRawDocParseOptions} options - Options for parsing the raw document.
    */
   public rawToSavedObject(
     doc: SavedObjectsRawDoc,
     options: SavedObjectsRawDocParseOptions = {}
   ): SavedObjectSanitizedDoc {
     if (!this.isRawSavedObject(doc, options)) {
       throw new Error(`Raw document '${doc._id}' is not a valid saved object`);
     }
     const { namespaceTreatment = 'strict' } = options;
     const { _id, _source, _seq_no, _primary_term } = doc;
     const {
       type,
       namespaces,
       originId,
       migrationVersion,
       references,
       coreMigrationVersion,
     } = _source;
 
     const version =
       _seq_no != null || _primary_term != null
         ? encodeVersion(_seq_no!, _primary_term!)
         : undefined;
     const { id, namespace } = this.trimIdPrefix(_source.namespace, type, _id, namespaceTreatment);
     const includeNamespace =
       namespace && (namespaceTreatment === 'lax' || this.registry.isSingleNamespace(type));
     const includeNamespaces = this.registry.isMultiNamespace(type);
 
     return {
       type,
       id,
       ...(includeNamespace && { namespace }),
       ...(includeNamespaces && { namespaces }),
       ...(originId && { originId }),
-      attributes: _source[type],
+      attributes: _source[type] ?? {},
       references: references || [],
       ...(migrationVersion && { migrationVersion }),
       ...(coreMigrationVersion && { coreMigrationVersion }),
       ...(_source.updated_at && { updated_at: _source.updated_at }),
       ...(version && { version }),
     };
   }
 
   /**
    * Converts a document from the saved object client format to the format that is stored in elasticsearch.
    *
    * @param {SavedObjectSanitizedDoc} savedObj - The saved object to be converted to raw ES format.
    */
   public savedObjectToRaw(savedObj: SavedObjectSanitizedDoc): SavedObjectsRawDoc {
     const {
       id,
       type,
       namespace,
       namespaces,
       originId,
       attributes,

Thoughts @elastic/kibana-core ?

Edit: I had to add the first part of this diff in 4421a06 due to CI failures.

This method will now throw an error if the input is not a valid raw
saved object, based on the output of the isRawSavedObject() method.
This previously required a [type] field to be present, but that breaks
object parsing for "find" operations that do not include any type-
specific fields.
@jportner jportner force-pushed the harden-saved-object-deserialization branch from e5fcd6d to 4421a06 Compare March 17, 2021 21:46
@jportner jportner requested a review from a team March 17, 2021 23:51
@jportner jportner marked this pull request as ready for review March 17, 2021 23:52
@jportner
Copy link
Contributor Author

@elasticmachine merge upstream

@@ -59,6 +59,9 @@ export class SavedObjectsSerializer {
doc: SavedObjectsRawDoc,
options: SavedObjectsRawDocParseOptions = {}
): SavedObjectSanitizedDoc {
if (!this.isRawSavedObject(doc, options)) {
throw new Error(`Raw document '${doc._id}' is not a valid saved object`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should show a reason for the error? There are several possible reasons: SO _source.type is not defined, id prefix doesn't match, probably we will have more in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done in cfa3520.

@jportner
Copy link
Contributor Author

@elasticmachine merge upstream

@jportner jportner requested a review from mshustov April 14, 2021 15:57
@jportner
Copy link
Contributor Author

I think the CI failure is transient, I'm going to retry it.

@jportner
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jportner jportner added the auto-backport Deprecated - use backport:version if exact versions are needed label Apr 15, 2021
@jportner jportner merged commit e675429 into elastic:master Apr 15, 2021
@jportner jportner deleted the harden-saved-object-deserialization branch April 15, 2021 15:57
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Apr 15, 2021
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Apr 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes v7.13.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants