Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🍄 Separate trash for the personal and shared drive #189

Merged
merged 25 commits into from
Oct 6, 2023
Merged

🍄 Separate trash for the personal and shared drive #189

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8dd1dc9
✨init model
shepilov Sep 4, 2023
4f861db
feat: updated document service and personal trash
MontaGhanmy Sep 11, 2023
4623daa
feat: updated item deletion and restore
MontaGhanmy Sep 12, 2023
e61edcf
feat: update scope when moving item
MontaGhanmy Sep 12, 2023
66c86a3
ref: refactoring item scope
MontaGhanmy Sep 15, 2023
6f462ca
ref: updated trash size logic
MontaGhanmy Sep 15, 2023
3363e60
feat: trash path switcher ui
MontaGhanmy Sep 15, 2023
6e68088
fix: public link tests
MontaGhanmy Sep 15, 2023
9c8a4cb
ref: document tests
MontaGhanmy Sep 15, 2023
46d9858
fix: e2e tests
MontaGhanmy Sep 18, 2023
dc6be2a
fix: cassandra orm
MontaGhanmy Sep 19, 2023
b4f0174
feat: updated cassandra query builder
MontaGhanmy Sep 19, 2023
cc933e2
feat: updated browse query
MontaGhanmy Sep 19, 2023
9b28b32
fix: object filtering
MontaGhanmy Sep 20, 2023
350c2de
feat: updated query tests
MontaGhanmy Sep 20, 2023
215b0bb
ref: updated orm query builder
MontaGhanmy Sep 20, 2023
85bc0ad
fix: orm test
MontaGhanmy Sep 20, 2023
75c676b
fix: updated query builder
MontaGhanmy Sep 20, 2023
9565442
feat: added is_in_trash index + ref
MontaGhanmy Sep 21, 2023
e9354d7
fix: trash ui bugs
MontaGhanmy Sep 22, 2023
0d7bbc0
🌐russian translation
shepilov Sep 25, 2023
7fa08a4
fix: context menu
MontaGhanmy Sep 26, 2023
3ead34a
merge: conflict
MontaGhanmy Sep 26, 2023
539bcd7
feat: restore/restore sub item/fix: submenu
MontaGhanmy Oct 4, 2023
35c055a
fix: bugs with navigation and restore
MontaGhanmy Oct 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion ...de/src/core/platform/services/database/services/orm/connectors/cassandra/query-builder.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export function buildSelectQuery<Entity>(
): string {
const instance = new (entityType as any)();
const { columnsDefinition, entityDefinition } = getEntityDefinition(instance);
let allowFiltering = false;

const where = Object.keys(filters)
.map(key => {
Expand All @@ -28,6 +29,7 @@ export function buildSelectQuery<Entity>(
return;
}
if (isObject(filter) && JSON.stringify(filter).includes("ne")) {
allowFiltering = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again have you been conducting serious performance benchmarks before conducting this change?

Due to the "log structured merge tree" architecture of the Cassandra storage, allow filtering can cause each request to scan through potentially terabytes of data.

As explained yesterday, I have big doubts that this change is compatible with anything looking to a serious production set up and would advise the team to look for a longer term solution.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally yes, but in our case we're restricting the partition and using indexes to query the data. We're also restricting the use of Allow filtering to a very specific scenario. We won't end up scanning through terabytes of data.

result = `${key} IN (NULL, false)`;
return result;
}
Expand All @@ -47,6 +49,7 @@ export function buildSelectQuery<Entity>(

result = `${key} IN (${inClause.join(",")})`;
} else {
if (key === "is_in_trash" || key === "scope") allowFiltering = true;
result = `${key} = ${transformValueToDbString(filter, columnsDefinition[key].type, {
columns: columnsDefinition[key].options,
secret: options.secret || "",
Expand Down Expand Up @@ -82,7 +85,7 @@ export function buildSelectQuery<Entity>(
whereClause.trim().length ? "WHERE " + whereClause : ""
} ${orderByClause.trim().length ? "ORDER BY " + orderByClause : ""}`
.trimEnd()
.concat(";");
.concat(allowFiltering ? " ALLOW FILTERING;" : ";");

return query;
}
Expand Down
2 changes: 1 addition & 1 deletion tdrive/backend/node/src/services/documents/entities/drive-file.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ export type DriveScope = "personal" | "shared";

@Entity(TYPE, {
globalIndexes: [["company_id", "parent_id"]],
primaryKey: [["company_id"], "is_in_trash", "scope", "id"],
primaryKey: [["company_id"], "id"],
type: TYPE,
search,
})
Expand Down
Loading