Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOP-5036 Refactor Search-Manifest integration #10

Merged
merged 40 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
0a023cb
DOP-5036 clean up some comments
anabellabuckvar Sep 27, 2024
fe44e16
DOP-5036 refactor delete stale
anabellabuckvar Sep 27, 2024
18c4b40
DOP-5036 refactor delete stale
anabellabuckvar Sep 27, 2024
35fafb3
DOP-5036 improve how to get collections
anabellabuckvar Sep 27, 2024
6625f51
DOP-5036 create getRepo function
anabellabuckvar Sep 27, 2024
a2fd57e
DOP-5036 create getDocsetEntry function
anabellabuckvar Sep 27, 2024
c126e41
DOP-5036 create getDocsetEntry function
anabellabuckvar Sep 27, 2024
eaa2126
DOP-5036 log client
anabellabuckvar Sep 30, 2024
1e75874
DOP-5036 unlog client
anabellabuckvar Sep 30, 2024
10ace66
DOP-5036 log client
anabellabuckvar Sep 30, 2024
99567dc
DOP-5036 test new teardown methods
anabellabuckvar Sep 30, 2024
752f767
DOP-5036 test new connection method
anabellabuckvar Sep 30, 2024
1858820
DOP-5036 use new connection methods
anabellabuckvar Sep 30, 2024
7a67d3e
DOP-5036 use new connection methods
anabellabuckvar Sep 30, 2024
c0b0847
DOP-5036 fix connection methods
anabellabuckvar Sep 30, 2024
f213067
DOP-5036 linting
anabellabuckvar Sep 30, 2024
2efa9f4
DOP-5036 use functions specific to each collection
anabellabuckvar Sep 30, 2024
a0609b5
DOP-5036 check for env vars
anabellabuckvar Sep 30, 2024
0557a19
DOP-5036 casing
anabellabuckvar Sep 30, 2024
b6f3b0a
DOP-5036 fix some casing
anabellabuckvar Oct 1, 2024
95e14f7
DOP-5036 fix tests
anabellabuckvar Oct 1, 2024
c3658d3
DOP-5036 add metadata type
anabellabuckvar Oct 1, 2024
6134626
DOP-5036 fix manifest typing
anabellabuckvar Oct 1, 2024
bf1dc81
DOP-5036 fix manifest typing
anabellabuckvar Oct 1, 2024
bb05afe
DOP-5036 fix facet formatting
anabellabuckvar Oct 1, 2024
6697b85
DOP-5036 fix facets
anabellabuckvar Oct 1, 2024
9c13000
DOP-5036 some formatting
anabellabuckvar Oct 1, 2024
ce383b4
DOP-5036 merge createFacets paths
anabellabuckvar Oct 2, 2024
fc3abd7
DOP-5036 cleaning a lot
anabellabuckvar Oct 4, 2024
3018c26
DOP-5036 updating tests
anabellabuckvar Oct 4, 2024
e6f8e5f
DOP-5036 add app services s3 manifest
anabellabuckvar Oct 4, 2024
8afdc8c
DOP-5036 add compass current s3 manifest
anabellabuckvar Oct 4, 2024
6365a05
DOP-5036 add compass upcoming s3 manifest
anabellabuckvar Oct 4, 2024
dc607e0
DOP-5036 fix some tests
anabellabuckvar Oct 4, 2024
95cc815
DOP-5036 change types to Pascal case
anabellabuckvar Oct 4, 2024
663b355
DOP-5036 simplify getBranch function
anabellabuckvar Oct 4, 2024
13777ec
DOP-5036 addressing nits
anabellabuckvar Oct 7, 2024
da8735e
DOP-5036 cleaning more
anabellabuckvar Oct 7, 2024
69c51cc
DOP-5036 change facet from class to interface
anabellabuckvar Oct 7, 2024
4550273
DOP-5036 last changes
anabellabuckvar Oct 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions search-manifest/src/assertEnvVars.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import type { EnvVars } from "./types";

const assertEnvVars = (vars: EnvVars) => {
const missingVars = Object.entries(vars)
.filter(([, value]) => !value)
.map(([key]) => `- ${key}`)
.join("\n");
if (missingVars)
throw new Error(`Missing env var(s) ${JSON.stringify(missingVars)}`);
return vars;
};

export const getEnvVars = () => {
const environmentVariables = assertEnvVars({
ATLAS_CLUSTER0_URI: `mongodb+srv://${process.env.MONGO_ATLAS_USERNAME}:${process.env.MONGO_ATLAS_PASSWORD}@${process.env.MONGO_ATLAS_CLUSTER0_HOST}/?retryWrites=true&w=majority`,
SNOOTY_DB_NAME: `${process.env.MONGO_ATLAS_POOL_DB_NAME}`,
ATLAS_SEARCH_URI: `mongodb+srv://${process.env.MONGO_ATLAS_USERNAME}:${process.env.MONGO_ATLAS_PASSWORD}@${process.env.MONGO_ATLAS_SEARCH_HOST}/?retryWrites=true&w=majority`,
SEARCH_DB_NAME: `${process.env.MONGO_ATLAS_SEARCH_DB_NAME}`,
REPOS_BRANCHES_COLLECTION: "repos_branches",
DOCSETS_COLLECTION: "docsets",
DOCUMENTS_COLLECTION: "documents",
});
return environmentVariables;
};
51 changes: 27 additions & 24 deletions search-manifest/src/generateManifest/createFacets.ts
Original file line number Diff line number Diff line change
@@ -1,25 +1,28 @@
import { NetlifyIntegration } from '@netlify/sdk';

export class Facet {
category: any;
value: any;
subFacets: any;

constructor(category: string, value: string, subFacets: []) {
this.category = category;
this.value = value;
this.subFacets = [];

if (subFacets) {
for (const subFacet of subFacets) {
this.subFacets.push(
new Facet(
subFacet['category'],
subFacet['value'],
subFacet['sub_facets'] ?? [],
),
);
}
}
}
export interface Facet {
category: string;
value: string;
subFacets: Array<Facet> | null;
}

export const createFacet = (facet: Facet) => {
const category = facet.category;
const value = facet.value;
const subFacetsArr = [];
if (facet.subFacets) {
for (const subFacet of facet.subFacets) {
subFacetsArr.push(
createFacet({
category: subFacet.category,
value: subFacet.value,
subFacets: subFacet.subFacets ?? [],
})
);
}
}
const newFacet: Facet = {
category: category,
value: value,
subFacets: subFacetsArr ?? null,
};
return newFacet;
};
Loading
Loading