Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: editors can select which planning constraints to check #3968

Merged
merged 16 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions api.planx.uk/modules/flows/validate/service/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ import * as jsondiffpatch from "jsondiffpatch";
import { dataMerged, getMostRecentPublishedFlow } from "../../../../helpers.js";
import { validateFileTypes } from "./fileTypes.js";
import { validateInviteToPay } from "./inviteToPay.js";
import { validateSections } from "./sections.js";
import { validatePlanningConstraints } from "./planningConstraints.js";
import { validateProjectTypes } from "./projectTypes.js";
import { validateSections } from "./sections.js";

type AlteredNode = {
id: string;
Expand Down Expand Up @@ -54,7 +55,14 @@ const validateAndDiffFlow = async (
const inviteToPay = validateInviteToPay(flattenedFlow);
const fileTypes = validateFileTypes(flattenedFlow);
const projectTypes = validateProjectTypes(flattenedFlow);
validationChecks.push(sections, inviteToPay, fileTypes, projectTypes);
const planningConstraints = validatePlanningConstraints(flattenedFlow);
validationChecks.push(
sections,
inviteToPay,
fileTypes,
projectTypes,
planningConstraints,
);

// Arrange list of validation checks in order of status: Fail, Warn, Pass, Not applicable
const failingChecks = validationChecks.filter((v) => v.status == "Fail");
Expand Down
38 changes: 38 additions & 0 deletions api.planx.uk/modules/flows/validate/service/planningConstraints.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import {
ComponentType,
type FlowGraph,
} from "@opensystemslab/planx-core/types";
import { numberOfComponentType } from "../helpers.js";
import type { FlowValidationResponse } from "./index.js";

const validatePlanningConstraints = (
flowGraph: FlowGraph,
): FlowValidationResponse => {
const numberOfPlanningConstraintNodes = numberOfComponentType(
flowGraph,
ComponentType.PlanningConstraints,
);
if (numberOfPlanningConstraintNodes > 1) {
return {
title: "Planning Constraints",
status: "Fail",
message:
"When using Planning Constraints, your flow must have exactly ONE Planning Constraints component",
};
} else if (numberOfPlanningConstraintNodes === 1) {
// In future, add extra `hasPlanningData` validation step here to ensure integration is available for this team
jessicamcinchak marked this conversation as resolved.
Show resolved Hide resolved
return {
title: "Planning Constraints",
status: "Pass",
message: "Your flow has valid Planning Constraints",
};
} else {
return {
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
};
}
};

export { validatePlanningConstraints };
185 changes: 185 additions & 0 deletions api.planx.uk/modules/flows/validate/validate.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,11 @@ describe("sections validation on diff", () => {
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -190,6 +195,11 @@ describe("sections validation on diff", () => {
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -240,6 +250,11 @@ describe("invite to pay validation on diff", () => {
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -300,6 +315,11 @@ describe("invite to pay validation on diff", () => {
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -356,6 +376,11 @@ describe("invite to pay validation on diff", () => {
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -414,6 +439,11 @@ describe("invite to pay validation on diff", () => {
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -468,6 +498,11 @@ describe("invite to pay validation on diff", () => {
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -555,6 +590,11 @@ describe("ODP Schema file type validation on diff", () => {
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
Expand Down Expand Up @@ -633,6 +673,151 @@ describe("ODP Schema file type validation on diff", () => {
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
{
title: "Planning Constraints",
status: "Not applicable",
message: "Your flow is not using Planning Constraints",
},
]);
});
});
});

describe("planning constraints validation on diff", () => {
it("passes if there is exactly one planning constraints component", async () => {
const alteredFlow = {
...mockFlowData,
PlanningConstraints: {
type: 11,
data: {
title: "Check all constraints",
fn: "property.constraints.planning",
},
},
};

queryMock.mockQuery({
name: "GetFlowData",
matchOnVariables: false,
data: {
flow: {
data: alteredFlow,
slug: "altered-flow-name",
team_id: 1,
team: {
slug: "testing",
},
publishedFlows: [{ data: alteredFlow }],
},
},
});

await supertest(app)
.post("/flows/1/diff")
.set(auth)
.expect(200)
.then((res) => {
expect(res.body.message).toEqual("Changes queued to publish");
expect(res.body.validationChecks).toEqual([
{
title: "Sections",
status: "Pass",
message: "Your flow has valid Sections",
},
{
title: "Planning Constraints",
status: "Pass",
message: "Your flow has valid Planning Constraints",
},
{
title: "Invite to Pay",
status: "Not applicable",
message: "Your flow is not using Invite to Pay",
},
{
title: "File types",
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Project types",
status: "Not applicable",
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
]);
});
});

it("warns if there is more than one planning constraints component", async () => {
const alteredFlow = {
...mockFlowData,
PlanningConstraints: {
type: 11,
data: {
title: "Check all constraints",
fn: "property.constraints.planning",
},
},
PlanningConstraintsTwo: {
type: 11,
data: {
title: "Check all constraints (dupe)",
fn: "property.constraints.planning",
},
},
};

queryMock.mockQuery({
name: "GetFlowData",
matchOnVariables: false,
data: {
flow: {
data: alteredFlow,
slug: "altered-flow-name",
team_id: 1,
team: {
slug: "testing",
},
publishedFlows: [{ data: alteredFlow }],
},
},
});

await supertest(app)
.post("/flows/1/diff")
.set(auth)
.expect(200)
.then((res) => {
expect(res.body.message).toEqual("Changes queued to publish");
expect(res.body.validationChecks).toEqual([
{
title: "Planning Constraints",
status: "Fail",
message:
"When using Planning Constraints, your flow must have exactly ONE Planning Constraints component",
},
{
title: "Sections",
status: "Pass",
message: "Your flow has valid Sections",
},
{
title: "Invite to Pay",
status: "Not applicable",
message: "Your flow is not using Invite to Pay",
},
{
title: "File types",
status: "Not applicable",
message: "Your flow is not using FileUpload or UploadAndLabel",
},
{
title: "Project types",
status: "Not applicable",
message:
'Your flow is not using Checklists which set "proposal.projectType"',
},
]);
});
});
Expand Down
34 changes: 25 additions & 9 deletions api.planx.uk/modules/gis/service/digitalLand.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,15 +71,30 @@
geom: string,
extras: Record<string, string>,
): Promise<GISResponse> {
// generate list of digital land datasets we should query based on 'active' planx schema variables
// generate list of digital land datasets we should query and their associated passport values
const activeDatasets: string[] = [];
Object.keys(baseSchema).forEach((key) => {
if (baseSchema[key]["active"]) {
baseSchema[key]["digital-land-datasets"]?.forEach((dataset: string) => {
activeDatasets.push(dataset);
});
}
});
let activeDataValues: string[] = [];
if (extras?.vals) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's tricky as these aren't 1:1 Planning.data datasets, but I think we should use a more meaningful param name here - maybe datasets, but this isn't ideal given the context here and the existing meaning datasets has.

We also need to add this new param to Swagger docs 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went with vals because the response includes a val property for each item - agree not most semantically meaningful though ! Continue to strongly oppose "datasets" though as that already corresponds to Planning Data source, not our curated constraint item which rely on one or many source datasets.

// if the editor selected constraints, prioritise their selection
activeDataValues = extras.vals.split(",");
Object.keys(baseSchema).forEach((key) => {
if (activeDataValues.includes(key)) {
baseSchema[key]["digital-land-datasets"]?.forEach((dataset: string) => {
activeDatasets.push(dataset);
});
}
});
} else {
// else default to the internally maintained list of all "active" datasets
Object.keys(baseSchema).forEach((key) => {
if (baseSchema[key]["active"]) {
activeDataValues.push(key);
baseSchema[key]["digital-land-datasets"]?.forEach((dataset: string) => {
activeDatasets.push(dataset);
});
}
});
}
jessicamcinchak marked this conversation as resolved.
Show resolved Hide resolved

// set up request query params per https://www.planning.data.gov.uk/docs
const options = {
Expand All @@ -97,8 +112,8 @@
options,
)}${datasets}`;
const res = await fetch(url)
.then((response: { json: () => any }) => response.json())

Check warning on line 115 in api.planx.uk/modules/gis/service/digitalLand.ts

View workflow job for this annotation

GitHub Actions / Run API Tests

Unexpected any. Specify a different type
.catch((error: any) => console.log(error));

Check warning on line 116 in api.planx.uk/modules/gis/service/digitalLand.ts

View workflow job for this annotation

GitHub Actions / Run API Tests

Unexpected any. Specify a different type

// if analytics are "on", store an audit record of the raw response
if (extras?.analytics !== "false") {
Expand Down Expand Up @@ -132,7 +147,7 @@
// check for & add any 'positive' constraints to the formattedResult
let formattedResult: Record<string, Constraint> = {};
if (res && res.count > 0 && res.entities) {
res.entities.forEach((entity: { dataset: any }) => {

Check warning on line 150 in api.planx.uk/modules/gis/service/digitalLand.ts

View workflow job for this annotation

GitHub Actions / Run API Tests

Unexpected any. Specify a different type
// get the planx variable that corresponds to this entity's 'dataset', should never be null because our initial request is filtered on 'dataset'
const key = Object.keys(baseSchema).find((key) =>
baseSchema[key]["digital-land-datasets"]?.includes(entity.dataset),
Expand All @@ -159,7 +174,8 @@
// TODO followup with digital land about how to return 'nots' via API (currently assumes any "active" metadata was successfully queried)
const nots = Object.keys(baseSchema).filter(
(key) =>
baseSchema[key]["active"] && !Object.keys(formattedResult).includes(key),
activeDataValues.includes(key) &&
!Object.keys(formattedResult).includes(key),
);
nots.forEach((not) => {
formattedResult[not] = {
Expand All @@ -180,7 +196,7 @@
formattedResult["designated.nationalPark"] &&
formattedResult["designated.nationalPark"].value
) {
formattedResult["designated.nationalPark"]?.data?.forEach((entity: any) => {

Check warning on line 199 in api.planx.uk/modules/gis/service/digitalLand.ts

View workflow job for this annotation

GitHub Actions / Run API Tests

Unexpected any. Specify a different type
if (
baseSchema[broads]["digital-land-entities"]?.includes(entity.entity)
) {
Expand Down
Loading
Loading