Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#175072065] EXPERIMENTAL: Start io-backend as Azure Function #722

Draft
wants to merge 32 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
93c5691
[#175072065] Start io-backend as Azure Function
BurnedMarshal Oct 20, 2020
8a8e8ae
[#175072065] Fix test on success unlockUserSession
BurnedMarshal Oct 20, 2020
9a5bd62
[#175072065] Use an existing url for test X_FORWARDED_PROTO_HEADER
BurnedMarshal Oct 20, 2020
67816b2
chore: release 7.5.0
balanza Oct 21, 2020
8028b3d
chore: release 7.6.0
balanza Oct 21, 2020
fa0147d
[#174877080] Upgrade to italia-utils 5.x (#711)
balanza Oct 27, 2020
70bc538
[#174877080] Upgrade io utils to v6 (#723)
balanza Oct 27, 2020
f59c56d
chore: release 7.7.0
balanza Oct 27, 2020
e641ee7
[#175072065] Remove DISABLE_BODY_PARSER env variables
BurnedMarshal Oct 27, 2020
94282ce
[#175072065] staticContentMiddleware become a middleware function
BurnedMarshal Oct 27, 2020
4a6d18a
[#175499507] update spid-commons (#725)
balanza Oct 29, 2020
6a7a1d8
chore: release 7.7.1
balanza Oct 29, 2020
07f66d5
[#175434645] Proxy abort user data processing (#724)
balanza Nov 2, 2020
95de2e8
chore: release 7.8.0
balanza Nov 2, 2020
a13fdeb
[#175679733] Add GetSupportToken API (#726)
AleDore Nov 20, 2020
26ccedf
[#175679733] Rename GetSupport Token endpoint (#727)
AleDore Nov 20, 2020
2e70348
chore: release 7.9.0
balanza Nov 24, 2020
2ddafb0
Update azure-pipelines.yml for Azure Pipelines
gunzip Nov 30, 2020
d2316d8
[#176003320] Return 500 instead of 401 on system errors (#728)
gunzip Dec 4, 2020
480050e
chore: release 7.9.1
Dec 4, 2020
4c11a09
add second app service (#729)
pasqualedevita Dec 5, 2020
7ec83a1
fix boolean (#730)
pasqualedevita Dec 5, 2020
61cb35a
fix parameters boolean (#731)
pasqualedevita Dec 5, 2020
a4fb41a
merged master
Dec 5, 2020
3ae06ab
[#175072065] Fix test on success unlockUserSession
BurnedMarshal Oct 20, 2020
7dba4e6
[#175072065] Use an existing url for test X_FORWARDED_PROTO_HEADER
BurnedMarshal Oct 20, 2020
52ee814
[#175072065] Remove DISABLE_BODY_PARSER env variables
BurnedMarshal Oct 27, 2020
08e37de
[#175072065] staticContentMiddleware become a middleware function
BurnedMarshal Oct 27, 2020
0f5a573
update yarn.lock
Dec 5, 2020
d4cf44a
update yarn.lock
Dec 5, 2020
972ede7
yarn.lock
Dec 5, 2020
d7312f2
yarn.lock
Dec 5, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .funcignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
*.js.map
*.ts
.git*
.vscode*
.circleci*
.prettierrc
local.settings.json*
tsconfig.json
tslint.json
README.md
yarn.lock
jest.config.js
__*
Dangerfile.js
CODEOWNERS
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Exclude Azure functions files
bin
obj

# Exclude OS Folders
.DS_Store

Expand Down Expand Up @@ -35,6 +39,7 @@ npm-debug.log

# Exclude certificates
*.pem
*.pfx

# Exclude local files
local.*
Expand Down
24 changes: 24 additions & 0 deletions StartBackend/function.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"route": "{*segments}",
"methods": [
"get",
"post",
"put",
"patch",
"delete"
]
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"scriptFile": "./index.js"
}
104 changes: 104 additions & 0 deletions StartBackend/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
import * as winston from "winston";

import { Context } from "@azure/functions";
import { secureExpressApp } from "io-functions-commons/dist/src/utils/express";
import { AzureContextTransport } from "io-functions-commons/dist/src/utils/logging";
import { setAppContext } from "io-functions-commons/dist/src/utils/middlewares/context_middleware";
import createAzureFunctionHandler from "io-functions-express/dist/src/createAzureFunctionsHandler";

/**
* Main entry point for the Digital Citizenship proxy.
*/

import { fromNullable } from "fp-ts/lib/Option";
import { newApp } from "../src/app";
import {
ALLOW_BPD_IP_SOURCE_RANGE,
ALLOW_MYPORTAL_IP_SOURCE_RANGE,
ALLOW_NOTIFY_IP_SOURCE_RANGE,
ALLOW_PAGOPA_IP_SOURCE_RANGE,
ALLOW_SESSION_HANDLER_IP_SOURCE_RANGE,
API_BASE_PATH,
AUTHENTICATION_BASE_PATH,
BONUS_API_BASE_PATH,
BPD_BASE_PATH,
DEFAULT_APPINSIGHTS_SAMPLING_PERCENTAGE,
ENV,
MYPORTAL_BASE_PATH,
PAGOPA_BASE_PATH
} from "../src/config";
import { initAppInsights } from "../src/utils/appinsights";
import {
getCurrentBackendVersion,
getValueFromPackageJson
} from "../src/utils/package";

const authenticationBasePath = AUTHENTICATION_BASE_PATH;
const APIBasePath = API_BASE_PATH;
const BonusAPIBasePath = BONUS_API_BASE_PATH;
const PagoPABasePath = PAGOPA_BASE_PATH;
const MyPortalBasePath = MYPORTAL_BASE_PATH;
const BPDBasePath = BPD_BASE_PATH;

/**
* If APPINSIGHTS_INSTRUMENTATIONKEY env is provided initialize an App Insights Client
* WARNING: When the key is provided several information are collected automatically
* and sent to App Insights.
* To see what kind of informations are automatically collected
* @see: utils/appinsights.js into the class AppInsightsClientBuilder
*/
const maybeAppInsightsClient = fromNullable(
process.env.APPINSIGHTS_INSTRUMENTATIONKEY
).map(k =>
initAppInsights(k, {
applicationVersion: getCurrentBackendVersion(),
cloudRole: getValueFromPackageJson("name"),
disableAppInsights: process.env.APPINSIGHTS_DISABLED === "true",
samplingPercentage: process.env.APPINSIGHTS_SAMPLING_PERCENTAGE
? parseInt(process.env.APPINSIGHTS_SAMPLING_PERCENTAGE, 10)
: DEFAULT_APPINSIGHTS_SAMPLING_PERCENTAGE
})
);

// tslint:disable-next-line: no-let
let logger: Context["log"] | undefined;
const contextTransport = new AzureContextTransport(() => logger, {
level: "debug"
});
winston.add(contextTransport);

// Setup Express
const init = newApp({
APIBasePath,
BPDBasePath,
BonusAPIBasePath,
MyPortalBasePath,
PagoPABasePath,
allowBPDIPSourceRange: ALLOW_BPD_IP_SOURCE_RANGE,
allowMyPortalIPSourceRange: ALLOW_MYPORTAL_IP_SOURCE_RANGE,
allowNotifyIPSourceRange: ALLOW_NOTIFY_IP_SOURCE_RANGE,
allowPagoPAIPSourceRange: ALLOW_PAGOPA_IP_SOURCE_RANGE,
allowSessionHandleIPSourceRange: ALLOW_SESSION_HANDLER_IP_SOURCE_RANGE,
appInsightsClient: maybeAppInsightsClient.toUndefined(),
authenticationBasePath,
env: ENV
}).then(app => {
secureExpressApp(app);
return {
app,
azureFunctionHandler: createAzureFunctionHandler(app)
};
});

// Binds the express app to an Azure Function handler
function httpStart(context: Context): void {
logger = context.log;
init
.then(({ app, azureFunctionHandler }) => {
setAppContext(app, context);
azureFunctionHandler(context);
})
.catch(context.log);
}

export default httpStart;
13 changes: 13 additions & 0 deletions extensions.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<AzureFunctionsVersion>v3</AzureFunctionsVersion>
<WarningsAsErrors></WarningsAsErrors>
<DefaultItemExcludes>**</DefaultItemExcludes>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.2.2" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="4.0.1" />
<PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.1.8" />
</ItemGroup>
</Project>
16 changes: 16 additions & 0 deletions host.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"version": "2.0",
"logging": {
"logLevel": {
"default": "Information"
}
},
"extensions": {
"http": {
"routePrefix": ""
},
"durableTask": {
"hubName": "%SLOT_TASK_HUBNAME%"
}
}
}
5 changes: 5 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@
"prettify": "prettier --write \"./**/*.ts\"",
"test": "jest -i",
"test:coverage": "dotenv -e .env.example -- jest -i --coverage",
"prestart:func": "func extensions install --javascript",
"start": "node src/server.js",
"start:func": "dotenv -e .env func start --javascript --useHttps --cert certs/cert.pfx --password secret",
"generate:proxy-models": "npm-run-all generate:proxy:* generate:api:* generate:messages:*",
"generate:proxy:api-models": "rimraf generated/backend && shx mkdir -p generated/backend && gen-api-models --api-spec api_backend.yaml --out-dir generated/backend",
"generate:proxy:auth-models": "rimraf generated/auth && shx mkdir -p generated/auth && gen-api-models --api-spec api_auth.yaml --out-dir generated/auth",
Expand Down Expand Up @@ -55,6 +57,7 @@
},
"homepage": "https://github.com/pagopa/io-backend#readme",
"dependencies": {
"@azure/functions": "^1.2.2",
"@azure/storage-queue": "^12.0.0",
"@pagopa/io-spid-commons": "^4.7.0",
"apicache": "^1.4.0",
Expand All @@ -68,6 +71,7 @@
"fp-ts": "~1.17.0",
"helmet": "3.21.1",
"http-graceful-shutdown": "^2.3.2",
"io-functions-commons": "^15.0.0",
"io-ts": "1.8.5",
"io-ts-types": "^0.4.7",
"italia-ts-commons": "^8.3.0",
Expand Down Expand Up @@ -97,6 +101,7 @@
"@types/helmet": "^0.0.43",
"@types/jest": "^23.3.3",
"@types/lolex": "2.1.3",
"@types/mime": "^1.3.1",
"@types/morgan": "^1.7.35",
"@types/node": "10.14.2",
"@types/node-fetch": "^2.1.2",
Expand Down
1 change: 1 addition & 0 deletions scripts/generate-test-certs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ else
echo "Generating certificates in $1"
cd $1
openssl req -sha256 -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -subj "/C=IT/ST=Italy/L=Rome/O=ACME/OU=IT Department/CN=api.italia.local"
openssl pkcs12 -inkey key.pem -in cert.pem -export -out cert.pfx -password pass:secret
fi
2 changes: 1 addition & 1 deletion src/__tests__/app.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ describe("Success app start", () => {
// test case: https forced. Already set: it trust the proxy and accept the header: X-Forwarded-Proto.
it("should respond 200 if forwarded from an HTTPS connection", () => {
return request(app)
.get("/")
.get("/info")
.set(X_FORWARDED_PROTO_HEADER, "https")
.expect(200);
});
Expand Down
44 changes: 37 additions & 7 deletions src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
appConfig,
BONUS_API_CLIENT,
CACHE_MAX_AGE_SECONDS,
DISABLE_BODY_PARSER,
ENV,
FF_BONUS_ENABLED,
getClientProfileRedirectionUrl,
Expand Down Expand Up @@ -54,6 +55,7 @@ import ServicesController from "./controllers/servicesController";
import SessionController from "./controllers/sessionController";
import UserMetadataController from "./controllers/userMetadataController";

import * as mime from "mime";
import { log } from "./utils/logger";
import checkIP from "./utils/middleware/checkIP";

Expand All @@ -64,6 +66,7 @@ import * as appInsights from "applicationinsights";
import { tryCatch2v } from "fp-ts/lib/Either";
import { isEmpty, StrMap } from "fp-ts/lib/StrMap";
import { fromLeft, taskEither, tryCatch } from "fp-ts/lib/TaskEither";
import * as fs from "fs";
import { VersionPerPlatform } from "../generated/public/VersionPerPlatform";
import BonusController from "./controllers/bonusController";
import SessionLockController from "./controllers/sessionLockController";
Expand Down Expand Up @@ -131,7 +134,7 @@ export interface IAppFactoryParameters {
BPDBasePath: string;
}

// tslint:disable-next-line: no-big-function
// tslint:disable-next-line: cognitive-complexity no-big-function
export function newApp({
env,
allowNotifyIPSourceRange,
Expand Down Expand Up @@ -250,18 +253,45 @@ export function newApp({
//
// Setup parsers
//
if (!DISABLE_BODY_PARSER) {
// Parse the incoming request body. This is needed by Passport spid strategy.
app.use(bodyParser.json());

// Parse the incoming request body. This is needed by Passport spid strategy.
app.use(bodyParser.json());

// Parse an urlencoded body.
app.use(bodyParser.urlencoded({ extended: true }));
// Parse an urlencoded body.
app.use(bodyParser.urlencoded({ extended: true }));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does SPID login work without this?

}

//
// Define the folder that contains the public assets.
//

app.use(express.static("public"));
app.get(/(\.html|\.svg|\.png)$/, async (req, res, next) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the result of this call can be cached using appcache, what about this?

try {
const path = "public" + req.path;
if (!fs.existsSync(path)) {
return next();
}
const stat = await fs.promises.stat(path);
if (stat.isDirectory()) {
return next();
}
const type = mime.lookup(path);
if (type) {
const charset = mime.charsets.lookup(type);
res.setHeader(
"Content-Type",
// tslint:disable-next-line: restrict-plus-operands
type + (charset ? `; charset=${charset}` : "")
);
}
const content = await fs.promises.readFile(path);
res.setHeader("Content-Length", stat.size);
res.status(200).send(content);
} catch (err) {
log.error(`static|Error retrieving static file asset|error:%s`, err);
return next(err);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me why we introduced this behaviour. Besides that, we can put this in a middleware function (req, res, next)=>void and we can give this chunk an explicit meaning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is required because the express default static folder (ref) doesn't work inside Azure functions. Here we emulate what static middleware does. We can create a specific middleware, but this code will be used only on this endpoint.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can just move it in a function declared inside the same module

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 94282ce

});

//
// Initializes Passport for incoming requests.
Expand Down
6 changes: 6 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -380,5 +380,11 @@ export const TEST_LOGIN_PASSWORD = NonEmptyString.decode(
process.env.TEST_LOGIN_PASSWORD
);

// When the application is running as Azure Function
// Express Body parser must be disabled.
export const DISABLE_BODY_PARSER = fromNullable(process.env.DISABLE_BODY_PARSER)
.map(_ => _.toLocaleLowerCase() === "true")
.getOrElse(false);
balanza marked this conversation as resolved.
Show resolved Hide resolved

// Feature flags
export const FF_BONUS_ENABLED = process.env.FF_BONUS_ENABLED === "1";
4 changes: 1 addition & 3 deletions src/controllers/__tests__/sessionLockController.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,6 @@ describe("SessionLockController#lockUserSession", () => {
});
});



describe("SessionLockController#unlockUserSession", () => {
it("should fail on invalid fiscal code", async () => {
const req = mockReq({ params: { fiscal_code: "invalid" } });
Expand Down Expand Up @@ -175,7 +173,7 @@ describe("SessionLockController#unlockUserSession", () => {
mockRedisUserMetadataStorage
);

const response = await controller.lockUserSession(req);
const response = await controller.unlockUserSession(req);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😅

response.apply(res);

expect(res.status).toHaveBeenCalledWith(200);
Expand Down
10 changes: 4 additions & 6 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,12 @@
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"sourceMap": true,
"baseUrl": "."
"baseUrl": ".",
"rootDir": "."
},
"include": [
"./src/**/*"
],
"exclude": [
"node_modules",
"src/**/__tests__/*",
"src/**/__mocks__/*"
"__tests__",
"__mocks__"
]
}
Loading