Handling CDN Stage

[auniverseaway]

Challenge

Historically, we have said:

https://business.stage.adobe.com      >>>   https://main--bacom--adobecom.hlx.page
https://www.stage.adobe.com/acrobat   >>>   https://main--dc--adobecom.hlx.page/acrobat

Note: Akamai Stage URLs point to "staged" content (i.e. hlx.page)

https://business.adobe.com      >>>   https://main--bacom--adobecom.hlx.live
https://www.adobe.com/acrobat   >>>   https://main--dc--adobecom.hlx.live/acrobat

Note: Akamai Prod URLs point to "prod" content (i.e. hlx.live)

In both scenarios, we point to the "main" branch which is considered prod / live.

The wisdom was, "keeping two branches in sync is a pain. Have both stage and prod point to your main branch." Everyone merges into main, and you can test your changes in your branch on hlx.page or hlx.live.

I think this is wrong going forward despite knowing it will cause additional QE and dev overhead. Why?

If a feature is built in milo, and an engineer points to /my-file.js or /my-asset.svg there's not a great way to cover every scenario where something would work on hlx.(page|live) and not on www.adobe.com. In both these scenarios, because the files references are not in /libs they will not be available on Akamai.

Additionally, there are not good methods (for now) to do .html testing... really, any testing that would maybe be impacted by the CDN.

My proposal

https://business.stage.adobe.com      >>>   https://main--bacom--adobecom.hlx.live
https://www.stage.adobe.com/acrobat   >>>   https://main--dc--adobecom.hlx.live/acrobat

https://business.adobe.com            >>>   https://prod--bacom--adobecom.hlx.live
https://www.adobe.com/acrobat         >>>   https://prod--dc--adobecom.hlx.live/acrobat

There are two changes here:

1. Stage now points to prod content.
2. Prod now points to a prod branch.

I think this solves a couple challenges we have faced:

1. Content migrations where stage is being used to check content viability (if it's live) is much better at surfacing this because the content must be previewed and published.
2. We can now do validations on code changes in an Akamai environment.
3. hlx.page is going to require authentication which means Akamai should no longer point to hlx.page.

Who this impacts

Mostly engineers and QEs. The way we validate code changes will change. The person who is on the hook to verify a PR would now be responsible for more tasks:

1. Validating the PR.
2. Merging into main.
3. Validating our first-tier consumers on Akamai.
4. Merging into 'prod`.
5. Validating on prod.

There's also the tooling aspect for Nala that would need to be handled. We are currently doing prod and PRs + hlx.live... we'd want to expand into stage as well... which would require a CI platform add.

[davidnuescheler]

the main branch is special as it is the only branch that triggers the push invalidation for the BYOCDN on code merges.

so, i think it would be easier with the current setup, if you introduced a stage branch, assuming that https://business.stage.adobe.com and https://www.stage.adobe.com/acrobat could easily be setup in a way to have push invalidation not setup and have a TTL defaulting to 5 min.

[JackySun9]

I have some questions on this:

1. validating the PR, we will validate on https://business.stage.adobe.com + https://stage--bacom--adobecom.hlx.live, right?
2. Merging into main, then we will validate, https://business.stage.adobe.com >>> https://main--bacom--adobecom.hlx.live
3. What about validating our first-tier consumers on Akamai?
4. if I want to validate content for stage, what should I do?

[apganapa]

Echo'ing @JackySun9 4th point. What is the best way to test in stage domain the content which are not supposed to be live yet? There could be 100 documents which are previewed but not published since they are not supposed to be live.
Testing just on ".page" domain would not give complete E2E flow behavior.
What needs to be done to test these content on "stage.adobe.com" domain? @davidnuescheler @auniverseaway

In Express, we have "stage" branch with code changes pushed to "stage branch" so codewise we have all the required code to load on "stage.adobe.com" domain.

Question is just w.r.t content which should not be published but verified on "stage.adobe.com" domain.
At present "stage.adobe.com/express" is pointing to "https://stage--express-website--adobe.hlx.page" so that we can accomplish the above ask.

[apganapa]

Proposal: @auniverseaway @davidnuescheler
Should we have a button "publish to stage" and have ".prelive" apart from ".page" , "live" ? this way clicking on "publish to stage" sends the stuff to ".prelive" origin, another cluster like "live" but something which we can use to connect from Stage Akamai.
So that,
"stage.adobe.com" Akamai domain points to ".prelive" origin? like "https://stage--express-website--adobe.hlx.prelive"
and "adobe.com" points to usual "live" domain.

that way, ".page" can be under firewall and outside the reach of Akamai.

[JackySun9]

From myside, I am using jenkins job to run test automation on "stage.adobe.com", here is an example: https://acom.ci.corp.adobe.com/view/Milo/job/MiloSingleUITestCase/

[auniverseaway]

@apganapa I think this over-complicates things. I think it would also be incompatible with how GWP has asked us to lock down content.

There's three types of testing I see needed on a site:

1. Content testing - Testing content changes (staged content)
2. Code testing - Testing code changes
3. CDN testing - Testing changes impacted by CDN (.html / libs).

In some cases, content & code need to be tested together, and I think you have that use case:

{your-feature-branch}--express-website--adobe.hlx.page

In this case, you can test "staged" content against any branch you need. I don't see a scenario where you would need to also test CDN changes at this point.

You can also test backwards compatibility by using prod / live content:

{your-feature-branch}--express-website--adobe.hlx.live

Worth noting: the protections we put in place on staged content are incompatible with CDN. Depending on how this is setup, you either need to be behind a firewall, or you need to have logged into MS's IDP. Neither of these are Akamai friendly. So, it's a non-starter to try and put staged content into the CDN if your staged content is protected. This would also apply even for a hypothetical "prelive" TLD.