Support for Serverless

[Nelrohd]

Is your feature request related to a problem? Please describe.

Currently next-i18next does not support serverless. It appears that the i18n-middleware create some troubles when used on lambda #271

Describe the solution you'd like

Not sure exactly what would be the final solution but:

• Support i18n without needing custom server.
• Support with custom server like express (note I didn't find an official repo for NextJS V8 serverless + express).

Describe alternatives you've considered

None.

Additional context

Open to suggestion for the work/final solution to be done/achieved.

[isaachinman]

A few things to consider - just my initial thoughts:

• Where would locales/translated content live? Seems we are going to have problems with a filesystem approach
• How would we support locale subpaths, if at all?
• How would we replace all the functionality that i18next-express-middleware gives us? See nextI18NextMiddleware for Koa? #9 (comment)

I am open to any/all suggestions on this topic, but in general regard it as a major refactor of the project from the ground up. If we can remove our dependency on Express/i18next-express-middleware, I think the package would become a lot more versatile.

However, I am not (currently) doing any work whatsoever with serverless NextJs, and have very little input to add here.

[beheh]

I've gotten next-i18next working in a serverless environment for one of our sites we're building right now (no full source available unfortunately), but it's pretty hacky because we're basically stitching together various parts from next-i18next in a more vanilla i18next setup.

Our approach is that we basically ship the JSON files once to a CDN (with an assetPrefix) and once inside the Lambda "executables". We reference them via import which makes them real Webpack chunks for the CDN and also allows it to pick them up and include them when packaging the serverless file and we're also running express infront of the bundles so the middleware can just be dropped. I can't really recommend this approach, but it can definitely work.

[isaachinman]

@beheh Sounds like you had some fun. Are there any lessons or approaches you'd like to bring back to next-i18next? Being able to support serverless is the eventual goal.

[yoeran]

Stumbled upon this issue because I'm researching for a new NextJS project that should have i18n and is planned to be deployed to Now 2.0. I've not used next-i18next yet, nor am I a serverless/now expert, but if someone has some smart ideas I'm willing to help with the implementation.

[isaachinman]

@yoeran I think the right place to start is rewriting/adapting i18next-express-middleware to use vanilla NodeJs HTTP interfaces instead of relying on Express.

[flearsnetwork]

Would also appreciate it if someone has the knowledge to optimize the package for serverless solutions.

[gurkerl83]

[RFC] serverless is still open (vercel/next.js#7208). The last entry mentions that a routing middleware is still missing in the current version of Next. Tracking the progress there is a merge request with the title "Example for API routes with middleware" https://github.com/zeit/next.js/pull/7958/files . I do not know anything about the internal of the express middleware currently utilized by next-i18next, but maybe the example shown in the merge request is enough to start a migration.

[isaachinman]

@gurkerl83 That example appears to show how one could use middleware on an API route, not a UI route.

[emirotin]

My request sounds different from the original title but I assume it's related cause it's also about reusing the module outside of next execution context. Sorry if I'm wrong though.

We have a successfully working Next.js project using your next-i18next (thanks!).

Now I also want to create a stand-alone node.js script that would reuse the entire infrastructure we already have and do some interpolations (in the static HTML files if you're curious).

For some reason (I have my brain kinda broken after hours of trying) it doesn't work.

Here's what I'm doing:

const nextConfig = require('next/config');
nextConfig.setConfig(require('./next.config'));

const { i18n } = require('./i18n');

And then trying to use i18n.t directly. I see that it's getting all the proper configs but does not load the data from the JSON files.

And thus an attempt to translate any key returns the key itself.

I'm trying to figure out what else next is doing behind the scene that makes your module fetch the needed JSON data and that I'm missing.

---

For ref, ./i18n is pretty standard:

const NextI18Next = require('next-i18next').default;
const i18nHelper = require('./lib/i18nHelper');

module.exports = new NextI18Next({
    localePath: 'static-src/locales',
    defaultLanguage: i18nHelper.getDefaultLanguage(),
    otherLanguages: i18nHelper.getOtherLanguages(),
    fallbackLng: i18nHelper.getDefaultLanguage(),
    defaultNS: i18nHelper.getDefaultNamespace(),
    fallbackNS: i18nHelper.getAllNamespaces(),
    ns: i18nHelper.getAllNamespaces(),
    localeSubpaths: 'foreign'
});

And I can confirm that all the i18nHelper methods are returning what expected.

[isaachinman]

@emirotin That's not something we're going to support. Unfortunately you'll need to go it alone. I would suggest simply initialising a normal i18next instance. Note that you can pass your identical next-i18next config object into the i18next.init function.

Let's keep this discussion on topic.

[emirotin]

Sorry for offtopic then. I truly assumed it's about the same root.
Just curious what's wrong with my current setup that makes it unusable?

P.S. barebones i18next doesn't work for me (yet) too normal i18next works!

[cryptiklemur]

It seems possible to spoof this with the following in a custom _app.js

import {handle} from 'i18next-express-middleware';
import {i18n, config} from '../i18n';

export default extends App {
    // ...    
    static async getInitialProps(context) {
        if (context.ctx.req && context.ctx.res) {
            res.set = res.setHeader;
            req.path = context.ctx.asPath;

            const i18nHandler = handle(i18n, {ignoreRoutes: config.ignoreRoutes});
            await new Promise((resolve, reject) => {
                try {
                    i18nHandler(req as any, res as any, resolve);
                } catch (e) {
                    reject(e);
                }
            });
        }
    }
    // ...    
}

Hopefully ZEIT follows through with adding middleware capabilities, and we can easily add this that way

[cryptiklemur]

If this is something you are cool with (and actually works? I'm still testing this), I can throw up a PR to introduce a helper function (plus some documentation)

[isaachinman]

@aequasi Not sure that'd work. The req and res objects that come off Next's context are vanilla HTTP interfaces, not Express req and res interfaces. What have you seen in your testing?

Because there are so many diverse use cases for this package, I'd rather we proceed in the least hacky way possible. We do have support from the Next team and they're aware of the request for serverless middleware.

[cryptiklemur]

The two express specific things (set and path) can easily be "mocked"

set actually needs to be:

req.set = (key, value) => {
    if (value !== undefined) {
        res.setHeader(key, value);
    }
}

[cryptiklemur]

From my testing, this works. I'm not having any issues with server-rendered content

[isaachinman]

To reiterate, I'd rather we proceed in the least hacky way possible. This means we need to:

1. Wait for serverless middleware support within NextJs
2. Fork i18next-express-middleware and refactor out the Express dep, and release a new package, eg i18next-node-middleware
3. Refactor next-i18next accordingly

If people want to roll their own serverless solutions in the meantime, feel free.

[samohovets]

@aequasi nice temporary solution 🤔 But I guess locale subpaths is not working, right?

UPD: yes, server-side locale subpaths is not working (naturally), but it seems fine on the client. I got client-side translations working so far without particularly critical bugs, but it can't work on Zeit Now serverless 🤔

Monkey patching all incoming requests in _app.jsx is not a best way to get things working, so as @isaachinman mentioned - we need to wait for serverless middlware support in Next.js 😞

[cryptiklemur]

Zeit compiles everything that it can find from static analysis into a single file. If i had to make an educated guess, the locales are not making it to the deployment, causing the fs.readdirSync to fail. You could look into: https://zeit.co/docs/v2/advanced/builders/overview#including-additional-files

[gurkerl83]

@aequasi I experienced the same error when a deployment with now get executed. With your last comment, do you mean specifying i18n-resource files within the includeFiles section of now deployment configuration (now.json) will resolve the fs.readdirSync error?

[isaachinman]

This comment: vercel/next.js#13624 (comment) potentially sheds some light on how to get Vercel to pick up filesystem deps.

@adrai I'm back to running into i18next module resolution issues:

2020-07-08T09:30:10.473Z	4e8d3e9a-1479-4f9d-b650-e1ab625ea951	ERROR	Unhandled error during request: Error: Cannot find module 'i18next-fs-backend/cjs'
Require stack:
- /var/task/node_modules/next-i18next/dist/commonjs/create-i18next-client.js
- /var/task/node_modules/next-i18next/dist/commonjs/index.js

Fairly sure this was previously working. Any ideas?

[adrai]

I'm back to running into i18next module resolution issues:

strange... is this reproducable with the example in the repo? Can't reproduce it.

[isaachinman]

@adrai I have been deploying this repo to Vercel. Feel free to fork and deploy. This issue isn't reproducible locally – it's definitely something to do with their strict module resolution.

[borispoehland]

I'm back to running into i18next module resolution issues:

strange... is this reproducable with the example in the repo? Can't reproduce it.

It only occurs when deploying to vercel. Here is a project that basically runs minimal configuration and has the filepath already included: https://github.com/borispoehland/next-i18next-boilerplate

Edit: Take Isaac's version

[adrai]

It only occurs when deploying to vercel.

Then they changed something again... 🤷‍♂️
Maybe now requiring /cjs is not needed anymore?

[adrai]

Seems eval('require...') is ignored by their bundler.... there are also others having this problem: saltyshiomix/nextron#78
vercel/next.js#13736

[adrai]

Probably adding something like this would force to bundle it:

const path = require('path');
path.resolve('./node_modules/i18next-fs-backend/')

🤷‍♂️

[borispoehland]

Probably adding something like this would force to bundle it:

const path = require('path');
path.resolve('./node_modules/i18next-fs-backend/')

🤷‍♂️

Sorry for the newbie question, but what stops you from importing the modules with the regular import syntax (instead of the eval(require...))?
Because with require they are handled as node modules?

[adrai]

I imagine importing them would bundle these dependencies also for the client...
Alternatively a dynamic import could work... but then that function would have been rewritten to be async...
@isaachinman is this correct?

[isaachinman]

Guess what? I just deployed a serverless next-i18next project successfully on Vercel:

https://next-i18next-vercel.now.sh/

Can we get some early adopters, of users both:

1. Deploying serverless projects with next-i18next@beta
2. Deploying custom server (Express) projects with next-i18next@beta

We want to make sure that both options are supported well and free of any major bugs.

[borispoehland]

Yeah, that suggestion works @borispoehland. The eval usage there is a nasty hack to exclude those deps from client bundles. I've always been open to other solutions, but no one has come forward.

Guess what? I just deployed a serverless next-i18next project successfully on Vercel:

https://next-i18next-vercel.now.sh/

Can we get some early adopters, of users both:

1. Deploying serverless projects with next-i18next@beta
2. Deploying custom server (Express) projects with next-i18next@beta

We want to make sure that both options are supported well and free of any major bugs.

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

[isaachinman]

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

I wasn't talking about Vercel. This project needs to support deployment on various platforms, and needs to maintain support for users that rely on custom servers.

[borispoehland]

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

I wasn't talking about Vercel. This project needs to support deployment on various platforms, and needs to maintain support for users that rely on custom servers.

I understand

[borispoehland]

@isaachinman the updated beta is not pushed yet, right?

[isaachinman]

@borispoehland What updates to the beta release are you expecting? I'm not convinced that platform-specific hacks like using path.resolve belong in the next-i18next source.

[iduuck]

Are you able to share the source code for how you solved the issue? The path.resolve('./node_modules/i18next-fs-backend/') "hack" does not tell me a lot.

[isaachinman]

The source code is here: https://github.com/isaachinman/next-i18next-vercel

[borispoehland]

@borispoehland What updates to the beta release are you expecting? I'm not convinced that platform-specific hacks like using path.resolve belong in the next-i18next source.

Your're absolutely right. Better put it in the i18n.js.

[isaachinman]

Update: I've spoken to Joe in the NextJs team, and we've identified fixes for both the locales dir path, and i18next-fs-backend. I will be updating next-i18next@beta such that no path.resolve tricks will be necessary at all – should have this done by the end of the week.

[adrai]

Update: I've spoken to Joe in the NextJs team, and we've identified fixes for both the locales dir path, and i18next-fs-backend. I will be updating next-i18next@beta such that no path.resolve tricks will be necessary at all – should have this done by the end of the week.

Cool, let me know if there is something that need to be done for other i18next modules...

[isaachinman]

So, I’ve undertaken the following changes:

1. Refactor localePath to be an absolute path (requirement), which users can pass via path.resolve. This is to get around bundling technologies used by Vercel, and other serverless platforms. The way that path.resolve falls back in browser envs is critical.
2. Refactor the create-client code to have two entry points: one for Node, and one for browsers. This way we can stop using eval hacks, and allow tools to properly analyse our dependencies (fixes the various i18next dep issues).

Changes here: 826a9a5.

I then released [email protected] and updated my example repo:

https://github.com/isaachinman/next-i18next-vercel

It's running smoothly on Vercel here:

https://next-i18next-vercel.now.sh/

So: we can now release next-i18next projects on Vercel (and other serverless platforms) without any compromises or hacks, straight out of the box. The main breaking change will be the requirement of path.resolve for localePath.

Please do let me know if anyone has any questions, or if any of you are able to beta test this release. I've been told by the NextJs team that rewrites will be stable very soon, so in the very near future this functionality will be available without the experimental flag.

I am going to (finally) close this issue in the near future, as serverless support is genuinely here. If anyone encounters bugs/problems with the serverless beta release, please open new issues as you normally would.

Thanks everyone!

[Progressandro]

So, I’ve undertaken the following changes:

1. Refactor localePath to be an absolute path (requirement), which users can pass via path.resolve. This is to get around bundling technologies used by Vercel, and other serverless platforms. The way that path.resolve falls back in browser envs is critical.
2. Refactor the create-client code to have two entry points: one for Node, and one for browsers. This way we can stop using eval hacks, and allow tools to properly analyse our dependencies (fixes the various i18next dep issues).

Changes here: 826a9a5.

I then released [email protected] and updated my example repo:

https://github.com/isaachinman/next-i18next-vercel

It's running smoothly on Vercel here:

https://next-i18next-vercel.now.sh/

So: we can now release next-i18next projects on Vercel (and other serverless platforms) without any compromises or hacks, straight out of the box. The main breaking change will be the requirement of path.resolve for localePath.

Please do let me know if anyone has any questions, or if any of you are able to beta test this release. I've been told by the NextJs team that rewrites will be stable very soon, so in the very near future this functionality will be available without the experimental flag.

I am going to (finally) close this issue in the near future, as serverless support is genuinely here. If anyone encounters bugs/problems with the serverless beta release, please open new issues as you normally would.

Thanks everyone!

Hey, thanks for coming up with an equally straightforward solution for the serverless option as well.

I'm trying to implement the update into my new project, however, I'm finding the following issue when importing the i18n file into any page-level component:

Error was not caught TypeError: path.resolve is not a function

I'm not a veteran with Next.js, but I don't see any obvious problem with my code.

my i18n.js:

const path = require('path')
const NextI18Next = require('next-i18next').default
const { localeSubpaths } = require('next/config').default().publicRuntimeConfig
module.exports = new NextI18Next({
  defaultLanguage: 'en',
  otherLanguages: ['es'],
  localeSubpaths,
  localePath: path.resolve('./public/locales'),
})

My "/test" page-level component:

import React from 'react'
import PropTypes from 'prop-types'
import { i18n, withTranslation } from '../i18n'
const Test = ({ t, namespacesRequired }) => {
  return (
    <React.Fragment>
      <main>
        <div>
          <button
            type="button"
            onClick={() => {
              console.log(i18n)
              i18n.changeLanguage(i18n.language === 'en' ? 'de' : 'en')
            }}
          >
            Change
          </button>
          <h1>{t('title')}</h1>
        </div>
      </main>
    </React.Fragment>
  )
}

Test.getInitialProps = async () => ({
  namespacesRequired: ['common'],
})

Test.propTypes = {
  t: PropTypes.func.isRequired,
}

export default withTranslation('common')(Test)

My next.config.js:

const withPlugins = require('next-compose-plugins');
const withOptimizedImages = require('next-optimized-images');
const withFonts = require('next-fonts');
const FilterWarningsPlugin = require('webpack-filter-warnings-plugin');
const { nextI18NextRewrites } = require('next-i18next/rewrites')

const localeSubpaths = {
  es: 'es'
}

const nextConfig = {
  env: {
    GOOGLE_API_KEY: 'API_KEY',
    REACT_APP_GOOGLE_MAP_API_KEY:
      'https://maps.googleapis.com/maps/api/js?v=3.exp&key=API_KEY&libraries=geometry,drawing,places',
    SERVER_API: `http://localhost:3001`,
  },
  webpack: (config, { isServer }) => {
    config.plugins.push(
      new FilterWarningsPlugin({
        exclude: /mini-css-extract-plugin[^]*Conflicting order between:/,
      })
    );
    config.resolve.modules.push(__dirname);

    return config;
  },
  publicRuntimeConfig: {
    localeSubpaths,
  },
  experimental: {
    async rewrites() {
      return [
        ...nextI18NextRewrites(localeSubpaths)
      ]
    }
  }
};

module.exports = withPlugins(
  [
    [
      withOptimizedImages,
      {
        mozjpeg: {
          quality: 90,
        },
        webp: {
          preset: 'default',
          quality: 90,
        },
      },
    ],
    withFonts,
  ],
  nextConfig
);

If you have any other question, I'll be glad to provide more information if possible.

Thanks in advance! 💪

[Lanny]

Thanks a ton @isaachinman , was able to deploy the 5.0.0-beta.3 to netlify so just wanted to chime in that other serverless type hosts seem to be working. I did have to explicitly list out my namespaces in the ns value to the NextI18Next constructor, otherwise it would call getAllNamespaces and try to read from the filesystem which failed in a serverless environment. Not sure if this was the intended behavior, or I misconfigured something, or it's a bug but listing out NSs isn't too bad and it's great to be able to run under the serverless target.

[lazidoca]

Thanks a ton @isaachinman , was able to deploy the 5.0.0-beta.3 to netlify so just wanted to chime in that other serverless type hosts seem to be working. I did have to explicitly list out my namespaces in the ns value to the NextI18Next constructor, otherwise it would call getAllNamespaces and try to read from the filesystem which failed in a serverless environment. Not sure if this was the intended behavior, or I misconfigured something, or it's a bug but listing out NSs isn't too bad and it's great to be able to run under the serverless target.

I just deployed to Netlify and it seems to return 404 when accessing the locale URL directly.

The demo is at: https://eager-shannon-a9d1c4.netlify.app/

If you access this URL, it shows 404

https://eager-shannon-a9d1c4.netlify.app/de/second-page

The repository for the demo is forked from @isaachinman with adding target: "serverless" https://github.com/welrn/next-i18next-vercel

[isaachinman]

@Progressandro @Lanny @lazidoca – as I wrote, please open separate issues.

[isaachinman]

Hey all – NextJs v9.5 came out today, and rewrites are no longer experimental. I've therefore just released [email protected], which supports serverless. Enjoy!