Edge cases with FormData body conversion

[ntjess]

Description

When hey-api converts a body payload into a form for multipart/form-data, nested arrays with one element lose their distinction as nested upon serialization. Instead, they look like the unnested data type. It is due to this code (and its equivalent for other backends):

openapi-ts/packages/client-axios/src/utils.ts

Line 498 in 7098555

value.forEach((v) => serializeFormDataPair(formData, key, v));

When only one value exists, the form has no indication to wrap the inner element as a list and instead treats it as a single list

Reproducible example or configuration

repro-heyapi.zip

• Ensure fastapi is installed with python >=3.10 as the server (optional, only to verify the 422 unprocessible entity error)
• Ensure npm packages are available (provided in the zip for node 22.5.1)
• Execute ./run_frontend_backend.sh and open localhost:5173 in browser
• Click the button and observe the POST data is incorrectly formatted (list[int] is created, not list[list[int]])

To regenerate the hey-api client api, ensure the fastapi server is running (it provides the openapi file), and run npx openapi-ts

OpenAPI specification (optional)

3.1.1

System information (optional)

macos

[ntjess]

Minor update: main.ts:16 should POST [[7]], not [7]. The 422 error still occurs when posting the correct data.

[mrlubos]

Hi @ntjess, are you able to provide the relevant part of your OpenAPI spec?

[ntjess]

The problem is a schema of number[][] doesn't properly convert a list with a single element ([[7]]) to form data.

Schema snippet:

  "Body_print_numbers__post": {
    "properties": {
      "numbers": {
        "items": {
          "items": {
            "type": "integer"
          },
          "type": "array"
        },
        "type": "array",
        "title": "Numbers"
      }
    },
    "type": "object",
    "required": ["numbers"],
    "title": "Body_print_numbers__post"
  }

Full schema

{
  "openapi": "3.1.0",
  "info": {
    "title": "FastAPI",
    "version": "0.1.0"
  },
  "paths": {
    "/": {
      "post": {
        "summary": "Print Numbers",
        "operationId": "print_numbers__post",
        "requestBody": {
          "content": {
            "application/x-www-form-urlencoded": {
              "schema": {
                "$ref": "#/components/schemas/Body_print_numbers__post"
              }
            }
          },
          "required": true
        },
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "items": {
                    "items": {
                      "type": "integer"
                    },
                    "type": "array"
                  },
                  "type": "array",
                  "title": "Response Print Numbers  Post"
                }
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "Body_print_numbers__post": {
        "properties": {
          "numbers": {
            "items": {
              "items": {
                "type": "integer"
              },
              "type": "array"
            },
            "type": "array",
            "title": "Numbers"
          }
        },
        "type": "object",
        "required": [
          "numbers"
        ],
        "title": "Body_print_numbers__post"
      },
      "HTTPValidationError": {
        "properties": {
          "detail": {
            "items": {
              "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
          }
        },
        "type": "object",
        "title": "HTTPValidationError"
      },
      "ValidationError": {
        "properties": {
          "loc": {
            "items": {
              "anyOf": [
                {
                  "type": "string"
                },
                {
                  "type": "integer"
                }
              ]
            },
            "type": "array",
            "title": "Location"
          },
          "msg": {
            "type": "string",
            "title": "Message"
          },
          "type": {
            "type": "string",
            "title": "Error Type"
          }
        },
        "type": "object",
        "required": [
          "loc",
          "msg",
          "type"
        ],
        "title": "ValidationError"
      }
    }
  }
}

[mrlubos]

Perfect, thank you!

[crtl]

related #1590 and possible workaround

[mrlubos]

Hey @ntjess can you try the build in #1609? It won't actually produce the example value you provided which is why I want you to try it.

1. Can you also let me know which API framework you're using?
2. Do you have multiple payloads with nested structures such as this one?

This might be difficult to get right for nested structures as different backends might expect different serialization, but let me know!

[mrlubos]

You might need to check out that branch locally, the build step for dependencies isn't good enough yet so the new packages didn't get published

[ntjess]

I tried out npm i https://pkg.pr.new/hey-api/openapi-ts/@hey-api/openapi-ts@1609 and reran the cli generator, but saw the same error.

Does the provided zip template + instructions work for you to verify? I attempted to set it up on StackBlitz but the python backend doesn't support installing FastAPI and I don't have familiarity with node server frameworks

[mrlubos]

@ntjess you'll need to install the client locally, openapi-ts doesn't need to be updated, the fix is in the client code. See my previous message about packages not being published properly!

You could also copy-paste the updated function into SDK and see if it works with that

[ntjess]

My apologies, I'm not a js/ts dev so this is new for me.

I cloned https://github.com/hey-api/openapi-ts into my working directory, uninstalled all prior deps, and ran

npm i ./openapi-ts

Which produced several critical warnings:

npm i ./openapi-ts
npm warn deprecated [email protected]: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated @humanwhocodes/[email protected]: Use @eslint/config-array instead
npm warn deprecated [email protected]: Rimraf versions prior to v4 are no longer supported
npm warn deprecated [email protected]: Glob versions prior to v9 are no longer supported
npm warn deprecated @humanwhocodes/[email protected]: Use @eslint/object-schema instead
npm warn deprecated [email protected]: This version is no longer supported. Please see https://eslint.org/version-support for other options.

added 527 packages, and audited 529 packages in 3s

132 packages are looking for funding
  run `npm fund` for details

3 high severity vulnerabilities

To address all issues, run:
  npm audit fix --force

Run `npm audit` for details.

I tried to recreate the api with npx ./openapi-ts/packages/openapi-ts but nothing happens. The --help flag prints information, but no combination of input flags recreates the API using local files.

Hopefully I'm just missing something simple, but how do I run the npx command to regenerate the api?

Thanks!

[mrlubos]

@ntjess no worries! It will be way simpler if you copy-paste formDataBodySerializer from the pull request and pass it manually to your SDK call. That way you can test if the serialized data gets accepted by your server https://github.com/hey-api/openapi-ts/blob/fix/form-data-body-serializer/packages/client-core/src/bodySerializer.ts

Also check out my 2 questions in #1585 (comment)

[ntjess]

which API framework you're using?

axios

Do you have multiple payloads with nested structures such as this one?

All my encountered cases are with 2D lists inside a toplevel dict. The real-world use case is providing x-y polygons to a backend during image segmentation requests. So the payloads are:

{
    image: "blob",
    foreground_xy: [[1,2], [10, 2], ...],
    background_xy: [],
}

The problems occur when either list contains just one point.

copy-paste formDataBodySerializer

I did this, but the .js.map files in dist are still outdated. My test project is run using vite, is there a simple way to regenerate node_modules/@hey-api/client-axios/dist/*.js.cmap with the change?

• Edit: chrome is pointing to node_modules/.vite/deps/@hey-api_client-axios.js which contains the old code even after deleting the .vite directory and pasting the updated .ts function

Again, apologies if this is straightforward; I'm just unfamiliar with the normal procedure