Skip to content

Commit

Permalink
feat(proxy-wasm) implement 'proxy_wasm_log_dispatch_errors' directive
Browse files Browse the repository at this point in the history
  • Loading branch information
thibaultcha committed Nov 20, 2024
1 parent bb315e7 commit e8328f5
Show file tree
Hide file tree
Showing 8 changed files with 232 additions and 51 deletions.
66 changes: 47 additions & 19 deletions docs/DIRECTIVES.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ By alphabetical order:
- [module](#module)
- [proxy_wasm](#proxy_wasm)
- [proxy_wasm_isolation](#proxy_wasm_isolation)
- [proxy_wasm_log_dispatch_errors](#proxy_wasm_log_dispatch_errors)
- [proxy_wasm_lua_resolver](#proxy_wasm_lua_resolver)
- [proxy_wasm_request_headers_in_access](#proxy_wasm_request_headers_in_access)
- [resolver](#resolver)
Expand Down Expand Up @@ -45,6 +46,8 @@ By context:
- [compiler](#compiler)
- [backtraces](#backtraces)
- [module](#module)
- [proxy_wasm_log_dispatch_errors](#proxy_wasm_log_dispatch_errors)
- [proxy_wasm_lua_resolver](#proxy_wasm_lua_resolver)
- [resolver](#resolver)
- [resolver_timeout](#resolver_timeout)
- [shm_kv](#shm_kv)
Expand Down Expand Up @@ -72,6 +75,7 @@ By context:
- `http{}`, `server{}`, `location{}`
- [proxy_wasm](#proxy_wasm)
- [proxy_wasm_isolation](#proxy_wasm_isolation)
- [proxy_wasm_log_dispatch_errors](#proxy_wasm_log_dispatch_errors)
- [proxy_wasm_lua_resolver](#proxy_wasm_lua_resolver)
- [proxy_wasm_request_headers_in_access](#proxy_wasm_request_headers_in_access)
- [resolver_add](#resolver_add)
Expand Down Expand Up @@ -260,7 +264,7 @@ Load a Wasm module from disk.
- `path` must point to a bytecode file whose format is `.wasm` (binary) or
`.wat` (text).
- `config` is an optional configuration string passed to `on_vm_start` when
`module` is a proxy-wasm filter.
`module` is a Proxy-Wasm filter.

If successfully loaded, the module can later be referred to by `name`.

Expand All @@ -286,11 +290,11 @@ proxy_wasm
**default** |
**example** | `proxy_wasm my_filter_module 'foo=bar';`

Add a proxy-wasm filter to the context's execution chain (see [Execution
Add a Proxy-Wasm filter to the context's execution chain (see [Execution
Chain]).

- `module` must be a Wasm module name declared by a [module](#module) directive.
This module must be a valid proxy-wasm filter.
This module must be a valid Proxy-Wasm filter.
- `config` is an optional configuration string passed to the filter's
`on_configure` phase.

Expand All @@ -305,25 +309,25 @@ start.
> Notes
Each instance of the `proxy_wasm` directive in the configuration will be
represented by a proxy-wasm root filter context in a Wasm instance.
represented by a Proxy-Wasm root filter context in a Wasm instance.

All root filter contexts of the same module share the same instance.

All root filter contexts will be initialized during nginx worker process
initialization, which will invoke the filters' `on_vm_start` and `on_configure`
phases. Each root context may optionally start a single background tick, as
specified by the [proxy-wasm SDK](#proxy-wasm).
specified by the [Proxy-Wasm SDK](#proxy-wasm).

Note that when the master process is in use as a daemon (default Nginx
configuration), the `nginx` exit code may be `0` when its worker processes fail
initialization. Since proxy-wasm filters are started on a per-process basis,
initialization. Since Proxy-Wasm filters are started on a per-process basis,
filter initialization takes place (and may fail) during worker process
initialization, which will be reflected in the error logs.

On incoming HTTP requests traversing the context (see [Contexts]), the
[Execution Chain] will resume execution for the current Nginx phase, which will
cause each configured filter to resume its corresponding proxy-wasm phase. Each
request gets associated with a proxy-wasm HTTP filter context, and a Wasm
cause each configured filter to resume its corresponding Proxy-Wasm phase. Each
request gets associated with a Proxy-Wasm HTTP filter context, and a Wasm
instance to execute into.

HTTP filter contexts can execute on instances with various lifecycles and
Expand All @@ -341,13 +345,13 @@ proxy_wasm_isolation
**default** | `none`
**example** | `proxy_wasm_isolation stream;`

Select the Wasm instance isolation mode for proxy-wasm filters.
Select the Wasm instance isolation mode for Proxy-Wasm filters.

- `isolation` must be one of `none`, `stream`, `filter`.

> Notes
Each proxy-wasm filter within the context's [Execution Chain] will be given an
Each Proxy-Wasm filter within the context's [Execution Chain] will be given an
instance to execute onto. The lifecycle and isolation of that instance depend on
the chosen `isolation` mode:

Expand All @@ -359,16 +363,40 @@ the chosen `isolation` mode:

[Back to TOC](#directives)

proxy_wasm_log_dispatch_errors
------------------------------

**usage** | `proxy_wasm_log_dispatch_errors <on\|off>;`
------------:|:----------------------------------------------------------------
**contexts** | `wasm{}`, `http{}`, `server{}`, `location{}`
**default** | `on`
**example** | `proxy_wasm_log_dispatch_errors off;`

Toggles TCP socket error logs on Proxy-Wasm dispatch calls failure.

When enabled, an `[error]` log will be produced on failure conditions such as
timeout, broken connection, resolver failure, etc.

When used in the `wasm{}` context, this directive has a global effect on all
`location{}` contexts (unless overridden) as well as root Proxy-Wasm dispatch
calls.

[Back to TOC](#directives)

proxy_wasm_lua_resolver
-----------------------

**usage** | `proxy_wasm_lua_resolver <on\|off>;`
------------:|:----------------------------------------------------------------
**contexts** | `http{}`, `server{}`, `location{}`
**contexts** | `wasm{}`, `http{}`, `server{}`, `location{}`
**default** | `off`
**example** | `proxy_wasm_lua_resolver on;`

Toggles the "Lua DNS resolver for proxy-wasm" feature within the context.
Toggles the "Lua DNS resolver for Proxy-Wasm" feature within the context.

When used in the `wasm{}` context, this directive has a global effect on all
`location{}` contexts (unless overridden) as well as root Proxy-Wasm dispatch
calls.

**Note:** this directive requires Lua support and will only have an effect if
ngx_wasm_module was compiled alongside [OpenResty].
Expand All @@ -389,7 +417,7 @@ If not, a default client instance will be created pointing to `8.8.8.8` with a
timeout value of `30s`.

When in use, any [resolver] directive in the effective context will be ignored
for proxy-wasm HTTP dispatches.
for Proxy-Wasm HTTP dispatches.

[Back to TOC](#directives)

Expand Down Expand Up @@ -431,7 +459,7 @@ This directive's arguments are identical to Nginx's [resolver] directive.
Wasm sockets usually rely on the configured Nginx [resolver] to resolve
hostname. However, some contexts do not support a [resolver] yet still provide
access to Wasm sockets (e.g. proxy-wasm's `on_vm_start` or `on_tick`). In such
access to Wasm sockets (e.g. Proxy-Wasm's `on_vm_start` or `on_tick`). In such
contexts, the global `wasm{}` resolver will be used.

The global resolver is also used as a fallback if no resolver is configured in
Expand Down Expand Up @@ -525,7 +553,7 @@ start.
Shared memory zones are shared between all nginx worker processes, and serve as
a means of storage and exchange for worker processes of a server instance.

Shared key/value memory zones can be used via the [proxy-wasm
Shared key/value memory zones can be used via the [Proxy-Wasm
SDK](#proxy-wasm)'s `[get\|set]_shared_data` API.

[Back to TOC](#directives)
Expand Down Expand Up @@ -557,7 +585,7 @@ start.
Shared memory zones are shared between all nginx worker processes, and serve as
a means of storage and exchange for worker processes of a server instance.

Shared queue memory zones can be used via the [proxy-wasm SDK](#proxy-wasm)'s
Shared queue memory zones can be used via the [Proxy-Wasm SDK](#proxy-wasm)'s
`[enqueue\|dequeue]_shared_queue` API.

**Note:** shared memory queues do not presently implement an automatic eviction
Expand Down Expand Up @@ -663,7 +691,7 @@ This directive is effective for all Wasm sockets in all contexts.

> Notes
When using the [proxy-wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
When using the [Proxy-Wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
`timeout` argument can be specified which will override this setting.

For configuring Wasm sockets in `http{}` contexts, see
Expand Down Expand Up @@ -708,7 +736,7 @@ Set a default timeout value for Wasm sockets read operations.

> Notes
When using the [proxy-wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
When using the [Proxy-Wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
`timeout` argument can be specified which will override this setting.

For configuring Wasm sockets in `http{}` contexts, see
Expand All @@ -729,7 +757,7 @@ Set a default timeout value for Wasm sockets send operations.

> Notes
When using the [proxy-wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
When using the [Proxy-Wasm SDK](#proxy-wasm) `dispatch_http_call()` method, a
`timeout` argument can be specified which will override this setting.

For configuring Wasm sockets in `http{}` contexts, see
Expand Down
8 changes: 5 additions & 3 deletions src/http/ngx_http_wasm.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ struct ngx_http_wasm_req_ctx_s {

unsigned ffi_attached:1;
unsigned pwm_lua_resolver:1; /* use Lua-land resolver in OpenResty */
unsigned pwm_log_dispatch_errors:1;
};


Expand All @@ -95,12 +96,13 @@ typedef struct {
ngx_bufs_t socket_large_buffers; /* wasm_socket_large_buffer_size */
ngx_bufs_t resp_body_buffers; /* wasm_response_body_buffers */

ngx_flag_t pwm_req_headers_in_access;
ngx_flag_t pwm_lua_resolver;

ngx_flag_t postpone_rewrite;
ngx_flag_t postpone_access;

ngx_flag_t pwm_req_headers_in_access;
ngx_flag_t pwm_lua_resolver;
ngx_flag_t pwm_log_dispatch_errors;

ngx_queue_t q; /* main_conf */
} ngx_http_wasm_loc_conf_t;

Expand Down
50 changes: 37 additions & 13 deletions src/http/ngx_http_wasm_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ ngx_wasm_subsystem_t ngx_http_wasm_subsystem = {

static ngx_command_t ngx_http_wasm_module_cmds[] = {

/* wasm */

{ ngx_string("wasm_call"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE3,
ngx_http_wasm_call_directive,
Expand Down Expand Up @@ -172,6 +174,22 @@ static ngx_command_t ngx_http_wasm_module_cmds[] = {
offsetof(ngx_http_wasm_loc_conf_t, resp_body_buffers),
NULL },

{ ngx_string("wasm_postpone_rewrite"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
NGX_HTTP_LOC_CONF_OFFSET,
offsetof(ngx_http_wasm_loc_conf_t, postpone_rewrite),
NULL },

{ ngx_string("wasm_postpone_access"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
NGX_HTTP_LOC_CONF_OFFSET,
offsetof(ngx_http_wasm_loc_conf_t, postpone_access),
NULL },

/* proxy_wasm */

{ ngx_string("proxy_wasm"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE12,
ngx_http_wasm_proxy_wasm_directive,
Expand All @@ -186,13 +204,6 @@ static ngx_command_t ngx_http_wasm_module_cmds[] = {
NGX_HTTP_MODULE,
NULL },

{ ngx_string("resolver_add"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
ngx_http_wasm_resolver_add_directive,
NGX_HTTP_LOC_CONF_OFFSET,
NGX_HTTP_MODULE,
NULL },

{ ngx_string("proxy_wasm_request_headers_in_access"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
Expand All @@ -207,18 +218,20 @@ static ngx_command_t ngx_http_wasm_module_cmds[] = {
offsetof(ngx_http_wasm_loc_conf_t, pwm_lua_resolver),
NULL },

{ ngx_string("wasm_postpone_rewrite"),
{ ngx_string("proxy_wasm_log_dispatch_errors"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
NGX_HTTP_LOC_CONF_OFFSET,
offsetof(ngx_http_wasm_loc_conf_t, postpone_rewrite),
offsetof(ngx_http_wasm_loc_conf_t, pwm_log_dispatch_errors),
NULL },

{ ngx_string("wasm_postpone_access"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
/* misc */

{ ngx_string("resolver_add"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
ngx_http_wasm_resolver_add_directive,
NGX_HTTP_LOC_CONF_OFFSET,
offsetof(ngx_http_wasm_loc_conf_t, postpone_access),
NGX_HTTP_MODULE,
NULL },

ngx_null_command
Expand Down Expand Up @@ -340,6 +353,7 @@ ngx_http_wasm_create_loc_conf(ngx_conf_t *cf)
loc->socket_buffer_reuse = NGX_CONF_UNSET;
loc->pwm_req_headers_in_access = NGX_CONF_UNSET;
loc->pwm_lua_resolver = NGX_CONF_UNSET;
loc->pwm_log_dispatch_errors = NGX_CONF_UNSET;
loc->postpone_rewrite = NGX_CONF_UNSET;
loc->postpone_access = NGX_CONF_UNSET;

Expand Down Expand Up @@ -404,6 +418,9 @@ ngx_http_wasm_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child)
ngx_conf_merge_value(conf->pwm_lua_resolver,
prev->pwm_lua_resolver, 0);

ngx_conf_merge_value(conf->pwm_log_dispatch_errors,
prev->pwm_log_dispatch_errors, 1);

ngx_conf_merge_value(conf->postpone_rewrite,
prev->postpone_rewrite, NGX_CONF_UNSET);

Expand Down Expand Up @@ -636,10 +653,17 @@ ngx_http_wasm_rctx(ngx_http_request_t *r, ngx_http_wasm_req_ctx_t **out)
rctx->pwm_lua_resolver = loc->pwm_lua_resolver != NGX_CONF_UNSET
? loc->pwm_lua_resolver
: wcf ? wcf->pwm_lua_resolver : 0;
rctx->pwm_log_dispatch_errors = loc->pwm_log_dispatch_errors
!= NGX_CONF_UNSET
? loc->pwm_log_dispatch_errors
: wcf ? wcf->pwm_log_dispatch_errors : 0;

} else {
/* fake request */
rctx->pwm_lua_resolver = wcf ? wcf->pwm_lua_resolver : 0;
rctx->pwm_log_dispatch_errors = wcf
? wcf->pwm_log_dispatch_errors
: 0;
}
}
#if (NGX_DEBUG)
Expand Down
6 changes: 4 additions & 2 deletions src/http/proxy_wasm/ngx_http_proxy_wasm_dispatch.c
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,10 @@ ngx_http_proxy_wasm_dispatch_err(ngx_http_proxy_wasm_dispatch_t *call,
#endif

if (!pwexec->ictx->instance->hostcall || rctx->fake_request) {
ngx_wasm_log_error(NGX_LOG_ERR, pwexec->log, 0,
"%*s", p - (u_char *) &errbuf, &errbuf);
if (rctx->pwm_log_dispatch_errors) {
ngx_wasm_log_error(NGX_LOG_ERR, pwexec->log, 0, "%*s",
p - (u_char *) &errbuf, &errbuf);
}

} else {
/* in-vm, executing a hostcall */
Expand Down
1 change: 1 addition & 0 deletions src/wasm/ngx_wasm.h
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ typedef struct {
ngx_resolver_t *user_resolver;

ngx_flag_t pwm_lua_resolver;
ngx_flag_t pwm_log_dispatch_errors;
} ngx_wasm_core_conf_t;


Expand Down
22 changes: 15 additions & 7 deletions src/wasm/ngx_wasm_core_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -166,13 +166,6 @@ static ngx_command_t ngx_wasm_core_commands[] = {
offsetof(ngx_wasm_core_conf_t, resolver_timeout),
NULL },

{ ngx_string("proxy_wasm_lua_resolver"),
NGX_WASM_CONF|NGX_CONF_TAKE1,
ngx_wasm_core_pwm_lua_resolver_directive,
NGX_WA_WASM_CONF_OFFSET,
offsetof(ngx_wasm_core_conf_t, pwm_lua_resolver),
NULL },

{ ngx_string("socket_connect_timeout"),
NGX_WASM_CONF|NGX_CONF_TAKE1,
ngx_conf_set_msec_slot,
Expand Down Expand Up @@ -215,6 +208,20 @@ static ngx_command_t ngx_wasm_core_commands[] = {
offsetof(ngx_wasm_core_conf_t, socket_large_buffers),
NULL },

{ ngx_string("proxy_wasm_lua_resolver"),
NGX_WASM_CONF|NGX_CONF_TAKE1,
ngx_wasm_core_pwm_lua_resolver_directive,
NGX_WA_WASM_CONF_OFFSET,
offsetof(ngx_wasm_core_conf_t, pwm_lua_resolver),
NULL },

{ ngx_string("proxy_wasm_log_dispatch_errors"),
NGX_WASM_CONF|NGX_CONF_TAKE1,
ngx_conf_set_flag_slot,
NGX_WA_WASM_CONF_OFFSET,
offsetof(ngx_wasm_core_conf_t, pwm_log_dispatch_errors),
NULL },

ngx_null_command
};

Expand Down Expand Up @@ -364,6 +371,7 @@ ngx_wasm_core_create_conf(ngx_conf_t *cf)
wcf->recv_timeout = NGX_CONF_UNSET_MSEC;

wcf->pwm_lua_resolver = NGX_CONF_UNSET;
wcf->pwm_log_dispatch_errors = NGX_CONF_UNSET;

wcf->socket_buffer_size = NGX_CONF_UNSET_SIZE;
wcf->socket_buffer_reuse = NGX_CONF_UNSET;
Expand Down
Loading

0 comments on commit e8328f5

Please sign in to comment.