Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add getJson and outputJson for plugin io util #25

Merged
merged 6 commits into from
Jan 5, 2024
Merged

feat: add getJson and outputJson for plugin io util #25

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f681b56
feat: add getJson and outputJson for plugin io util
nilslice Jan 4, 2024
6ff6fe6
chore: add example and test
nilslice Jan 4, 2024
eac6e1e
fix: build.zig updates
nilslice Jan 4, 2024
1fe7a67
feat: simplify getJson and add more flexible getJsonOpt
nilslice Jan 4, 2024
95dc74e
chore: update example to use new functions
nilslice Jan 4, 2024
8e1cbb7
chore: update docs, examples, build script
nilslice Jan 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,10 @@ jobs:
echo $TEST | grep 'A debug log!'
echo $TEST | grep 'A warning log!'
echo $TEST | grep 'An error log!'

TEST=$(extism call zig-out/bin/basic-example.wasm json_input --input '{"name": "Yoda", "age": 900}')
echo $TEST | grep 'Hello, Yoda. You are 900 years old!'

TEST=$(extism call zig-out/bin/basic-example.wasm json_output)
echo $TEST | grep '["first thing","second thing","third thing"]'

36 changes: 36 additions & 0 deletions examples/basic.zig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,42 @@ export fn count_vowels() i32 {
return 0;
}

const Input = struct {
name: []const u8,
age: u16,
};

export fn json_input() i32 {
const plugin = Plugin.init(allocator);
const json = plugin.getJson(Input, .{}) catch unreachable; // you must call .deinit() when done
defer json.deinit();

const input: Input = json.value();
const out = std.fmt.allocPrint(allocator, "Hello, {s}. You are {d} years old!\n", .{ input.name, input.age }) catch unreachable;

plugin.output(out);
return 0;
}

const Result = struct {
things: [3][]const u8,
};

export fn json_output() i32 {
const plugin = Plugin.init(allocator);
const data = [_][]const u8{
"first thing",
"second thing",
"third thing",
};

const result = Result{ .things = data };

plugin.outputJson(result, .{}) catch unreachable;

return 0;
}

export fn http_get() i32 {
const plugin = Plugin.init(allocator);
// create an HTTP request via Extism built-in function (doesn't require WASI)
Expand Down
29 changes: 29 additions & 0 deletions src/main.zig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,21 @@ pub const http = @import("http.zig");

const LogLevel = enum { Info, Debug, Warn, Error };

pub fn Json(comptime T: type) type {
return struct {
parsed: std.json.Parsed(T),
slice: []const u8,

pub fn value(self: @This()) T {
return self.parsed.value;
}

pub fn deinit(self: @This()) void {
self.parsed.deinit();
}
};
}

pub const Plugin = struct {
allocator: std.mem.Allocator,

Expand Down Expand Up @@ -34,6 +49,15 @@ pub const Plugin = struct {
return buf;
}

// IMPORTANT: It's the caller's responsibility to free the returned struct
pub fn getJson(self: Plugin, comptime T: type, options: std.json.ParseOptions) !Json(T) {
const bytes = try self.getInput();
const out = try std.json.parseFromSlice(T, self.allocator, bytes, options);
const FromJson = Json(T);
const input = FromJson{ .parsed = out, .slice = bytes };
Comment on lines +56 to +57
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of a way to better handle this such that the end-user can get the automatic type from the JSON, and have enough control over freeing the memory so it doesn't deallocate before a value is used.

In the example here:

const Input = struct {
    name: []const u8,
    age: u16,
};

export fn json_input() i32 {
    const plugin = Plugin.init(allocator);
    const json = plugin.getJson(Input, .{}) catch unreachable; // you must call .deinit() when done
    defer json.deinit();

    const input: Input = json.value();
    const out = std.fmt.allocPrint(allocator, "Hello, {s}. You are {d} years old!\n", .{ input.name, input.age }) catch unreachable;

    plugin.output(out);
    return 0;
}

The input.name (a []const u8) would be freed unless json.deinit() is called after this function returns and the value is used outside this function. Initially I thought I could handle this for the end-user, but I don't think I can.

So, this returns the Json wrapped type, with methods .value() (to get the parsed struct), and .deinit() to give the end-user the ability to free the slice used to parse the struct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can avoid this by using alloc_always when parsing the JSON:

std.json.parseFromSlice(Config, allocator, data, .{.allocate = .alloc_always})

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zshipko thanks, this works great!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, now there is a getJson which returns the comptime T and handles the memory management. kind of the "expected / easy-mode" path.

and, there's a getJsonOpt which is the previous implementation and allows the end user to control the parsing and allocation.

return input;
}

pub fn allocate(self: Plugin, length: usize) Memory {
_ = self; // to make the interface consistent

Expand Down Expand Up @@ -73,6 +97,11 @@ pub const Plugin = struct {
extism.output_set(offset, c_len);
}

pub fn outputJson(self: Plugin, T: anytype, options: std.json.StringifyOptions) !void {
const out = try std.json.stringifyAlloc(self.allocator, T, options);
self.output(out);
}

pub fn setErrorMemory(self: Plugin, mem: Memory) void {
_ = self; // to make the interface consistent
extism.error_set(mem.offset);
Expand Down
Loading