-
Notifications
You must be signed in to change notification settings - Fork 268
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
30 changed files
with
828 additions
and
16 deletions.
There are no files selected for viewing
Binary file added
BIN
+79 KB
packages/docs/docs/node-reference/assets/extract-yaml-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+77.4 KB
packages/docs/docs/node-reference/assets/extract-yaml-node-example-02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+62.5 KB
packages/docs/docs/node-reference/assets/get-all-datasets-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+21.3 KB
packages/docs/docs/node-reference/assets/get-dataset-row-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+34.6 KB
packages/docs/docs/node-reference/assets/get-dataset-row-node-example-02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+24.6 KB
packages/docs/docs/node-reference/assets/get-embedding-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+104 KB
packages/docs/docs/node-reference/assets/get-global-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+85.8 KB
packages/docs/docs/node-reference/assets/get-global-node-example-02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+70.4 KB
packages/docs/docs/node-reference/assets/gpt-function-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+13.5 KB
packages/docs/docs/node-reference/assets/graph-input-node-example-01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+9.8 KB
packages/docs/docs/node-reference/assets/graph-input-node-example-02-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+19.8 KB
packages/docs/docs/node-reference/assets/graph-input-node-example-02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,126 @@ | ||
--- | ||
title: 'Extract YAML' | ||
id: extract-yaml | ||
title: Extract YAML Node | ||
sidebar_label: Extract YAML | ||
--- | ||
|
||
# Extract YAML Node | ||
import Tabs from '@theme/Tabs'; | ||
import TabItem from '@theme/TabItem'; | ||
|
||
![Extract YAML Node Screenshot](./assets/extract-yaml-node.png) | ||
|
||
## Overview | ||
|
||
The Extract YAML Node is used to parse a YAML object from a string. It is particularly useful when you want to extract structured data from a text string that includes a YAML object. | ||
|
||
The node looks for a predefined root property name in the input text and considers all indented text after that as part of the YAML object. By default, the root property name is set to `yamlDocument`, but it can be configured in the node settings or via an input port. | ||
|
||
You may also specify a path to a specific property in the YAML object using a [JSONPath](https://goessner.net/articles/JsonPath/) expression. The node will then extract the value of that property from the YAML object. This is a shorthand for using the [Extract Object Path Node](./extract-object-path.mdx) after the Extract YAML Node. | ||
|
||
Note that the entire document includes the root property name. For example, if the root property name is set to `yamlDocument`, then the object path `$.yamlDocument` will return the entire document. | ||
|
||
The Extract YAML Node can only extract a single YAML object from a string. If you want to extract multiple YAML objects from a string, you may want to use the [Code Node](./code.mdx) or [External Call Node](./external-call.mdx) to do more advanced text extraction. | ||
|
||
Text outside of the YAML object will be ignored. If you want to extract text outside of the YAML object, you can use the [Extract with Regex Node](./extract-with-regex.mdx) instead. | ||
|
||
<Tabs | ||
defaultValue="inputs" | ||
values={[ | ||
{label: 'Inputs', value: 'inputs'}, | ||
{label: 'Outputs', value: 'outputs'}, | ||
{label: 'Editor Settings', value: 'settings'}, | ||
] | ||
}> | ||
|
||
<TabItem value="inputs"> | ||
|
||
## Inputs | ||
|
||
| Title | Data Type | Description | Default Value | Notes | | ||
| ------------------ | --------- | -------------------------------------------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------- | | ||
| Input | `string` | The string that contains the YAML object. | (required) | The input will be coerced into a string if it is not a string. | | ||
| Root Property Name | `string` | The root property name of the YAML object in the input string. | (optional) | The input will be coerced into a string if it is not a string. This input is only available if enabled in the node settings. | | ||
| Object Path | `string` | The JSONPath expression to extract data from the parsed YAML object. | (optional) | The input will be coerced into a string if it is not a string. This input is only available if enabled in the node settings. | | ||
|
||
</TabItem> | ||
|
||
<TabItem value="outputs"> | ||
|
||
## Outputs | ||
|
||
| Title | Data Type | Description | Notes | | ||
| -------- | --------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | ||
| Output | `object` | The parsed YAML object or the value extracted from the object using the JSONPath expression. | If the YAML object or the value is not found, this port will not be ran. | | ||
| Matches | `any[]` | When using a JSONPath expression that can match multiple times, represents all paths of the object that match. | If the YAML object or the value is not found, this port will run with an empty array. | | ||
| No Match | `string` | If no YAML object is found in the input text, this port will run, with the full contents of the input string. | If a YAML object is found in the input string, this port will not be ran. | | ||
|
||
</TabItem> | ||
|
||
<TabItem value="settings"> | ||
|
||
## Editor Settings | ||
|
||
| Setting | Description | Default Value | Use Input Toggle | Input Data Type | | ||
| ------------------ | -------------------------------------------------------------------- | -------------- | ---------------- | --------------- | | ||
| Root Property Name | The root property name of the YAML object in the input string. | `yamlDocument` | Yes | `string` | | ||
| Object Path | The JSONPath expression to extract data from the parsed YAML object. | (empty) | Yes | `string` | | ||
|
||
</TabItem> | ||
|
||
</Tabs> | ||
|
||
## Example 1: Extract a YAML object from a string | ||
|
||
1. Create a [Text Node](./text.mdx) and set the text to the following: | ||
|
||
``` | ||
yamlDocument: | ||
name: John Doe | ||
age: 30 | ||
job: Engineer | ||
``` | ||
|
||
2. Create an Extract YAML Node and connect the Text Node to its `Input` input. | ||
3. Run the graph. The `Output` of the Extract YAML Node should contain the parsed YAML object. | ||
|
||
![Extract YAML Node Example 1](./assets/extract-yaml-node-example-01.png) | ||
|
||
## Example 2: Extract a property from a YAML object | ||
|
||
1. Create a [Text Node](./text.mdx) and set the text to the following: | ||
|
||
``` | ||
yamlDocument: | ||
name: John Doe | ||
age: 30 | ||
job: Engineer | ||
``` | ||
|
||
2. Create an Extract YAML Node and connect the Text Node to its `Input` input. | ||
3. Set the `Object Path` of the Extract YAML Node to `$.yamlDocument.name`. | ||
4. Run the graph. The `Output` of the Extract YAML Node should contain the value `John Doe`. | ||
|
||
![Extract YAML Node Example 2](./assets/extract-yaml-node-example-02.png) | ||
|
||
## Error Handling | ||
|
||
If the JSONPath expression is invalid, the Extract YAML Node will error. If the YAML object or the value is not found, the `Output` and `Matches` outputs will not be ran, and the `No Match` output will run with the full contents of the input string. | ||
|
||
## FAQ | ||
|
||
**Q: Can I use the Extract YAML Node to extract data from a JSON object?** | ||
|
||
A: No, the Extract YAML Node is specifically designed to parse YAML objects. If you want to extract data from a JSON object, you can use the [Extract JSON Node](./extract-json.mdx) instead. | ||
|
||
**Q: Can I use the Extract YAML Node to extract an array from a YAML object?** | ||
|
||
A: Yes, you can use the Extract YAML Node to extract any valid YAML structure, including arrays. You can use a JSONPath expression to specify the path to the array in the YAML object. Alternatively you can specify a multi-match such as `$.yamlDocument.someArray[*]` and use the `Matches` output to get all the matches. | ||
|
||
## See Also | ||
|
||
- [Extract JSON Node](./extract-json.mdx) | ||
- [Extract Object Path Node](./extract-object-path.mdx) | ||
- [Object Node](./object.mdx) | ||
- [Array Node](./array.mdx) | ||
- [Extract with Regex Node](./extract-with-regex.mdx) | ||
- [Data Types](../user-guide/data-types.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,101 @@ | ||
--- | ||
title: 'Filter' | ||
id: filter | ||
title: Filter Node | ||
sidebar_label: Filter | ||
--- | ||
|
||
# Filter Node | ||
import Tabs from '@theme/Tabs'; | ||
import TabItem from '@theme/TabItem'; | ||
|
||
![Filter Node Screenshot](./assets/filter-node.png) | ||
|
||
## Overview | ||
|
||
The Filter Node is used to filter an array based on a corresponding array of boolean values. The node takes in an array of any data type and an array of boolean values of the same length. It then filters the array, only including the elements where the corresponding boolean value is `true`. | ||
|
||
This node is particularly useful when you want to filter an array based on some condition. For example, you could use a split (see [Splitting](../user-guide/splitting.md)) [Compare Node](./compare.mdx) to compare each element in an array to a value, and then use the Filter Node to filter the array based on the results of the comparison. | ||
|
||
<Tabs | ||
defaultValue="inputs" | ||
values={[ | ||
{label: 'Inputs', value: 'inputs'}, | ||
{label: 'Outputs', value: 'outputs'}, | ||
{label: 'Editor Settings', value: 'settings'}, | ||
] | ||
}> | ||
|
||
<TabItem value="inputs"> | ||
|
||
## Inputs | ||
|
||
| Title | Data Type | Description | Default Value | Notes | | ||
| ------- | ----------- | ----------------------------------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| Array | `any[]` | The array to be filtered. | (required) | The input will be coerced into an array if it is not an array. | | ||
| Include | `boolean[]` | An array of boolean values indicating whether to include each element in the array. | (required) | The input will be coerced into a boolean array if it is not a boolean array. The length of this array should match the length of the Array input. | | ||
|
||
</TabItem> | ||
|
||
<TabItem value="outputs"> | ||
|
||
## Outputs | ||
|
||
| Title | Data Type | Description | Notes | | ||
| -------- | --------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| Filtered | `any[]` | The filtered array, containing only the elements where the corresponding boolean value in the Include input was `true`. | The output will be an array of the same data type as the Array input. If the Array input is not an array, the output will be an array of `any`. | | ||
|
||
</TabItem> | ||
|
||
<TabItem value="settings"> | ||
|
||
## Editor Settings | ||
|
||
This node has no configurable editor settings. | ||
|
||
</TabItem> | ||
|
||
</Tabs> | ||
|
||
## Example 1: Filter an array of numbers | ||
|
||
1. Create an [Array Node](./array.mdx) and set the values to `[1, 2, 3, 4, 5]`. | ||
2. Create another Array Node and set the values to `[true, false, true, false, true]`. | ||
3. Create a Filter Node and connect the first Array Node to the `Array` input and the second Array Node to the `Include` input. | ||
4. Run the graph. The `Filtered` output of the Filter Node should be `[1, 3, 5]`. | ||
|
||
![Filter Node Example 1](./assets/filter-node-example-01.png) | ||
|
||
## Example 2: Filter an array based on a condition | ||
|
||
1. Create an [Array Node](./array.mdx) and set the values to `[1, 2, 3, 4, 5]`. | ||
2. Create a [Compare Node](./compare.mdx) and set the comparison to `>`. Enable splitting on the Compare node. | ||
3. Create a [Number Node](./number.mdx) and set the value to `3`. | ||
4. Connect the Array Node to the `A` input of the Compare Node, and connect the Number Node to the `B` input of the Compare Node. | ||
5. Create a Filter Node and connect the Compare Node to the `Array` input and the `Result` output of the Compare Node to the `Include` input. | ||
6. Run the graph. The `Filtered` output of the Filter Node should be `[4, 5]`. | ||
|
||
![Filter Node Example 2](./assets/filter-node-example-02.png) | ||
|
||
## Error Handling | ||
|
||
The Filter Node will error if the `Array` or `Include` inputs are not provided, or if the lengths of the arrays do not match. | ||
|
||
## FAQ | ||
|
||
**Q: What happens if the lengths of the Array and Include inputs do not match?** | ||
|
||
A: The Filter Node will error. The lengths of the Array and Include inputs should always match. | ||
|
||
**Q: Can I use the Filter Node to filter an array of objects?** | ||
|
||
A: Yes, you can use the Filter Node to filter an array of any data type, including objects. Just make sure to provide an array of boolean values indicating whether to include each object in the filtered array. | ||
|
||
## See Also | ||
|
||
- [Array Node](./array.mdx) | ||
- [Compare Node](./compare.mdx) | ||
- [Number Node](./number.mdx) | ||
- [Splitting](../user-guide/splitting.md) | ||
- [Extract Object Path Node](./extract-object-path.mdx) | ||
- [Evaluate Node](./evaluate.mdx) | ||
- [Coalesce Node](./coalesce.mdx) | ||
- [Join Node](./join.mdx) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,97 @@ | ||
--- | ||
title: 'Get All Dataset' | ||
id: get-all-datasets | ||
title: Get All Datasets Node | ||
sidebar_label: Get All Datasets | ||
--- | ||
|
||
# Get All Dataset Node | ||
import Tabs from '@theme/Tabs'; | ||
import TabItem from '@theme/TabItem'; | ||
|
||
![Get All Datasets Node Screenshot](./assets/get-all-datasets-node.png) | ||
|
||
## Overview | ||
|
||
The Get All Datasets Node is used to retrieve all datasets available in the current project. This node is particularly useful when you want to access and manipulate multiple datasets in your graph, or search through all datasets for a specific dataset. | ||
|
||
The node requires a dataset provider to be available in the context when the graph is being run. The dataset provider is responsible for managing the storage and retrieval of datasets. In the Rivet application, the dataset provider is handled for you automatically. | ||
|
||
For more information on datasets, see the [Data Studio](../user-guide/features/data-studio.md) section of the user guide. | ||
|
||
<Tabs | ||
defaultValue="inputs" | ||
values={[ | ||
{label: 'Inputs', value: 'inputs'}, | ||
{label: 'Outputs', value: 'outputs'}, | ||
{label: 'Editor Settings', value: 'settings'}, | ||
] | ||
}> | ||
|
||
<TabItem value="inputs"> | ||
|
||
The Get All Datasets Node does not have any inputs. | ||
|
||
</TabItem> | ||
|
||
<TabItem value="outputs"> | ||
|
||
## Outputs | ||
|
||
| Title | Data Type | Description | Notes | | ||
| -------- | ---------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | | ||
| Datasets | `object[]` | An array of all datasets in the current project. Each dataset is represented as an object. | If no datasets exist in the current project, the output will be an empty array. | | ||
|
||
### Object Structure | ||
|
||
Each dataset in the `Datasets` output is represented as an object with the following properties: | ||
|
||
| Title | Data Type | Description | Notes | | ||
| ----------- | --------- | ------------------------------- | ----- | | ||
| id | `string` | The ID of the dataset. | | | ||
| name | `string` | The name of the dataset. | | | ||
| description | `string` | The description of the dataset. | | | ||
|
||
Note that the actual data of the dataset is not included in the output. To retrieve the data of a dataset, use the [Load Dataset Node](./load-dataset.mdx). | ||
|
||
</TabItem> | ||
|
||
<TabItem value="settings"> | ||
|
||
The Get All Datasets Node does not have any configurable editor settings. | ||
|
||
</TabItem> | ||
|
||
</Tabs> | ||
|
||
## Example: Retrieve all datasets in a project | ||
|
||
1. In the [Data Studio](../user-guide/features/data-studio.md), create a few datasets with different IDs. | ||
2. Create a Get All Datasets Node in your graph. | ||
3. Run the graph. The `Datasets` output of the Get All Datasets Node should contain an array of all datasets in the current project. | ||
|
||
![Get All Datasets Node Example](./assets/get-all-datasets-node-example-01.png) | ||
|
||
## Error Handling | ||
|
||
The Get All Datasets Node will error if the dataset provider is not available in the context when the graph is being run. | ||
|
||
## FAQ | ||
|
||
**Q: What is a dataset provider?** | ||
|
||
A: A dataset provider is an object that is responsible for managing the storage and retrieval of datasets. It is passed to the context when the graph is being run. The dataset provider must implement the `DatasetProvider` interface, which includes methods for getting, putting, and deleting datasets. See the [API Reference](../api-reference.md) for more information. | ||
|
||
**Q: Can I filter the datasets returned by the Get All Datasets Node?** | ||
|
||
A: No, the Get All Datasets Node will return all datasets in the current project. If you want to filter the datasets, you can use other nodes to process the output of the Get All Datasets Node, such as the [Filter Node](../filter.mdx) or [Extract Object Path Node](./extract-object-path.mdx) | ||
|
||
**Q: What happens if there are no datasets in the current project?** | ||
|
||
A: If there are no datasets in the current project, the `Datasets` output of the Get All Datasets Node will be an empty array. | ||
|
||
## See Also | ||
|
||
- [Load Dataset Node](./load-dataset.mdx) | ||
- [Append to Dataset Node](./append-to-dataset.mdx) | ||
- [Create Dataset Node](./create-dataset.mdx) | ||
- [KNN Dataset Node](./knn-dataset.mdx) | ||
- [Get Dataset Row Node](./get-dataset-row.mdx) |
Oops, something went wrong.