diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 00000000..5edb0ac6 --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,16 @@ +name: Test JSONata +on: + push: + branches: [stedi-main] + pull_request: + branches: [stedi-main] +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-node@v3 + with: + node-version: 20 + - run: npm install + - run: npm test diff --git a/CHANGELOG.md b/CHANGELOG.md index 38d9f7f4..1880ccbe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,18 @@ +#### 2.0.4 Maintenance Release + +- Prevent writing to the object prototype or constructor (PR https://github.com/jsonata-js/jsonata/pull/676) +- Add upper/lower presentation format for am/pm in fromMillis (PR https://github.com/jsonata-js/jsonata/pull/644) +- Various documentation additions and corrections + +#### 2.0.3 Maintenance Release + +- Fix regex termination lexer (PR https://github.com/jsonata-js/jsonata/pull/623) +- Fix TypeScript definition (PR https://github.com/jsonata-js/jsonata/pull/633) + +#### 2.0.2 Maintenance Release + +- Typescript definition: fix return type of evaluate method (PR https://github.com/jsonata-js/jsonata/pull/615) + #### 2.0.1 Maintenance Release - Small update to pick up README changes with 2.0.0 changes diff --git a/docs/construction.md b/docs/construction.md index 4705ee5d..81a42d58 100644 --- a/docs/construction.md +++ b/docs/construction.md @@ -44,7 +44,7 @@ __Examples__ ## Object constructors -In a similar manner to the way arrays can be constructed, JSON objects can also be constructed in the output. At any point in a location path where a field reference is expected, a pair of braces `{}` containing key/value pairs separated by commas, with each key and value separated by a colon: `{key1: value2, key2:value2}`. The keys and values can either be literals or can be expressions. The key must either be a string or an expression that evaluates to a string. +In a similar manner to the way arrays can be constructed, JSON objects can also be constructed in the output. At any point in a location path where a field reference is expected, a pair of braces `{}` containing key/value pairs separated by commas, with each key and value separated by a colon: `{key1: value1, key2:value2}`. The keys and values can either be literals or can be expressions. The key must either be a string or an expression that evaluates to a string. When an object constructor follows an expression that selects multiple values, the object constructor will create a single object that contains a key/value pair for each of those context values. If an array of objects is required (one for each context value), then the object constructor should immediately follow the dot '.' operator. diff --git a/docs/numeric-operators.md b/docs/numeric-operators.md index 41263a38..ca9582f5 100644 --- a/docs/numeric-operators.md +++ b/docs/numeric-operators.md @@ -13,7 +13,7 @@ __Example__ `5 + 2` => `7` -## `-` (Substraction/Negation) +## `-` (Subtraction/Negation) The subtraction operator subtracts the RHS value from the LHS value to produce the numerical difference It is an error if either operand is not a number. diff --git a/docs/overview.md b/docs/overview.md index 86bfc870..a087678a 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -16,7 +16,22 @@ JSONata is a lightweight query and transformation language for JSON data. Inspir * Install the module from [NPM](https://www.npmjs.com/package/jsonata) * Fork the repo on [GitHub](https://github.com/jsonata-js/jsonata) +## Implementations of JSONata + +The following are known implementations of JSONata in addition to the primary implementation in JavaScript in the above repo. + +|Language|Link|Notes|JSONata version| +|---|---|---|---| +|C|https://github.com/qlyoung/jsonata-c|Runs JSONata in embedded JS engine|1.8.3| +|Go|https://github.com/blues/jsonata-go|Native implementation|1.5.4| +|Go|https://github.com/yxuco/gojsonata|Native implementation| | +|Java|https://github.com/IBM/JSONata4Java|Native implementation| | +|Java|https://github.com/dashjoin/jsonata-java|Native port of reference|2.0.3| +|.NET|https://github.com/mikhail-barg/jsonata.net.native|Native implementation|1.8.5| +|Python|https://github.com/qlyoung/pyjsonata|API bindings based on C bindings|1.8.3| +|Rust|https://github.com/johanventer/jsonata-rust|Implementation work in progress| | +|Rust|https://github.com/Stedi/jsonata-rs|Actively-developed fork of jsonata-rust| | + ## Find out more * Introduction at [London Node User Group meetup](https://www.youtube.com/watch?v=TDWf6R8aqDo) -* IBM developerWorks [Tech Talk](https://www.youtube.com/watch?v=ZRtlkIj0uDY) diff --git a/docs/predicate.md b/docs/predicate.md index 380d4afb..4a518a03 100644 --- a/docs/predicate.md +++ b/docs/predicate.md @@ -31,7 +31,7 @@ At any step in a location path, the selected items can be filtered using a predi ## Singleton array and value equivalence -Within a JSONata expression or subexpression, any value (which is not itself an array) and an array containing just that value are deemed to be equivalent. This allows the language to be composable such that location paths that extract a single value from and object and location paths that extract multiple values from arrays can both be used as inputs to other expressions without needing to use different syntax for the two forms. +Within a JSONata expression or subexpression, any value (which is not itself an array) and an array containing just that value are deemed to be equivalent. This allows the language to be composable such that location paths that extract a single value from an object and location paths that extract multiple values from arrays can both be used as inputs to other expressions without needing to use different syntax for the two forms. Consider the following examples: diff --git a/docs/regex.md b/docs/regex.md index 2bed89ed..91c59607 100644 --- a/docs/regex.md +++ b/docs/regex.md @@ -31,7 +31,7 @@ Regexes are often used in query predicates (filter expressions) when selecting o `path.to.object[stringProperty ~> /regex/]` -The `~>` is the [chain operator](control-operators#chain), and its use here implies that the result of `/regex/` is a function. We'll see below that this is in fact the case. +The `~>` is the [chain operator](other-operators#-chain), and its use here implies that the result of `/regex/` is a function. We'll see below that this is in fact the case. __Examples__ diff --git a/docs/sorting-grouping.md b/docs/sorting-grouping.md index 85f1c24b..8c7e0ae6 100644 --- a/docs/sorting-grouping.md +++ b/docs/sorting-grouping.md @@ -12,7 +12,7 @@ Arrays contain an ordered collection of values. If you need to re-order the val 2. Using the [order-by](path-operators#order-by-) operator. -The [order-by](path-operators#order-by-) operator is a convenient syntax that can used directly in a path expression to sort the result sequences in ascending or descending order. The [`$sort()`](array-functions#sort) function requires more syntax to be written, but is more flexible and supports custom comparator functions. +The [order-by](path-operators#order-by-) operator is a convenient syntax that can be used directly in a path expression to sort the result sequences in ascending or descending order. The [`$sort()`](array-functions#sort) function requires more syntax to be written, but is more flexible and supports custom comparator functions. ## Grouping diff --git a/jsonata.d.ts b/jsonata.d.ts index e71583a4..257aac07 100644 --- a/jsonata.d.ts +++ b/jsonata.d.ts @@ -2,9 +2,14 @@ // Project: https://github.com/jsonata-js/jsonata // Definitions by: Nick and Michael M. Tiller -declare function jsonata(str: string): jsonata.Expression; +declare function jsonata(str: string, options?: jsonata.JsonataOptions): jsonata.Expression; declare namespace jsonata { + interface JsonataOptions { + recover?: boolean, + RegexEngine?: RegExp + } + interface ExprNode { type: string; value?: any; @@ -15,7 +20,7 @@ declare namespace jsonata { steps?: ExprNode[]; expressions?: ExprNode[]; stages?: ExprNode[]; - lhs?: ExprNode; + lhs?: ExprNode[]; rhs?: ExprNode; } diff --git a/package.json b/package.json index b9ad6ffa..1d5a0ff5 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@stedi/jsonata", - "version": "2.0.1", + "version": "2.0.4", "description": "JSON query and transformation language", "module": "jsonata.js", "main": "jsonata.js", diff --git a/src/datetime.js b/src/datetime.js index 00d80eab..01217658 100644 --- a/src/datetime.js +++ b/src/datetime.js @@ -904,6 +904,13 @@ const dateTime = (function () { if (offset === 0 && markerSpec.presentation2 === 't') { componentValue = 'Z'; } + } else if (markerSpec.component === 'P') { + // ยง9.8.4.7 Formatting Other Components + // Formatting P for am/pm + // getDateTimeFragment() always returns am/pm lower case so check for UPPER here + if (markerSpec.names === tcase.UPPER) { + componentValue = componentValue.toUpperCase(); + } } return componentValue; }; diff --git a/src/jsonata.js b/src/jsonata.js index 43aa61a9..845ce25d 100644 --- a/src/jsonata.js +++ b/src/jsonata.js @@ -1293,6 +1293,13 @@ var jsonata = (function() { } for(var ii = 0; ii < matches.length; ii++) { var match = matches[ii]; + if (match && (match.isPrototypeOf(result) || match instanceof Object.constructor)) { + throw { + code: "D1010", + stack: (new Error()).stack, + position: expr.position + }; + } // evaluate the update value for each match var update = await evaluate(expr.update, match, environment); // update must be an object @@ -1539,7 +1546,7 @@ var jsonata = (function() { if (typeof err.token == 'undefined' && typeof proc.token !== 'undefined') { err.token = proc.token; } - err.position = proc.position; + err.position = proc.position || err.position; } throw err; } @@ -1972,6 +1979,7 @@ var jsonata = (function() { "T1007": "Attempted to partially apply a non-function. Did you mean ${{{token}}}?", "T1008": "Attempted to partially apply a non-function", "D1009": "Multiple key definitions evaluate to same key: {{value}}", + "D1010": "Attempted to access the Javascript object prototype", // Javascript specific "T1010": "The matcher function argument passed to function {{token}} does not return the correct object structure", "T2001": "The left side of the {{token}} operator must evaluate to a number", "T2002": "The right side of the {{token}} operator must evaluate to a number", diff --git a/src/parser.js b/src/parser.js index adb525ad..629ff82c 100644 --- a/src/parser.js +++ b/src/parser.js @@ -76,9 +76,23 @@ const parser = (() => { var depth = 0; var pattern; var flags; + + var isClosingSlash = function (position) { + if (path.charAt(position) === '/' && depth === 0) { + var backslashCount = 0; + while (path.charAt(position - (backslashCount + 1)) === '\\') { + backslashCount++; + } + if (backslashCount % 2 === 0) { + return true; + } + } + return false; + }; + while (position < length) { var currentChar = path.charAt(position); - if (currentChar === '/' && path.charAt(position - 1) !== '\\' && depth === 0) { + if (isClosingSlash(position)) { // end of regex found pattern = path.substring(start, position); if (pattern === '') { diff --git a/test/implementation-tests.js b/test/implementation-tests.js index 4d85f1e7..8d8951d7 100644 --- a/test/implementation-tests.js +++ b/test/implementation-tests.js @@ -749,7 +749,38 @@ describe("Tests that are specific to a Javascript runtime", () => { }); }); + describe("empty regex: Escaped termination", function() { + it("should throw error", function() { + expect(function() { + var expr = jsonata("/\\/"); + expr.evaluate(); + }) + .to.throw() + .to.deep.contain({ position: 3, code: "S0302" }); + }); + }); + + describe("empty regex: Escaped termination", function() { + it("should throw error", function() { + expect(function() { + var expr = jsonata("/\\\\\\/"); + expr.evaluate(); + }) + .to.throw() + .to.deep.contain({ position: 5, code: "S0302" }); + }); + }); + describe("Functions - $match", function() { + describe('$match("test escape \\\\", /\\\\/)', function() { + it("should find \\", async function() { + var expr = jsonata('$match("test escape \\\\", /\\\\/)'); + var result = await expr.evaluate(); + var expected = { match: "\\", index: 12, groups: []}; + expect(result).to.deep.equal(expected); + }); + }); + describe('$match("ababbabbcc",/ab/)', function() { it("should return result object", async function() { var expr = jsonata('$match("ababbabbcc",/ab/)'); @@ -924,6 +955,35 @@ describe("Tests that are specific to a Javascript runtime", () => { }); }); }); + describe("Expressions that attempt to pollute the object prototype", function() { + it("should throw an error with __proto__", async function() { + const expr = jsonata('{} ~> | __proto__ | {"is_admin": true} |'); + expect( + expr.evaluate() + ).to.eventually.be.rejected.to.deep.contain({ + position: 7, + code: "D1010", + }); + }); + it("should throw an error with __lookupGetter__", async function() { + const expr = jsonata('{} ~> | __lookupGetter__("__proto__")() | {"is_admin": true} |'); + expect( + expr.evaluate() + ).to.eventually.be.rejected.to.deep.contain({ + position: 7, + code: "D1010", + }); + }); + it("should throw an error with constructor", async function() { + const expr = jsonata('{} ~> | constructor | {"is_admin": true} |'); + expect( + expr.evaluate() + ).to.eventually.be.rejected.to.deep.contain({ + position: 7, + code: "D1010", + }); + }); + }); }); describe("Test that yield platform specific results", () => { diff --git a/test/test-suite/groups/function-fromMillis/formatDateTime.json b/test/test-suite/groups/function-fromMillis/formatDateTime.json index 96e1962d..c08a9eaf 100644 --- a/test/test-suite/groups/function-fromMillis/formatDateTime.json +++ b/test/test-suite/groups/function-fromMillis/formatDateTime.json @@ -539,6 +539,22 @@ "2018-10-21T13:05:00.000Z" ] }, + { + "function": "#fromMillis", + "category": "Upper case AM/PM presentation", + "description": "am/pm presentation should be set to uppercase AM", + "expr": "$fromMillis(1521801216617, '[F], [D]/[M]/[Y] [h]:[m]:[s] [PN]')", + "data": {}, + "result": "friday, 23/3/2018 10:33:36 AM" + }, + { + "function": "#fromMillis", + "category": "Lower case AM/PM presentation", + "description": "am/pm presentation should be set to lowercase am", + "expr": "$fromMillis(1521801216617, '[F], [D]/[M]/[Y] [h]:[m]:[s] [Pn]')", + "data": {}, + "result": "friday, 23/3/2018 10:33:36 am" + }, { "function": "#fromMillis", "category": "error",