From ecbe15a13091f069108d6a4ce60a69a1f954d601 Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Wed, 12 Jun 2024 10:24:53 +0530 Subject: [PATCH 01/19] Update package.md file --- ballerina/Package.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/ballerina/Package.md b/ballerina/Package.md index 0d45c01..bfbe238 100644 --- a/ballerina/Package.md +++ b/ballerina/Package.md @@ -446,3 +446,12 @@ In this instance, all other elements in the XML data, such as `author` and `pric This behavior extends to arrays as well. The process of projecting XML data into a record supports various use cases, including the filtering out of unnecessary elements. This functionality is anticipated to be enhanced in the future to accommodate more complex scenarios, such as filtering values based on regular expressions, among others. + +## Report issues + +To report bugs, request new features, start new discussions, view project boards, etc., go to the [Ballerina library parent repository](https://github.com/ballerina-platform/ballerina-library). + +## Useful links + +- Chat live with us via our [Discord server](https://discord.gg/ballerinalang). +- Post all technical questions on Stack Overflow with the [#ballerina](https://stackoverflow.com/questions/tagged/ballerina) tag. From c5b862325d3100334312f3c6de18075d5a1ee5ee Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Wed, 12 Jun 2024 10:25:07 +0530 Subject: [PATCH 02/19] Add module.md file --- ballerina/Module.md | 448 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 448 insertions(+) create mode 100644 ballerina/Module.md diff --git a/ballerina/Module.md b/ballerina/Module.md new file mode 100644 index 0000000..0d45c01 --- /dev/null +++ b/ballerina/Module.md @@ -0,0 +1,448 @@ +# Ballerina XML Data Library + +The Ballerina XML Data Library is a comprehensive toolkit designed to facilitate the handling and manipulation of XML data within Ballerina applications. It streamlines the process of converting XML data to native Ballerina data types, enabling developers to work with XML content seamlessly and efficiently. + +This library is the refined successor of the `ballerina/xmldata` module, incorporating enhanced functionalities and improved performance. + +## Features + +- **Versatile XML Data Input**: Accept XML data as a xml, a string, byte array, or a stream and convert it into a Record value. +- **XML to Record Value Conversion**: Transform XML data into Ballerina records with ease in compliance with OpenAPI 3 standards. +- **Projection Support**: Perform selective conversion of XML data subsets into Record values through projection. + +## Usage + +### Converting an XML value to a Record value + +To convert an XML value to a Record value, you can utilize the `parseAsType` function provided by the library. The example below showcases the transformation of an XML value into a Record value. + +```ballerina +import ballerina/data.xmldata; +import ballerina/io; + +public function main() returns error? { + xml data = xml ` + 0 + string + string + `; + + Book book = check xmldata:parseAsType(data); + io:println(book); +} + +type Book record { + int id; + string title; + string author; +}; +``` + +### Converting an external XML document to a Record value + +For transforming XML content from an external source into a Record value, the `parseString`, `parseBytes`, `parseStream` functions can be used. This external source can be in the form of a string or a byte array/byte-block-stream that houses the XML data. This is commonly extracted from files or network sockets. The example below demonstrates the conversion of an XML value from an external source into a Record value. + +```ballerina +import ballerina/data.xmldata; +import ballerina/io; + +public function main() returns error? { + string xmlContent = check io:fileReadString("path/to/file.xml"); + Book book = check xmldata:parseString(xmlContent); + io:println(book); +} + +type Book record { + int id; + string title; + string author; +}; +``` + +Make sure to handle possible errors that may arise during the file reading or XML to record conversion process. The `check` keyword is utilized to handle these errors, but more sophisticated error handling can be implemented as per your requirements. + +## XML to Record Canonical Representation + +The translation of XML to a Record representation is a fundamental feature of the library. It facilitates a structured and type-safe approach to handling XML data within Ballerina applications. + +Take for instance the following XML snippet: + +```xml + + 601970 + string + string + +``` + +XML data is inherently hierarchical, forming a tree structure. In the given example, the root element is `book`, which encompasses three child elements: `id`, `title`, and `author`. The `id` element harbors a numeric value `0`, whereas both the `title` and `author` elements contain string values. + +A straightforward record representation of the above XML data is: + +```ballerina +type Book record { + int id; + string title; + string author; +}; +``` + +In this representation, the XML data is efficiently translated into a record value. The `book` element is mapped to a record of type `Book`, and the child elements `id`, `title`, and `author` are converted into record fields of types `int` and `string` correspondingly. + +This record type definition can be further refined through annotations. Moreover, utilizing open and closed records grants control over the translation process, which is elaborated in subsequent sections. + +### XML Element Names + +The name of the XML element serves as the name of the record field, altered to fit a valid Ballerina identifier. Notably, the record field name corresponds to the local name of the XML element, with any namespace prefixes being disregarded. + +Consider the XML snippet: + +```xml + + 601970 + string + string + +``` + +The canonical representation of the above XML as a Ballerina record is: + +```ballerina +type Book record { + int id; + string title\-name; + string author\-name; +}; +``` + +Observe how the XML element names `title-name` and `author-name` are represented using delimited identifiers in Ballerina; the `-` characters in the XML element names are escaped using the `\ ` character. + +Moreover, the `@Name` annotation can be utilized to explicitly specify the name of the record field, providing control over the translation process: + +```ballerina +import ballerina/data.xmldata; + +type Book record { + int id; + @xmldata:Name { value: "title-name" } + string title; + @xmldata:Name { value: "author-name" } + string author; +}; +``` + +### XML Attributes + +Similarly to XML elements, XML attributes are also represented into record fields within the corresponding parent Record type. The name of the XML attribute is converted into the name of the record field, ensuring it is a valid Ballerina identifier. It is crucial to emphasize that the record field name aligns with the local name of the XML attribute, and any namespace prefixes are ignored. + +Consider the following XML snippet: + +```xml + + 601970 + string + string + +``` + +The canonical representation of the above XML as a Ballerina record is: + +```ballerina +type Book record { + string lang; + decimal price; + int id; + string title; + string author; +}; +``` + +Additionally, the `@Attribute` annotation can be used to explicitly specify the field as an attribute providing control over the translation process. When element and attribute have same name in the same scope the priority is given to the element unless the expected record field has the `@Attribute` annotation. + +### Child Elements + +Child elements are mapped to record fields, with the type reflecting that of the corresponding child element. + +Examine the XML snippet below: + +```xml + + 601970 + string + + string + string + + +``` + +The canonical representation of the above XML as a Ballerina record is: + +```ballerina +type Book record { + int id; + string title; + Author author; +}; + +type Author record { + string name; + string country; +}; +``` + +In this transformation, child elements, like the `author` element containing its own sub-elements, are converted into nested records. This maintains the hierarchical structure of the XML data within the Ballerina type system, enabling intuitive and type-safe data manipulation. + +Alternatively, inline type definitions offer a compact method for representing child elements as records within their parent record. This approach is particularly beneficial when the child record does not require reuse elsewhere and is unique to its parent record. + +Consider the subsequent Ballerina record definition, which employs inline type definition for the `author` field: + +```ballerina +type Book record { + int id; + string title; + record { + string name; + string country; + } author; +}; +``` + +### XML Text Content + +The transformation of XML text content into record fields typically involves types like `string`, `boolean`, `int`, `float`, or `decimal`, depending on the textual content. For numeric values where type information is not explicitly defined, the default conversion type is `decimal`. Conversely, for non-numeric content, the default type is `string`. + +Consider the XML snippet below: + +```xml + + 601970 + string + string + true + 10.5 + +``` + +The translation into a Ballerina record would be as follows: + +```ballerina +type Book record { + int id; + string title; + string author; + boolean available; + decimal price; +}; +``` + +In scenarios where the parent XML element of text content also includes attributes, the XML text content can be represented by a `string` type field named `#content` within a record type, with the attributes being mapped to their respective fields. + +For instance, examine this XML: + +```xml + + 601970 + string + 10.5 + +``` + +The canonical translation of XML to a Ballerina record is as such: + +```ballerina +type Book record { + int id; + Title title; + decimal price; +}; + +type Title record { + string \#content; + string lang; +}; +``` + +### XML Namespaces + +XML namespaces are accommodated by the library, supporting the translation of XML data that contains namespace prefixes. However, the presence of XML namespaces is not mandatory, and the library is capable of processing XML data without namespaces. Should namespaces be present, they will be utilized to resolve the names of XML elements and attributes. + +It's important to note that, unlike in the `xmldata` module, the namespace prefixes do not reflect in the record field names, as the record field names align with the local names of the XML elements. + +Examine the XML snippet below with default namespaces: + +```xml + + 601970 + string + string + +``` + +The translation into a Ballerina record would be: + +```ballerina +type Book record { + int id; + string title; + string author; +}; +``` + +Incorporating namespace validation yields: + +```ballerina +import ballerina/data.xmldata; + +@xmldata:Namespace { + uri: "http://example.com/book" +} +type Book record { + int id; + string title; + string author; +}; +``` + +Here is the same XML snippet with a namespace prefix: + +```xml + + 601970 + string + string + +``` + +The translation into a Ballerina record would be: + +```ballerina +import ballerina/data.xmldata; + +@xmldata:Namespace { + prefix: "bk", + uri: "http://example.com/book" +} +type Book record {| + @xmldata:Namespace { + prefix: "bk", + uri: "http://example.com/book" + } + int id; + @xmldata:Namespace { + prefix: "bk", + uri: "http://example.com/book" + } + string title; + @xmldata:Namespace { + prefix: "bk", + uri: "http://example.com/book" + } + string author; +|}; +``` + +Here is the same XML snippet with a namespace prefix: + +```xml + + 601970 + string + string + +``` + +The translation into a Ballerina record would be: + +```ballerina +import ballerina/data.xmldata; + +@xmldata:Namespace { + uri: "http://example.com/book", + prefix: "bk" +} +type Book record {| + @xmldata:Namespace { + uri: "http://example.com/book", + prefix: "bk" + } + int id; + @xmldata:Namespace { + uri: "http://example.com/book", + prefix: "bk" + } + string title; + @xmldata:Namespace { + uri: "http://example.com/author", + prefix: "au" + } + string author; +|}; +``` + +In these examples, the XML namespaces are appropriately acknowledged, ensuring the integrity of the XML structure within the Ballerina records. + +### Working with Arrays + +The library is equipped to handle the transformation of XML data containing arrays into Ballerina records. + +Take the following XML snippet as an example: + +```xml + + 601970 + string + string + string + string + +``` + +The canonical representation of this XML as a Ballerina record is: + +```ballerina +type Book record { + int id; + string title; + string[] author; +}; +``` + +### Controlling Which Elements to Convert + +The library allows for selective conversion of XML elements into records through the use of rest fields. This is beneficial when the XML data contains elements that are not necessary to be transformed into record fields. + +Take this XML snippet as an example: + +```xml + + 601970 + string + string + 10.5 + +``` + +Suppose that only the book `id`, and `title` elements are needed for conversion into record fields. This can be achieved by defining only the required fields in the record type and omitting the rest field: + +```ballerina +type Book record {| + int id; + string title; +|}; +``` + +However, if the rest field is utilized (or if the record type is defined as an open record), all elements in the XML data will be transformed into record fields: + +```ballerina +type Book record { + int id; + string title; +}; +``` + +In this instance, all other elements in the XML data, such as `author` and `price` along with their attributes, will be transformed into `string` type fields with the corresponding element name as the key. + +This behavior extends to arrays as well. + +The process of projecting XML data into a record supports various use cases, including the filtering out of unnecessary elements. This functionality is anticipated to be enhanced in the future to accommodate more complex scenarios, such as filtering values based on regular expressions, among others. From 2b9e98c9561fb70855d47568ff79d7c0b32bc204 Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Wed, 12 Jun 2024 05:57:14 +0000 Subject: [PATCH 03/19] [Automated] Update the native jar versions --- ballerina/Ballerina.toml | 2 +- ballerina/CompilerPlugin.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ballerina/Ballerina.toml b/ballerina/Ballerina.toml index 1577886..afd9674 100644 --- a/ballerina/Ballerina.toml +++ b/ballerina/Ballerina.toml @@ -13,4 +13,4 @@ export = ["data.xmldata"] groupId = "io.ballerina.lib" artifactId = "data-native" version = "0.1.1" -path = "../native/build/libs/data.xmldata-native-0.1.1-SNAPSHOT.jar" +path = "../native/build/libs/data.xmldata-native-0.1.1.jar" diff --git a/ballerina/CompilerPlugin.toml b/ballerina/CompilerPlugin.toml index 98592f5..2a0250b 100644 --- a/ballerina/CompilerPlugin.toml +++ b/ballerina/CompilerPlugin.toml @@ -3,4 +3,4 @@ id = "constraint-compiler-plugin" class = "io.ballerina.lib.data.xmldata.compiler.XmldataCompilerPlugin" [[dependency]] -path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.1-SNAPSHOT.jar" +path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.1.jar" From 13e8b6e90e5e96318444bead3350446bcdaeeafd Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Wed, 12 Jun 2024 05:57:14 +0000 Subject: [PATCH 04/19] [Gradle Release Plugin] - pre tag commit: 'v0.1.1'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index 68e8d52..23427e1 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.1-SNAPSHOT +version=0.1.1 ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0 From 76f0153cade3fcf27b43dab9229ab168fa5266bf Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Wed, 12 Jun 2024 05:57:16 +0000 Subject: [PATCH 05/19] [Gradle Release Plugin] - new version commit: 'v0.1.2-SNAPSHOT'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index 23427e1..ee41fe4 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.1 +version=0.1.2-SNAPSHOT ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0 From 63e7b21131ff9c8ea289d586f28931698ba80a7d Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Thu, 4 Jul 2024 17:33:52 +0530 Subject: [PATCH 06/19] [Automated] Update the native jar versions --- ballerina/Ballerina.toml | 6 +++--- ballerina/CompilerPlugin.toml | 2 +- ballerina/Dependencies.toml | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/ballerina/Ballerina.toml b/ballerina/Ballerina.toml index afd9674..f948987 100644 --- a/ballerina/Ballerina.toml +++ b/ballerina/Ballerina.toml @@ -1,7 +1,7 @@ [package] org = "ballerina" name = "data.xmldata" -version = "0.1.1" +version = "0.1.2" authors = ["Ballerina"] keywords = ["xml"] repository = "https://github.com/ballerina-platform/module-ballerina-data-xmldata" @@ -12,5 +12,5 @@ export = ["data.xmldata"] [[platform.java17.dependency]] groupId = "io.ballerina.lib" artifactId = "data-native" -version = "0.1.1" -path = "../native/build/libs/data.xmldata-native-0.1.1.jar" +version = "0.1.2" +path = "../native/build/libs/data.xmldata-native-0.1.2-SNAPSHOT.jar" diff --git a/ballerina/CompilerPlugin.toml b/ballerina/CompilerPlugin.toml index 2a0250b..0ee6f03 100644 --- a/ballerina/CompilerPlugin.toml +++ b/ballerina/CompilerPlugin.toml @@ -3,4 +3,4 @@ id = "constraint-compiler-plugin" class = "io.ballerina.lib.data.xmldata.compiler.XmldataCompilerPlugin" [[dependency]] -path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.1.jar" +path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.2-SNAPSHOT.jar" diff --git a/ballerina/Dependencies.toml b/ballerina/Dependencies.toml index 57bd1b1..bcfbdef 100644 --- a/ballerina/Dependencies.toml +++ b/ballerina/Dependencies.toml @@ -10,7 +10,7 @@ distribution-version = "2201.9.0" [[package]] org = "ballerina" name = "data.xmldata" -version = "0.1.1" +version = "0.1.2" dependencies = [ {org = "ballerina", name = "io"}, {org = "ballerina", name = "jballerina.java"}, From 532f953b5f351c23e2240573fa205914bc93650e Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Thu, 4 Jul 2024 18:04:25 +0530 Subject: [PATCH 07/19] Fix record to xml conversion missing namespace --- ballerina/tests/toXml_test.bal | 56 +++++++++++++++++++ .../lib/data/xmldata/utils/DataUtils.java | 29 +++++++++- 2 files changed, 82 insertions(+), 3 deletions(-) diff --git a/ballerina/tests/toXml_test.bal b/ballerina/tests/toXml_test.bal index 0950703..76e9338 100644 --- a/ballerina/tests/toXml_test.bal +++ b/ballerina/tests/toXml_test.bal @@ -1165,6 +1165,62 @@ isolated function testRecordWithNamespaceAnnotationToXml1() returns error? { test:assertEquals(result.toString(), expected, msg = "testComplexRecordToXml result incorrect"); } +type AddressR record {| + string city; + int code; +|}; + +type Wsa_ReplyTo record { + @Namespace {prefix: "wsa", uri: "example1.com"} + AddressR Address; +}; + +type Htng_ReplyTo record { + @Namespace {prefix: "htng", uri: "example2.com"} + AddressR Address; +}; + +@Name {value: "soap"} +type Soap record { + @Name {value: "ReplyTo"} + @Namespace {prefix: "wsa", uri: "example1.com"} + Wsa_ReplyTo wsaReplyTo; + @Name {value: "ReplyTo"} + @Namespace {prefix: "htng", uri: "example2.com"} + Htng_ReplyTo htngReplyTo; +}; + +@test:Config { + groups: ["toXml"] +} +isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? { + Soap val = { + htngReplyTo: { + Address: { + code: 40000, + city: "Colombo" + } + }, + wsaReplyTo: { + Address: { + code: 10000, + city: "Kandy" + } + } + }; + + xml xmlVal = check toXml(val); + string expected = "" + + "" + + "Kandy10000" + + "" + + "" + + "Colombo40000" + + "" + + ""; + test:assertEquals(xmlVal.toString(), expected); +} + @test:Config { groups: ["toXml"] } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java index dfe0d70..e124c4e 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java @@ -459,14 +459,20 @@ private static BMap addFields(BMap input, Type processRecord(key, annotations, recordValue, value, fieldType); } else if (fieldType.getTag() == TypeTags.TYPE_REFERENCED_TYPE_TAG) { Type referredType = TypeUtils.getReferredType(fieldType); + BMap namespaceAnnotRecord = ValueCreator.createMapValue(Constants.JSON_MAP_TYPE); if (annotations.size() > 0) { + String fieldName = key; key = getKeyNameFromAnnotation(annotations, key); + QName qName = addFieldNamespaceAnnotation(fieldName, key, annotations, namespaceAnnotRecord); + String localPart = qName.getLocalPart(); + key = qName.getPrefix().isBlank() ? localPart : qName.getPrefix() + ":" + localPart; } BMap subRecordAnnotations = ((RecordType) referredType).getAnnotations(); key = getElementName(subRecordAnnotations, key); BMap annotationRecord = ValueCreator.createMapValue(Constants.JSON_MAP_TYPE); processSubRecordAnnotation(subRecordAnnotations, annotationRecord); BMap subRecordValue = addFields(((BMap) value), referredType); + addNamespaceToSubRecord(key, namespaceAnnotRecord, subRecordValue); if (annotationRecord.size() > 0) { subRecordValue.put(annotationRecord.getKeys()[0], annotationRecord.get(annotationRecord.getKeys()[0])); @@ -475,7 +481,7 @@ private static BMap addFields(BMap input, Type } else if (fieldType.getTag() == TypeTags.ARRAY_TAG) { processArray(fieldType, annotations, recordValue, entry); } else { - addPrimitiveValue(addFieldNamespaceAnnotation(key, annotations, recordValue), + addPrimitiveValue(addFieldNamespaceAnnotation(key, key, annotations, recordValue), annotations, recordValue, value); } } else { @@ -486,10 +492,27 @@ private static BMap addFields(BMap input, Type } @SuppressWarnings("unchecked") - private static QName addFieldNamespaceAnnotation(String key, BMap annotations, + private static void addNamespaceToSubRecord(String key, BMap namespaceAnnotRecord, + BMap subRecord) { + if (namespaceAnnotRecord.isEmpty()) { + return; + } + + Object value = namespaceAnnotRecord.get(StringUtils.fromString(key)); + if (value == null) { + return; + } + + for (Map.Entry nsAnnotEntry: ((BMap) value).entrySet()) { + subRecord.put(nsAnnotEntry.getKey(), nsAnnotEntry.getValue()); + } + } + + @SuppressWarnings("unchecked") + private static QName addFieldNamespaceAnnotation(String fieldName, String key, BMap annotations, BMap recordValue) { BString annotationKey = StringUtils.fromString(Constants.FIELD - + (key.replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0"))); + + (fieldName.replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0"))); boolean isAttributeField = isAttributeField(annotationKey, annotations); if (annotations.containsKey(annotationKey)) { BMap annotationValue = (BMap) annotations.get(annotationKey); From 60fad59d7dc18836f2f45cd40ed9a31a6636ab8d Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Thu, 4 Jul 2024 22:08:32 +0530 Subject: [PATCH 08/19] Fix incorrect xml when field name with underscore --- ballerina/tests/toXml_test.bal | 26 +++++++++++++++++++ .../lib/data/xmldata/utils/Constants.java | 2 +- .../lib/data/xmldata/utils/DataUtils.java | 8 +++--- 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/ballerina/tests/toXml_test.bal b/ballerina/tests/toXml_test.bal index 76e9338..374b734 100644 --- a/ballerina/tests/toXml_test.bal +++ b/ballerina/tests/toXml_test.bal @@ -1221,6 +1221,32 @@ isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? test:assertEquals(xmlVal.toString(), expected); } +type RequestorID record { + @Attribute + string ID; + @Attribute + string ID_Context; + @Attribute + string Type; +}; + +type Source record { + RequestorID RequestorID; +}; + +@test:Config { + groups: ["toXml"] +} +isolated function testUnderscoreInTheFieldName() returns error? { + Source s = { + RequestorID: { + ID: "1", + ID_Context: "2", + Type: "3"}}; + xml xmlVal = check toXml(s); + test:assertEquals(xmlVal.toString(), ""); +} + @test:Config { groups: ["toXml"] } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java index 2e3e477..bbf227b 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java @@ -58,6 +58,6 @@ private Constants() {} public static final BString ATTRIBUTE_PREFIX = StringUtils.fromString("attributePrefix"); public static final BString TEXT_FIELD_NAME = StringUtils.fromString("textFieldName"); public static final BString ALLOW_DATA_PROJECTION = StringUtils.fromString("allowDataProjection"); - public static final String NON_NUMERIC_STRING_REGEX = "[^a-zA-Z\\d\s]"; + public static final String RECORD_FIELD_NAME_ESCAPE_CHAR_REGEX = "[^a-zA-Z\\d\s_]"; public static final String NS_ANNOT_NOT_DEFINED = "$$ns_annot_not_defined$$"; } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java index e124c4e..1f3a4f6 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java @@ -512,7 +512,7 @@ private static void addNamespaceToSubRecord(String key, BMap na private static QName addFieldNamespaceAnnotation(String fieldName, String key, BMap annotations, BMap recordValue) { BString annotationKey = StringUtils.fromString(Constants.FIELD - + (fieldName.replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0"))); + + (fieldName.replaceAll(Constants.RECORD_FIELD_NAME_ESCAPE_CHAR_REGEX, "\\\\$0"))); boolean isAttributeField = isAttributeField(annotationKey, annotations); if (annotations.containsKey(annotationKey)) { BMap annotationValue = (BMap) annotations.get(annotationKey); @@ -547,7 +547,7 @@ private static BMap getFieldNamespaceAndNameAnnotations(String BMap nsFieldAnnotation = ValueCreator.createMapValue(Constants.JSON_MAP_TYPE); BString annotationKey = StringUtils.fromString((Constants.FIELD - + (key.replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0")))); + + (key.replaceAll(Constants.RECORD_FIELD_NAME_ESCAPE_CHAR_REGEX, "\\\\$0")))); if (!parentAnnotations.containsKey(annotationKey)) { return nsFieldAnnotation; } @@ -591,7 +591,7 @@ private static void addPrimitiveValue(QName qName, BMap annotat BString key = qName.getPrefix().isBlank() ? localPart : StringUtils.fromString(qName.getPrefix() + ":" + localPart); BString annotationKey = StringUtils.fromString(Constants.FIELD - + (localPart.getValue().replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0"))); + + (localPart.getValue().replaceAll(Constants.RECORD_FIELD_NAME_ESCAPE_CHAR_REGEX, "\\\\$0"))); BMap currentValue; if (record.containsKey(key)) { currentValue = (BMap) record.get(key); @@ -648,7 +648,7 @@ private static void processArray(Type childType, BMap annotatio @SuppressWarnings("unchecked") private static String getKeyNameFromAnnotation(BMap annotations, String keyName) { BString annotationKey = StringUtils.fromString(Constants.FIELD - + (keyName.replaceAll(Constants.NON_NUMERIC_STRING_REGEX, "\\\\$0"))); + + (keyName.replaceAll(Constants.RECORD_FIELD_NAME_ESCAPE_CHAR_REGEX, "\\\\$0"))); if (annotations.containsKey(annotationKey)) { BMap annotationValue = (BMap) annotations.get(annotationKey); return processFieldAnnotation(annotationValue, keyName); From 0f46dd1adad1875fe337070e0b9e95b614a9301a Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Thu, 4 Jul 2024 23:29:00 +0530 Subject: [PATCH 09/19] Handle namespace attached to the field and type --- ballerina/tests/toXml_test.bal | 88 +++++++++++++++---- .../lib/data/xmldata/utils/DataUtils.java | 17 +++- 2 files changed, 87 insertions(+), 18 deletions(-) diff --git a/ballerina/tests/toXml_test.bal b/ballerina/tests/toXml_test.bal index 374b734..689260c 100644 --- a/ballerina/tests/toXml_test.bal +++ b/ballerina/tests/toXml_test.bal @@ -15,6 +15,7 @@ // under the License. import ballerina/test; +import ballerina/io; @test:Config { groups: ["toXml"] @@ -1165,36 +1166,36 @@ isolated function testRecordWithNamespaceAnnotationToXml1() returns error? { test:assertEquals(result.toString(), expected, msg = "testComplexRecordToXml result incorrect"); } -type AddressR record {| +type AddressR1 record {| string city; int code; |}; -type Wsa_ReplyTo record { +type Wsa_ReplyTo1 record { @Namespace {prefix: "wsa", uri: "example1.com"} - AddressR Address; + AddressR1 Address; }; -type Htng_ReplyTo record { - @Namespace {prefix: "htng", uri: "example2.com"} - AddressR Address; +type Htng_ReplyTo1 record { + @Namespace {prefix: "wsa", uri: "example1.com"} + AddressR1 Address; }; @Name {value: "soap"} -type Soap record { +type Soap1 record { @Name {value: "ReplyTo"} @Namespace {prefix: "wsa", uri: "example1.com"} - Wsa_ReplyTo wsaReplyTo; + Wsa_ReplyTo1 wsaReplyTo; @Name {value: "ReplyTo"} @Namespace {prefix: "htng", uri: "example2.com"} - Htng_ReplyTo htngReplyTo; + Htng_ReplyTo1 htngReplyTo; }; @test:Config { groups: ["toXml"] } isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? { - Soap val = { + Soap1 val = { htngReplyTo: { Address: { code: 40000, @@ -1213,14 +1214,71 @@ isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? string expected = "" + "" + "Kandy10000" + - "" + - "" + - "Colombo40000" + - "" + - ""; + "" + + "Colombo40000" + + ""; + io:println(xmlVal); test:assertEquals(xmlVal.toString(), expected); } +@Namespace {prefix: "wsa", uri: "example1.com"} +type AddressR2 record {| + string city; + int code; +|}; + +@Namespace {prefix: "wsa", uri: "example1.com"} +type Wsa_ReplyTo2 record { + @Namespace {prefix: "wsa", uri: "example1.com"} + AddressR2 Address; +}; + +@Namespace {prefix: "htng", uri: "example2.com"} +type Htng_ReplyTo2 record { + @Namespace {prefix: "wsa", uri: "example1.com"} + AddressR2 Address; +}; + +@Name {value: "soap"} +type Soap2 record { + @Name {value: "ReplyTo"} + @Namespace {prefix: "wsa", uri: "example1.com"} + Wsa_ReplyTo2 wsaReplyTo; + @Name {value: "ReplyTo"} + @Namespace {prefix: "htng", uri: "example2.com"} + Htng_ReplyTo2 htngReplyTo; +}; + +@test:Config { + groups: ["toXml"] +} +isolated function testXmlToRecordWithNamespaceAttachedToFieldsAndTypes() returns error? { + Soap2 val = { + htngReplyTo: { + Address: { + code: 40000, + city: "Colombo" + } + }, + wsaReplyTo: { + Address: { + code: 10000, + city: "Kandy" + } + } + }; + + xml xmlVal = check toXml(val); + string expected = "" + + "" + + "Kandy10000" + + "" + + "Colombo40000" + + ""; + io:println(xmlVal); + test:assertEquals(xmlVal.toString(), expected); +} + type RequestorID record { @Attribute string ID; diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java index 1f3a4f6..95e7e27 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java @@ -460,17 +460,25 @@ private static BMap addFields(BMap input, Type } else if (fieldType.getTag() == TypeTags.TYPE_REFERENCED_TYPE_TAG) { Type referredType = TypeUtils.getReferredType(fieldType); BMap namespaceAnnotRecord = ValueCreator.createMapValue(Constants.JSON_MAP_TYPE); + boolean doesNamespaceDefinedInField = false; if (annotations.size() > 0) { String fieldName = key; key = getKeyNameFromAnnotation(annotations, key); QName qName = addFieldNamespaceAnnotation(fieldName, key, annotations, namespaceAnnotRecord); + if (!qName.getNamespaceURI().equals("")) { + doesNamespaceDefinedInField = true; + } String localPart = qName.getLocalPart(); key = qName.getPrefix().isBlank() ? localPart : qName.getPrefix() + ":" + localPart; } - BMap subRecordAnnotations = ((RecordType) referredType).getAnnotations(); - key = getElementName(subRecordAnnotations, key); + BMap annotationRecord = ValueCreator.createMapValue(Constants.JSON_MAP_TYPE); - processSubRecordAnnotation(subRecordAnnotations, annotationRecord); + if (!doesNamespaceDefinedInField) { + BMap subRecordAnnotations = ((RecordType) referredType).getAnnotations(); + key = getElementName(subRecordAnnotations, key); + processSubRecordAnnotation(subRecordAnnotations, annotationRecord); + } + BMap subRecordValue = addFields(((BMap) value), referredType); addNamespaceToSubRecord(key, namespaceAnnotRecord, subRecordValue); if (annotationRecord.size() > 0) { @@ -744,6 +752,9 @@ private static String getElementName(BMap annotation, String ke key = prefix.getValue().concat(Constants.COLON).concat(key); } } + } + + for (BString value : keys) { if (value.getValue().endsWith(Constants.NAME)) { key = processNameAnnotation(annotation, key, value, hasNamespaceAnnotation); } From b281d17237ca45f6f2e263aa2a8f56b07101df1f Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Fri, 5 Jul 2024 00:10:34 +0530 Subject: [PATCH 10/19] Fix annotation lookup logic --- ballerina/tests/toXml_test.bal | 33 ++++++++++++-- .../lib/data/xmldata/utils/Constants.java | 1 + .../lib/data/xmldata/utils/DataUtils.java | 44 ++++++++++++------- 3 files changed, 57 insertions(+), 21 deletions(-) diff --git a/ballerina/tests/toXml_test.bal b/ballerina/tests/toXml_test.bal index 689260c..e48ee94 100644 --- a/ballerina/tests/toXml_test.bal +++ b/ballerina/tests/toXml_test.bal @@ -15,7 +15,6 @@ // under the License. import ballerina/test; -import ballerina/io; @test:Config { groups: ["toXml"] @@ -1192,7 +1191,7 @@ type Soap1 record { }; @test:Config { - groups: ["toXml"] + groups: ["toXml", "testFail"] } isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? { Soap1 val = { @@ -1217,7 +1216,6 @@ isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? "" + "Colombo40000" + ""; - io:println(xmlVal); test:assertEquals(xmlVal.toString(), expected); } @@ -1275,7 +1273,6 @@ isolated function testXmlToRecordWithNamespaceAttachedToFieldsAndTypes() returns "" + "Colombo40000" + ""; - io:println(xmlVal); test:assertEquals(xmlVal.toString(), expected); } @@ -1305,6 +1302,34 @@ isolated function testUnderscoreInTheFieldName() returns error? { test:assertEquals(xmlVal.toString(), ""); } +@Namespace { + uri: "example.com" +} +type File record {| + @Namespace { + uri: "example.com" + } + string fileName; + @Namespace { + uri: "example.com" + } + string fileNamespace; +|}; + +@test:Config { + groups: ["toXml"] +} +isolated function testToRecordFieldNameEndsWithNameOrNamespace() returns error? { + File file = { + fileName: "test.bal", + fileNamespace: "wso2.com" + }; + + xml result = check toXml(file); + string expected = "test.balwso2.com"; + test:assertEquals(result.toString(), expected); +} + @test:Config { groups: ["toXml"] } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java index bbf227b..04830ac 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/Constants.java @@ -42,6 +42,7 @@ private Constants() {} public static final ArrayType JSON_ARRAY_TYPE = TypeCreator.createArrayType(PredefinedTypes.TYPE_JSON); public static final String FIELD = "$field$."; public static final String NAMESPACE = "Namespace"; + public static final String MODULE_NAME = "ballerina/data.xmldata"; public static final BString URI = StringUtils.fromString("uri"); public static final BString PREFIX = StringUtils.fromString("prefix"); public static final String ATTRIBUTE = "Attribute"; diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java index 95e7e27..f9c3b95 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java @@ -66,8 +66,7 @@ public static QualifiedName validateAndGetXmlNameFromRecordAnnotation(RecordType BMap annotations = recordType.getAnnotations(); String localName = recordName; for (BString annotationsKey : annotations.getKeys()) { - String key = annotationsKey.getValue(); - if (!key.contains(Constants.FIELD) && key.endsWith(Constants.NAME)) { + if (isNameAnnotationKey(annotationsKey.getValue())) { String name = ((BMap) annotations.get(annotationsKey)).get(Constants.VALUE).toString(); String localPart = elementName.getLocalPart(); if (!name.equals(localPart)) { @@ -81,7 +80,7 @@ public static QualifiedName validateAndGetXmlNameFromRecordAnnotation(RecordType // Handle the namespace annotation. for (BString annotationsKey : annotations.getKeys()) { String key = annotationsKey.getValue(); - if (!key.contains(Constants.FIELD) && key.endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(key)) { Map namespaceAnnotation = ((Map) annotations.get(StringUtils.fromString(key))); BString uri = (BString) namespaceAnnotation.get(Constants.URI); @@ -106,8 +105,7 @@ private static ArrayList getNamespace(RecordType recordType) { BMap annotations = recordType.getAnnotations(); ArrayList namespace = new ArrayList<>(); for (BString annotationsKey : annotations.getKeys()) { - String key = annotationsKey.getValue(); - if (!key.contains(Constants.FIELD) && key.endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(annotationsKey.getValue())) { BMap namespaceAnnotation = (BMap) annotations.get(annotationsKey); namespace.add(namespaceAnnotation.containsKey(Constants.PREFIX) ? ((BString) namespaceAnnotation.get(Constants.PREFIX)).getValue() : ""); @@ -184,7 +182,7 @@ public static Map getAllAttributesInRecordType(RecordType @SuppressWarnings("unchecked") public static QualifiedName getFieldNameFromRecord(Map fieldAnnotation, String fieldName) { for (BString key : fieldAnnotation.keySet()) { - if (key.getValue().endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(key.getValue())) { Map namespaceAnnotation = ((Map) fieldAnnotation.get(key)); BString uri = (BString) namespaceAnnotation.get(Constants.URI); BString prefix = (BString) namespaceAnnotation.get(Constants.PREFIX); @@ -198,7 +196,7 @@ public static QualifiedName getFieldNameFromRecord(Map fieldAnn @SuppressWarnings("unchecked") private static String getModifiedName(Map fieldAnnotation, String attributeName) { for (BString key : fieldAnnotation.keySet()) { - if (key.getValue().endsWith(Constants.NAME)) { + if (isNameAnnotationKey(key.getValue())) { return ((Map) fieldAnnotation.get(key)).get(Constants.VALUE).toString(); } } @@ -525,7 +523,7 @@ private static QName addFieldNamespaceAnnotation(String fieldName, String key, B if (annotations.containsKey(annotationKey)) { BMap annotationValue = (BMap) annotations.get(annotationKey); for (BString fieldKey : annotationValue.getKeys()) { - if (fieldKey.toString().endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(fieldKey.getValue())) { return processFieldNamespaceAnnotation(annotationValue, key, fieldKey, recordValue, isAttributeField); } @@ -542,7 +540,7 @@ public static boolean isAttributeField(BString annotationKey, BMap annotationValue = (BMap) annotations.get(annotationKey); for (BString fieldKey : annotationValue.getKeys()) { - if (fieldKey.toString().endsWith(Constants.ATTRIBUTE)) { + if (isAttributeAnnotationKey(fieldKey.getValue())) { return true; } } @@ -563,7 +561,7 @@ private static BMap getFieldNamespaceAndNameAnnotations(String BMap annotationValue = (BMap) parentAnnotations.get(annotationKey); for (BString fieldKey : annotationValue.getKeys()) { String keyName = fieldKey.getValue(); - if (keyName.endsWith(Constants.NAMESPACE) || keyName.endsWith(Constants.NAME)) { + if (isNamespaceAnnotationKey(keyName) || isNameAnnotationKey(keyName)) { nsFieldAnnotation.put(fieldKey, annotationValue.get(fieldKey)); break; } @@ -694,7 +692,7 @@ private static BMap processParentAnnotation(Type type, BMap annotation, String key) { for (BString value : annotation.getKeys()) { String stringValue = value.getValue(); - if (stringValue.endsWith(Constants.NAME)) { + if (isNameAnnotationKey(stringValue)) { BMap names = (BMap) annotation.get(value); String name = names.get(StringUtils.fromString(VALUE)).toString(); if (key.contains(Constants.COLON)) { @@ -705,7 +703,7 @@ private static String processFieldAnnotation(BMap annotation, S key = name; } } - if (stringValue.endsWith(Constants.ATTRIBUTE)) { + if (isAttributeAnnotationKey(stringValue)) { key = ATTRIBUTE_PREFIX.concat(key); } } @@ -718,10 +716,10 @@ private static BString processAnnotation(BMap annotation, Strin for (BString value : annotation.getKeys()) { if (!value.getValue().contains(Constants.FIELD)) { String stringValue = value.getValue(); - if (stringValue.endsWith(Constants.NAME)) { + if (isNameAnnotationKey(stringValue)) { key = processNameAnnotation(annotation, key, value, hasNamespaceAnnotation); } - if (stringValue.endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(stringValue)) { hasNamespaceAnnotation = true; key = processNamespaceAnnotation(annotation, key, value, namespaces); } @@ -733,7 +731,7 @@ private static BString processAnnotation(BMap annotation, Strin private static void processSubRecordAnnotation(BMap annotation, BMap subRecord) { BString[] keys = annotation.getKeys(); for (BString value : keys) { - if (value.getValue().endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(value.getValue())) { processNamespaceAnnotation(annotation, "", value, subRecord); } } @@ -744,7 +742,7 @@ private static String getElementName(BMap annotation, String ke BString[] keys = annotation.getKeys(); boolean hasNamespaceAnnotation = false; for (BString value : keys) { - if (value.getValue().endsWith(Constants.NAMESPACE)) { + if (isNamespaceAnnotationKey(value.getValue())) { hasNamespaceAnnotation = true; BMap namespaceAnnotation = (BMap) annotation.get(value); BString prefix = (BString) namespaceAnnotation.get(Constants.PREFIX); @@ -755,7 +753,7 @@ private static String getElementName(BMap annotation, String ke } for (BString value : keys) { - if (value.getValue().endsWith(Constants.NAME)) { + if (isNameAnnotationKey(value.getValue())) { key = processNameAnnotation(annotation, key, value, hasNamespaceAnnotation); } } @@ -816,6 +814,18 @@ private static String addAttributeToRecord(BString prefix, BString uri, String k return prefix.getValue().concat(Constants.COLON).concat(key); } + private static boolean isNamespaceAnnotationKey(String key) { + return key.startsWith(Constants.MODULE_NAME) && key.endsWith(Constants.NAMESPACE); + } + + private static boolean isNameAnnotationKey(String key) { + return key.startsWith(Constants.MODULE_NAME) && key.endsWith(Constants.NAME); + } + + private static boolean isAttributeAnnotationKey(String key) { + return key.startsWith(Constants.MODULE_NAME) && key.endsWith(Constants.ATTRIBUTE); + } + /** * Holds data required for the traversing. * From 526ecf2aed2c57e892493a54d8d9a56a8ba983a5 Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Sat, 6 Jul 2024 03:35:30 +0000 Subject: [PATCH 11/19] [Automated] Update the native jar versions --- ballerina/Ballerina.toml | 2 +- ballerina/CompilerPlugin.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ballerina/Ballerina.toml b/ballerina/Ballerina.toml index f948987..6e20373 100644 --- a/ballerina/Ballerina.toml +++ b/ballerina/Ballerina.toml @@ -13,4 +13,4 @@ export = ["data.xmldata"] groupId = "io.ballerina.lib" artifactId = "data-native" version = "0.1.2" -path = "../native/build/libs/data.xmldata-native-0.1.2-SNAPSHOT.jar" +path = "../native/build/libs/data.xmldata-native-0.1.2.jar" diff --git a/ballerina/CompilerPlugin.toml b/ballerina/CompilerPlugin.toml index 0ee6f03..0e62175 100644 --- a/ballerina/CompilerPlugin.toml +++ b/ballerina/CompilerPlugin.toml @@ -3,4 +3,4 @@ id = "constraint-compiler-plugin" class = "io.ballerina.lib.data.xmldata.compiler.XmldataCompilerPlugin" [[dependency]] -path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.2-SNAPSHOT.jar" +path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.2.jar" From e2ed4cf7ccab6e16beaea8b11f377ee2b24c96d9 Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Sat, 6 Jul 2024 03:35:30 +0000 Subject: [PATCH 12/19] [Gradle Release Plugin] - pre tag commit: 'v0.1.2'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index ee41fe4..f7afc51 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.2-SNAPSHOT +version=0.1.2 ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0 From 1ea69d3ab4f307a24a5b76fb2b33b55fb31af8bc Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Sat, 6 Jul 2024 03:35:32 +0000 Subject: [PATCH 13/19] [Gradle Release Plugin] - new version commit: 'v0.1.3-SNAPSHOT'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index f7afc51..b84f076 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.2 +version=0.1.3-SNAPSHOT ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0 From c75203ce3c4e29d082c9e0015543e4ba7075ce44 Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Sun, 7 Jul 2024 12:58:26 +0530 Subject: [PATCH 14/19] [Automated] Update the native jar versions --- ballerina/Ballerina.toml | 6 +++--- ballerina/CompilerPlugin.toml | 2 +- ballerina/Dependencies.toml | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/ballerina/Ballerina.toml b/ballerina/Ballerina.toml index 6e20373..5c74e53 100644 --- a/ballerina/Ballerina.toml +++ b/ballerina/Ballerina.toml @@ -1,7 +1,7 @@ [package] org = "ballerina" name = "data.xmldata" -version = "0.1.2" +version = "0.1.3" authors = ["Ballerina"] keywords = ["xml"] repository = "https://github.com/ballerina-platform/module-ballerina-data-xmldata" @@ -12,5 +12,5 @@ export = ["data.xmldata"] [[platform.java17.dependency]] groupId = "io.ballerina.lib" artifactId = "data-native" -version = "0.1.2" -path = "../native/build/libs/data.xmldata-native-0.1.2.jar" +version = "0.1.3" +path = "../native/build/libs/data.xmldata-native-0.1.3-SNAPSHOT.jar" diff --git a/ballerina/CompilerPlugin.toml b/ballerina/CompilerPlugin.toml index 0e62175..2299c31 100644 --- a/ballerina/CompilerPlugin.toml +++ b/ballerina/CompilerPlugin.toml @@ -3,4 +3,4 @@ id = "constraint-compiler-plugin" class = "io.ballerina.lib.data.xmldata.compiler.XmldataCompilerPlugin" [[dependency]] -path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.2.jar" +path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.3-SNAPSHOT.jar" diff --git a/ballerina/Dependencies.toml b/ballerina/Dependencies.toml index bcfbdef..d920883 100644 --- a/ballerina/Dependencies.toml +++ b/ballerina/Dependencies.toml @@ -10,7 +10,7 @@ distribution-version = "2201.9.0" [[package]] org = "ballerina" name = "data.xmldata" -version = "0.1.2" +version = "0.1.3" dependencies = [ {org = "ballerina", name = "io"}, {org = "ballerina", name = "jballerina.java"}, From 6109556bfd65b285eef24aa6560c4526bbdd7dd3 Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Mon, 8 Jul 2024 17:47:03 +0530 Subject: [PATCH 15/19] Handle attribute and element with same name --- ballerina/tests/fromXml_test.bal | 128 ++++++++++++++++++ ballerina/tests/toXml_test.bal | 2 +- .../lib/data/xmldata/compiler/Constants.java | 1 + .../compiler/XmldataRecordFieldValidator.java | 11 +- .../compiler/objects/QualifiedName.java | 6 +- .../lib/data/xmldata/utils/DataUtils.java | 33 ++++- .../lib/data/xmldata/xml/QualifiedName.java | 26 +++- .../data/xmldata/xml/QualifiedNameMap.java | 24 +++- .../lib/data/xmldata/xml/XmlParser.java | 29 ++-- 9 files changed, 231 insertions(+), 29 deletions(-) diff --git a/ballerina/tests/fromXml_test.bal b/ballerina/tests/fromXml_test.bal index 1163c0e..7ae07bc 100644 --- a/ballerina/tests/fromXml_test.bal +++ b/ballerina/tests/fromXml_test.bal @@ -2812,6 +2812,75 @@ function testProjectionWithXmlAttributeForParseAsType() returns error? { test:assertEquals(rec.A, "2"); } +type RecType record { + string name; + @Name { + value: "name" + } + @Attribute + string duplicateName; +}; + +@test:Config +isolated function testElementAndAttributeInSameScopeHaveSameName() returns error? { + string xmlStr = string ` + + Kanth + + `; + RecType rec = check parseString(xmlStr); + test:assertEquals(rec.name, "Kanth"); + test:assertEquals(rec.duplicateName, "Kevin"); + + xml xmlVal = xml ` + + Kanth + + `; + RecType rec2 = check parseAsType(xmlVal); + test:assertEquals(rec2.name, "Kanth"); + test:assertEquals(rec2.duplicateName, "Kevin"); +} + +type RecNs3 record {| + @Namespace { + prefix: "ns1", + uri: "example1.com" + } + string name; + @Name { + value: "name" + } + @Namespace { + prefix: "ns2", + uri: "example2.com" + } + string duplicateName; +|}; + +@test:Config +isolated function testElementWithDifferentNamespace() returns error? { + string xmlStr = string ` + + Kevin + Kanth + + `; + RecNs3 rec = check parseString(xmlStr); + test:assertEquals(rec.name, "Kevin"); + test:assertEquals(rec.duplicateName, "Kanth"); + + xml xmlVal = xml ` + + Kevin + Kanth + + `; + RecNs3 rec2 = check parseAsType(xmlVal); + test:assertEquals(rec2.name, "Kevin"); + test:assertEquals(rec2.duplicateName, "Kanth"); +} + // Negative cases type DataN1 record {| int A; @@ -3240,3 +3309,62 @@ function testInvalidNamespaceInOpenRecordForParseAsType2() { test:assertTrue(err is error); test:assertEquals((err).message(), "undefined field 'name' in record 'data.xmldata:AuthorOpen'"); } + +type RecTypeDup1 record { + string name; + @Name { + value: "name" + } + string duplicateName; +}; + +type RecTypeDup2 record { + @Namespace { + prefix: "ns", + uri: "example.com" + } + string name; + @Name { + value: "name" + } + string duplicateName; +}; + +@test:Config +isolated function testDuplicateField() { + string xmlStr = string ` + + Kanth + + `; + RecTypeDup1|Error err = parseString(xmlStr); + test:assertTrue(err is Error); + test:assertEquals(( err).message(), "duplicate field 'name'"); + + xml xmlVal = xml ` + + Kanth + + `; + RecTypeDup1|Error err2 = parseAsType(xmlVal); + test:assertTrue(err2 is Error); + test:assertEquals(( err2).message(), "duplicate field 'name'"); + + string xmlStr2 = string ` + + Kanth + + `; + RecTypeDup2|Error err3 = parseString(xmlStr2); + test:assertTrue(err3 is Error); + test:assertEquals(( err3).message(), "duplicate field 'name'"); + + xml xmlVal2 = xml ` + + Kanth + + `; + RecTypeDup2|Error err4 = parseAsType(xmlVal2); + test:assertTrue(err4 is Error); + test:assertEquals(( err4).message(), "duplicate field 'name'"); +} diff --git a/ballerina/tests/toXml_test.bal b/ballerina/tests/toXml_test.bal index e48ee94..9dc91e9 100644 --- a/ballerina/tests/toXml_test.bal +++ b/ballerina/tests/toXml_test.bal @@ -1191,7 +1191,7 @@ type Soap1 record { }; @test:Config { - groups: ["toXml", "testFail"] + groups: ["toXml"] } isolated function testXmlToRecordWithNamespaceAttachedToFields() returns error? { Soap1 val = { diff --git a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/Constants.java b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/Constants.java index d630676..b6c330e 100644 --- a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/Constants.java +++ b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/Constants.java @@ -13,6 +13,7 @@ public class Constants { static final String TO_XML = "toXml"; static final String NAME = "Name"; static final String NAMESPACE = "Namespace"; + static final String ATTRIBUTE = "Attribute"; static final String XMLDATA = "xmldata"; static final String BALLERINA = "ballerina"; static final String DATA_XMLDATA = "data.xmldata"; diff --git a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/XmldataRecordFieldValidator.java b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/XmldataRecordFieldValidator.java index bea7ac4..170ed26 100644 --- a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/XmldataRecordFieldValidator.java +++ b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/XmldataRecordFieldValidator.java @@ -83,7 +83,7 @@ public void perform(SyntaxNodeAnalysisContext ctx) { boolean erroneousCompilation = diagnostics.stream() .anyMatch(d -> d.diagnosticInfo().severity().equals(DiagnosticSeverity.ERROR)); if (erroneousCompilation) { - rest(); + reset(); return; } @@ -99,10 +99,10 @@ public void perform(SyntaxNodeAnalysisContext ctx) { } } - rest(); + reset(); } - private void rest() { + private void reset() { semanticModel = null; allDiagnosticInfo.clear(); modulePrefix = Constants.XMLDATA; @@ -376,6 +376,7 @@ private QualifiedName getQNameFromAnnotation(String fieldName, String uri = ""; String name = fieldName; String prefix = ""; + boolean isAttribute = false; for (AnnotationAttachmentSymbol annotAttSymbol : annotationAttachments) { AnnotationSymbol annotation = annotAttSymbol.typeDescriptor(); if (!getAnnotModuleName(annotation).contains(Constants.XMLDATA)) { @@ -397,9 +398,11 @@ private QualifiedName getQNameFromAnnotation(String fieldName, } uri = ((LinkedHashMap) annotAttSymbol.attachmentValue().orElseThrow().value()) .get("uri").toString(); + } else if (value.equals(Constants.ATTRIBUTE)) { + isAttribute = true; } } - return new QualifiedName(uri, name, prefix); + return new QualifiedName(uri, name, prefix, isAttribute); } private String getAnnotModuleName(AnnotationSymbol annotation) { diff --git a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/objects/QualifiedName.java b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/objects/QualifiedName.java index 7a2ab6b..f023435 100644 --- a/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/objects/QualifiedName.java +++ b/compiler-plugin/src/main/java/io/ballerina/lib/data/xmldata/compiler/objects/QualifiedName.java @@ -22,11 +22,13 @@ public class QualifiedName { private final String localPart; private final String namespaceURI; private final String prefix; + private boolean isAttributeDefined; - public QualifiedName(String namespaceURI, String localPart, String prefix) { + public QualifiedName(String namespaceURI, String localPart, String prefix, boolean isAttributeDefined) { this.localPart = localPart; this.namespaceURI = namespaceURI; this.prefix = prefix; + this.isAttributeDefined = isAttributeDefined; } @Override @@ -46,6 +48,6 @@ public boolean equals(Object objectToTest) { QualifiedName qName = (QualifiedName) objectToTest; return localPart.equals(qName.localPart) && namespaceURI.equals(qName.namespaceURI) && - prefix.equals(qName.prefix); + prefix.equals(qName.prefix) && (isAttributeDefined == qName.isAttributeDefined); } } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java index f9c3b95..3d26869 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/utils/DataUtils.java @@ -50,6 +50,10 @@ import javax.xml.namespace.QName; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.ATTRIBUTE; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.ELEMENT; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.NOT_DEFINED; + /** * A util class for the Data package's native implementation. * @@ -137,24 +141,29 @@ public static Map getAllFieldsInRecordType(RecordType reco Map> fieldNames = new HashMap<>(); Map recordFields = recordType.getFields(); for (String key : recordFields.keySet()) { + QualifiedNameMap attributeMap = analyzerData.attributeHierarchy.peek(); QualifiedName modifiedQName = modifiedNames.getOrDefault(key, new QualifiedName(Constants.NS_ANNOT_NOT_DEFINED, key, "")); String localName = modifiedQName.getLocalPart(); - if (fieldMap.containsKey(modifiedQName)) { + if (attributeMap.contains(modifiedQName) && modifiedQName.getAttributeState() == NOT_DEFINED) { + if (!key.equals(attributeMap.get(modifiedQName).getFieldName())) { + modifiedQName.setAttributeState(ELEMENT); + fieldMap.put(modifiedQName, recordFields.get(key)); + fieldNames.put(localName, new ArrayList<>(List.of(modifiedQName))); + } + } else if (fieldMap.containsKey(modifiedQName)) { throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); } else if (fieldNames.containsKey(localName)) { - if (modifiedQName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { - throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); - } List qNames = fieldNames.get(localName); qNames.forEach(qName -> { - if (qName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (DataUtils.isSameAttributeFlag(qName.getAttributeState(), modifiedQName.getAttributeState()) + && DataUtils.isSameNamespace(qName, modifiedQName)) { throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); } }); fieldMap.put(modifiedQName, recordFields.get(key)); fieldNames.get(localName).add(modifiedQName); - } else if (!analyzerData.attributeHierarchy.peek().contains(modifiedQName)) { + } else if (!attributeMap.contains(modifiedQName)) { fieldMap.put(modifiedQName, recordFields.get(key)); fieldNames.put(localName, new ArrayList<>(List.of(modifiedQName))); } @@ -172,6 +181,7 @@ public static Map getAllAttributesInRecordType(RecordType String attributeName = keyStr.split(Constants.FIELD_REGEX)[1].replaceAll("\\\\", ""); Map fieldAnnotation = (Map) annotations.get(annotationKey); QualifiedName fieldQName = getFieldNameFromRecord(fieldAnnotation, attributeName); + fieldQName.setAttributeState(ATTRIBUTE); fieldQName.setLocalPart(getModifiedName(fieldAnnotation, attributeName)); attributes.put(fieldQName, recordType.getFields().get(attributeName)); } @@ -365,6 +375,17 @@ public static void logArrayMismatchErrorIfProjectionNotAllowed(boolean allowData throw DiagnosticLog.error(DiagnosticErrorCode.ARRAY_SIZE_MISMATCH); } + public static boolean isSameNamespace(QualifiedName q1, QualifiedName q2) { + String ns1 = q1.getNamespaceURI(); + String ns2 = q2.getNamespaceURI(); + return (ns1.equals(ns2) && q1.getPrefix().equals(q2.getPrefix())) + || ns1.equals(Constants.NS_ANNOT_NOT_DEFINED) || ns2.equals(Constants.NS_ANNOT_NOT_DEFINED); + } + + public static boolean isSameAttributeFlag(QualifiedName.AttributeState flag1, QualifiedName.AttributeState flag2) { + return flag1 == NOT_DEFINED || flag2 == NOT_DEFINED || flag1.equals(flag2); + } + @SuppressWarnings("unchecked") public static Object getModifiedRecord(BMap input, BString textFieldName, BTypedesc type) { Type describingType = type.getDescribingType(); diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedName.java b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedName.java index 77e0dc9..198cf7d 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedName.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedName.java @@ -27,6 +27,20 @@ public class QualifiedName { private String localPart; private String namespaceURI; private String prefix; + private AttributeState attributeState = AttributeState.NOT_DEFINED; + + public enum AttributeState { + ATTRIBUTE, + ELEMENT, + NOT_DEFINED + } + + public QualifiedName(String namespaceURI, String localPart, String prefix, AttributeState attributeState) { + this.localPart = localPart; + this.namespaceURI = namespaceURI; + this.prefix = prefix; + this.attributeState = attributeState; + } public QualifiedName(String namespaceURI, String localPart, String prefix) { this.localPart = localPart; @@ -56,9 +70,17 @@ public String getPrefix() { return prefix; } + public void setAttributeState(AttributeState attributeState) { + this.attributeState = attributeState; + } + + public AttributeState getAttributeState() { + return this.attributeState; + } + @Override public int hashCode() { - return prefix.hashCode() ^ namespaceURI.hashCode() ^ localPart.hashCode(); + return prefix.hashCode() ^ namespaceURI.hashCode() ^ localPart.hashCode() ^ attributeState.hashCode(); } @Override @@ -72,6 +94,6 @@ public boolean equals(Object objectToTest) { } return localPart.equals(qName.localPart) && namespaceURI.equals(qName.namespaceURI) && - prefix.equals(qName.prefix); + prefix.equals(qName.prefix) && attributeState.equals(qName.attributeState); } } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java index 2a69d27..91e05c6 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java @@ -32,7 +32,8 @@ public V remove(QualifiedName qName) { List qNames = fields.get(localName); for (QualifiedName qualifiedName : fields.get(localName)) { - if (qualifiedName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (isSameNamespace(qualifiedName, qName) + && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { field = this.members.remove(qualifiedName); qNames.remove(qualifiedName); break; @@ -56,13 +57,26 @@ public boolean contains(QualifiedName qName) { return false; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (qualifiedName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (isSameNamespace(qualifiedName, qName) + && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { return true; } } return false; } + private boolean isSameNamespace(QualifiedName q1, QualifiedName q2) { + String ns1 = q1.getNamespaceURI(); + String ns2 = q2.getNamespaceURI(); + return (ns1.equals(ns2) && q1.getPrefix().equals(q2.getPrefix())) + || ns1.equals(Constants.NS_ANNOT_NOT_DEFINED) || ns2.equals(Constants.NS_ANNOT_NOT_DEFINED); + } + + private boolean isSameAttributeFlag(QualifiedName.AttributeState flag1, QualifiedName.AttributeState flag2) { + return (flag1 == QualifiedName.AttributeState.NOT_DEFINED + || flag2 == QualifiedName.AttributeState.NOT_DEFINED) || (flag1.equals(flag2)); + } + public boolean contains(String localName) { return stringToQNameMap.containsKey(localName); } @@ -89,7 +103,8 @@ public V get(QualifiedName qName) { return null; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (qualifiedName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (isSameNamespace(qualifiedName, qName) + && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { return members.get(qualifiedName); } } @@ -115,7 +130,8 @@ public QualifiedName getMatchedQualifiedName(QualifiedName elementQName) { return null; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (qualifiedName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (isSameNamespace(qualifiedName, elementQName) + && isSameAttributeFlag(qualifiedName.getAttributeState(), elementQName.getAttributeState())) { return qualifiedName; } } diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/XmlParser.java b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/XmlParser.java index 4c16820..a7e8084 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/XmlParser.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/XmlParser.java @@ -62,6 +62,9 @@ import static javax.xml.stream.XMLStreamConstants.END_ELEMENT; import static javax.xml.stream.XMLStreamConstants.PROCESSING_INSTRUCTION; import static javax.xml.stream.XMLStreamConstants.START_ELEMENT; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.ATTRIBUTE; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.ELEMENT; +import static io.ballerina.lib.data.xmldata.xml.QualifiedName.AttributeState.NOT_DEFINED; /** * Convert Xml string to a ballerina record. @@ -892,24 +895,29 @@ private Map getAllFieldsInRecordType(RecordType recordType Map> fieldNames = new HashMap<>(); Map recordFields = recordType.getFields(); for (String key : recordFields.keySet()) { + QualifiedNameMap attributeMap = xmlParserData.attributeHierarchy.peek(); QualifiedName modifiedQName = modifiedNames.getOrDefault(key, new QualifiedName(Constants.NS_ANNOT_NOT_DEFINED, key, "")); String localName = modifiedQName.getLocalPart(); - if (fieldMap.containsKey(modifiedQName)) { + if (attributeMap.contains(modifiedQName) && modifiedQName.getAttributeState() == NOT_DEFINED) { + if (!key.equals(attributeMap.get(modifiedQName).getFieldName())) { + modifiedQName.setAttributeState(ELEMENT); + fieldMap.put(modifiedQName, recordFields.get(key)); + fieldNames.put(localName, new ArrayList<>(List.of(modifiedQName))); + } + } else if (fieldMap.containsKey(modifiedQName)) { throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); } else if (fieldNames.containsKey(localName)) { - if (modifiedQName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { - throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); - } List qNames = fieldNames.get(localName); qNames.forEach(qName -> { - if (qName.getNamespaceURI().equals(Constants.NS_ANNOT_NOT_DEFINED)) { + if (DataUtils.isSameAttributeFlag(qName.getAttributeState(), modifiedQName.getAttributeState()) + && DataUtils.isSameNamespace(qName, modifiedQName)) { throw DiagnosticLog.error(DiagnosticErrorCode.DUPLICATE_FIELD, localName); } }); fieldMap.put(modifiedQName, recordFields.get(key)); fieldNames.get(localName).add(modifiedQName); - } else if (!xmlParserData.attributeHierarchy.peek().contains(modifiedQName)) { + } else if (!attributeMap.contains(modifiedQName)) { fieldMap.put(modifiedQName, recordFields.get(key)); fieldNames.put(localName, new ArrayList<>(List.of(modifiedQName))); } @@ -928,6 +936,7 @@ private Map getAllAttributesInRecordType(RecordType record Map fieldAnnotation = (Map) annotations.get(annotationKey); QualifiedName fieldQName = DataUtils.getFieldNameFromRecord(fieldAnnotation, attributeName); fieldQName.setLocalPart(getModifiedName(fieldAnnotation, attributeName)); + fieldQName.setAttributeState(ATTRIBUTE); attributes.put(fieldQName, recordType.getFields().get(attributeName)); } } @@ -948,7 +957,8 @@ private void handleAttributes(XMLStreamReader xmlStreamReader, XmlParserData xml for (int i = 0; i < xmlStreamReader.getAttributeCount(); i++) { QName attributeQName = xmlStreamReader.getAttributeName(i); QualifiedName attQName = new QualifiedName(attributeQName.getNamespaceURI(), - xmlParserData.attributePrefix + attributeQName.getLocalPart(), attributeQName.getPrefix()); + xmlParserData.attributePrefix + attributeQName.getLocalPart(), attributeQName.getPrefix(), + ATTRIBUTE); Field field = xmlParserData.attributeHierarchy.peek().remove(attQName); if (field == null) { Optional f = getFieldFromFieldHierarchy(attQName, xmlParserData); @@ -984,8 +994,7 @@ private void handleAttributesRest(XMLStreamReader xmlStreamReader, Type restType for (int i = 0; i < xmlStreamReader.getAttributeCount(); i++) { QName attributeQName = xmlStreamReader.getAttributeName(i); QualifiedName attQName = new QualifiedName(attributeQName.getNamespaceURI(), - attributeQName.getLocalPart(), attributeQName.getPrefix()); - + attributeQName.getLocalPart(), attributeQName.getPrefix(), ATTRIBUTE); try { mapNode.put(StringUtils.fromString(attQName.getLocalPart()), convertStringToRestExpType( StringUtils.fromString(xmlStreamReader.getAttributeValue(i)), restType)); @@ -1043,7 +1052,7 @@ private void updateStacksWhenRecordAsRestType(QualifiedName elementQName, XmlPar private QualifiedName getElementName(XMLStreamReader xmlStreamReader) { QName qName = xmlStreamReader.getName(); - return new QualifiedName(qName.getNamespaceURI(), qName.getLocalPart(), qName.getPrefix()); + return new QualifiedName(qName.getNamespaceURI(), qName.getLocalPart(), qName.getPrefix(), ELEMENT); } /** From efe4da67d8f15b671d88b7b9179f7793c15280fb Mon Sep 17 00:00:00 2001 From: prakanth <50439067+prakanth97@users.noreply.github.com> Date: Tue, 9 Jul 2024 11:05:49 +0530 Subject: [PATCH 16/19] Address review suggestions --- ballerina/tests/fromXml_test.bal | 33 +++++++++++++++---- .../data/xmldata/xml/QualifiedNameMap.java | 32 +++++++----------- 2 files changed, 37 insertions(+), 28 deletions(-) diff --git a/ballerina/tests/fromXml_test.bal b/ballerina/tests/fromXml_test.bal index 7ae07bc..9785d5f 100644 --- a/ballerina/tests/fromXml_test.bal +++ b/ballerina/tests/fromXml_test.bal @@ -2812,7 +2812,7 @@ function testProjectionWithXmlAttributeForParseAsType() returns error? { test:assertEquals(rec.A, "2"); } -type RecType record { +type RecType1 record { string name; @Name { value: "name" @@ -2821,6 +2821,17 @@ type RecType record { string duplicateName; }; +type RecType2 record { + record {| + string \#content; + |} name; + @Name { + value: "name" + } + @Attribute + string duplicateName; +}; + @test:Config isolated function testElementAndAttributeInSameScopeHaveSameName() returns error? { string xmlStr = string ` @@ -2828,18 +2839,26 @@ isolated function testElementAndAttributeInSameScopeHaveSameName() returns error Kanth `; - RecType rec = check parseString(xmlStr); - test:assertEquals(rec.name, "Kanth"); - test:assertEquals(rec.duplicateName, "Kevin"); + RecType1 rec11 = check parseString(xmlStr); + test:assertEquals(rec11.name, "Kanth"); + test:assertEquals(rec11.duplicateName, "Kevin"); + + RecType2 rec12 = check parseString(xmlStr); + test:assertEquals(rec12.name.\#content, "Kanth"); + test:assertEquals(rec12.duplicateName, "Kevin"); xml xmlVal = xml ` Kanth `; - RecType rec2 = check parseAsType(xmlVal); - test:assertEquals(rec2.name, "Kanth"); - test:assertEquals(rec2.duplicateName, "Kevin"); + RecType1 rec21 = check parseAsType(xmlVal); + test:assertEquals(rec21.name, "Kanth"); + test:assertEquals(rec21.duplicateName, "Kevin"); + + RecType2 rec22 = check parseAsType(xmlVal); + test:assertEquals(rec22.name.\#content, "Kanth"); + test:assertEquals(rec22.duplicateName, "Kevin"); } type RecNs3 record {| diff --git a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java index 91e05c6..f32cc75 100644 --- a/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java +++ b/native/src/main/java/io/ballerina/lib/data/xmldata/xml/QualifiedNameMap.java @@ -1,6 +1,6 @@ package io.ballerina.lib.data.xmldata.xml; -import io.ballerina.lib.data.xmldata.utils.Constants; +import io.ballerina.lib.data.xmldata.utils.DataUtils; import java.util.ArrayList; import java.util.HashMap; @@ -32,8 +32,9 @@ public V remove(QualifiedName qName) { List qNames = fields.get(localName); for (QualifiedName qualifiedName : fields.get(localName)) { - if (isSameNamespace(qualifiedName, qName) - && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { + if (DataUtils.isSameNamespace(qualifiedName, qName) + && DataUtils.isSameAttributeFlag(qualifiedName.getAttributeState(), + qName.getAttributeState())) { field = this.members.remove(qualifiedName); qNames.remove(qualifiedName); break; @@ -57,26 +58,14 @@ public boolean contains(QualifiedName qName) { return false; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (isSameNamespace(qualifiedName, qName) - && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { + if (DataUtils.isSameNamespace(qualifiedName, qName) + && DataUtils.isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { return true; } } return false; } - private boolean isSameNamespace(QualifiedName q1, QualifiedName q2) { - String ns1 = q1.getNamespaceURI(); - String ns2 = q2.getNamespaceURI(); - return (ns1.equals(ns2) && q1.getPrefix().equals(q2.getPrefix())) - || ns1.equals(Constants.NS_ANNOT_NOT_DEFINED) || ns2.equals(Constants.NS_ANNOT_NOT_DEFINED); - } - - private boolean isSameAttributeFlag(QualifiedName.AttributeState flag1, QualifiedName.AttributeState flag2) { - return (flag1 == QualifiedName.AttributeState.NOT_DEFINED - || flag2 == QualifiedName.AttributeState.NOT_DEFINED) || (flag1.equals(flag2)); - } - public boolean contains(String localName) { return stringToQNameMap.containsKey(localName); } @@ -103,8 +92,8 @@ public V get(QualifiedName qName) { return null; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (isSameNamespace(qualifiedName, qName) - && isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { + if (DataUtils.isSameNamespace(qualifiedName, qName) + && DataUtils.isSameAttributeFlag(qualifiedName.getAttributeState(), qName.getAttributeState())) { return members.get(qualifiedName); } } @@ -130,8 +119,9 @@ public QualifiedName getMatchedQualifiedName(QualifiedName elementQName) { return null; } for (QualifiedName qualifiedName : stringToQNameMap.get(localName)) { - if (isSameNamespace(qualifiedName, elementQName) - && isSameAttributeFlag(qualifiedName.getAttributeState(), elementQName.getAttributeState())) { + if (DataUtils.isSameNamespace(qualifiedName, elementQName) + && DataUtils.isSameAttributeFlag(qualifiedName.getAttributeState(), + elementQName.getAttributeState())) { return qualifiedName; } } From 1c59408bf65d7a2bbb5adc6b91c0ebee83b829e5 Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Tue, 9 Jul 2024 11:19:46 +0000 Subject: [PATCH 17/19] [Automated] Update the native jar versions --- ballerina/Ballerina.toml | 2 +- ballerina/CompilerPlugin.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ballerina/Ballerina.toml b/ballerina/Ballerina.toml index 5c74e53..89f5455 100644 --- a/ballerina/Ballerina.toml +++ b/ballerina/Ballerina.toml @@ -13,4 +13,4 @@ export = ["data.xmldata"] groupId = "io.ballerina.lib" artifactId = "data-native" version = "0.1.3" -path = "../native/build/libs/data.xmldata-native-0.1.3-SNAPSHOT.jar" +path = "../native/build/libs/data.xmldata-native-0.1.3.jar" diff --git a/ballerina/CompilerPlugin.toml b/ballerina/CompilerPlugin.toml index 2299c31..edc2310 100644 --- a/ballerina/CompilerPlugin.toml +++ b/ballerina/CompilerPlugin.toml @@ -3,4 +3,4 @@ id = "constraint-compiler-plugin" class = "io.ballerina.lib.data.xmldata.compiler.XmldataCompilerPlugin" [[dependency]] -path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.3-SNAPSHOT.jar" +path = "../compiler-plugin/build/libs/data.xmldata-compiler-plugin-0.1.3.jar" From 319ee053ce2f821fba37de1303ceeebd9b58f4d3 Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Tue, 9 Jul 2024 11:19:46 +0000 Subject: [PATCH 18/19] [Gradle Release Plugin] - pre tag commit: 'v0.1.3'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index b84f076..b2b09e1 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.3-SNAPSHOT +version=0.1.3 ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0 From 8f6e01a3f3ae2cacb91e4e2021f80443b3757d4d Mon Sep 17 00:00:00 2001 From: ballerina-bot Date: Tue, 9 Jul 2024 11:19:47 +0000 Subject: [PATCH 19/19] [Gradle Release Plugin] - new version commit: 'v0.1.4-SNAPSHOT'. --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index b2b09e1..89e0012 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ org.gradle.caching=true group=io.ballerina.lib -version=0.1.3 +version=0.1.4-SNAPSHOT ballerinaLangVersion=2201.9.0 checkstyleToolVersion=10.12.0