Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correction de la page magento, ajout de la refonte des filtres #74

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Correction de la page magento, ajout de la refonte des filtres #74

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6916492
Add a chapter on the long filter for magento plugin
JoMessina Jan 10, 2024
8d9be30
Improve description, add some fancy things
JoMessina Jan 10, 2024
02352cd
Merge branch 'master' into feature/magento-2-logical-cursor-for-long-…
JoMessina Jan 24, 2024
c3f4f79
Edit the chapter on magento to include the refacto of filters, fix th…
JoMessina Jan 24, 2024
935b394
Edit the chapter on magento to include the refacto of filters, fix th…
JoMessina Feb 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 44 additions & 39 deletions content/connectivity/magento-2/_index.en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ This package includes classes to extract data from Magento, using a [custom conn
```shell
composer require php-etl/magento2-flow:'*'
```
if you are still on satellite 0.6, use this :
```shell
composer require php-etl/magento2-flow:'v0.2'
```

This package includes classes and code that you will be able to use in your custom connector.

Expand All @@ -35,12 +39,12 @@ The package includes the following extractor classes: `CustomerExtractor`, `Invo

Extractor classes take 4 arguments:

| name | description | type | default value |
|---------------|--------------------------------------------------------------------------------------------------|--------------------------|---------------|
| logger | the service that will log exceptions | \Psr\Log\LoggerInterface | |
| client | client to choose depending on the Magento version. Available clients are: V2_1, V2_2, V2_3, V2_4 | Client | |
| page size | (Optional) maximum amount of entities to retrieve in a single payload | int | 100 |
| filter groups | (Optional) groups of filters to use when searching for entities | array | [] |
| name | description | type | default value |
|------------------|-------------------------------------------------------------------------------------------------------------|--------------------------|---------------|
| logger | the service that will log exceptions | \Psr\Log\LoggerInterface | |
| client | client parameter to communicate with magento2 api. Compatibility magento2 api versions : 2.1, 2.2, 2.3, 2.4 | Client | |
| query parameters | query parameters sent to the api which contains groups of filters to use when searching for entities | array | [] |
| page size | (Optional) maximum amount of entities to retrieve in a single payload | int | 100 |

```yaml
custom:
Expand All @@ -51,13 +55,13 @@ custom:
public: true
arguments:
- '@Monolog\Logger' # Logger
- '@Kiboko\Magento\V2_1\Client' # Client
- '@Kiboko\Magento\Client' # Client
- [] # QueryParameters, contains filters
- 500 # Page size
- [] # Filter groups

Kiboko\Magento\V2_1\Client:
Kiboko\Magento\Client:
factory:
class: 'Kiboko\Magento\V2_1\Client' # Client
class: 'Kiboko\Magento\Client' # Client
method: 'create'
arguments:
- '@Http\Client\Common\PluginClient'
Expand Down Expand Up @@ -89,7 +93,7 @@ custom:
- 300 # Log level. 300 for Warning, 200 for Info...
```

#### With filters
#### With ScalarFilters
Filters and filter groups can be specified.
Filters in a group are chained with `OR`. Groups are chained with `AND`.

Expand All @@ -100,19 +104,23 @@ In this example we will search for customers that were updated after 1985 (`@dat
public: true
arguments:
- '@Monolog\Logger'
- '@Kiboko\Magento\V2_1\Client'
- '@Kiboko\Magento\Client'
- '@query_parameters'
- 500
- [ '@date_filter_group', '@id_filter_group' ]
# updated_at >= 1985-10-26 11:25:00 AND (entity_id = 17 OR entity_id = 46)

# ...

query_parameters:
class: Kiboko\Component\Flow\Magento2\QueryParameters
calls:
- withGroups: [ '@id_filter_group', '@date_filter_group' ]
# updated_at >= 1985-10-26 11:25:00 AND (entity_id = 17 OR entity_id = 46)
date_filter_group:
class: Kiboko\Component\Flow\Magento2\FilterGroup
calls:
- withFilter: [ '@last_execution' ]
last_execution:
class: Kiboko\Component\Flow\Magento2\Filter
class: Kiboko\Component\Flow\Magento2\Filter\ScalarFilter
arguments:
- 'updated_at'
- 'gteq'
Expand All @@ -123,48 +131,49 @@ In this example we will search for customers that were updated after 1985 (`@dat
calls:
- withFilter: [ '@id_to_check', '@other_id' ]
id_to_check:
class: Kiboko\Component\Flow\Magento2\Filter
class: Kiboko\Component\Flow\Magento2\Filter\ScalarFilter
arguments:
- 'entity_id'
- 'eq'
- '17'
other_id:
class: Kiboko\Component\Flow\Magento2\Filter
class: Kiboko\Component\Flow\Magento2\Filter\ScalarFilter
arguments:
- 'entity_id'
- 'eq'
- '46'
# ...
```

#### With long filter
#### With ArrayFilter
Filters are passed to the url.
But the most popular web browsers will not work with URLs over 2000 characters, and would return a 414 (Request-URI Too Long).
You can use the method `withLongFilter` to avoid this limitation and batch your request in multiple smaller requests.
You can use the class `ArrayFilter` to avoid this limitation and batch your request in multiple smaller requests.

In this example we will search for specific orders with a lot of elements in the request's filter.
We have 214 increment_id, and we use a `withLongFilter` with parameters:
- `@order_increment_id` references our order's filter.
- `offset`, starts the request at the chosen index, by default we have 0.
- `length`, defines a batch length, by default we have 200.
We have 214 increment_id, and we use the `ArrayFilter` with parameters:
- `increment_id`: name of the field to filter.
- `in` operator to the filter, you need the 'in' operator to use the ArrayFilter.
- `000000526,4000000026,00000918,000001754,6000000123,4000000150,6000000185,000003798,6000000211,[..],5000000445` defines the target values.
- `150` defines the lenght of your smaller request (by default set to 200).

Here we have set an offset to 0 and a length to 150, it means we are starting the request from the first element and make multiple requests with 150 items max.
```yaml
# ...
order_filter_group:
class: Kiboko\Component\Flow\Magento2\FilterGroup
query_parameters:
class: Kiboko\Component\Flow\Magento2\QueryParameters
calls:
- withLongFilter: [ '@order_filter' ]
- withGroup: [ '@order_filter' ]
order_filter:
class: Kiboko\Component\Flow\Magento2\FilterGroup
calls:
- withLongFilter: ['@order_increment_id', 0, 150]
- withFilter: ['@order_increment_id']
order_increment_id:
class: Kiboko\Component\Flow\Magento2\Filter
class: Kiboko\Component\Flow\Magento2\Filter\ArrayFilter
arguments:
- 'increment_id'
- 'in'
- '000000526,4000000026,00000918,000001754,6000000123,4000000150,6000000185,000003798,6000000211,[..],5000000445'
- 150
# ...
```

Expand All @@ -182,17 +191,15 @@ custom:
public: true
arguments:
- '@Monolog\Logger'
- '@Kiboko\Magento\V2_3\Client' # Client to use depending on the Magento version.
# Available clients are:
# V2_1, V2_2, V2_3, V2_4
- '@Kiboko\Magento\Client'
- '@Symfony\Component\Cache\Psr16Cache'
- 'category.%s'
- '@Acme\Custom\LookupMapper' # Your custom mapper class
- 'category_name' # Index of the category ID, in your line.

Kiboko\Magento\V2_3\Client:
Kiboko\Magento\Client:
factory:
class: 'Kiboko\Magento\V2_3\Client' # Client
class: 'Kiboko\Magento\Client' # Client
method: 'create'
arguments:
- '@Http\Client\Common\PluginClient'
Expand Down Expand Up @@ -241,18 +248,16 @@ custom:
public: true
arguments:
- '@Monolog\Logger'
- '@Kiboko\Magento\V2_3\Client' # Client to use depending on the Magento version.
# Available clients are:
# V2_1, V2_2, V2_3, V2_4
- '@Kiboko\Magento\Client'
- '@Symfony\Component\Cache\Psr16Cache'
- 'collection.%s' # Cache key
- '@Acme\Custom\LookupMapper' # Your custom mapper class
- 'Collection' # Index of the attribute ID, in your line.
- 'qv_collection' # Attribute code

Kiboko\Magento\V2_3\Client:
Kiboko\Magento\Client:
factory:
class: 'Kiboko\Magento\V2_3\Client' # Client
class: 'Kiboko\Magento\Client' # Client
method: 'create'
arguments:
- '@Http\Client\Common\PluginClient'
Expand Down
Loading