Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data transfer #1587

Open
wants to merge 44 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
4b9bdf0
data transfer module added.
suraj-webkul Sep 6, 2024
681e238
seting up the datatransfer.
suraj-webkul Sep 6, 2024
9110883
person importing wihout queue is completed.
suraj-webkul Sep 9, 2024
30a6f97
fix the ui of index and create page.
suraj-webkul Sep 9, 2024
376b97d
Person import is done.
suraj-webkul Sep 9, 2024
59a918f
fix failed pint test cases.
suraj-webkul Sep 9, 2024
b41fa33
prevent to duplicate.
suraj-webkul Sep 9, 2024
f201e30
product import.
suraj-webkul Sep 9, 2024
5699325
product importation.
suraj-webkul Sep 9, 2024
53704c4
product importing doen.
suraj-webkul Sep 9, 2024
ffcf880
upload sample.
suraj-webkul Sep 9, 2024
d51e235
update samples.
suraj-webkul Sep 9, 2024
23aacac
minor changes.
suraj-webkul Sep 9, 2024
7b11d35
Minor changes.
suraj-webkul Sep 9, 2024
e3c960d
Leads imports done.
suraj-webkul Sep 9, 2024
144ac3e
write missing translate.
suraj-webkul Sep 9, 2024
c5646cf
publish assets and fir minor issue.
suraj-webkul Sep 9, 2024
5475ed2
Minor changes and fix faild pint test cases.
suraj-webkul Sep 10, 2024
715f525
refactor some code base.
suraj-webkul Sep 10, 2024
fad2885
proper commenting.
suraj-webkul Sep 10, 2024
1218c64
fix importer switch button color.
suraj-webkul Sep 10, 2024
5aef37a
ui issues fixed.
suraj-webkul Sep 10, 2024
a1413a2
add attributes validations to datatransfers.
suraj-webkul Sep 10, 2024
d92f378
fix breadcrumbs and fix icons of data transfer.
suraj-webkul Sep 10, 2024
d519e76
update the broken icon of cross for error.
suraj-webkul Sep 10, 2024
69499df
update view render event.
suraj-webkul Sep 10, 2024
143d27e
add contact number to person of column of unique_id.
suraj-webkul Sep 11, 2024
352105c
Update leads, products and person importer.
suraj-webkul Sep 11, 2024
ac51174
remove whitespace.
suraj-webkul Sep 11, 2024
1cf0132
refactor comments.
suraj-webkul Sep 11, 2024
adb096f
refactor the translation and remove unique_id from controller for upd…
suraj-webkul Sep 11, 2024
8eab186
rename helpers.
suraj-webkul Sep 11, 2024
3f71632
Minor changes.
suraj-webkul Sep 12, 2024
21b90b6
add leads csv.
suraj-webkul Sep 16, 2024
e3ec8cc
resolved conflicts.
suraj-webkul Sep 16, 2024
6e621d3
resolved raised bugs.
suraj-webkul Sep 16, 2024
9c8da35
Fix product imports.
suraj-webkul Sep 18, 2024
9280448
minor chnags.
suraj-webkul Sep 18, 2024
35e91e9
Revert migration code.
suraj-webkul Sep 18, 2024
bb56af2
fix faild pint test caes.
suraj-webkul Sep 18, 2024
847df0f
minor changes.
suraj-webkul Sep 18, 2024
6771215
add update support.
suraj-webkul Sep 18, 2024
0ef0e38
Update csv file.
suraj-webkul Sep 18, 2024
3c04e30
conflicts resolved.
suraj-webkul Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@
"Webkul\\Attribute\\": "packages/Webkul/Attribute/src",
"Webkul\\Contact\\": "packages/Webkul/Contact/src",
"Webkul\\Core\\": "packages/Webkul/Core/src",
"Webkul\\DataGrid\\": "packages/Webkul/DataGrid/src",
"Webkul\\DataTransfer\\": "packages/Webkul/DataTransfer/src",
"Webkul\\Email\\": "packages/Webkul/Email/src",
"Webkul\\EmailTemplate\\": "packages/Webkul/EmailTemplate/src",
"Webkul\\Installer\\": "packages/Webkul/Installer/src",
Expand All @@ -69,8 +71,7 @@
"Webkul\\User\\": "packages/Webkul/User/src",
"Webkul\\Warehouse\\": "packages/Webkul/Warehouse/src",
"Webkul\\WebForm\\": "packages/Webkul/WebForm/src",
"Webkul\\Automation\\": "packages/Webkul/Automation/src",
"Webkul\\DataGrid\\": "packages/Webkul/DataGrid/src"
"Webkul\\Automation\\": "packages/Webkul/Automation/src"
}
},
"autoload-dev": {
Expand Down
1 change: 1 addition & 0 deletions config/app.php
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@
Webkul\Contact\Providers\ContactServiceProvider::class,
Webkul\Core\Providers\CoreServiceProvider::class,
Webkul\DataGrid\Providers\DataGridServiceProvider::class,
Webkul\DataTransfer\Providers\DataTransferServiceProvider::class,
Webkul\EmailTemplate\Providers\EmailTemplateServiceProvider::class,
Webkul\Email\Providers\EmailServiceProvider::class,
Webkul\Installer\Providers\InstallerServiceProvider::class,
Expand Down
1 change: 1 addition & 0 deletions config/concord.php
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
\Webkul\User\Providers\ModuleServiceProvider::class,
\Webkul\Warehouse\Providers\ModuleServiceProvider::class,
\Webkul\WebForm\Providers\ModuleServiceProvider::class,
\Webkul\DataTransfer\Providers\ModuleServiceProvider::class,
],

'register_route_models' => true,
Expand Down
39 changes: 39 additions & 0 deletions database/migrations/2024_09_09_094040_create_job_batches_table.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('job_batches', function (Blueprint $table) {
$table->string('id')->primary();
$table->string('name');
$table->integer('total_jobs');
$table->integer('pending_jobs');
$table->integer('failed_jobs');
$table->text('failed_job_ids');
$table->mediumText('options')->nullable();
$table->integer('cancelled_at')->nullable();
$table->integer('created_at');
$table->integer('finished_at')->nullable();
});
}

/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
Schema::dropIfExists('job_batches');
}
};
36 changes: 36 additions & 0 deletions database/migrations/2024_09_09_094042_create_jobs_table.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('jobs', function (Blueprint $table) {
$table->bigIncrements('id');
$table->string('queue')->index();
$table->longText('payload');
$table->unsignedTinyInteger('attempts');
$table->unsignedInteger('reserved_at')->nullable();
$table->unsignedInteger('available_at');
$table->unsignedInteger('created_at');
});
}

/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
Schema::dropIfExists('jobs');
}
};
32 changes: 32 additions & 0 deletions packages/Webkul/Admin/src/Config/acl.php
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,39 @@
'name' => 'admin::app.acl.delete',
'route' => ['admin.settings.tags.delete', 'admin.settings.tags.mass_delete', 'admin.leads.tags.detach'],
'sort' => 2,
],
[
'key' => 'settings.data_transfer',
'name' => 'admin::app.acl.data-transfer',
'route' => 'admin.settings.data_transfer.imports.index',
'sort' => 10,
], [
'key' => 'settings.data_transfer.imports',
'name' => 'admin::app.acl.imports',
'route' => 'admin.settings.data_transfer.imports.index',
'sort' => 1,
], [
'key' => 'settings.data_transfer.imports.create',
'name' => 'admin::app.acl.create',
'route' => 'admin.settings.data_transfer.imports.create',
'sort' => 1,
], [
'key' => 'settings.data_transfer.imports.edit',
'name' => 'admin::app.acl.edit',
'route' => 'admin.settings.data_transfer.imports.edit',
'sort' => 2,
], [
'key' => 'settings.data_transfer.imports.delete',
'name' => 'admin::app.acl.delete',
'route' => 'admin.settings.data_transfer.imports.delete',
'sort' => 3,
], [
'key' => 'settings.data_transfer.imports.import',
'name' => 'admin::app.acl.import',
'route' => 'admin.settings.data_transfer.imports.imports',
'sort' => 4,
],
[
'key' => 'configuration',
'name' => 'admin::app.acl.configuration',
'route' => 'admin.configuration.index',
Expand Down
9 changes: 9 additions & 0 deletions packages/Webkul/Admin/src/Config/menu.php
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,14 @@
'route' => 'admin.settings.workflows.index',
'sort' => 3,
'icon-class' => 'icon-settings-flow',
],
[
'key' => 'settings.automation.data_transfer',
'name' => 'admin::app.layouts.data_transfer',
'info' => 'admin::app.layouts.data_transfer_info',
'route' => 'admin.settings.data_transfer.imports.index',
'sort' => 4,
'icon-class' => 'icon-download',
], [
'key' => 'settings.other_settings',
'name' => 'admin::app.layouts.other-settings',
Expand All @@ -273,4 +281,5 @@
'sort' => 9,
'icon-class' => 'icon-configuration',
],

];
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
<?php

namespace Webkul\Admin\DataGrids\Settings\DataTransfer;

use Illuminate\Database\Query\Builder;
use Illuminate\Support\Facades\DB;
use Webkul\DataGrid\DataGrid;

class ImportDataGrid extends DataGrid
{
/**
* Prepare query builder.
*/
public function prepareQueryBuilder(): Builder
{
return DB::table('imports')
->select(
'id',
'state',
'file_path',
'error_file_path',
'started_at',
'completed_at',
'type',
'summary',
);
}

/**
* Prepare Columns.
*/
public function prepareColumns(): void
{
$this->addColumn([
'index' => 'id',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.id'),
'type' => 'integer',
'filterable' => true,
'sortable' => true,
]);

$this->addColumn([
'index' => 'type',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.type'),
'type' => 'string',
'filterable' => true,
'sortable' => true,
]);

$this->addColumn([
'index' => 'state',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.state'),
'type' => 'string',
'filterable' => true,
'sortable' => true,
]);

$this->addColumn([
'index' => 'file_path',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.uploaded-file'),
'type' => 'string',
'closure' => function ($row) {
return '<a href="'.route('admin.settings.data_transfer.imports.download', $row->id).'" class="cursor-pointer text-blue-600 hover:underline">'.$row->file_path.'<a>';
},
]);

$this->addColumn([
'index' => 'error_file_path',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.error-file'),
'type' => 'string',
'closure' => function ($row) {
if (empty($row->error_file_path)) {
return '';
}

return '<a href="'.route('admin.settings.data_transfer.imports.download_error_report', $row->id).'" class="cursor-pointer text-blue-600 hover:underline">'.$row->error_file_path.'<a>';
},
]);

$this->addColumn([
'index' => 'started_at',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.started-at'),
'type' => 'date',
'filterable' => true,
'filterable_type' => 'date_range',
'sortable' => true,
]);

$this->addColumn([
'index' => 'completed_at',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.completed-at'),
'type' => 'date',
'filterable' => true,
'filterable_type' => 'date_range',
'sortable' => true,
]);

$this->addColumn([
'index' => 'summary',
'label' => trans('admin::app.settings.data-transfer.imports.index.datagrid.summary'),
'type' => 'string',
'closure' => function ($row) {
if (empty($row->summary)) {
return '';
}

$summary = json_decode($row->summary, true);

$stats = [];

foreach ($summary as $type => $value) {
$stats[] = trans('admin::app.settings.data-transfer.imports.index.datagrid.'.$type).': '.$summary[$type];
}

return implode(', ', $stats);
},
]);
}

/**
* Prepare actions.
*/
public function prepareActions(): void
{
if (bouncer()->hasPermission('settings.data_transfer.imports.import')) {
$this->addAction([
'index' => 'import',
'icon' => 'icon-import',
'title' => trans('admin::app.settings.data-transfer.imports.index.datagrid.import'),
'method' => 'GET',
'url' => function ($row) {
return route('admin.settings.data_transfer.imports.import', $row->id);
},
]);
}

if (bouncer()->hasPermission('settings.data_transfer.imports.edit')) {
$this->addAction([
'index' => 'edit',
'icon' => 'icon-edit',
'title' => trans('admin::app.settings.data-transfer.imports.index.datagrid.edit'),
'method' => 'GET',
'url' => function ($row) {
return route('admin.settings.data_transfer.imports.edit', $row->id);
},
]);
}

if (bouncer()->hasPermission('settings.data_transfer.imports.delete')) {
$this->addAction([
'index' => 'delete',
'icon' => 'icon-delete',
'title' => trans('admin::app.settings.data-transfer.imports.index.datagrid.delete'),
'method' => 'DELETE',
'url' => function ($row) {
return route('admin.settings.data_transfer.imports.delete', $row->id);
},
]);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -186,8 +186,12 @@ private function sanitizeRequestedPersonData(array $data): array
$data['organization_id'] = null;
}

$data['unique_id'] = $data['user_id'].'|'.$data['organization_id'].'|'.$data['emails'][0]['value'];

if (isset($data['contact_numbers'])) {
$data['contact_numbers'] = collect($data['contact_numbers'])->filter(fn ($number) => ! is_null($number['value']))->toArray();

$data['unique_id'] .= '|'.$data['contact_numbers'][0]['value'];
}

return $data;
Expand Down
Loading
Loading