Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

req(IMP): speed up file to database sync #256

Open
chrillep opened this issue Oct 29, 2022 · 1 comment
Open

req(IMP): speed up file to database sync #256

chrillep opened this issue Oct 29, 2022 · 1 comment

Comments

@chrillep
Copy link
Contributor

chrillep commented Oct 29, 2022

addGroupTranslation (and or more places) could benefit from upsert https://laravel.com/docs/9.x/eloquent#upserts

would scale and speed up sync 3x-10x times.
syncing truncated translation table with 17765 translations takes 15,288.38 ms

If you would like to perform multiple "upserts" in a single query, then you should use the upsert method instead.

All databases except SQL Server require the columns in the second argument of the upsert method to have a "primary" or "unique" index. In addition, the MySQL database driver ignores the second argument of the upsert method and always uses the "primary" and "unique" indexes of the table to detect existing records.

Is your feature request related to a problem? Please describe.
I'm always frustrated when it takes to long to sync lots of translations.

Describe the solution you'd like
use upsert instead of looping through every key

Describe alternatives you've considered
using a translationSeeder that uses upsert

example

alter table translations
    add constraint translations_language_group_key
        unique (`language_id`, `group`, `key`);
<?php

declare(strict_types=1);

use Illuminate\Support\Collection;
use JoeDixon\Translation\Language;
use JoeDixon\Translation\Translation as TranslationModel;

public function addGroupTranslations(string $language, Collection $groups): void
{
    if (! $this->languageExists(language: $language)) {
        $this->addLanguage(language: $language);
    }
    $languageId = Language::where(column: 'language', operator: $language)
        ->firstOrFail()
        ->id;

    $translationGroups= [];
    foreach ($groups as $group => $translations) {
        foreach ($translations as $key => $value) {
            if (is_array($value)) {
                continue;
            }
            $translationGroups[] = [
                'language_id' => $languageId,
                'group' => $group,
                'key' => $key,
                'value' => $value,
            ];
        }
    }

    TranslationModel::upsert(
        values: $translationGroups,
        uniqueBy: ['language_id', 'group', 'key'],
        update: ['value']
    );
}
@chrillep chrillep changed the title use upsert to speed up file to database sunc req(IMP): speed up file to database sunc Oct 29, 2022
@chrillep chrillep changed the title req(IMP): speed up file to database sunc req(IMP): speed up file to database sync Oct 29, 2022
@chrillep
Copy link
Contributor Author

chrillep commented Oct 31, 2022

@joedixon any thoughts on how to accomplish this without breaking the current API for File|Database driver without duplication?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant