Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new translations providers added #5

Open
drkhateeb opened this issue Jun 7, 2024 · 32 comments
Open

new translations providers added #5

drkhateeb opened this issue Jun 7, 2024 · 32 comments

Comments

@drkhateeb
Copy link

Hello dear Dev
I appreciate your work providing this amazing plugin
Kindly add DeepL with Arabic Language Support, Built-in, similar to Google and Bing
update to : GPT4o to Azure / Open AI

Gemini Flash - Pro from Gemini AI Studio API
For local LLM, LM studio will be great
https://lmstudio.ai/docs/local-server
AI model functions to do - Paraphrase - Proofread - and custom promts , Add Tags to the current segment using AI model

@JuchiaLu
Copy link
Owner

JuchiaLu commented Jun 7, 2024

I will add some or all of the requested features, but due to a holiday in my area, I might not be able to start working on this for a few days.

@drkhateeb
Copy link
Author

Thank you very much and have nice holidays

@drkhateeb
Copy link
Author

drkhateeb commented Jun 7, 2024

it is also a great feature if you add a plugin to connect to WordFast Anywhere TM, TB online

https://www.wordfast.net/wiki/Wordfast_Anywhere_TMs_and_glossaries_API
momoQ already had a plugin to connect to Wordfast Server
but has not a plugin to online version via Wordfast Anywhere

this feature will enable collaborative translation by sharing TM and TB and access it anytime

https://www.wordfast.com/myaccount

image

@drkhateeb
Copy link
Author

this may help
https://github.com/songquanpeng/one-api

OpenAI key management & redistribution system, supports Azure, Anthropic Claude, Google PaLM 2 & Gemini, Zhipu ChatGLM, Baidu Wenxin Yiyan, iFlytek Spark Cognition, Alitong Yiqianwen, 360 Zhinao and Tencent Hunyuan, which can be used for secondary distribution management key. Only a single executable file, Docker image has been packaged, one-click deployment, out of the box. OpenAI key management & redistribution system, using a single API for all LLMs, and features an English UI.

@JuchiaLu
Copy link
Owner

I'll start working on this task now, but there are a few details we need to go over first. If everything looks good, I'll proceed with the development as follows:

  • DeepL (built-in): Got it.
  • Lingvanex, Yandex: Sure, but I don't have accounts for these, so I might not be able to test them properly.
  • GPT4o to Azure / Open AI: I'll add the GPT4o model and update the OpenAI GUI to allow for custom model names.
  • Gemini Flash - Pro: I'll integrate one-api and use the OpenAI GUI to connect to various large model service providers.
  • Local LLM: I think Ollama is more popular, and I've already set up the integration. However, to simplify future maintenance, I might remove it and use one-api.
  • AI model functions: I'm not quite clear on the specific use cases. Could you provide more details?
  • TM, TB: They seem to work differently from MT, meaning I'll need to use another SDK from memoQ, which is beyond the current plugin's scope.

@drkhateeb
Copy link
Author

DeepL (built-in): Got it. very good
Lingvanex, Yandex: Sure, but I don't have accounts for these, so I might not be able to test them properly. Could you make it similar to DeepL, via browser
GPT4o to Azure / Open AI: I'll add the GPT4o model and update the OpenAI GUI to allow for custom model names.
Gemini Flash - Pro: I'll integrate one-api and use the OpenAI GUI to connect to various large model service providers.
Local LLM: I think Ollama is more popular, and I've already set up the integration. However, to simplify future maintenance, I might remove it and use one-api.
AI model functions: I'm not quite clear on the specific use cases. Could you provide more details?

for this, I meant to create a list of prompts to do linguistic functions using AI: the translator may select or create a prompt to

  • Proofread the Active segment
  • Polish / Paraphrase it
  • Diactricise
  • provide 3 different translations in different styles, then the translator selects the proper one
  • save prompts in XML, import/export

TM, TB: They seem to work differently from MT, meaning I'll need to use another SDK from memoQ, which is beyond the current plugin's scope. For this, I will provide some documentation in the next reply

thanks in advance for your magnificent work

@JuchiaLu
Copy link
Owner

DeepL (built-in): done.
Lingvanex(built-in), Yandex(built-in): done.
GPT4o to Azure / Open AI: done.
Gemini Flash - Pro: done, use the OpenAI GUI to connect "one-api".
Local LLM: done, use the OpenAI GUI to connect "one-api".

This is a temporary test plugin. I am adding custom rate limiting and adaptive rate limiting features. A new version will be released after completion, and the temporary test plugin will be removed.

Multi-Supplier-MT-Plugin.zip

@drkhateeb
Copy link
Author

Good morning and excellent news
kindly add the name of the provider here
image

how to run this plugin on MemoQ Server 10.6.12

@drkhateeb
Copy link
Author

add prompt list to save - re-use prompts

image

@drkhateeb
Copy link
Author

how to set
Base URL and Path for
Gemini flash /pro latest
Onae-API
LLM
any demo / tutorial
Best

@drkhateeb
Copy link
Author

Kindly check this:
Autocomplete (Interactive Machine Translation)
https://docs.basiccat.org/en/latest/advancedFeatures.html#section-7

Export bilingual paragraph files

Sharing translation memory and term
I think it needs TM/TB SDK from memoQ to share online
https://docs.basiccat.org/en/latest/collaboration.html
source code:
https://github.com/xulihang/BasicCAT/issues

Documentations:
https://docs.memoq.com/current/api-docs/wsapi/api/tmservice/MemoQServices.html

https://docs.memoq.com/current/sdk-docs/
https://docs.memoq.com/current/sdk-docs/memoQ-TB-SDK-2.4.1.zip

https://docs.memoq.com/current/sdk-docs/memoQ-TM-SDK-2.4.1.zip
create TM/TB plugin for sharing and collaboration team
GIThub repository
wordfastanywhere API to store and share TM/TB
https://www.wordfast.net/wiki/Wordfast_Anywhere_TMs_and_glossaries_API

Best

@JuchiaLu
Copy link
Owner

"kindly add the name of the provider here":
Due to memoQ's SDK limitations, the plugin cannot modify memoQ's interface, so this is not possible.

"how to run this plugin on MemoQ Server 10.6.12":
As I am not a translator and have not used MemoQ Server, I'm afraid I am not familiar with this process.

"add prompt list to save - re-use prompts":
I might consider this once I have completed current tasks.

"how to set, Base URL and Path for, Gemini flash /pro latest":
You need to download one-api.exe and run it. Then, access http://127.0.0.1:3000 in your browser, login with username 'root' and password '123456', and follow these steps to configure it, using gemini-1.5-pro as an example:
20240613031609
20240613031833
20240613032341
sorry, one-api does not provide an option to change the language, so the screenshots can only be displayed in Chinese.

@drkhateeb
Copy link
Author

thank you very much

is there any English version of OneAPI
based on this
https://github.com/songquanpeng/one-api/blob/ed717211aaa1b1fa968aed493b80402de204df39/i18n/en.json

@JuchiaLu
Copy link
Owner

One-API supports I18N, but I couldn't find an option to change the language or if it switches automatically based on the environment.

@drkhateeb
Copy link
Author

Trados Studio
it is a CAT tool, similar to memoQ,, and widely used by 275000 users worldwide
this is the GitHub
https://github.com/RWS/Sdl-Community
lots of plugins, many of NMT providers

DeepL:
https://github.com/RWS/Sdl-Community/tree/master/DeepLMTProvider
Bing
https://github.com/RWS/Sdl-Community/tree/master/MicrosoftTranslatorProvider
Google:
https://github.com/RWS/Sdl-Community/tree/master/GoogleCloudTranslationProvider

Yandex

LingvaNex

Systran:
https://www.systransoft.com/translate/

U-Dictionary
http://u-dictionary.com/#/Translator/English

ModernMT
https://www.modernmt.com/translate

Packages:
https://www.nuget.org/packages/Trados.Community.Toolkit.LanguagePlatform

could you please create a plugin project for Trados Studio
to GET TRANSLATION as built-in, similar to memoQ MultiSupplier project

REf:
https://www.youtube.com/watch?v=i7jocOn0MGc

@drkhateeb
Copy link
Author

Hello dear dev
kindly create a
what is the cost of this plugin:

plugin to connect Trados Studio to Wordfast Anywhere
Wordfast Anywhere TMs and glossaries API
https://www.wordfast.net/wiki/Wordfast_Anywhere_TMs_and_glossaries_API
Wordfast Anywhere API is REST and all methods are accessed via: https://anywhere.wordfast.com/wfa/api.
First of all you need to register to the free online Wordfast Anywhere application at http://anywhere.wordfast.com/
The API will allow to connect any external application to Wordfast Anywhere TMs and glossaries.
A key is required for all the commands of the API.
The API key is generated by making a share of TMs and glossaries in Wordfast Anywhere.
The API usage is free but the number of keys is limited to 5.

Go to Wordfast Anywhere menu => Setup TM&Glo. Open the sharing dialog by Clicking on the Manage Sharing button => Add button for My shares to the application


Trados:
https://github.com/RWS/Sdl-Community

https://www.nuget.org/packages/Trados.Community.Toolkit.LanguagePlatform

Trados Translation Memory Api
Terminology Provider Api
Similar Plugins:
https://github.com/RWS/Sdl-Community/tree/master/TAUS%20Data%20Search
https://github.com/RWS/Sdl-Community/tree/master/IATETerminologyProvider

the requested plugin will connect Studio to Wordfast Anywhere TM and TB via API

Best,

@JuchiaLu
Copy link
Owner

Sorry for the delay; I've been busy with other things lately.

I haven't used Trados Studio before, so I need to learn how it works first. From what I know, it only provides source code for reference, which takes time to go through.

I'm writing this plugin just for fun and not looking for any payment. If I find some free time, I'll be happy to work on it, but right now, I'm tied up with other commitments.

@drkhateeb
Copy link
Author

I am satisfied with the efforts you have put in and the consistent follow-up. I enjoy this kind of engaging and enjoyable interaction and coding, and I hope it continues in the future. Please take your time, and I eagerly anticipate positive updates from you. Trados offers access to source code for plugins and C# packages, enabling developers to leverage these resources. Once again, I express my gratitude for dedicating your time to assist junior students like myself in their learning journey. Warm regards.

@JuchiaLu
Copy link
Owner

MemoQ.WordPreview.zip,Here's another memoQ plugin I developed:

preview.mp4

Due to some unresolved performance issues(like it cannot directly modify text in Word), I haven't made it open source. I use it for bilingual reading, and I'm not sure if it would be useful to you, but it might help with final bilingual proofreading. It should work with Office Word 2016 and later versions.

@drkhateeb
Copy link
Author

Thank you very much
it is very useful, as live review

could you create a plugin to export bilingual word docx with the same original format
for example : left column for source and right for target translation]
https://github.com/bookfere/Ebook-Translator-Calibre-Plugin
image

@drkhateeb drkhateeb changed the title DeepL , Gemini Flash/Pro 1.5 , LingVaNex, Yandex new translations providers added Jun 25, 2024
@JuchiaLu
Copy link
Owner

"kindly add the name of the provider here"、"add prompt list to save - re-use prompts"

A new version has been released,now you can "multi Install" and "custom display name".
multi Install

@drkhateeb
Copy link
Author

drkhateeb commented Jul 24, 2024

Great news,
Thanks a lot for amazing MultiSupplier

Kindly add Claude Sonnet 3.5, it is very accurate and provide human-like translation
add it a separate provide such as OpenAI API LLM, not through One-API

Best Regards

@drkhateeb
Copy link
Author

Personalize and adjust translations using a Glossary
A glossary is a CSV file
containing source language (SL) and target language (TL) terms, which helps guide the language model to deliver accurate term translations.
I look forward to this feature in the upcoming version.

@JuchiaLu
Copy link
Owner

JuchiaLu commented Jul 24, 2024

"add Claude Sonnet 3.5: "

Claude Support Test.zip done, But I don't have an API KEY, so I can't test.

"Personalize and adjust translations using a Glossary: "

I don't know how to pass a Glossary to the language model. Should I put it in the Prompt? If the file is large, won't it waste Tokens?

@drkhateeb
Copy link
Author

drkhateeb commented Jul 24, 2024

Thank you for the updates.
Regarding the CSV, my usual process involves uploading it, adding the prompts, and then pasting the text that needs translation. Claude retains the terms used in the translation for all subsequent requests, so there's no need to re-add them for each new translation request. I believe you could implement a similar approach within the plugin.

add a text box in the plugin below the prompt box, to add terms, separated by TAB delimited or semicolon
and inform Claude to use terms

image

@JuchiaLu
Copy link
Owner

"Claude retains the terms used in the translation for all subsequent requests, so there's no need to re-add them for each new translation request. I believe you could implement a similar approach within the plugin."

Actually LLM do not have memory. Their principle is that each request sends all historical conversation information, making it seem like they have memory. Claude hides this operation from the user. You can search for the keyword 'chatgpt api remember context' to find related information.

@drkhateeb
Copy link
Author

so, it is better to fine-tune the sonnet 3.5

https://claude3.pro/claude-3-5-sonnet-fine-tuning/#:~:text=While%20Claude%203.5%20Sonnet%20is,quickly%20adapt%20to%20new%20scenarios.

could the plugin connect the fine-tuned models?

image

@JuchiaLu
Copy link
Owner

"Fine-tuned" refers to the slight "retraining" of a model using data from a specific domain, making it more proficient in that field. However, you have to deploy or host the fine-tuned model yourself, which is a cost that most people can hardly afford. Additionally, fine-tuned models still do not have memory capabilities; they are just more proficient in certain areas.

@drkhateeb
Copy link
Author

That is a very informative reply, thank you
so, the Glossary is still an option, to get a proficient and accurate translation,
I am looking forward to seeing it in the next version

@JuchiaLu
Copy link
Owner

Yes, if you don't care about token usage and carry all glossaries on each request, then we can do this.

@drkhateeb
Copy link
Author

I think it is still affordable, 1M tokens input =3USD, and 1M output =15USD
but hosting and fine-tuning is very expensive

@JuchiaLu
Copy link
Owner

actually now, you can place glossaries directly in the Prompt, which comes with each request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants