Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

複数エンジン対応: すべてのエンジンからキャラクター情報を取得する #883

Merged
merged 135 commits into from
Aug 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
a135eff
add runEngineAll
aoirint Mar 8, 2022
f81a774
fix log
aoirint Mar 8, 2022
22e0168
fix log
aoirint Mar 8, 2022
c5202dd
add killEngineAll
aoirint Mar 8, 2022
6e46ad9
fix log
aoirint Mar 8, 2022
31bfe9f
fix log
aoirint Mar 8, 2022
eb919bd
add restartEngineAll
aoirint Mar 8, 2022
39b81df
call killEngineAll
aoirint Mar 8, 2022
c8a7c22
fix log unit
aoirint Mar 8, 2022
9778831
npm run fmt
aoirint Mar 8, 2022
7d7b41c
add START_WAITING_ENGINE_ALL, RESTART_ENGINE_ALL
aoirint Mar 8, 2022
e0198fe
npm run fmt
aoirint Mar 8, 2022
5f33a07
DETECTED_ENGINE_ERROR engineKey
aoirint Mar 8, 2022
e23e9fb
npm run fmt
aoirint Mar 8, 2022
9d593e0
fix check all ready logic
aoirint Mar 8, 2022
4f0c2ee
fix engineState
aoirint Mar 8, 2022
c9a2272
fix test
aoirint Mar 8, 2022
23bdd06
type with undefined
aoirint Mar 8, 2022
8272997
fix allEngineState logic
aoirint Mar 8, 2022
63c5816
fix test
aoirint Mar 8, 2022
4754d9f
add comment
aoirint Mar 8, 2022
f3777c8
multiple engine: SET_CHARACTER_INFOS, USER_ORDERED_CHARACTER_INFOS
aoirint Mar 8, 2022
67591d6
multiple engine: LOAD_CHARACTER_ALL
aoirint Mar 8, 2022
a97a3c6
fix buildFileName logic
aoirint Mar 8, 2022
d0bad7b
fix default style id
aoirint Mar 8, 2022
9b3defa
fix characterInfos from vue
aoirint Mar 9, 2022
579f08b
pass engineInfos to vue
aoirint Mar 9, 2022
d963590
fix log
aoirint Mar 9, 2022
b1c6fda
fix changeStyleId
aoirint Mar 9, 2022
7333e5a
fix synthesis to use engineKey/Id
aoirint Mar 9, 2022
825a0fb
fix character portrait to use engineId
aoirint Mar 9, 2022
2f35f0a
fix audiocell icon to use engineId
aoirint Mar 9, 2022
b76e2b8
fix character voice testplay
aoirint Mar 9, 2022
f7c8746
wip fix voicesample
aoirint Mar 9, 2022
a02fd17
fix test
aoirint Mar 9, 2022
46c1228
use <0.12 for project migration
aoirint Mar 10, 2022
e21c23e
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 10, 2022
cc83cd9
use semver for migration
aoirint Mar 10, 2022
2102710
assume to be killed
aoirint Mar 10, 2022
c24c7e4
add memo comment
aoirint Mar 10, 2022
5c1cf7c
fix comment
aoirint Mar 10, 2022
086bd30
call onAllKilled after final process kill errored
aoirint Mar 10, 2022
9399dd1
fix log
aoirint Mar 10, 2022
db81c2b
fix log
aoirint Mar 10, 2022
26e4b2d
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 11, 2022
b681ec4
fix infinite loop when using external engine
aoirint Mar 11, 2022
abe3453
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
5aa3eef
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
cb968e6
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
334f77f
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
19aca50
add logs
aoirint Mar 13, 2022
d33a330
call IS_ENGINE_READY from IS_ALL_ENGINE_READY
aoirint Mar 13, 2022
858f650
fix comment
aoirint Mar 13, 2022
521252e
call IS_ENGINE_READY with 0th engineKey
aoirint Mar 13, 2022
88d9fc9
fixme to todo
aoirint Mar 13, 2022
a75322e
fix comment
aoirint Mar 13, 2022
f5204a7
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 13, 2022
a6f1922
assert engineKey, engineId, styleId is not undefined
aoirint Mar 13, 2022
110a762
fix comment
aoirint Mar 14, 2022
2c754de
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
da2e26d
remove unused import
aoirint Mar 14, 2022
6d30f34
Merge branch 'patch-multiple-engine-api-log' into patch-multiple-engi…
aoirint Mar 14, 2022
af19eae
fix CharacterOrderDialog
aoirint Mar 14, 2022
a719ae5
CharacterOrderDialog: change icon based on selected style
aoirint Mar 14, 2022
7175447
remove .value in template
aoirint Mar 14, 2022
7cac6e5
DefaultStyleSelectDialog: multiple engine (allow duplicated styleId)
aoirint Mar 14, 2022
8319276
LibraryPolicy: multiple engine
aoirint Mar 14, 2022
bf31b88
fix log
aoirint Mar 14, 2022
4453c8e
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
222792d
CharacterOrderDialog: allow duplication of styleId when speakerUuid i…
aoirint Mar 14, 2022
695ef6f
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 14, 2022
8261eb6
fix comment
aoirint Mar 14, 2022
a240b63
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
8c2b543
split icon change pr
aoirint Mar 14, 2022
ac6310b
fix comment
aoirint Mar 14, 2022
09332c1
state.engineInfos to Record<string, EngineInfo> from EngineInfo[]
aoirint Mar 24, 2022
5d0ee83
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 24, 2022
7031f3d
fix dictionary actions to use state.engineKeys instead of engineInfos
aoirint Mar 24, 2022
c37757c
npm run fmt
aoirint Mar 24, 2022
f7e2bdf
initialize engineStates with STARTING
aoirint Mar 24, 2022
a075309
npm run fmt
aoirint Mar 25, 2022
8c24a7e
GENERATE_AUDIO_ITEM: add engineKey undefined check
aoirint Mar 25, 2022
966f013
use strict equality for engineKey undefined check
aoirint Mar 25, 2022
f86a7d5
add comment to describe loop behavior
aoirint Mar 25, 2022
4e745d5
add STARTING assert test
aoirint Mar 25, 2022
b152e36
remove unneeded undefined check
aoirint Mar 26, 2022
1d15f02
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 26, 2022
d4db483
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 26, 2022
56063cb
DictionaryManageDialog: temporarily use first engineKey
aoirint Mar 26, 2022
51f19f1
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 26, 2022
571d980
flatMap engineInfos -> engineKeys
aoirint Mar 26, 2022
d610af5
loop engineInfos -> engineKeys
aoirint Mar 26, 2022
8d6e974
add characterInfos assert.isObject
aoirint Mar 26, 2022
d30dfbb
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 22, 2022
2fa8690
fix DictionaryManageDialog working for first engine
aoirint Jun 22, 2022
c0af3b5
Merge remote-tracking branch 'origin/patch-multiple-engine-api' into …
aoirint Jun 22, 2022
7e5e1e1
SETUP_ENGINE_SPEAKER: add engineKey (fix character init)
aoirint Jun 22, 2022
b7759cd
fix assertion compare
aoirint Jun 22, 2022
f137521
fix assert message
aoirint Jun 22, 2022
4afe467
project migration version 0.12 -> 0.13
aoirint Jun 22, 2022
0a34c0d
add comment
aoirint Jun 22, 2022
a0a812a
LOAD_USER_DICT: add engineKey
aoirint Jun 22, 2022
70c863c
getFlattenCharacterInfos func
aoirint Jun 22, 2022
2c4bed5
GENERATE_AUDIO_ITEM: (engineId, styleId) assertion
aoirint Jun 22, 2022
34fc2dc
fix message
aoirint Jun 22, 2022
7747878
add comment
aoirint Jun 26, 2022
bf28087
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 26, 2022
95ebf6c
Merge branch 'main' into patch-multiple-engine-api
Hiroshiba Jul 23, 2022
a66db4e
動くように
Hiroshiba Jul 23, 2022
c357917
engineKey -> engineId
Hiroshiba Jul 23, 2022
6e4ce29
engineKey to engineId
Hiroshiba Jul 23, 2022
ba7b039
key -> id
Hiroshiba Jul 23, 2022
53cbef8
Merge branch 'engineKey-to-engineId' into patch-multiple-engine-api
Hiroshiba Jul 23, 2022
3f2b511
id -> uuid
Hiroshiba Jul 24, 2022
547e506
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
Hiroshiba Aug 3, 2022
9dc5185
default engine id
Hiroshiba Aug 3, 2022
ef389b9
Update src/store/audio.ts
Hiroshiba Aug 3, 2022
96e114a
getFlattenCharacterInfos関数をGETTERSに
Hiroshiba Aug 4, 2022
965468c
engineidなどの設計方針を記載
Hiroshiba Aug 4, 2022
eaaf42f
Merge branch 'patch-multiple-engine-api' of github.com:Hiroshiba/voic…
Hiroshiba Aug 4, 2022
941ebd9
USER_ORDERED_CHARACTER_INFOSのとこ修正
Hiroshiba Aug 4, 2022
1e8faec
if ((payload.engineId === undefined) != (payload.styleId === undefined))
Hiroshiba Aug 4, 2022
7f26fad
home.vueのとこをflattenCharacterInfosに
Hiroshiba Aug 4, 2022
85357f5
style.styleName || "ノーマル"
Hiroshiba Aug 4, 2022
0814300
speakerUuid・styleIdが一致するCharacterInfoを持つエンジンを探す
Hiroshiba Aug 4, 2022
9b721cc
ID->Id
Hiroshiba Aug 4, 2022
69f2053
Merge branch 'main' into patch-multiple-engine-api
Hiroshiba Aug 15, 2022
ca57b3e
追加修正
Hiroshiba Aug 15, 2022
ef7d083
下のengineIdで上書き定義しないように
Hiroshiba Aug 21, 2022
67e0448
FIXMEコメント追加、0.14に変更
Hiroshiba Aug 21, 2022
d24a860
AudioItemにSpeakerIdを追加したい旨を追記
Hiroshiba Aug 21, 2022
1ed710f
ですます調に
Hiroshiba Aug 21, 2022
0397be3
StyleId は現在整数値型になっていますが、将来的にはUuidにしたいと考えています。
Hiroshiba Aug 22, 2022
763c676
.
Hiroshiba Aug 23, 2022
1a92d93
Merge branch 'main' into patch-multiple-engine-api
Hiroshiba Aug 23, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/設計方針.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,18 @@
ダイアログは外部 API とみなし、Vuex の Store の Action 内から呼ぶことを許容しています。
これは、ダイアログの結果によってダイアログ前後の挙動を 1 つの Action に書けるという利便性を取ったためです。
ダイアログの種類によっては View に直接作用してダイアログ UI を表示するものもありますが、これも許容することにしています。

## EngineId、SpeakerId、StyleId

EngineId はエンジンが持つ ID で、世界で唯一かつ不変です。
SpeakerId は話者が持つ ID で、世界で唯一かつ不変。エンジン間でも同じ ID を持ちます。
StyleId はスタイルごとの ID で、話者ごとに唯一であれば良いです。

声を一意に決めるには、(EngineId, SpeakerId, StyleId)の3組が揃っている必要がある、という仕様を目指しています。
現状は、音声合成APIに SpeakerId 引数が無いため、(EngineId, StyleId)の2組で一意に声が決まっています。

VOICEVOX は歴史的経緯により、 SpeakerId と StyleId が混同していることがあります。
特に型が整数値になっている SpeakerId は StyleId と混同している場合があります。
(エンジン API の SpeakerId 引数に StyleId を渡したりなど。)

StyleId は現在整数値型になっていますが、将来的にはUuidにしたいと考えています。
40 changes: 33 additions & 7 deletions src/components/AudioCell.vue
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
"
@click="
changeStyleId(
characterInfo.metas.speakerUuid,
getDefaultStyle(characterInfo.metas.speakerUuid).styleId
)
"
Expand Down Expand Up @@ -97,7 +98,12 @@
v-close-popup
active-class="selected-character-item"
:active="style.styleId === selectedStyle.styleId"
@click="changeStyleId(style.styleId)"
@click="
changeStyleId(
characterInfo.metas.speakerUuid,
style.styleId
)
"
>
<q-avatar rounded size="2rem" class="q-mr-md">
<q-img
Expand Down Expand Up @@ -198,11 +204,11 @@ export default defineComponent({

const selectedCharacterInfo = computed(() =>
userOrderedCharacterInfos.value !== undefined &&
audioItem.value.engineId !== undefined &&
audioItem.value.styleId !== undefined
? userOrderedCharacterInfos.value.find((info) =>
info.metas.styles.find(
(style) => style.styleId === audioItem.value.styleId
)
? store.getters.CHARACTER_INFO(
audioItem.value.engineId,
audioItem.value.styleId
)
: undefined
);
Expand Down Expand Up @@ -256,13 +262,26 @@ export default defineComponent({
}
};

const changeStyleId = (styleId: number) => {
const changeStyleId = (speakerUuid: string, styleId: number) => {
// FIXME: 同一キャラが複数エンジンにまたがっているとき、順番が先のエンジンが必ず選択される
const engineId = store.state.engineIds.find((_engineId) =>
(store.state.characterInfos[_engineId] ?? []).some(
(characterInfo) => characterInfo.metas.speakerUuid === speakerUuid
)
);
if (engineId === undefined)
throw new Error(
`No engineId for target character style (speakerUuid == ${speakerUuid}, styleId == ${styleId})`
);

store.dispatch("COMMAND_CHANGE_STYLE_ID", {
audioKey: props.audioKey,
engineId,
styleId,
});
};
const getDefaultStyle = (speakerUuid: string) => {
// FIXME: 同一キャラが複数エンジンにまたがっているとき、順番が先のエンジンが必ず選択される
const characterInfo = userOrderedCharacterInfos.value?.find(
(info) => info.metas.speakerUuid === speakerUuid
);
Expand Down Expand Up @@ -316,10 +335,17 @@ export default defineComponent({
await pushAudioText();
}

const engineId = audioItem.value.engineId;
if (engineId === undefined)
throw new Error("assert engineId !== undefined");

const styleId = audioItem.value.styleId;
if (styleId == undefined) throw new Error("styleId == undefined");
if (styleId === undefined)
throw new Error("assert styleId !== undefined");

const audioKeys = await store.dispatch("COMMAND_PUT_TEXTS", {
texts,
engineId,
styleId,
prevAudioKey,
});
Expand Down
11 changes: 5 additions & 6 deletions src/components/CharacterPortrait.vue
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,17 @@ export default defineComponent({
const store = useStore();

const characterInfo = computed(() => {
const characterInfos = store.state.characterInfos || [];
const activeAudioKey: string | undefined = store.getters.ACTIVE_AUDIO_KEY;
const audioItem = activeAudioKey
? store.state.audioItems[activeAudioKey]
: undefined;

const engineId = audioItem?.engineId;
const styleId = audioItem?.styleId;

return styleId !== undefined
? characterInfos.find((info) =>
info.metas.styles.find((style) => style.styleId === styleId)
)
: undefined;
if (engineId === undefined || styleId === undefined) return undefined;

return store.getters.CHARACTER_INFO(engineId, styleId);
});

const characterName = computed(() => {
Expand Down
25 changes: 24 additions & 1 deletion src/components/DictionaryManageDialog.vue
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,8 @@ export default defineComponent({
const store = useStore();
const $q = useQuasar();

const engineIdComputed = computed(() => store.state.engineIds[0]); // TODO: 複数エンジン対応

const dictionaryManageDialogOpenedComputed = computed({
get: () => props.modelValue,
set: (val) => emit("update:modelValue", val),
Expand All @@ -282,10 +284,16 @@ export default defineComponent({
};

const loadingDictProcess = async () => {
const engineId = engineIdComputed.value;
if (engineId === undefined)
throw new Error(`assert engineId !== undefined`);

loadingDict.value = true;
try {
userDict.value = await createUILockAction(
store.dispatch("LOAD_USER_DICT")
store.dispatch("LOAD_USER_DICT", {
engineId,
})
);
} catch {
$q.dialog({
Expand Down Expand Up @@ -356,6 +364,10 @@ export default defineComponent({
surface.value = convertHankakuToZenkaku(text);
};
const setYomi = async (text: string, changeWord?: boolean) => {
const engineId = engineIdComputed.value;
if (engineId === undefined)
throw new Error(`assert engineId !== undefined`);

// テキスト長が0の時にエラー表示にならないように、テキスト長を考慮する
isOnlyHiraOrKana.value = !text.length || kanaRegex.test(text);
// 読みが変更されていない場合は、アクセントフレーズに変更を加えない
Expand All @@ -378,6 +390,7 @@ export default defineComponent({
await createUILockAction(
store.dispatch("FETCH_ACCENT_PHRASES", {
text: text + "ガ'",
engineId,
styleId: styleId.value,
isKana: true,
})
Expand All @@ -396,12 +409,17 @@ export default defineComponent({
};

const changeAccent = async (_: number, accent: number) => {
const engineId = engineIdComputed.value;
if (engineId === undefined)
throw new Error(`assert engineId !== undefined`);

if (accentPhrase.value) {
accentPhrase.value.accent = accent;
accentPhrase.value = (
await createUILockAction(
store.dispatch("FETCH_MORA_DATA", {
accentPhrases: [accentPhrase.value],
engineId,
styleId: styleId.value,
})
)
Expand All @@ -413,6 +431,10 @@ export default defineComponent({
audioElem.pause();

const play = async () => {
const engineId = engineIdComputed.value;
if (engineId === undefined)
throw new Error(`assert engineId !== undefined`);

if (!accentPhrase.value) return;
nowGenerating.value = true;
const query: AudioQuery = {
Expand All @@ -429,6 +451,7 @@ export default defineComponent({

const audioItem: AudioItem = {
text: yomi.value,
engineId,
styleId: styleId.value,
query,
};
Expand Down
6 changes: 4 additions & 2 deletions src/components/LibraryPolicy.vue
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ export default defineComponent({
const store = useStore();
const md = useMarkdownIt();

const characterInfos = computed(() => store.state.characterInfos);
const flattenCharacterInfos = computed(
() => store.getters.GET_FLATTEN_CHARACTER_INFOS
);

const convertMarkdown = (text: string) => {
return md.render(text);
Expand All @@ -62,7 +64,7 @@ export default defineComponent({
};

return {
characterInfos,
characterInfos: flattenCharacterInfos,
convertMarkdown,
selectCharacterInfIndex,
detailIndex,
Expand Down
Loading