synthesizer 합성 중 ckpt 불러오기에 오류가 발생합니다.

[L0ire]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1300, in restore
names_to_keys = object_graph_key_mapping(save_path)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1618, in object_graph_key_mapping
object_graph_string = reader.get_tensor(trackable.OBJECT_GRAPH_PROTO_KEY)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/pywrap_tensorflow_internal.py", line 915, in get_tensor
return CheckpointReader_GetTensor(self, compat.as_bytes(tensor_str))
tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "synthesizer.py", line 342, in
synthesizer.load(config.load_path, config.num_speakers, config.checkpoint_step,inference_prenet_dropout=False)
File "synthesizer.py", line 81, in load
saver.restore(self.sess, checkpoint_path)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1306, in restore
err, "a Variable name or other graph key that is missing")
tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing fro
m the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

2 root error(s) found.
(0) Not found: Key model/Eembedding/decoder_init_dense_0/bias not found in checkpoint

(1) Not found: Key model/Eembedding/decoder_init_dense_0/bias not found in checkpoint

0 successful operations.
0 derived errors ignored.

Original stack trace for 'save/RestoreV2':
File "synthesizer.py", line 342, in
synthesizer.load(config.load_path, config.num_speakers, config.checkpoint_step,inference_prenet_dropout=False)
File "synthesizer.py", line 80, in load
saver = tf.train.Saver()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/gen_io_ops.py", line 1696, in restore_v2
name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

synthesizer.py를 이용해 음성을 생성하려 시도했으나, 위와 같은 오류가 발생했습니다.

에러 로그 도중, tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint 라는 문장이 신경쓰여 tf.train.list_variables 함수를 이용해 지금껏 저장된 ckpt 파일들을 확인했는데, 실제로 _CHECKPOINTABLE_OBJECT_GRAPH 값이 존재하지 않는걸 확인했습니다.

해당 오류 발생 및 ckpt 파일 확인 이후, synthesizer.py의 약 80 라인에서, saver 변수에 tf.train.import_meta_graph 함수를 이용해 해당 ckpt 파일의 metaGraph 데이터를 가져와 합성을 시도했으나, 합성이 정상적으로 이루어지지 않는것으로 추정이 됩니다.(1초 남짓의 노이즈 wav파일이 생성됩니다.)

60개의 데이터를 약 3만회 학습한 체크포인트였으며, 소스코드를 크게 수정하진 않았습니다.(커스텀 text cleaner 함수를 사용했습니다.)

해당 오류를 해결할 수 있게 도와주시면 정말 감사하겠습니다.