How to do a version upgrade and keep the existings projects

[haluwong]

Hi All, I'm running v0.84.2 and it's a bit old and i would like to ask the guideline to upgrade to the latest release v0.107.1?
I tried to delete all the existing unstract container and keep the volume.
Then, download the current latest release: v0.107.1, decompress and ./run-platform.sh
The login screen can be shown but show error after login

Attached with the container's log for unstract-db
_unstract-db_logs.txt

![Image](https://github.com/user-attachments/assets/6636e865-90a4-4a1c-bc2d-a5ff4ae59f76

unstract_db has tables but unstract is empty

Please kindly advise how to do the version upgrade and keep the existing projects/workflow data.
Thanks,
Halu

[chandrasekharan-zipstack]

Hello @haluwong

Since v0.93.0 there has been some changes in how we store data. Can you attempt to follow this migration guide to move existing data into the new structure and make it usable for the latest versions?

Since it involves significant changes in the schema structure we didn't want to automate this directly for the user. Please try it out and let us know if you face any issues

[haluwong]

Thanks for the migration guide.
I'm trying to upgrade from v0.84.2 to v0.92.1. Then upgrade to the latest (v0.107.6) by

./run-platform.sh -u -v v0.107.6

Then, i run the following command in unstract-backend's shell

python manage.py create_schema

python manage.py migrate

python manage.py migrate_to_v2

at last it said:
INFO : [2025-02-05 15:42:42,224]{module:migrate_to_v2 process:123 thread:140408638462848 request_id:none} :- Migration not run since SCHEMAS_TO_MIGRATE env seems empty.Set the value as _ALL_ to migrate complete data

(app-3.9) root@cbf6501ae6a7:/app# export SCHEMAS_TO_MIGRATE='ALL'

run again

python manage.py migrate_to_v2

Error as follow:

INFO : [2025-02-05 15:44:12,566]{module:migrate_to_v2 process:145 thread:140661642136448 request_id:none} :- Migration 'migration_mock_org_003_workflow' started
INFO : [2025-02-05 15:44:12,579]{module:migrate_to_v2 process:145 thread:140661642136448 request_id:none} :- [migration_mock_org_003_workflow] Processing batch 1: 10 rows
ERROR : [2025-02-05 15:44:12,579]{module:migrate_to_v2 process:145 thread:140661642136448 request_id:none} :- Unexpected error: tuple index out of range
ERROR : [2025-02-05 15:44:12,580]{module:migrate_to_v2 process:145 thread:140661642136448 request_id:none} :- Unexpected error: tuple index out of range
Traceback (most recent call last):
  File "/app/manage.py", line 29, in <module>
    main()
  File "/app/manage.py", line 25, in main
    execute_from_command_line(sys.argv)
  File "/app/.venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/app/.venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/app/.venv/lib/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/app/.venv/lib/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/app/migrating/v2/management/commands/migrate_to_v2.py", line 521, in handle
    migrator.migrate(migrations, organization_id)
  File "/app/migrating/v2/management/commands/migrate_to_v2.py", line 437, in migrate
    self._migrate_rows(
  File "/app/migrating/v2/management/commands/migrate_to_v2.py", line 304, in _migrate_rows
    dest_cursor.executemany(dest_query, converted_rows)
IndexError: tuple index out of range

Anything wrong in my migration steps? Please advise~