Dependencies: Update requirement to psycopg~=3.0

[sphuber]

Fixes #5864

[sphuber]

Before merging, I want to see if I can perform some meaningful benchmarks to see if performance is affected at all.

[sphuber]

Although psycopg is supposed to be more efficient, there are a few sources online that suggest this is perhaps not always trivially the case:

• Discussion by the author of Sqlalchemy
• Extensive benchmark

Not sure if these differences would even be relevant compared to other overhead added by AiiDA's code. I have not found any clear indication on the psycopg2 documentation about its lifetime and if/when there will be an end-of-life. But it is pretty sure that support will stop at some point and we might be forced to move to psycopg v3 regardless.

[sphuber]

Another interesting thing is that since the very beginning of AiiDA (I remember this being there pre-v1.0) profiles (core.psql_dos ones) contain the database_engine key. This is still an option in verdi setup and verdi quicksetup, except it is not really an option as it is hardcoded to postgresql_psycopg2. So all core.psql_dos profiles out there contain:

"main": {
    "storage": {
        "backend": "core.psql_dos",
        "config": {
            "database_engine": "postgresql_psycopg2",
            ...
        }
    },
}

The funny thing is that this is not actuall used whatsoever.

We could have used this in the connection string when creating the sqlalchemy engine. We currently hardcode this to postgres:// which is the default psycopg dialect and maps to postgres+psycopg2://. So we are currently targeting psycopg. This is why this PR updates the hardcoded protocol to postgres+psycopg:// to target the new version of psycopg.

Because it is hardcoded, we don't need a migration for the existing configs, although it would perhaps be nice to clean it up from existing profiles. Since verdi setup and verdi quicksetup are deprecated now anyway, we don't have to remove the options there I would say. The field is still present in the pydantic model of the PsqlDosStorage, but since it is not needed anyway, we might as well get rid of it.

[codecov]

Codecov Report

Attention: Patch coverage is 81.81818% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.81%. Comparing base (ef60b66) to head (423d2b8).
Report is 116 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/manage/tests/pytest_fixtures.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6362      +/-   ##
==========================================
+ Coverage   77.51%   77.81%   +0.31%     
==========================================
  Files         560      562       +2     
  Lines       41444    41888     +444     
==========================================
+ Hits        32120    32593     +473     
+ Misses       9324     9295      -29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mbercx]

As discussed during the AiiDA meeting, let's merge this version bump into main so we can start testing it in production projects.