Cannot convert the same OracleDataFrame to a PyArrow table twice

[cpcloud]

1. What versions are you using?

platform.platform: Linux-6.13.6-x86_64-with-glibc2.40
sys.maxsize > 2**32: True
platform.python_version: 3.12.9

oracledb.__version__: 3.0.0

2. Is it an error or a hang or a crash? An error.

3. What error(s) or behavior you are seeing?

ArrowInvalid: Cannot import released ArrowSchema

4. Does your application call init_oracle_client()?

No

5. Include a runnable Python script that shows the problem.

In [26]: import oracledb, pyarrow as pa

In [27]: con = oracledb.connect(oracledb.makedsn("localhost", 1521, service_name="IBIS_TESTING", sid=None), user="IBIS
       ⋮ ", password="ibis", stmtcachesize=0)

In [28]: odf = con.fetch_df_all("SELECT 1 AS \"a\"")

In [29]: t = pa.Table.from_arrays(arrays=odf.column_arrays(), names=odf.column_names())

In [30]: t = pa.Table.from_arrays(arrays=odf.column_arrays(), names=odf.column_names())
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1                                                                                    │
│                                                                                                  │
│ in pyarrow.lib.Table.from_arrays:4851                                                            │
│                                                                                                  │
│ in pyarrow.lib._sanitize_arrays:1593                                                             │
│                                                                                                  │
│ in pyarrow.lib._schema_from_arrays:1574                                                          │
│                                                                                                  │
│ in pyarrow.lib.array:271                                                                         │
│                                                                                                  │
│ in pyarrow.lib.Array._import_from_c_capsule:1875                                                 │
│                                                                                                  │
│ in pyarrow.lib.pyarrow_internal_check_status:155                                                 │
│                                                                                                  │
│ in pyarrow.lib.check_status:92                                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ArrowInvalid: Cannot import released ArrowSchema

[cjbj]

I believe this is a features/limitation of nanoarrow. I'll let @aosingh comment.

[aosingh]

@cpcloud

The lifetime semantics is defined by Arrow C Data Interface and Arrow PyCapsule Interface. The following note is from the documentation:

If the capsule has been passed to a consumer, the consumer should have moved the data and marked the release callback as null, so there isn’t a risk of releasing data the consumer is using. Read more in the C Data Interface specification.

In the example shown, pyarrow marks the release callback as NULL after invoking it during the first conversion and raises an error during the second. If twice conversion is a valid usecase, we will have to check how to do that.

[cjbj]

@cpcloud do you have a practical use case for converting twice?