replace index and state with one (workspace? fs?) index

[efiop]

Both our state and index db, do the same thing: they store information about files that exist on a particular filesystem along with some metadata. They currently use different formats, but they could be unified into one general
filesystem index. This is kinda similar to git's index, but is much more generalized for arbitrary filesystems/clouds.

Having one fs index, will also allow us to generalize our state for any possible filesystem, which will speedup external dependecies/outputs and import-url operations, since we won't have to rehash files every time.

Fsspec filesystems use dircache that does something similar, but is not persistent. Might be worth inheriting from it, but making it persistent for us.

Ideally the format of the storage should be language agnostic. Sqlite db with fspath as a key and metadata as a value (e.g. in string or binary form, to avoid complicated table structures).

[skshetry]

state and index were replaced with one single index from sqltrie. We haven't completed the migration yet, but this issue can be closed.

The progress of dvc-data index migration can be tracked in #9333.