Client load data directly from csvs.

[KFilippopolitis]

Update data processing for client so they load data from csv and not the database.
New mipdb version.

[ThanKarab]

I think we need a little bit more work on passing the csv paths and how to distinguish on what to pass on each worker.

Why do we need to keep the workerId on the dataset location since we know what node gives us which dataset.

Let's discuss this on Monday, to explain it to me a bit.

[ThanKarab]

I think this should be a separate commit specifying the fix, the reason, etc.

Also max_retries could be an env variable, hard coded in the Dockerfile or somewhere else, whatever you prefer.

Finally this:

        except Exception as e:
            print(f"Connection attempt {attempt + 1} failed: {e}")
            if attempt < max_retries - 1:
                time.sleep(1)
            else:
                print("Max retries reached. Exiting.")
                raise

could be rewritten like:

        except Exception as e:
            if attempt >= max_retries - 1:
                print("Could not establish connection to the server.")
                raise ex
            print(f"Connection with the server failed. Attempt {attempt} failed: {e}")
            time.sleep(1) // USE GLOBAL VARIABLE
                

You can also start the range() from 1 instead of doing the +1 -1

[ThanKarab]

A key represents the code of a dataset.

[ThanKarab]

Awesome work! 💯

As discussed in the call we have 2 remaining things:

1. Make the csv_path nullable
2. In the mipdb, when importing from socket, the csv_path should be null.