Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Error When Returning Large Data from API Endpoints #9334

Merged
merged 4 commits into from
Oct 1, 2024

Conversation

IonesioJunior
Copy link
Member

@IonesioJunior IonesioJunior commented Oct 1, 2024

Description

This PR addresses an issue where returning large datasets from API endpoints caused an error. The root cause was the set_result_store function breaking due to an incorrect path: context.server.blob_storage was non-existent. The fix involved updating the path to context.server.services.blob_storage, ensuring proper access to the blob storage service.

Affected Dependencies

  • packages/syft/src/syft/service/action/action_service.py

How has this been tested?

<...>

@sy.api_endpoint(path="test.serde")
def test_serde(context, mb):
    import numpy as np
    import pandas as pd
    
    def create_random_dataframe(num_mb: int, num_columns: int = 10):
        size_bytes = num_mb * 1e6
        num_elements = int(size_bytes / 8) # float64 = 8b
        return pd.DataFrame(np.random.rand(num_elements // num_columns, num_columns))
    
    
    df = create_random_dataframe(mb)
    return df

root_client.custom_api.add(test_serde)
x = root_client.api.services.test.serde(mb=1000)  # 1 GB
x.get()

….server.blob_storage doesn't exist. Fix the path to context.server.services.blob_storage

This solves the issue when trying to store a huge blob in blob storage
@IonesioJunior IonesioJunior marked this pull request as ready for review October 1, 2024 14:09
@IonesioJunior IonesioJunior merged commit 072c629 into dev Oct 1, 2024
39 checks passed
@IonesioJunior IonesioJunior deleted the hotfix/set_result_store branch October 1, 2024 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants