Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disabled all deployment package upload to S3 and S3 table creation #296

Merged
merged 3 commits into from
Aug 21, 2024

Conversation

Danidite
Copy link
Contributor

@Danidite Danidite commented Aug 20, 2024

Address the following issue:

…ixed issue of deployment_manager crashing IF some region does not possess carbon data.
@Danidite Danidite self-assigned this Aug 20, 2024
Copy link

github-actions bot commented Aug 20, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
caribou/common
   utils.py37878%43–58
caribou/common/models
   endpoints.py34391%18–19, 50
caribou/common/models/remote_client
   aws_remote_client.py4826886%42, 65, 105, 151, 216, 219, 255, 272–273, 301–302, 341, 352, 362, 395–397, 405–406, 425–427, 446, 449, 473, 486–487, 490, 496, 536–538, 576–578, 596–597, 638, 644, 654, 658, 669–670, 678–679, 688, 692, 703, 713–715, 735, 746–748, 768, 783–785, 800–805, 833–834, 838, 854–858, 864–865
   remote_client_factory.py21386%21–23
caribou/data_collector/components
   data_collector.py18194%23
   data_exporter.py34294%49, 56
caribou/data_collector/components/carbon
   carbon_collector.py21290%34–35
   carbon_retriever.py1341887%25, 59, 93, 135–144, 148, 153, 160, 168–169, 215, 246, 249, 256
caribou/data_collector/components/performance
   performance_retriever.py38684%44, 50–51, 56–59
caribou/data_collector/components/provider
   provider_collector.py24292%43–44
   provider_retriever.py2945083%24, 32, 34, 47, 59, 77, 84, 100, 132–152, 183–281, 299–304, 336, 344, 346, 348, 350, 352, 354, 356, 432, 446, 556, 649
caribou/data_collector/components/workflow
   workflow_collector.py25196%34
   workflow_retriever.py30510964%216–257, 280, 330–337, 339, 371–414, 436, 467–526, 533–563, 566–583, 587–604, 615, 676, 698
caribou/data_collector/utils/latency_retriever
   aws_latency_retriever.py74988%59, 63, 65, 67, 70, 74, 76, 78, 81
   latency_retriever.py6183%8
caribou/deployment/client
   caribou.py16194%22
   caribou_workflow.py46513571%54, 70–71, 80–83, 129, 201–223, 267–268, 282, 315–326, 347–354, 380–391, 434–454, 562, 578, 581, 636–662, 667–683, 705–715, 793, 796, 798, 800, 802, 804, 831, 866–881, 916–917, 927–931, 961, 996, 1004, 1016–1096, 1109–1138, 1172–1181, 1212, 1245, 1273–1282, 1302–1320, 1340, 1353–1359, 1365
caribou/deployment/common/config
   config.py83199%104
caribou/deployment/common/deploy
   deployer.py1401689%43, 102–109, 199, 240–242, 249, 256, 267
   deployment_packager.py1924577%31, 34, 39, 63–70, 73–84, 90, 96, 101, 106, 128, 156, 169, 182, 190, 251–253, 259–274, 284, 286, 288, 291
   executor.py53492%32–33, 66, 68
   workflow_builder.py1711392%43, 59, 146, 178, 186, 189, 191, 196, 205, 208, 210, 324, 330
caribou/deployment/common/deploy/models
   function.py42295%42, 53
   function_instance.py12283%16, 22
   iam_role.py34391%19–21
   instructions.py18289%17, 26
   resource.py18194%25
   workflow.py1943284%39, 44, 67, 77, 99, 120, 166, 171, 206, 219–231, 272, 281, 289, 293, 298–327
caribou/deployment/common/deploy_instructions
   aws_deploy_instructions.py24292%37, 83
   deploy_instructions.py54983%57, 95, 145, 151, 164, 168, 172, 176, 182
caribou/deployment/common/factories
   deploy_instruction_factory.py18383%20–22
   deployer_factory.py1071685%24–27, 41, 52–53, 82, 90, 99, 108, 119, 126–129
caribou/deployment/server
   re_deployment_server.py61592%36, 48, 51, 58, 107
caribou/deployment_solver
   workflow_config.py751185%24, 26, 32–33, 54–56, 61, 79, 82, 107
caribou/deployment_solver/deployment_algorithms
   coarse_grained_deployment_algorithm.py25292%27, 32
   deployment_algorithm.py981189%52, 91, 105, 118, 132, 149, 184–190, 219
   fine_grained_deployment_algorithm.py25292%38, 43
   stochastic_heuristic_deployment_algorithm.py921485%18–25, 49, 59, 71, 92, 95, 100, 106–109, 119, 136, 163, 170
caribou/deployment_solver/deployment_input
   input_manager.py1995572%126, 132, 135, 194, 200, 246–254, 276–282, 291–294, 304–307, 320–339, 352–373, 379–387, 441–442, 476–487, 493–495, 501
caribou/deployment_solver/deployment_input/components/calculators
   carbon_calculator.py96496%124–128, 214, 261
   cost_calculator.py70199%163
   runtime_calculator.py1091190%130–145, 152, 201, 209, 262, 295
caribou/deployment_solver/deployment_input/components/loaders
   carbon_loader.py21386%22, 43, 46
   datacenter_loader.py49590%106–111, 114
   performance_loader.py18194%38
   region_viability_loader.py17288%15, 32
   workflow_loader.py1472682%86, 102–112, 122, 145, 165, 183, 193, 210, 264–276, 292, 364–369, 392, 430–432, 437
caribou/deployment_solver/deployment_metrics_calculator
   deployment_metrics_calculator.py40198%49
   go_deployment_metrics_calculator.py513139%18–21, 25–27, 41–50, 53, 64–73, 76–80, 83–85, 88–89
   simple_deployment_metrics_calculator.py1291787%125–142, 170–171, 203–204, 220–223, 235–236, 251
caribou/deployment_solver/deployment_metrics_calculator/models
   workflow_instance.py1948258%84–95, 107, 141–144, 183–186, 194–196, 203–227, 238–269, 275–289, 308–441, 520
caribou/deployment_solver/models
   indexer.py17194%25
caribou/deployment_solver/ranker
   ranker.py62494%14, 26–27, 95
caribou/endpoint
   client.py1302085%46, 56, 104, 126–129, 140, 167, 183, 185, 223–227, 232–233, 237–240
caribou/monitors
   deployment_manager.py1341887%64, 70, 81, 100, 122–127, 164–169, 184, 207, 222, 228, 263, 283
   monitor.py8188%12
caribou/syncers
   log_sync_workflow.py52129044%108, 163–173, 178, 186, 193, 203, 210, 248–254, 256, 258, 260, 262, 264, 268, 272, 274, 276, 279–285, 288, 292, 302–369, 378–385, 395–463, 472–475, 480–495, 506–597, 610–664, 674–700, 705–722, 727–744, 758, 765–788, 797–798, 808, 813–819, 846–847, 849–850, 855–856, 890–897, 901, 938, 940, 945, 979, 983–986, 994–1004, 1007–1013, 1016–1027, 1030
   log_syncer.py42198%46
caribou/syncers/components
   execution_data.py74495%76, 86, 129, 137
   workflow_run_sample.py83693%136, 145–149
TOTAL6378119981% 

Tests Skipped Failures Errors Time
642 0 💤 0 ❌ 0 🔥 20.269s ⏱️

Copy link
Collaborator

@vGsteiger vGsteiger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@vGsteiger vGsteiger merged commit c0eb0c9 into main Aug 21, 2024
7 checks passed
@vGsteiger vGsteiger deleted the features/Danidite/disable_deployment_resources branch August 21, 2024 05:43
vGsteiger pushed a commit that referenced this pull request Aug 21, 2024
)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.
vGsteiger added a commit that referenced this pull request Aug 21, 2024
* Fix some parts of the readme

* Update documentation

* chore: Update documentation and fix some parts of the readme

* updated setup documentation

* updated readme files

* Update poetry

* Update quick start and readme

* updated naming restrictions.

* Added additional restrictions to sourrce_code limitation regarding reserved log levels and invoke_serverless_function levels.

* Updated metrics manager with workflow collector and region collector changes.

* Remote Client Caching and Benchmark Updates (#288)

* Updated benchmarks, and caching created remove clients.

* Removed override permission for benchmarks.

* Added a unit test for _get_remote_client

* linting

* Slightly improved benchmarks labeling.

* Caribou-Go: Moving Monte-Carlo Simulation to Go (#282)

* paper information added

* Update QUICK_START.md

* move docker installation to INSTALL.md

* minor refine documentations

* Added caching to client CLI (#295)

* Added caching to client CLI

* linting

* Also modified remove_function instance to also use the cached remote client.

* Disabled all deployment package upload to S3 and S3 table creation (#296)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.

* Add contributing

* chore: Retry table creation on error

* Fix some last docu issues

---------

Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Danidite <[email protected]>
Co-authored-by: Daniel Long <[email protected]>
Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Mohammad Shahrad <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Danidite added a commit that referenced this pull request Aug 21, 2024
)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.
Danidite added a commit that referenced this pull request Aug 21, 2024
* Fix some parts of the readme

* Update documentation

* chore: Update documentation and fix some parts of the readme

* updated setup documentation

* updated readme files

* Update poetry

* Update quick start and readme

* updated naming restrictions.

* Added additional restrictions to sourrce_code limitation regarding reserved log levels and invoke_serverless_function levels.

* Updated metrics manager with workflow collector and region collector changes.

* Remote Client Caching and Benchmark Updates (#288)

* Updated benchmarks, and caching created remove clients.

* Removed override permission for benchmarks.

* Added a unit test for _get_remote_client

* linting

* Slightly improved benchmarks labeling.

* Caribou-Go: Moving Monte-Carlo Simulation to Go (#282)

* paper information added

* Update QUICK_START.md

* move docker installation to INSTALL.md

* minor refine documentations

* Added caching to client CLI (#295)

* Added caching to client CLI

* linting

* Also modified remove_function instance to also use the cached remote client.

* Disabled all deployment package upload to S3 and S3 table creation (#296)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.

* Add contributing

* chore: Retry table creation on error

* Fix some last docu issues

---------

Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Danidite <[email protected]>
Co-authored-by: Daniel Long <[email protected]>
Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Mohammad Shahrad <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disable Deployment Resources Bucket
2 participants