Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation and Setup #285

Merged
merged 24 commits into from
Aug 21, 2024
Merged

Conversation

vGsteiger
Copy link
Collaborator

@vGsteiger vGsteiger commented Aug 18, 2024

@vGsteiger vGsteiger added the documentation Improvements or additions to documentation label Aug 18, 2024
@vGsteiger vGsteiger self-assigned this Aug 18, 2024
@Danidite Danidite marked this pull request as ready for review August 20, 2024 16:44
Copy link

github-actions bot commented Aug 20, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
caribou/common
   utils.py37878%43–58
caribou/common/models
   endpoints.py34391%18–19, 50
caribou/common/models/remote_client
   aws_remote_client.py4826886%42, 65, 105, 151, 216, 219, 255, 272–273, 301–302, 341, 352, 362, 395–397, 405–406, 425–427, 446, 449, 473, 486–487, 490, 496, 536–538, 576–578, 596–597, 638, 644, 654, 658, 669–670, 678–679, 688, 692, 703, 713–715, 735, 746–748, 768, 783–785, 800–805, 833–834, 838, 854–858, 864–865
   remote_client_factory.py21386%21–23
caribou/data_collector/components
   data_collector.py18194%23
   data_exporter.py34294%49, 56
caribou/data_collector/components/carbon
   carbon_collector.py21290%34–35
   carbon_retriever.py1341887%25, 59, 93, 135–144, 148, 153, 160, 168–169, 215, 246, 249, 256
caribou/data_collector/components/performance
   performance_retriever.py38684%44, 50–51, 56–59
caribou/data_collector/components/provider
   provider_collector.py24292%43–44
   provider_retriever.py2945083%24, 32, 34, 47, 59, 77, 84, 100, 132–152, 183–281, 299–304, 336, 344, 346, 348, 350, 352, 354, 356, 432, 446, 556, 649
caribou/data_collector/components/workflow
   workflow_collector.py25196%34
   workflow_retriever.py30510964%216–257, 280, 330–337, 339, 371–414, 436, 467–526, 533–563, 566–583, 587–604, 615, 676, 698
caribou/data_collector/utils/latency_retriever
   aws_latency_retriever.py74988%59, 63, 65, 67, 70, 74, 76, 78, 81
   latency_retriever.py6183%8
caribou/deployment/client
   caribou.py16194%22
   caribou_workflow.py46513571%54, 70–71, 80–83, 129, 201–223, 267–268, 282, 315–326, 347–354, 380–391, 434–454, 562, 578, 581, 636–662, 667–683, 705–715, 793, 796, 798, 800, 802, 804, 831, 866–881, 916–917, 927–931, 961, 996, 1004, 1016–1096, 1109–1138, 1172–1181, 1212, 1245, 1273–1282, 1302–1320, 1340, 1353–1359, 1365
caribou/deployment/common/config
   config.py83199%104
caribou/deployment/common/deploy
   deployer.py1401689%43, 102–109, 199, 240–242, 249, 256, 267
   deployment_packager.py1924577%31, 34, 39, 63–70, 73–84, 90, 96, 101, 106, 128, 156, 169, 182, 190, 251–253, 259–274, 284, 286, 288, 291
   executor.py53492%32–33, 66, 68
   workflow_builder.py1711392%43, 59, 146, 178, 186, 189, 191, 196, 205, 208, 210, 324, 330
caribou/deployment/common/deploy/models
   function.py42295%42, 53
   function_instance.py12283%16, 22
   iam_role.py34391%19–21
   instructions.py18289%17, 26
   resource.py18194%25
   workflow.py1943284%39, 44, 67, 77, 99, 120, 166, 171, 206, 219–231, 272, 281, 289, 293, 298–327
caribou/deployment/common/deploy_instructions
   aws_deploy_instructions.py24292%37, 83
   deploy_instructions.py54983%57, 95, 145, 151, 164, 168, 172, 176, 182
caribou/deployment/common/factories
   deploy_instruction_factory.py18383%20–22
   deployer_factory.py1071685%24–27, 41, 52–53, 82, 90, 99, 108, 119, 126–129
caribou/deployment/server
   re_deployment_server.py61592%36, 48, 51, 58, 107
caribou/deployment_solver
   workflow_config.py751185%24, 26, 32–33, 54–56, 61, 79, 82, 107
caribou/deployment_solver/deployment_algorithms
   coarse_grained_deployment_algorithm.py25292%27, 32
   deployment_algorithm.py981189%52, 91, 105, 118, 132, 149, 184–190, 219
   fine_grained_deployment_algorithm.py25292%38, 43
   stochastic_heuristic_deployment_algorithm.py921485%18–25, 49, 59, 71, 92, 95, 100, 106–109, 119, 136, 163, 170
caribou/deployment_solver/deployment_input
   input_manager.py1995572%126, 132, 135, 194, 200, 246–254, 276–282, 291–294, 304–307, 320–339, 352–373, 379–387, 441–442, 476–487, 493–495, 501
caribou/deployment_solver/deployment_input/components/calculators
   carbon_calculator.py96496%124–128, 214, 261
   cost_calculator.py70199%163
   runtime_calculator.py1091190%130–145, 152, 201, 209, 262, 295
caribou/deployment_solver/deployment_input/components/loaders
   carbon_loader.py21386%22, 43, 46
   datacenter_loader.py49590%106–111, 114
   performance_loader.py18194%38
   region_viability_loader.py17288%15, 32
   workflow_loader.py1472682%86, 102–112, 122, 145, 165, 183, 193, 210, 264–276, 292, 364–369, 392, 430–432, 437
caribou/deployment_solver/deployment_metrics_calculator
   deployment_metrics_calculator.py40198%49
   go_deployment_metrics_calculator.py513139%18–21, 25–27, 41–50, 53, 64–73, 76–80, 83–85, 88–89
   simple_deployment_metrics_calculator.py1291787%125–142, 170–171, 203–204, 220–223, 235–236, 251
caribou/deployment_solver/deployment_metrics_calculator/models
   workflow_instance.py1948258%84–95, 107, 141–144, 183–186, 194–196, 203–227, 238–269, 275–289, 308–441, 520
caribou/deployment_solver/models
   indexer.py17194%25
caribou/deployment_solver/ranker
   ranker.py62494%14, 26–27, 95
caribou/endpoint
   client.py1392185%50, 60, 110, 120, 142–145, 156, 183, 199, 201, 239–243, 248–249, 253–256
caribou/monitors
   deployment_manager.py1341887%64, 70, 81, 100, 122–127, 164–169, 184, 207, 222, 228, 263, 283
   monitor.py8188%12
caribou/syncers
   log_sync_workflow.py52129044%108, 163–173, 178, 186, 193, 203, 210, 248–254, 256, 258, 260, 262, 264, 268, 272, 274, 276, 279–285, 288, 292, 302–369, 378–385, 395–463, 472–475, 480–495, 506–597, 610–664, 674–700, 705–722, 727–744, 758, 765–788, 797–798, 808, 813–819, 846–847, 849–850, 855–856, 890–897, 901, 938, 940, 945, 979, 983–986, 994–1004, 1007–1013, 1016–1027, 1030
   log_syncer.py42198%46
caribou/syncers/components
   execution_data.py74495%76, 86, 129, 137
   workflow_run_sample.py83693%136, 145–149
TOTAL6387120081% 

Tests Skipped Failures Errors Time
643 0 💤 0 ❌ 0 🔥 16.466s ⏱️

Danidite and others added 2 commits August 20, 2024 20:02
* Updated benchmarks, and caching created remove clients.

* Removed override permission for benchmarks.

* Added a unit test for _get_remote_client

* linting

* Slightly improved benchmarks labeling.
@vGsteiger vGsteiger requested review from Danidite and jerryyiransun and removed request for Danidite August 20, 2024 19:05
@engshahrad
Copy link
Contributor

engshahrad commented Aug 21, 2024

@vGsteiger README file seems to links to a non-existent contribution guide at the end.

Copy link
Collaborator

@jerryyiransun jerryyiransun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I've made a few modifications

* Added caching to client CLI

* linting

* Also modified remove_function instance to also use the cached remote client.
Danidite and others added 5 commits August 21, 2024 06:46
)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.
@vGsteiger vGsteiger merged commit 5c030ba into main Aug 21, 2024
7 checks passed
@vGsteiger vGsteiger deleted the feature/vGsteiger/documentation branch August 21, 2024 08:10
Danidite added a commit that referenced this pull request Aug 21, 2024
* Fix some parts of the readme

* Update documentation

* chore: Update documentation and fix some parts of the readme

* updated setup documentation

* updated readme files

* Update poetry

* Update quick start and readme

* updated naming restrictions.

* Added additional restrictions to sourrce_code limitation regarding reserved log levels and invoke_serverless_function levels.

* Updated metrics manager with workflow collector and region collector changes.

* Remote Client Caching and Benchmark Updates (#288)

* Updated benchmarks, and caching created remove clients.

* Removed override permission for benchmarks.

* Added a unit test for _get_remote_client

* linting

* Slightly improved benchmarks labeling.

* Caribou-Go: Moving Monte-Carlo Simulation to Go (#282)

* paper information added

* Update QUICK_START.md

* move docker installation to INSTALL.md

* minor refine documentations

* Added caching to client CLI (#295)

* Added caching to client CLI

* linting

* Also modified remove_function instance to also use the cached remote client.

* Disabled all deployment package upload to S3 and S3 table creation (#296)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.

* Add contributing

* chore: Retry table creation on error

* Fix some last docu issues

---------

Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Danidite <[email protected]>
Co-authored-by: Daniel Long <[email protected]>
Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Mohammad Shahrad <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update Setup Documentation Update Documentation Document the naming Restrictions
5 participants