A LazyBroadcast as a dependency

[charleskawczynski]

A step towards #3594.

This PR adds LazyBroadcast as a dependency, to help prevent #3540 from becoming stale.

[charleskawczynski]

@Sbozzolo, can you please update documentation about how to add dependencies? It's not clear, or working for me locally.

For example, I'm running julia --project=test, and

julia> using Revise; include("test/dependencies.jl")
Detected stale dependencies: Set(["LazyBroadcast"])
Please, edit the dependencies.jl file
Test Failed at /Users/charliekawczynski/Dropbox/Caltech/work/dev/CliMA/ClimaAtmos.jl/test/dependencies.jl:88
  Expression: direct_dependencies == known_dependencies
   Evaluated: Set(["ClimaCore", "Statistics", "ClimaUtilities", "Thermodynamics", "ClimaDiagnostics", "Random", "CloudMicrophysics", "ClimaComms", "FastGaussQuadrature", "Interpolations"  …  "Dates", "StaticArrays", "ClimaParams", "NCDatasets", "UnrolledUtilities", "SurfaceFluxes", "Logging", "DiffEqBase", "ArtifactWrappers", "ArgParse"]) == Set(["ClimaCore", "Statistics", "ClimaUtilities", "Thermodynamics", "ClimaDiagnostics", "Random", "CloudMicrophysics", "ClimaComms", "FastGaussQuadrature", "Interpolations"  …  "Dates", "StaticArrays", "ClimaParams", "NCDatasets", "UnrolledUtilities", "SurfaceFluxes", "Logging", "DiffEqBase", "ArtifactWrappers", "ArgParse"])

ERROR: LoadError: There was an error during testing
in expression starting at /Users/charliekawczynski/Dropbox/Caltech/work/dev/CliMA/ClimaAtmos.jl/test/dependencies.jl:88

Also, Pkg.project is experimental, and I don't think we should be using that in our test suite:

julia> using Pkg

help?> Pkg.project
  Pkg.project()::ProjectInfo

  This feature is considered experimental.

  Request a ProjectInfo struct which contains information about the active project.

  ProjectInfo fields

[Sbozzolo]

Would extending the comment to this help?

# This test follows the spirit of the reproducibility tests and tries to capture
# the cost of adding a new dependency by having developers explicitly come here
# and declare their intents. If you want to add a new dependency, add the name
# to the list below. Possibly, also add a short explanation about why that
# dependency is required.

I checked your branch locally and it works on my end. Could it be that you had a stale Manifest?

[Sbozzolo]

Also, Pkg.project is experimental, and I don't think we should be using that in our test suite:

I didn't realize it was experimental. If I remember correctly, the alternative was a much uglier code that had to traverse trees to find the dependencies. I can look to see if there's other ways.

[charleskawczynski]

Would extending the comment to this help?

# This test follows the spirit of the reproducibility tests and tries to capture
# the cost of adding a new dependency by having developers explicitly come here
# and declare their intents. If you want to add a new dependency, add the name
# to the list below. Possibly, also add a short explanation about why that
# dependency is required.

I might suggest updating the comment to:

# We believe in slim dependencies. We strive to achieve this by slightly resisting
# new dependencies. If you want to add a new dependency, add the name to the list
# below. Comments justifying dependencies are welcome.

? I can open this suggestion in a separate PR

I checked your branch locally and it works on my end. Could it be that you had a stale Manifest?

Ah, no, sorry, it was my fault. I didn't realize we upgraded ClimaAtmos to Julia 1.11, which seemed to be the issue.

[Sbozzolo]

Just leaving a comment that this PR to me indicates that there's a problem with our workflows.

We shouldn't need to add a dependency to ClimaAtmos that is now pushed onto every developer/user just because it simplifies rebasing someone's development branch.

I think that removing the number of checked Manifests would help with this (and other things as well)

[charleskawczynski]

I think that removing the number of checked Manifests would help with this (and other things as well)

The manifests are an important ingredient for reproducibility, I'm not sure that we can get rid of them, unfortunately.

[Sbozzolo]

I think that removing the number of checked Manifests would help with this (and other things as well)

The manifests are an important ingredient for reproducibility, I'm not sure that we can get rid of them, unfortunately.

I think we can remove most of them and only have a .buildkite one. For example, we don't need a Manifest.toml if buildkite only runs using Julia 1.11. We can also compactify examples with perf. I'll open a PR for this

[charleskawczynski]

Yeah, I guess that's true. Thanks!