Skip to content

Commit

Permalink
Add split_into_chunks for ProgressLoggingFoldable
Browse files Browse the repository at this point in the history
* Calling collect on a Foldable that is wrapped with withprogress to
allow progress logging fails because ProgressLoggingFoldable is not
iterable.
* We add a method of split_into_chunks for when coll is a
ProgressLoggingFodlable that partitions the foldable enclosed withing
the ProgressLoggingFoldable struct, collects it, and then rewraps with
a call to withprogress to allow logging to continue.
* This allows the creation of transducers with progress logging and fixes JuliaFolds2/Transduces.jl#10
  • Loading branch information
andreas-theo committed May 6, 2024
1 parent f3892cc commit 45c2dff
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
1 change: 1 addition & 0 deletions src/progress.jl
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ end
Base.IteratorSize(::Type{ProgressLoggingFoldable{T}}) where {T} = Base.IteratorSize(T)
Base.IteratorEltype(::Type{ProgressLoggingFoldable{T}}) where {T} = Base.IteratorEltype(T)
Base.length(foldable::ProgressLoggingFoldable) = length(foldable.foldable)
Base.size(foldable::ProgressLoggingFoldable) = size(foldable.foldable)
Base.eltype(::Type{ProgressLoggingFoldable{T}}) where {T} = eltype(T)

# Use Juno/Atom-compatible log-level. See:
Expand Down
4 changes: 4 additions & 0 deletions src/reduce.jl
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,10 @@ function split_into_chunks(coll, sz)
collect(Iterators.partition(coll, sz))
end

function split_into_chunks(coll::Transducers.ProgressLoggingFoldable, sz)
withprogress(collect(Iterators.partition(coll.foldable, sz)); interval=coll.interval)
end

tcopy(xf, reducible; kwargs...) = tcopy(xf, _materializer(reducible), reducible; kwargs...)

function tcopy(::Type{T}, itr; kwargs...) where {T}
Expand Down

0 comments on commit 45c2dff

Please sign in to comment.