[Feature Request] Progress indicators for iterators with unknown length

[kescobo]

I'm really enjoying getting to know Term.jl, and the docs are great! One thing that I'm missing is a progress bar for an iterator or loop with unknown length. Clearly, you could not have a percent or fraction, but something like a spinning ball + a counter for the number of iterations that have passed or time elapse would be really useful.

You can see ProgressMeter.jl for inspiration.

julia> mutable struct MyItr
           count
       end

julia> Base.iterate(mi::MyItr, state=mi.count) = state == 0 ? nothing : (state, state-1)

julia> let prog = ProgressUnknown("Here we go...", spinner=true)
       for i in MyItr(100)
           ProgressMeter.next!(prog)
           sleep(0.1)
       end
       ProgressMeter.finish!(prog)
       end
✓ Here we go...          Time: 0:00:10

[FedeClaudi]

Thanks for opening the first discussion!

This would be very good indeed, added to the list.

You can almost get something like this done with the current code if you need it urgently"

# create custom columns layout
import Term.progress: DescriptionColumn, ElapsedColumn, AbstractColumn, ProgressBar, update, start, stop
import Term

# define a new column type
struct CountsColumn <: AbstractColumn
    segments::Vector
    measure::Measure
    style::String
end

# constructor
CountsColumn(style::String) = CountsColumn([], Measure(10, 1), style)

# update method
update(col::CountsColumn, i::Int, args...)::String = apply_style("[$(col.style)]$i iterations[/$(col.style)]")


columns = [
    DescriptionColumn("[green bold]Job running[/green bold]"),
    CountsColumn("blue"),
    ElapsedColumn(),
]


# create iterable
mutable struct MyItr
    count
end

Base.iterate(mi::MyItr, state=mi.count) = state == 0 ? nothing : (state, state-1)


# ----------------------------- run progress bar ----------------------------- #
pbar = ProgressBar(; N=1000, columns=columns)  # set to very high value so that it gets ignored
start(pbar)

for i in MyItr(100)
    update(pbar)
    sleep(0.1)
end
stop(pbar)

The problem is that ProgressBar still expects an N argument and it will add a BarColumn if one is not provided. This is very easy to change: just let ProgressBar accept N=nothing and don't add the BarColumn in case.

I can add it soon, but feel free to send a PR if you'd like!

[kescobo]

No rush! I have not had a chance to dive into the internals (and likely won't have time for at least a bit), so a PR is probably not going to happen in the short term, but I'm already imagining a lot of uses for this package! I definitely hope to contribute in the future.

[FedeClaudi]

Awesome! Looking forward to that!

Also let me know what you end up using Term for!

[FedeClaudi]

Working on it, keep an eye here #62

[FedeClaudi]

It will come with: #62

Usage:

pbar = ProgressBar(; columns=:spinner)

to use these preset columns. If you don't pass N you can still use any set of columns you'd like, but it just won't show a ProgressColumn with the progress bar any longer.

The spinner visualization is handled by a new SpinnerColumn and multiple types of spinners are available, as shown (also you can apply markup style to the spinner).

Hope you'll like it!

[kescobo]

Love 'bar' - reminds me of Kit from Knight rider 😆

[kescobo]

Or maybe cylons...

[nathanrboyer]

I'd like to see the spinner change to a check mark when it is done. I'm also wondering how you got your previous spinners to stop. My first spinner restarts when I start the second one.

function f()
    with(progressbar) do
        readjob = addjob!(progressbar, description = "Reading Input File")
        start!(readjob)
        sleep(5)
        stop!(readjob)

        transformjob = addjob!(progressbar, description = "Transforming Tables")
        start!(transformjob)
        sleep(5)
        stop!(transformjob)
    end
end
progressbar = ProgressBar(; columns=:minimal, columns_kwargs = Dict(:SpinnerColumn => Dict(:spinnertype => :circle)))
f()

[FedeClaudi]

In the example above I was creating entirely new progress bars, this was before the new API with addjob! and the rest.
It looks like it's a bug and the spinner doesn't stop when calling stop! as one would would expect. It will require a small change to the update method of the spinner column here.

In the meanwhile you can remove the job to hide the spinner:

function f()
  with(progressbar) do
      readjob = addjob!(progressbar, description = "Reading Input File")
      start!(readjob)
      sleep(1)
      stop!(readjob)
      removejob!(progressbar, readjob)

      println("done")
      sleep(2)

      transformjob = addjob!(progressbar, description = "Transforming Tables")
      start!(transformjob)
      sleep(2)
      stop!(transformjob)
  end
end

I'd like to see the spinner change to a check mark when it is done.

That's a neat idea, I can add that in.

[FedeClaudi]

From the next release, calling stop! on a spinner progress bar not only stops the spinner, but also turns it into a checkmark:

Thanks for the suggestion @nathanrboyer

[nathanrboyer]

This is working great for me with the sleep test case. However, the spinner never shows up if I am doing actual processing like computing something large or generating a plot. It just jumps right to the checkmark when it is done. MWE:

function f(sleeptime)
    progressbar = ProgressBar(; columns=:minimal, columns_kwargs = Dict(:SpinnerColumn => Dict(:spinnertype => :circle)))
    with(progressbar) do
        readjob = addjob!(progressbar, description = "Reading Input File")
        start!(readjob)
        sleep(sleeptime)
        stop!(readjob)

        transformjob = addjob!(progressbar, description = "Processing Input")
        start!(transformjob)
        collect(1:10^8) .^ (10^9) # Doing real work
        stop!(transformjob)

        writejob = addjob!(progressbar, description = "Writing Output File")
        start!(writejob)
        sleep(sleeptime)
        stop!(writejob)
    end
end

Also, does the progress bar have to jump down to the bottom of the terminal? Thanks!

[FedeClaudi]

It's something that came up before: #77 (comment)

I think I'm doing something wrong with how the progress bar display is updated in the background (by a separate process). I'm not very strong on multi-processing in Julia yet, but it's something that I aim to fix with the next major release.
Adding a sleep statement to your loops should fix that - not a very elegant solution but it should do for now I hope.

Also, does the progress bar have to jump down to the bottom of the terminal? Thanks!

It does, because that allows for any output to stdout to be printed above the progress bar without breaking the progress bar itself.

[nathanrboyer]

Adding sleep to each section helped somewhat but didn't fix everything. Three of my spinners showed up but didn't spin. The last spinner (for plotting) never showed up. The spinners aren't critical I suppose, but it is nice to know what the code is working on.

[FedeClaudi]

I agree, I always add progress information or I get too impatient if I don't know what is being run... sorry that this is still a bit buggy!

[FedeClaudi]

This was fixed thanks to the brilliant folks at Julia discourse. Will come out with the next version v0.4.