Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buffer of Finch suggestions #9

Open
2 of 4 tasks
bennn opened this issue May 9, 2023 · 2 comments
Open
2 of 4 tasks

buffer of Finch suggestions #9

bennn opened this issue May 9, 2023 · 2 comments
Labels
low-priority Issues not requring immediate attention

Comments

@bennn
Copy link
Member

bennn commented May 9, 2023 •
edited
Loading

Low priority / unpromising ideas that might be worth suggesting to Finch:

  • init_stepper check that 1st arg is positive, non-NaN
  • when a mesh contains a NaN, why does Finch run 4x slower and not crash? (TF logging, for sure)
  • same thing happens when coefficient returns NaN --- why's there a big slowdown? should it crash?! (turn this into a small example; EDIT: it's a non-issue after all. Plain Finch runs the same with a NaN. TF without logging runs the same too. The slowdown is due to logging Improve performance of logging TrackedFloats.jl#19 )
  • ....
@bennn
Copy link
Member Author

bennn commented May 9, 2023

The init_stepper suggestion came after I tried injecting 10 times and saw the same result (link) each time:

  • inject at call to minimum in finch_interface.jl:1188
  • send the NaN as 1st arg to init_stepper
  • immediate crash at ceil call time_steppers.jl:27
ERROR: LoadError: InexactError: Int64(NaN)
Stacktrace:
 [1] Int64
   @ ./float.jl:788 [inlined]
 [2] convert(#unused#::Type{Int64}, x::Float64)
   @ Base ./number.jl:7
 [3] setproperty!(x::Stepper, f::Symbol, v::Float64)
   @ Base ./Base.jl:39
 [4] init_stepper(dx::TrackedFloat64, stepper::Stepper)
   @ Finch ~/.julia/dev/Finch/src/time_steppers.jl:27
 [5] macro expansion
   @ ~/.julia/dev/Finch/src/finch_interface.jl:1190 [inlined]
 [6] macro expansion
   @ ~/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237 [inlined]
 [7] conservationForm(var::Variable{TrackedFloat64}, cf::String)
   @ Finch ~/.julia/dev/Finch/src/finch_interface.jl:1116
 [8] top-level scope
   @ ~/code/uu/fpx/examples/examples/finch/inj-adv2d-fv.jl:63
in expression starting at /home/ben/code/uu/fpx/examples/examples/finch/inj-adv2d-fv.jl:63
Original program: 
#=
2D advection using higher order FV or structured or unstructured mesh
=#

### If the Finch package has already been added, use this line #########
using Finch # Note: to add the package, first do: ]add "https://github.com/paralab/Finch.git"

using Dates
using FloatTracker: TrackedFloat64, write_log_to_file, set_inject_nan, set_logger, set_exclude_stacktrace, FunctionRef

set_exclude_stacktrace([:prop])
set_logger(filename="inj-adv2d", buffersize=20, cstg=true, cstgArgs=false, cstgLineNum=true)
fns = [] ## [FunctionRef(:refel_nodes!, Symbol("refel_nodes.jl"))]
libs = [] ##["NBodySimulator", "OrdinaryDiffEq"]
now_str = Dates.format(now(), "yyyymmddHHMMss")
recording_file = "rand-adv2d-recording_$now_str"
println("Recording to $recording_file...")

### If not, use these four lines (working from the examples directory) ###
# if !@isdefined(Finch)
#     include("../Finch.jl");
#     using .Finch
# end
##########################################################################

initFinch("inj-adv2d", TrackedFloat64);

# Configuration setup
domain(2)
solverType(FV)

timeStepper(EULER_EXPLICIT,cfl=20000)
# NaN-making timestepper

println("begin MESH")
# a uniform grid of quads on a 0.1 x 0.3 rectangle domain
mesh(QUADMESH, elsperdim=[15, 45], bids=4, interval=[0, 0.1, 0, 0.3])
println("end MESH")

########
set_inject_nan(true, 20, 1, fns, libs, record=recording_file)
########

# Variables and BCs
u = variable("u", location=CELL)
boundary(u, 1, FLUX, "(abs(y-0.06) < 0.033 && sin(3*pi*t)>0) ? 1 : 0") # x=0
boundary(u, 2, NO_BC) # x=0.1
boundary(u, 3, NO_BC) # y=0
boundary(u, 4, NO_BC) # y=0.3

# Time interval and initial condition
T = 1.3;
# T=10
timeInterval(T)
initial(u, "0")

# Coefficients
coefficient("a", ["0.1*cos(pi*x/2/0.1)","0.3*sin(pi*x/2/0.1)"], type=VECTOR) # advection velocity
coefficient("s", ["0.1 * sin(pi*x)^4 * sin(pi*y)^4"]) # source

# The "upwind" function applies upwinding to the term (a.n)*u with flow velocity a.
# The optional third parameter is for tuning. Default upwind = 0, central = 1. Choose something between these.
conservationForm(u, "s + surface(upwind(a,u))");

println("begin SOLVE")
solve(u)
println("end SOLVE")

finalizeFinch()

write_log_to_file()

@ashton314 ashton314 added the low-priority Issues not requring immediate attention label May 9, 2023
@bennn
Copy link
Member Author

bennn commented May 17, 2023

Outputs from two poisson1d injections below.

Both injections happen in a similar way:

  • when building a grid (grid_from_mesh),
  • when inverting a refel_nodes matrix,
  • put a NaN inside one cell

Leads to a big slowdown: up from 70s to 300s and from 10GB to 23GB.

Why is it a slowdown and not an error?

Original code, with random injections 
#=
# 1D Poisson, Dirichlet bc
# CG, Linear element
# Simplest test possible
=#

using Finch
using Dates
using FloatTracker: TrackedFloat64, FunctionRef, write_log_to_file, set_inject_nan, set_logger, set_exclude_stacktrace

set_exclude_stacktrace([:prop])
set_logger(filename="inj-poisson", buffersize=20, cstg=true, cstgArgs=false, cstgLineNum=true)
fns = [] ## [FunctionRef(:refel_nodes!, Symbol("refel_nodes.jl"))]
libs = [] ##["NBodySimulator", "OrdinaryDiffEq"]
now_str = Dates.format(now(), "yyyymmddHHMMss")
recording_file = "rand-poisson-recording_$now_str"
println("Recording to $recording_file...")

########
set_inject_nan(true, 20, 1, fns, libs, record=recording_file)
########

initFinch("poisson1d", TrackedFloat64);

# Set up the configuration
domain(1) # dimension

mesh(LINEMESH, elsperdim=180)   # build uniform LINEMESH with 180 elements

u = variable("u")              # make a scalar variable with symbol u
testSymbol("v")                # sets the symbol for a test function

boundary(u, 1, DIRICHLET, 0)  # boundary condition for BID 1 is Dirichlet with value 0

# Write the weak form 
coefficient("f", "-100*pi*pi*sin(10*pi*x)*sin(pi*x) - pi*pi*sin(10*pi*x)*sin(pi*x) + 20*pi*pi*cos(10*pi*x)*cos(pi*x)")
weakForm(u, "-grad(u)*grad(v) - f*v")

# exportCode("poisson1dcode");
# importCode("poisson1dcode");

solve(u);

finalizeFinch()

write_log_to_file()

1

Recording file:

21, lu.jl

Injection site + stacktrace, final output:

INJECT - (1.224744871391589, -1.224744871391589)...
run_or_inject at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:39
- at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:105
#generic_lufact!#178 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:181
generic_lufact!##kw at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:137
#lu!#177 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:135
lu!##kw at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:135
#lu#179 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:279
lu at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:278
lu at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:278
inv at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/dense.jl:893
build_refel at /home/ben/.julia/dev/Finch/src/refel.jl:257
#grid_from_mesh#21 at /home/ben/.julia/dev/Finch/src/grid.jl:159
grid_from_mesh##kw at /home/ben/.julia/dev/Finch/src/grid.jl:80
macro expansion at /home/ben/.julia/dev/Finch/src/Finch.jl:242
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#add_mesh#134 at /home/ben/.julia/dev/Finch/src/Finch.jl:145
add_mesh##kw at /home/ben/.julia/dev/Finch/src/Finch.jl:142
macro expansion at /home/ben/.julia/dev/Finch/src/finch_interface.jl:414
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#mesh#81 at /home/ben/.julia/dev/Finch/src/finch_interface.jl:359
mesh##kw at /home/ben/.julia/dev/Finch/src/finch_interface.jl:357
top-level scope at /home/ben/code/uu/fpx/examples/examples/finch/inj-poisson.jl:28
eval at ./boot.jl:368
include_string at ./loading.jl:1428
_include at ./loading.jl:1488
include at ./Base.jl:419
exec_options at ./client.jl:303
_start at ./client.jl:522

 ──────────────────────────────────────────────────────────────────────────
                                  Time                    Allocations      
                         ───────────────────────   ────────────────────────
    Tot / % measured:          300s /  88.6%           23.5GiB /  71.4%    

 Section         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────
 Solve                1     239s   89.8%    239s   14.9GiB   88.7%  14.9GiB
   FE_solve           1     238s   89.7%    238s   14.9GiB   88.5%  14.9GiB
     lin_solve        1     228s   85.8%    228s   13.2GiB   78.5%  13.2GiB
     assembly         1   78.6ms    0.0%  78.6ms   39.0MiB    0.2%  39.0MiB
     scatter          1   1.44μs    0.0%  1.44μs     0.00B    0.0%    0.00B
 Mesh                 1    17.8s    6.7%   17.8s   1.34GiB    8.0%  1.34GiB
   mesh2grid          1    11.9s    4.5%   11.9s    811MiB    4.7%   811MiB
   gen/read           1    142ms    0.1%   142ms   6.75MiB    0.0%  6.75MiB
   geo factors        1   85.4ms    0.0%  85.4ms   14.9MiB    0.1%  14.9MiB
 CodeGen              1    9.35s    3.5%   9.35s    579MiB    3.4%   579MiB
 ──────────────────────────────────────────────────────────────────────────
Finch has completed.

2

Recording File:

24, triangular.jl

Injection site + stacktrace, final output:

INJECT - (0.0, 1.0)...
run_or_inject at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:39
- at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:105
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/triangular.jl:1245
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/triangular.jl:1258
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:409
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/factorization.jl:126
inv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:490
inv at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/dense.jl:893
build_refel at /home/ben/.julia/dev/Finch/src/refel.jl:257
#grid_from_mesh#21 at /home/ben/.julia/dev/Finch/src/grid.jl:159
grid_from_mesh##kw at /home/ben/.julia/dev/Finch/src/grid.jl:80
macro expansion at /home/ben/.julia/dev/Finch/src/Finch.jl:242
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#add_mesh#134 at /home/ben/.julia/dev/Finch/src/Finch.jl:145
add_mesh##kw at /home/ben/.julia/dev/Finch/src/Finch.jl:142
macro expansion at /home/ben/.julia/dev/Finch/src/finch_interface.jl:414
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#mesh#81 at /home/ben/.julia/dev/Finch/src/finch_interface.jl:359
mesh##kw at /home/ben/.julia/dev/Finch/src/finch_interface.jl:357
top-level scope at /home/ben/code/uu/fpx/examples/examples/finch/inj-poisson.jl:28
eval at ./boot.jl:368
include_string at ./loading.jl:1428
_include at ./loading.jl:1488
include at ./Base.jl:419
exec_options at ./client.jl:303
_start at ./client.jl:522

 ──────────────────────────────────────────────────────────────────────────
                                  Time                    Allocations      
                         ───────────────────────   ────────────────────────
    Tot / % measured:          300s /  88.6%           23.5GiB /  71.4%    

 Section         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────
 Solve                1     239s   89.9%    239s   14.9GiB   88.7%  14.9GiB
   FE_solve           1     238s   89.8%    238s   14.9GiB   88.5%  14.9GiB
     lin_solve        1     229s   86.2%    229s   13.2GiB   78.6%  13.2GiB
     assembly         1   76.6ms    0.0%  76.6ms   38.0MiB    0.2%  38.0MiB
     scatter          1   1.50μs    0.0%  1.50μs     0.00B    0.0%    0.00B
 Mesh                 1    17.6s    6.6%   17.6s   1.34GiB    8.0%  1.34GiB
   mesh2grid          1    11.7s    4.4%   11.7s    808MiB    4.7%   808MiB
   gen/read           1    142ms    0.1%   142ms   6.75MiB    0.0%  6.75MiB
   geo factors        1   83.6ms    0.0%  83.6ms   13.3MiB    0.1%  13.3MiB
 CodeGen              1    9.10s    3.4%   9.10s    579MiB    3.4%   579MiB
 ──────────────────────────────────────────────────────────────────────────
Finch has completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
low-priority Issues not requring immediate attention
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bennn @ashton314