Unable to run BAB attack with any given configuration files

[yusf1013]

Describe the bug
Running with any configuration file in exp_configs/bab_attack/ fails with error

To Reproduce
Please provide us with the following to receive timely help:

1. Clone official repository
2. Set up conda env
3. Run conda activate alpha-beta-crown
4. Run python abcrown.py --config exp_configs/bab_attack/cifar_cnn_a_adv.yaml

Just running the official reposity with the given configuration file is failing for bab attack.
Output:

/home/yusuf/miniconda3/envs/alpha-beta-crown/lib/python3.9/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
/home/yusuf/miniconda3/envs/alpha-beta-crown/lib/python3.9/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/home/yusuf/miniconda3/envs/alpha-beta-crown/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: /home/yusuf/miniconda3/envs/alpha-beta-crown/lib/python3.9/site-packages/torchvision/image.so: undefined symbol: _ZNK3c1010TensorImpl36is_contiguous_nondefault_policy_implENS_12MemoryFormatE
  warn(f"Failed to load image Python extension: {e}")
Configurations:

general:
  device: cuda
  seed: 100
  conv_mode: patches
  deterministic: false
  double_fp: false
  loss_reduction_func: sum
  sparse_alpha: true
  sparse_interm: true
  save_adv_example: false
  eval_adv_example: false
  show_adv_example: false
  precompile_jit: false
  complete_verifier: bab
  enable_incomplete_verification: true
  csv_name: null
  results_file: out.txt
  root_path: ''
  deterministic_opt: false
  graph_optimizer: 'Customized("custom_graph_optimizer", "default_optimizer")'
  no_batchdim_buffers: false
  save_output: false
  output_file: out.pkl
model:
  name: cnn_4layer_adv
  path: models/sdp/cifar_cnn_a_adv.model
  onnx_path: null
  onnx_path_prefix: ''
  cache_onnx_conversion: false
  debug_onnx: false
  onnx_quirks: null
  input_shape: null
  onnx_loader: default_onnx_and_vnnlib_loader
  onnx_optimization_flags: none
  onnx_vnnlib_joint_optimization_flags: none
  check_optmized: false
  flatten_final_output: false
  optimize_graph: null
data:
  start: 537
  end: 538
  select_instance: null
  num_outputs: 10
  mean: [0.4914, 0.4824, 0.4467]
  std: [0.2471, 0.2435, 0.2616]
  pkl_path: null
  dataset: CIFAR
  data_filter_path: null
  data_idx_file: null
specification:
  type: lp
  robustness_type: verified-acc
  norm: .inf
  epsilon: 0.00784313725
  epsilon_min: 0.0
  vnnlib_path: null
  vnnlib_path_prefix: ''
  rhs_offset: null
solver:
  batch_size: 8192
  auto_enlarge_batch_size: false
  min_batch_size_ratio: 0.1
  use_float64_in_last_iteration: false
  early_stop_patience: 10
  start_save_best: 0.5
  bound_prop_method: alpha-crown
  init_bound_prop_method: same
  prune_after_crown: false
  crown:
    batch_size: 1000000000
    max_crown_size: 1000000000
  alpha-crown:
    alpha: true
    lr_alpha: 0.1
    iteration: 100
    share_alphas: false
    lr_decay: 0.98
    full_conv_alpha: true
    max_coeff_mul: .inf
    matmul_share_alphas: false
    apply_output_constraints_to: []
    disable_optimization: []
  beta-crown:
    lr_alpha: 0.01
    lr_beta: 0.05
    lr_decay: 0.999
    optimizer: adam
    iteration: 100
    beta: true
    beta_warmup: true
    enable_opt_interm_bounds: false
    all_node_split_LP: false
  forward:
    refine: false
    dynamic: false
    max_dim: 10000
  multi_class:
    label_batch_size: 32
    skip_with_refined_bound: true
  mip:
    parallel_solvers: 8
    solver_threads: 1
    refine_neuron_timeout: 15
    refine_neuron_time_percentage: 0.8
    early_stop: true
    adv_warmup: true
    mip_solver: gurobi
    skip_unsafe: false
bab:
  initial_max_domains: 1
  max_domains: 5000000
  decision_thresh: 0
  timeout: 1800
  timeout_scale: 1
  override_timeout: null
  get_upper_bound: true
  dfs_percent: 0.0
  pruning_in_iteration: true
  pruning_in_iteration_ratio: 0.2
  sort_targets: false
  batched_domain_list: true
  optimized_interm: ''
  interm_transfer: true
  recompute_interm: false
  sort_domain_interval: -1
  vanilla_crown: false
  cut:
    enabled: false
    implication: false
    bab_cut: false
    lp_cut: false
    method: null
    lr: 0.01
    lr_decay: 1.0
    iteration: 100
    bab_iteration: -1
    early_stop_patience: -1
    lr_beta: 0.02
    number_cuts: 50
    topk_cuts_in_filter: 1000
    batch_size_primal: 100
    max_num: 1000000000
    patches_cut: false
    cplex_cuts: false
    cplex_cuts_wait: 0
    cplex_cuts_revpickup: true
    cut_reference_bounds: true
    fix_intermediate_bounds: false
  branching:
    method: kfsb
    candidates: 3
    reduceop: max
    enable_intermediate_bound_opt: false
    branching_input_and_activation: false
    branching_input_and_activation_order: [input, relu]
    branching_input_iterations: 30
    branching_relu_iterations: 50
    sb_coeff_thresh: 0.001
    nonlinear_split:
      method: shortcut
      branching_point_method: uniform
      num_branches: 2
      branching_point_refinement: false
      filter: false
      filter_beta: false
      filter_batch_size: 10000
      filter_iterations: 25
      shortlist_size: 500
      loose_tanh_threshold: null
    new_input_split:
      enable: false
      batch_size: 2
      rounds: 1
      init_alpha_batch_size: 8192
      full_alpha: false
    input_split:
      enable: false
      enhanced_bound_prop_method: alpha-crown
      enhanced_branching_method: naive
      enhanced_bound_patience: 100000000.0
      attack_patience: 100000000.0
      adv_check: 0
      split_partitions: 2
      sb_margin_weight: 1.0
      sb_primary_spec: null
      sb_primary_spec_iter: 1
      sb_sum: false
      bf_backup_thresh: -1
      bf_rhs_offset: 0
      bf_zero_crossing_score: false
      ibp_enhancement: false
      catch_assertion: false
      compare_with_old_bounds: false
      update_rhs_with_attack: false
  attack:
    enabled: true
    beam_candidates: 128
    beam_depth: 6
    max_dive_fix_ratio: 0.5
    min_local_free_ratio: 0.5
    mip_start_iteration: 2
    mip_timeout: 360
    adv_pool_threshold: null
    refined_mip_attacker: true
    refined_batch_size: null
attack:
  pgd_order: before
  pgd_steps: 100
  pgd_restarts: 100
  pgd_batch_size: 100000000
  pgd_early_stop: true
  pgd_lr_decay: 0.99
  pgd_alpha: auto
  pgd_loss_mode: null
  enable_mip_attack: false
  adv_saver: default_adv_saver
  early_stop_condition: default_early_stop_condition
  adv_example_finalizer: default_adv_example_finalizer
  pgd_loss: default_pgd_loss
  cex_path: ./test_cex.txt
  attack_mode: diverse_pgd
  attack_tolerance: 0.0
  attack_func: attack_with_general_specs
  gama_lambda: 10.0
  gama_decay: 0.9
  check_clean: false
  input_split:
    pgd_steps: 100
    pgd_restarts: 30
    pgd_alpha: auto
  input_split_enhanced:
    pgd_steps: 200
    pgd_restarts: 500000
    pgd_alpha: auto
  input_split_check_adv:
    pgd_steps: 5
    pgd_restarts: 5
    pgd_alpha: auto
    max_num_domains: 10
debug:
  view_model: false
  lp_test: null
  rescale_vnnlib_ptb: null
  test_optimized_bounds: false
  test_optimized_bounds_after_n_iterations: 0

Experiments at Tue Feb 20 22:08:43 2024 on potato
Sequential(
  (0): Conv2d(3, 16, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (1): ReLU()
  (2): Conv2d(16, 32, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (3): ReLU()
  (4): Flatten(start_dim=1, end_dim=-1)
  (5): Linear(in_features=2048, out_features=100, bias=True)
  (6): ReLU()
  (7): Linear(in_features=100, out_features=10, bias=True)
)
Parameters:
  0.weight: shape torch.Size([16, 3, 4, 4])
  0.bias: shape torch.Size([16])
  2.weight: shape torch.Size([32, 16, 4, 4])
  2.bias: shape torch.Size([32])
  5.weight: shape torch.Size([100, 2048])
  5.bias: shape torch.Size([100])
  7.weight: shape torch.Size([10, 100])
  7.bias: shape torch.Size([10])
Trying generic MNIST/CIFAR data loader.
Files already downloaded and verified
Internal results will be saved to out.txt.

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% idx: 0, vnnlib ID: 537 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Attack parameters: initialization=uniform, steps=100, restarts=100, alpha=0.008052505552768707, initialization=uniform, GAMA=False
Model output of first 5 examples:
 tensor([[ -7.39000368, -13.28085136,  -4.82924271,  -7.23044920,  -5.35508966,
          -8.15784931,  -6.39428473,  -8.74342728, -11.18965244, -12.32971764]],
       device='cuda:0')
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.07it/s]
Adv example prediction (first 2 examples and 2 restarts):
 tensor([[[ -6.71217108, -12.28457832,  -4.12656498,  -6.24762297,  -4.17495298,
           -7.11022472,  -5.33284330,  -7.67359495, -10.33115959, -11.30035210],
         [ -6.71217108, -12.28457832,  -4.12656498,  -6.24762297,  -4.17495298,
           -7.11022472,  -5.33284330,  -7.67359495, -10.33115959, -11.30035210]]],
       device='cuda:0')
PGD attack margin (first 2 examles and 10 specs):
 tensor([[[2.58560610, 8.15801334, 2.12105799, 0.04838800, 2.98365974,
          1.20627832, 3.54702997, 6.20459461, 7.17378712]]], device='cuda:0')
number of violation:  0
Attack finished in 1.0545 seconds.
PGD attack failed
/home/yusuf/Desktop/abc_fresh/complete_verifier/auto_LiRPA/parse_graph.py:154: FutureWarning: 'torch.onnx.symbolic_helper._set_opset_version' is deprecated in version 1.13 and will be removed in version 1.14. Please remove its usage and avoid setting internal variables directly.
  _set_opset_version(12)
Model: BoundedModule(
  (/input.1): BoundInput(name=/input.1, inputs=[], perturbed=True)
  (/1): BoundParams(name=/1, inputs=[], perturbed=False)
  (/2): BoundParams(name=/2, inputs=[], perturbed=False)
  (/3): BoundParams(name=/3, inputs=[], perturbed=False)
  (/4): BoundParams(name=/4, inputs=[], perturbed=False)
  (/5): BoundParams(name=/5, inputs=[], perturbed=False)
  (/6): BoundParams(name=/6, inputs=[], perturbed=False)
  (/7): BoundParams(name=/7, inputs=[], perturbed=False)
  (/8): BoundParams(name=/8, inputs=[], perturbed=False)
  (/input): BoundConv(name=/input, inputs=[/input.1, /1, /2], perturbed=True)
  (/input.4): BoundRelu(name=/input.4, inputs=[/input], perturbed=True)
  (/input.8): BoundConv(name=/input.8, inputs=[/input.4, /3, /4], perturbed=True)
  (/12): BoundRelu(name=/12, inputs=[/input.8], perturbed=True)
  (/13): BoundFlatten(name=/13, inputs=[/12], perturbed=True)
  (/input.12): BoundLinear(name=/input.12, inputs=[/13, /5, /6], perturbed=True)
  (/15): BoundRelu(name=/15, inputs=[/input.12], perturbed=True)
  (/16): BoundLinear(name=/16, inputs=[/15, /7, /8], perturbed=True)
)
Original output: tensor([[ -7.39000368, -13.28085136,  -4.82924271,  -7.23044920,  -5.35508966,
          -8.15784931,  -6.39428473,  -8.74342632, -11.18965340, -12.32971668]],
       device='cuda:0')
Split layers:
  BoundConv(name=/input, inputs=[/input.1, /1, /2], perturbed=True): [(BoundRelu(name=/input.4, inputs=[/input], perturbed=True), 0)]
  BoundConv(name=/input.8, inputs=[/input.4, /3, /4], perturbed=True): [(BoundRelu(name=/12, inputs=[/input.8], perturbed=True), 0)]
  BoundLinear(name=/input.12, inputs=[/13, /5, /6], perturbed=True): [(BoundRelu(name=/15, inputs=[/input.12], perturbed=True), 0)]
Nonlinear functions:
   BoundRelu(name=/input.4, inputs=[/input], perturbed=True)
   BoundRelu(name=/12, inputs=[/input.8], perturbed=True)
   BoundRelu(name=/15, inputs=[/input.12], perturbed=True)
layer /input.4 using sparse-features alpha with shape [686]; unstable size 686; total size 4096 ([1, 16, 16, 16])
layer /input.4 start_node /input.8 using sparse-spec alpha [2, 458, 1, 686] with unstable size 457 total_size 2048 output_shape (32, 8, 8)
layer /input.4 start_node /input.12 using sparse-spec alpha [2, 25, 1, 686] with unstable size 24 total_size 100 output_shape torch.Size([100])
layer /input.4 start_node /16 using full alpha [2, 9, 1, 686] with unstable size None total_size 9 output_shape 9
layer /12 using sparse-features alpha with shape [457]; unstable size 457; total size 2048 ([1, 32, 8, 8])
layer /12 start_node /input.12 using sparse-spec alpha [2, 25, 1, 457] with unstable size 24 total_size 100 output_shape torch.Size([100])
layer /12 start_node /16 using full alpha [2, 9, 1, 457] with unstable size None total_size 9 output_shape 9
layer /15 using sparse-features alpha with shape [24]; unstable size 24; total size 100 ([1, 100])
layer /15 start_node /16 using full alpha [2, 9, 1, 24] with unstable size None total_size 9 output_shape 9
Optimizable variables initialized.
initial CROWN bounds: tensor([[ 0.77253890,  5.35441875,  0.88030529, -0.71995872,  1.51813030,
         -0.45570922,  1.82640553,  3.47526670,  4.63339520]], device='cuda:0') None
best_l after optimization: 18.567962646484375
alpha/beta optimization time: 2.470933437347412
initial alpha-crown bounds: tensor([[ 0.91563702,  5.49638367,  1.00899839, -0.58640003,  1.65252423,
         -0.34389973,  2.01817369,  3.65779972,  4.74874640]], device='cuda:0')
Worst class: (+ rhs) -0.586400032043457
Total VNNLIB file length: 9, max property batch size: 1, total number of batches: 9
lA shape: [torch.Size([9, 1, 16, 16, 16]), torch.Size([9, 1, 32, 8, 8]), torch.Size([9, 1, 100])]

Properties batch 0, size 1
Remaining timeout: 1795.2805857658386
##### Instance 0 first 10 spec matrices: 
tensor([[[ 0.,  0.,  1.,  0., -1.,  0.,  0.,  0.,  0.,  0.]]])
thresholds: tensor([0.], device='cuda:0') ######
Remaining spec index tensor([0], device='cuda:0') with bounds tensor([[-0.58640003]], device='cuda:0') need to verify.
Model prediction is: tensor([ -7.39000368, -13.28085136,  -4.82924271,  -7.23044920,  -5.35508966,
         -8.15784931,  -6.39428473,  -8.74342632, -11.18965340, -12.32971668],
       device='cuda:0')
build_with_refined_bounds batch [1/1]
setting alpha for layer /input.4 start_node /16 with alignment adjustment
setting alpha for layer /12 start_node /16 with alignment adjustment
setting alpha for layer /15 start_node /16 with alignment adjustment
all alpha initialized
true A is required, we do a full backward CROWN pass to obtain it
(alpha-)CROWN with fixed intermediate bounds: tensor([[-0.58639872]], device='cuda:0') tensor([[1.64917374]], device='cuda:0')
Intermediate layers: /input,/input.8,/input.12,/16
Keeping alphas for these layers: ['/16']
Keeping alphas for these layers: ['/16']
Traceback (most recent call last):
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/abcrown.py", line 612, in <module>
    abcrown.main()
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/abcrown.py", line 591, in main
    verified_status = self.complete_verifier(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/abcrown.py", line 398, in complete_verifier
    l, nodes, ret = self.bab(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/abcrown.py", line 241, in bab
    result = general_bab(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/bab.py", line 302, in general_bab
    domains = DomainClass(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/attack/domains.py", line 206, in __init__
    candidate_domains = [to_cpu(ReLUDomain(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/attack/domains.py", line 206, in <listcomp>
    candidate_domains = [to_cpu(ReLUDomain(
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/attack/domains.py", line 94, in to_cpu
    self.lA = [lA.to(device='cpu', non_blocking=True) for lA in self.lA]
  File "/home/yusuf/Desktop/abc_fresh/complete_verifier/attack/domains.py", line 94, in <listcomp>
    self.lA = [lA.to(device='cpu', non_blocking=True) for lA in self.lA]
AttributeError: 'str' object has no attribute 'to'

System configuration:

• OS: Ubuntu 23.04
• Python version: Python 3.9.17
• Pytorch Version: 1.13.1+cu117
• Hardware: intel 13900h, rtx 4070
• Have you tried to reproduce the problem in a cleanly created conda/virtualenv environment using official installation instructions and the latest code on the main branch?: Yes