Skip to content

Commit

Permalink
MPIEvaluator: Fix logging by explicitly passing logging level
Browse files Browse the repository at this point in the history
Pass the log level explicitly to the run_experiment_mpi() function.

The level passed to log_to_stderr isn't available when the _logger object is initiated earlier. To migrate this problem, we explicitly pass it.

The initializer first took care of that, but since that wouldn't allow multiple consecutive calls to the MPIEvaluator it had to be removed again.

Now there are two sets of options:
- ema_logging.log_to_stderr(level=n, set_root_logger_levels=True) --> Everything level n
- ema_logging.log_to_stderr(level=n, set_root_logger_levels=False) --> Evaluator module level n, internal MPI / EMA process level 10 (default)

When level=10, set_root_logger_levels True or False doesn't matter.

Might have performance impact by setting up a logging.basicConfig for every experiment.
  • Loading branch information
EwoutH committed Nov 2, 2023
1 parent dff46cd commit f67b194
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions ema_workbench/em_framework/evaluators.py
Original file line number Diff line number Diff line change
Expand Up @@ -443,8 +443,11 @@ def finalize(self):
def evaluate_experiments(self, scenarios, policies, callback, combine="factorial"):
ex_gen = experiment_generator(scenarios, self._msis, policies, combine=combine)
experiments = list(ex_gen)
log_level = _logger.getEffectiveLevel()

packed = [(experiment, experiment.model_name, self._msis) for experiment in experiments]
packed = [
(experiment, experiment.model_name, self._msis, log_level) for experiment in experiments
]

_logger.info(
f"MPIEvaluator: Starting {len(packed)} experiments using MPI pool with {self._pool._max_workers} workers"
Expand All @@ -462,7 +465,9 @@ def run_experiment_mpi(packed_data):

rank = COMM_WORLD.Get_rank()

experiment, model_name, msis = packed_data
experiment, model_name, msis, level = packed_data

logging.basicConfig(level=level, format="[%(processName)s/%(levelname)s] %(message)s")
_logger.debug(f"MPI Rank {rank}: starting {repr(experiment)}")

models = NamedObjectMap(AbstractModel)
Expand Down

0 comments on commit f67b194

Please sign in to comment.