Issue with Calibration process

[Areza622]

Hi,

I tried running the calibration process. First, I installed all the required libraries and dependencies. Then, I followed the example for calibration provided in the instructions on the main page (Section 5: CALIBRATION: Automatic calibration for MATSim scenario).

After running the code, I encountered an error related to missing initial values, which I added manually. Upon running the code again, the MATSim GUI opened. After providing the config file and running the simulation, I received an error regarding some files in the output folder (persons, plans, and trips files). After struggling for a while, I modified the code to provide a direct path to each of these required files. However, this approach seems illogical, and I wonder if there’s a way to run the Python code without manually specifying the paths to the output files.

Additionally, it seems the process repeats each time, requiring me to reopen the MATSim GUI, provide the config file again, and update the output plans file from the previous iteration.

Despite these efforts, I still couldn't achieve logical results or reach my target modal share.

One more question: Is it possible to use the MATSim Run class instead of relying on the MATSim GUI for this process?

I would greatly appreciate more detailed instructions on how to properly run the calibration code.

[rakow]

Hello,

there should be no GUI involved when running the calibration. I suspect your jar is not executing the correct scenario class.

Please double-check your generated jar and make sure it runs the scenario class and not the gui.

[Areza622]

Dear @rakow , thank you for your response. Indeed in the code provided in the main page the corresponding part is:

study, obj = create_calibration("calib", ASCCalibrator(modes, initial, target, fixed_mode=fixed_mode,
lr=utils.linear_scheduler(start=0.25, interval=10)),
"./matsim-scenario-1.0-SNAPSHOT.jar",
"./scenarios/input/scenario-v1.0-10pct.config.xml",

I just adrresed the default "matsim-example-project-0.0.1-SNAPSHOT.jar" from my matsim-example-project. How can I check if it executes the RunMatsim.java ?

[rakow]

First, you need to have a run class like this:

https://github.com/matsim-org/matsim-example-project/blob/master/src/main/java/org/matsim/project/RunMatsimApplication.java

In the pom.xml you can specify which class is executed:

https://github.com/matsim-org/matsim-example-project/blob/1f18869c2be8e070b046ea48ed7d1a14f7730445/pom.xml#L194C1-L194C60

You need to replace this reference with your scenario class and rebuild the jar. You may check if it is correct by just running
java -jar matsim-example-project-0.0.1-SNAPSHOT.jar run on the command line.

[Areza622]

Dear @rakow , thank you so much for your help. I try rebuilding the jar file and see how things will work!

[Areza622]

Dear @rakow , I rebuilt the JAR file and tested if it runs correctly. After executing the Python code (attached as "Python_Code.txt"), I encountered the following issues:

1. Subpopulation Error: Initially, I received an error related to the "Subpopulation." In my case, different subpopulations were not required, but I added a single subpopulation called "default," which resolved the issue. However, when I used a name other than "default," it did not work and simulation crashed after reading vehicle file.

2. KeyError 'n' : After resolving the subpopulation issue, I ran the code, and it generated output in the "Runs" folder of the matsim-python-tools directory. However, the code then crashed with the following error:

File "C:\Users\Al\anaconda3\Lib\site-packages\pandas\core\indexes\base.py", line 3812, in get_loc raise KeyError(key) from err KeyError: 'n' [W 2024-12-13 18:21:15,407] Trial 191 failed with value None.

To fix this, I added a default value for 'n' as follows:

if "n" not in aggr.columns: aggr["n"] = 0

3. This resolved the KeyError above, but it led to a new error:

File "C:\Users\Al\matsim-python-tools\main_page_code.py", line 33, in study.optimize(obj, 10) KeyError: 'walk_share'

Unfortunately, I have not been able to resolve this issue. Could you please help with this?

Additionally, is there an existing scenario in the GitHub repository that fully works with this calibration code? It would be helpful for me to check the input files and their formatting in order to compare them with my own scenario.

I've attached the Python code and error logs in text files for your reference.

Thank you in advance for your help!
First_Error.txt
Phyton_Code.txt
Second_Error.txt

[rakow]

Please try with a subpopulation called "person". This one is also mentioned in your code.

[Areza622]

Dear @rakow ,
I followed your suggestion and set all plans as the "person" subpopulation. However, MATSim does not run with only "person" as the subpopulation; I had to define a "default" subpopulation as well.

After verifying that the code runs properly in the command line (cmd), I attempted to execute the Python code, but unfortunately, I encountered the error that I have attached to this message.

I am unsure whether the issue relates to the following part of the error, since input files should be sourced from the specified config file and not this folder:
--config:plans.inputPlansFile=C:\Users\Al\matsim-python-tools\runs\107\107.output_plans.xml.gz

And probably in the next iterations input files are taken from the address below(which does not exist):
--yaml params\run233.yaml --output runs/233 --runId 233

I also tried the suggestion to "Alternatively, run the cmd from the log above manually and check for errors." but the code ran properly without any errors.

Third_Error.txt

[rakow]

Normally, only having a "person" subpopulation should be no problem. I would advise cleaning all existing failed simulations runs and stored information. Delete the calib.db file, runs and params directory to start the calibration from scratch.
You may also set debug=True to see the output of the simulation in case it is still failing after starting anew.