-
Notifications
You must be signed in to change notification settings - Fork 12
GEM5Integration
Step 1: Get a patched GEM5
You can get a modified clone of gem5-devel repository at:
shell$ hg clone https://code.google.com/p/tpzsimul.gem5/ gem5
If you are want to have a updated gem5 (to the latest stable version), to incorporate our interface you should do:
shell$ hg clone https://code.google.com/p/tpzsimul.gem5/ gem5
shell$ cd gem5
shell$ hg pull http://repo.gem5.org/gem5
shell$ hg merge
shell$ hg ci -m "topaz interface added to gem5"
The modifications of Topaz patch are fairly small and localized in 4 files of ruby. Therefore in most cases, there will be a clean merge. Nevertheless, sometimes Gem5 guys changes key parts, such as statistics, or similar. In such occasions, to compile successfully the tool, it will be needed to "modify" the gem5-topaz interface.
Even if you want to use gem5-stable repository, you can use Topaz. The procedure, although "unelegant", it might works (with some effort in the patch):
shell$ hg clone http://repo.gem5.org/gem5-stable/
shell$ hg clone https://code.google.com/p/tpzsimul.gem5/
shell$ cd tpzsimul.gem5/
shell$ hg diff 8811:tip > ../patch #We asume latest gem5 devel change-set is 8811
shell$ cd ../gem-stable/
shell$ cat ../patch | patch -p1 #(at least) up to changeset 8642 the patch applies clearly
shell$ hg ci -m "Topaz interface added to gem5-stable"
In order to get TOPAZ, you have to download it directly to the directory gem5/ext using:
cd gem5/ext
hg clone https://code.google.com/p/tpzsimul/ TOPAZ/
cd ..
NOTE: GEM5 Network tester is broken since this changeset 10089. Network-only mode Doesn't work anymore (last time checked was with dec 2014 stable version).
To perform this compilation you should have a file at the 'build_opt' directory where the env variable USE_TOPAZ is set to True. For example, a simple network tester can be compiled using the 'ALPHA_Network_Topaz_test' configuration file already created which contains:
SS_COMPATIBLE_FP = 1
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU,InOrderCPU'
PROTOCOL = 'Network_test'
USE_TOPAZ = True
To compile ´ALPHA_Network_Topaz_test´ (you need a g++ > 4.3 and the remaining dependencies listed here):
scons build/ALPHA_Network_Topaz_test/gem5.opt -j 8
After a bit, you get the build and to test topaz, you'll have to run a command like the following:
./build/ALPHA_Network_Topaz_test/gem5.opt ./configs/example/ruby_network_test.py --num-cpus=16 \
--num-dirs=16--topology=Mesh --mesh-rows=4 --sim-cycles=1000 --injectionrate=0.1 --synthetic=0 \
--fixed-pkts --maxpackets=10000 --topaz-network=T44-CT-NOC
This may report an error indicating that the SGML component does not exist. This occurs because Topaz requires an ini file indicating which sgml files to use. You need to make use of GEM5 Parameters to specify, among other parameters, where the initialization file of TOPAZ is. All the parameters defined at gem5/config/Ruby.py are listed in the following table:
| Option | type | help | Default value |
|---|---|---|---|
| --topaz-init-file | string | File that declares simulation.sgm, network.sgm and router.sgm" | "./TPZSimul.ini" |
| --topaz-network | string | Network listed in simulation.sgm to be used by TOPAZ | unset |
| --topaz-flit-size | int | Number of bytes per physical router-to-router wire | 16 |
| --topaz-clock-ratio | int | Memory-network clock multiplier | 1 |
As seen in the table, the option by default for network inicialization is ./TPZSimul.ini, so we have two options: creating a file in the directory where GEM5 is invoked and repeat the previous command or creating it somewhere else indicating where it is. For the first option you should create a file called TPZSimul.ini in the directory where gem5 is invoked with the following content:
<!-- Paths can be absolute or relative. In this case, we are assuming -->
<!-- that TPZSimul.ini is located in gem5 root -->
<RouterFile id="./ext/TOPAZ/sgm/RouterGem5.sgm" >
<NetworkFile id="./ext/TOPAZ/sgm/NetworkGem5.sgm" >
<SimulationFile id="./ext/TOPAZ/sgm/SimulaGem5.sgm" >
The downloaded Topaz already provides a file with this content. For the second option you need to use one of the command line options of gem5 to make Topaz point to the appropriate directory. This command line option is 'topaz-init-file'. Let's suppose you create this file at top gem5 directory, then you will have to indicate this directory to the topaz-init-file parameter when invoking gem5. If you do so from its parent directory, this would be the command (notice the new paremeter at the end of the line):
./build/ALPHA_Network_Topaz_test/gem5.opt ./configs/example/ruby_network_test.py --num-cpus=16 \
--num-dirs=16 --topology=Mesh --mesh-rows=4 --sim-cycles=1000 --injectionrate=0.1 --synthetic=0 \
--fixed-pkts --maxpackets=10000 --topaz-network=T44-CT-MC --topaz-init-file="./TPZSimul.ini"
Either of both options is correct and it is your choice which one to use.
The 4 network configurations availabe in ./ext/TOPAZ/sgm/SimulaGem5.sgm are:
- T44-CT-MC Torus 4x4 using VCT, Bubble deadlock avoidance, DOR routing and with multicast support
- M44-CT-MC Mesh 4x4 using VCT, DOR routing and with multicast support
- T44-CT-UC Torus 4x4 using VCT, Bubble deadlock avoidance, DOR routing and without multicast support
- M44-CT-UC Mesh 4x4 using VCT, DOR routing and without multicast support
For sake of simplicity we restrict to 4 configurations, but all of the depicted in ./ext/TOPAZ/sgm/Simula.sgm can be used. Any of these can be passed through --topaz-network variable.
Let's check a different network:
./build/ALPHA_Network_Topaz_test/gem5.opt ./configs/example/ruby_network_test.py --num-cpus=16 \
--num-dirs=16 --topology=Mesh --mesh-rows=4 --sim-cycles=1000 --injectionrate=0.1 --synthetic=0 \
--fixed-pkts --maxpackets=10000 --topaz-network=M44-CT-MC --topaz-init-file="./TPZSimul.ini"
After the simulation, TOPAZ results are printed at the end of gem5/ruby.stats file.
If at some point you want to center your attention in network simulation, TOPAZ is able to work standalone (with higher level of flexibility). Just follow this guide . We do not recommend to use GEM5 wrap-up to work with synthetic traffic or traces.
We can use any of the Ruby config files. For example, if we want to use MOESI_CMP_token protocol (multicast and inordered traffic) we should create a file such as build_opt/ALPHA_token_topaz:
SS_COMPATIBLE_FP = 1
CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU,InOrderCPU'
PROTOCOL = 'MOESI_CMP_token'
USE_TOPAZ = True
We compile then with:
scons build/ALPHA_token_topaz/gem5.opt -j 8
And eventually, we can check it with:
./build/ALPHA_token_topaz/gem5.opt ./configs/example/ruby_mem_test.py --maxloads 1000 --num-cpus=16 \
--num-dirs=16 --topology=Mesh --num-l2caches=16 --mesh-rows=4 --topaz-network=T44-CT-UC \
--topaz-flit-size=16 --topaz-adaptive-interface-threshold=0 > m5out/topaz.out
If everything was is correct, you should see something like:
<TOPAZ> +++++++ Topaz Enabled! +++++++ </TOPAZ>
info: Entering event queue @ 0. Starting simulation...
system.cpu14: completed 1000 read, 540 write accesses @327673
hack: be nice to actually delete the event here
Exiting @ tick 327673 because maximum number of loads reached
And you can see Topaz out at m5out/topaz.out (sorry but not unified stats with m5 ATT)
.....
.....
.....
block_inputs: 0
block_outputs: 512
<TOPAZ>
Ratio Processor Clock/Network Clock = 1
Flit size in bytes = 16 bytes
******************* NET CONFIGURATION *********************
Network : Torus(4,4,1)
Buffer control : CT
Routing control: TORUS-DOR
Started at : Wed Apr 17 20:37:28 2013
Ended at : Wed Apr 17 20:37:28 2013
***********************************************************
Simulation time = 00:00:11 (11 secs)
Memory Footprint = 370.871 MBytes)
Traffic Pattern = Empty
Seed = 113
Cycles simulated = 123376
Cycles deprecated = 0
Buffers size = 31
Messages length = 1 packet(s)
Packets length = 5 flits
************************ PERFORMANCE ***********************
Supply Thr. Norm = 0.146074 flits/cycle/router
Accept Thr. Norm = 0.353563 f/c/r (m: 0.334984, M:0.365728)
Supply Thr. = 2.33718 f/c
Throughput = 5.65702 f/c (m: 5.35975, M:5.85164)
Average Distance = 2.13373
Messages generated = 99064
Messages received = 508736
Messages to inject = 94
...
...
...
TIME=123316
TOPAZ NETWORK USAGE
Usage TOPAZ network: 99%
Total TOPAZ messages: 518680
Total Number of messages: 518680
</TOPAZ>
SimpleNetwork Stats: The traffic you see here should be null
if your are not using the adaptive-interfaz)
It is possible to disable TOPAZ on-the-fly if the network is lightly loaded. In such situations, the network contention will be almost zero, being therefore the effect of using the original simple network model in network latency almost negligible. When the applications changes from a compute-intensive phase to a communication-intensice phase, the load will be higher and TOPAZ will be reenabled on-the-fly. This feature is very important, specially when highly detailed models are used for the routers (otherwise, the slowdown will be substantial). (more info):
./build/ALPHA_token_topaz/gem5.opt ./configs/example/ruby_mem_test.py -l 1000 --num-cpus=16 \
--num-dirs=16 --topology=Mesh --num-l2caches=16 --mesh-rows=4 --topaz-network=T44-CT-UC \
--topaz-flit-size=16 --topaz-adaptive-interface-threshold=100
Although topology= is ignored internally by TOPAZ, it is required to perform the map of ruby components to TOPAZ routers. Since we are using a 4x4 torus, a 4x4 Mesh will be enough to have correct mapping. Because GEM5 does not support the flexibility of GEMS, in terms of topology construction, at this point there is not possible to use File_specified topologies. In the future it could be interesting to add such feature, since in SoC context, could be very useful.
Good luck!