forked from CosmoStat/MRS3D
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME
executable file
·235 lines (158 loc) · 8.46 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
=====================================================================================
MRS3D - MultiResolution in Spherical 3D
Library and tools
v1.0b 09/2013
=====================================================================================
MRS3D is a C++ package providing a library and a set of tools allowing the manipulation
of Spherical Fourier-Bessel Coefficients and the implementation of MultiResolution
algorithms adapted to 3D data in spherical coordinates.
In particular, MRS3D implements a Spherical 3D Isotropic Wavelet Decomposition.
As a part of MRS3D, this package also contains the FastDSBT C++ library wich implements
a Discrete Spherical Bessel Transform based on a matrix formalism.
MRS3D is distributed under the CeCILL License (see LICENSE.CeCILL) in the hope that it
will be usefull but without any warranty.
The following softwares are used in MRS3D:
- HEALPix (http://healpix.jpl.nasa.gov/) distributed under the GNU GPL
- 3DEX (https://github.com/ixkael/3DEX) developed by Boris Leistedt and released
under the CeCiLL License. See the ArXiV paper http://arxiv.org/abs/1111.3591.
Author: François Lanusse ([email protected])
Copyright CEA 2013
===================
Requirements
===================
In order to be able to compile the MRS3D package, the following softwares are required:
- cmake (http://cmake.org/cmake/resources/software.html)
- HEALPix (http://sourceforge.net/projects/healpix/files/)
- cfitsio (http://heasarc.gsfc.nasa.gov/fitsio/)
- OpenMPI or other MPI implementation
- gfortran and gcc
The following softwares are optional:
- Doxygen
- IDL
===================
Building
===================
Before building the MRS3D package, it is necessary to build and install the HEALPix
package (Fortran AND CXX versions).
First, unpack the tar.gz archive:
$ tar -xvzf MRS3D.tar.gz
$ cd MRS3D
Set the CFITSIO_DIR environment variable to the directory containing libcfitsio.a:
$ export CFITSIO_DIR=/usr/lib
If IDL is installed on your system, you can build the MRS3D IDL interface by setting
the environment variable IDL_DIR to the IDL root directory:
$ export IDL_DIR=/Applications/itt/idl
By default, the binary tools, headers and libraries will be installed in the MRS3D
directory. To install them elsewhere on your sytem (e.g. /usr/local), you can set
the INSTALL_DIR environment variable:
$ export INSTALL_DIR=/usr/local [ Optional ]
Compile the package:
$ mkdir build
$ cd build
$ cmake ..
$ make all
$ make install
To build the documentation:
$ make doc
The html documentation will be available at doc/html/index.html.
If LaTeX is installed on your system, you can also build the pdf documentation by
going to the doc/latex subdirectory and typing make.
===================
Running the code
===================
The transformations implemented in MRS3D require the values of zeros of Bessel functions.
Instead of behing calculated each time, these values are tabulated inside a binary FITS
file. This file, referred to as qlnTable, must be provided as first argument of all MRS3D
commands.
A precomputed qlnTable is provided with MRS3D and contains the first 3000 zeros of Bessel
functions up to the order 2000. This file should be sufficient for most applications.
To compute a larger qlnTable, the gen_qln tool is provided. The following command will
generate a table containing the first 2000 zeros of Bessel functions up to order 4000.
$ gen_qln qlnTable.fits 4000 2000
MRS3D provides two main binary tools almn2wavelet and waveletThresholding:
> almn2wavelet : Computes an Isotropic Undecimated Spherical 3D Wavelet decomposition
from an almn .fits file.
> waveletThresholding : Applies wavelet hard thresholding to the input almn .fits file.
These tools work on Spherical Fourier-Bessel coefficients stored in "almn" FITS files.
In order to compute almn coefficients to test these functions, two additional tools are
provided field2almn and almn2field:
> field2almn : Computes the Discrete Spherical Fourier-Bessel Transform from a 3D
cartesian field .fits file.
> almn2field : Reconstructs a cartesian 3D field from an almn .fits file.
As an example, to decompose a field into wavelet scales from the MRS3D root directory run:
$ bin/field2almn test/qlnTable.fits test/VirgoField256.fits test/VirgoAlmn.fits 512 250.0 256 256 256
Will store in "VirgoAlmn.fits" coefficients computed from the field "VirgoField256.fits"
using a HEALPix resolution of nside=512, a boundary condition rmax=250.0 Mpc/h, up to
the orders lmax=256, mmax=256 and nmax=256.
$ bin/almn2wavelet test/qlnTable.fits test/VirgoAlmn.fits wavelet 512 4 3.5
This command will produce 5 FITS files (wavelet_0.fits wavelet_1.fits...), one for each
wavelet scale, plus one for the smoothed density. The cutoff frequency of the low pass
filter is set to 3.5
$ bin/almn2field test/qlnTable.fits test/wavelet_0.fits test/waveletField_0.fits 256 479.0 512
Reconstructs the density field of the first wavelet scale inside a cube of size
256x256x256 and physical size 479.0 Mpc/h using a Healpix resolution of nside=512.
Now, to visualize the density fields, the IDL interface can be used, see the next section.
===================
IDL Interface
===================
To work on spherical Fourier-Bessel coefficients and 3D fields from IDL, MRS3D comes with
an IDL interface. The IDL procedures are in src/IDL. In order to work these procedures
load libMRS3DIDL and by default expect the library to be in the same directory.
To setup the IDL procedures, go to src/IDL and create a symlink to the libMRS3DIDL library:
$ ln -s ../../lib/libmrs3d_IDL.dylib ./libmrs3d_IDL.dylib
To setup the FastDSBT IDL interface use as well:
$ ln -s ../../lib/libfastDSBT_IDL.dylib ./libfastDSBT_IDL.dylib
(under linux, the libraries in /lib will be .so and not .dylib but keep the name in .dylib
for the symlinks)
In the /src/IDL directory, launch IDL.
Available features:
- Compute power spectrum from almn
- Save/Load power spectrum to/from FITS files
- Save/Load almn to/from FITS files
- Save/Load cartesian density fiels to/from FITS files
- Convert almn to field and field to almn (The use of the binary tools is prefered)
Example of how to visualize the wavelet scale computed at the previous section:
IDL> fieldWavelet = mrs3d_load_field('/path-to-MRS3D/test/waveletField_0.fits')
IDL> ivolume, fieldWavelet
To get help on the mrs3d_* commands, set the help keyword (e.g. mrs3d_save_field,/help)
===================
Parallel execution
===================
MRS3D is designed to take full advantage of OpenMP and MPI.
All the tools provided with MRS3D are parallelized using OpenMP and will by default run using a thread by processor core.
To manually set the number of threads used by the command set the environment variable OMP_NUM_THREADS:
export OMP_NUM_THREADS=4 (bash)
or setenv OMP_NUM_THREADS 4 (tcsh)
To launch a command using several MPI processes:
mpirun -np 4 [command followed by arguments]
!!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!!
When using MPI, be sure to use powers of 2 for the number
of l,m and n coefficients.
Since l goes from 0 to lmax, you must set l=2^i -1 !!!!
The same goes for m.
Since n varies from 1 to nmax you can directly set nmax= 2^l
A valid set of (lmax,mmax,nmax) is : 1023 1023 512
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An example of a PBS script to run a command on a computer cluster is provided below:
-----------------------------------waveletThresholding.job----------------------------------
#!/bin/sh
#PBS -l pmem=15G
#PBS -l pvmem=16G
#PBS -l mem=120G
#PBS -l vmem=128G
#PBS -l nodes=8:ppn=1
#PBS -W x=NACCESSPOLICY:SINGLEJOB
#PBS -l walltime=100:00:00
#PBS -M [your email address]
#PBS -m abe
export OMP_NUM_THREADS=8
cd [path to working directory]
cat $PBS_NODEFILE | sort | uniq > machinefile.txt
echo "----Begining job----" > log
date >> log
echo "--------------------" >> log
mpiexec -machinefile machinefile.txt -np 8 ./waveletThresholding qlnTable_2000_3000.fits Almn.fits ThresholdedAlmn.fits 2048 250.0 5 5 9.0 0.75 >> log
echo "----Job done--------" >> log
date >> log
----------------------------------------------------------------------------------------------
To launch the job : qsub waveletThresholding.job