Skip to content

Commit

Permalink
Round database unit to the nearest attometer when reading from GDS2
Browse files Browse the repository at this point in the history
Currently if you create a layout with a DBU of (e.g.) 16, write it to GDS2,
and then read it again, you don't get exactly 16, but 16.000000000000004.
It would be nice if reasonable database unit values roundtrip exactly.
  • Loading branch information
rocallahan committed Oct 2, 2024
1 parent 7953cad commit 182553e
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions src/plugins/streamers/gds2/db_plugin/dbGDS2ReaderBase.cc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@


#include "dbGDS2ReaderBase.h"
#include <cmath>
#include "dbGDS2Format.h"
#include "dbGDS2.h"
#include "dbArray.h"
Expand Down Expand Up @@ -240,6 +241,11 @@ GDS2ReaderBase::do_read (db::Layout &layout)

m_dbuu = dbuu;
m_dbu = dbum * 1e6; /*in micron*/
// round to (approximately) the nearest attometer. This enables
// preserving DBU values precisely during round trips through GDS2.
// Use a power of two so that the division doesn't introduce errors.
const double factor = 1024.0 * 1024.0 * 1024.0 * 1024.0;
m_dbu = round (m_dbu * factor) / factor;
layout.dbu (m_dbu);

} else {
Expand Down

0 comments on commit 182553e

Please sign in to comment.