RDI work

This page is for changes in handling RDI format, occasioned by issue 777

Here, 'doc' means OS_TM_Apr14.pdf from Teledyne/RDI

Task 1. find the data chunks

• Fixed Leader Data format the documented length changes across firmware versions. What a mess. The 2014 doc says this is 50 bytes long (firmware 23.17; doc figure 46, page 145) but Gonzalez' file (firmware 23.19) has 60 bytes. I think the only answer is to ignore the length given in the tables and work with the data-offset information ... if I can figure that out.

Browse[1]> header$dataOffset
 [1]   28   88  148  790 1112 1434 1756 2078 2159 2193 2243
Browse[2]> LEN<-diff(c(0,header$dataOffset))
Browse[2]> LEN
 [1]  28  60  60 642 322 322 322 322  81  34  50
Browse[2]> LEN[1]==6+2*header$numberOfDataTypes # header OK
[1] TRUE
Browse[2]> LEN[2]==50 # FLD wrong length
[1] FALSE
Browse[2]> LEN[3]==60 # FLD right length
[1] TRUE
Browse[2]> LEN[4]==2+8*header$numberOfCells ## velo is ok
[1] TRUE
Browse[2]> LEN[5]==2+4*header$numberOfCells ## corr is ok
[1] TRUE
Browse[2]> LEN[6]==2+4*header$numberOfCells ## intensity is ok
[1] TRUE
Browse[2]> LEN[7]==2+4*header$numberOfCells ## percent-good is ok
[1] TRUE
Browse[2]> LEN[8]==2+4*header$numberOfCells ## status is ok
[1] TRUE
Browse[2]> LEN[9]==81 ## bottom track is ok
[1] TRUE

Then we have something 34 bytes long (meaning?) and something 50 bytes long (meaning?)

Hm. Let's look at codes, since this is how I now plan to scan the file; I'm writing my interpretation to the right of the codes (based on table 33, page 146 of doc)

Browse[12]> cbind(buf[1+c(0,header$dataOffset)],buf[2+c(0,header$dataOffset)])
      [,1] [,2]
 [1,]   7f   7f Header
 [2,]   00   00 Fixed leader
 [3,]   80   00 variable leader
 [4,]   00   01 velocity profile data
 [5,]   00   02 correlation profile data
 [6,]   00   03 echo intensity profile data
 [7,]   00   04 percent good profile data
 [8,]   00   05 status profile data
 [9,]   00   06 bottom track data
[10,]   00   30 binary fixed attitude
[11,]   d8   30 binary variable attitude (since d8 is between 0x40 and 0xF0)
[12,]   00   20 navigation

so that looks like this is going to work ... the plan therefore is to work through the file, using the data offsets and IGNORING the old method I had, which skipped based on what I had previously thought were known lengths of data chunks.

Hm... we have ensembleStart as a vector of pointers to the 7F7F flags. So that's what we need already ... I had thought I would need to recode something.