fw.txt not created correctly

[dsiganos]

I am seeing warning of the sort:

../../../buildroot/board/laird/mkfwtxt.sh http://e6430/wb/wb45n/20151229
../../../buildroot/board/laird/mkfwtxt.sh: line 89: warning: here-document at line 53 delimited by end-of-file (wanted `-transfer-list--')

And my fw.txt files are not correct. This is what my fw.txt file looks like:

# e6430-?
# b9a6744e449baec282e1e3f880f91097  at91bs.bin  9280
# 304fa37de589b743844a5e8d7ce04d7b  u-boot.bin  358960
  6cf0d6ff16de114bcbce88f8cc20af09  kernel.bin  2091674
  59cac36796438753318a32846a62df31  rootfs.bin  30670848

  flags -c

# transfer-list
  /etc/summit/profiles.conf
  /etc/network/interfaces
  /etc/ssl/certs
  /root/.ssh

--transfer-list--

# display file
echo fw.txt:
cat fw.txt


#####################################
# duplicate update-list in old-format
# for backwards compatibility
echo >>fw.txt
echo >>fw.txt
echo "# old format..." >>fw.txt
for n in 1 2 3 4
do
  # construct image var
  eval name=$image4     && image=rootfs.bin

  # skip non-existant files
  [ -f rootfs.bin ] || continue

  # set line prefix as hash or space
  [ rootfs.bin != rootfs.bin ] && x='#' || x=' '

  # write image line: <{url/}name>  
  md5sum rootfs.bin |sed "s,^\(.*[^ ]\)[ ]\+\(.*\),  http://e6430/wb/wb45n/20151229/\2  \1," >>fw.txt
done
echo >>fw.txt

I believe the problem is in this piece of code. The use of the backslash is maybe causing a problem? I don't know and I have not investigated but the way the here-doc is setup certainly looks much more complicated than it needs to be.

cat >>$fwul<<\
--transfer-list--
# transfer-list
  /etc/summit/profiles.conf
  /etc/network/interfaces
  /etc/ssl/certs
  /root/.ssh
--transfer-list--

The code comes from the file:

buildroot/board/laird/mkfwtxt.sh

[derosier]

I wonder if it has something to do with the differences between different distro's /bin/sh. Our internal systems use dash (Ubuntu 14.04 LTS) and don't have trouble.

In any case, I see no need for the escaping of the newline. Could you edit mkfwtxt.sh to remove the escape+nl? So:

cat >>$fwul<<--transfer-list--

Give that a try in your environment. If it works, we'll likely fix it.

[derosier]

I did some checking on this. Turns out that with dash (what we use on Ubuntu), and ash (busybox), the heredoc delimiter is considered OK, but with bash it does indeed fail.

The reason is simple: <<- is considered special and with the line-continuation, bash interprets the line to be essentially <<-"-transfer-list--" (quotes for illustration, obviously they're not included). dash interprets thus: <<"--transfer-list--".

The proper fix is changing the lines thus:

cat >>$fwul<<TRANSFER-LIST
# transfer-list
  /etc/summit/profiles.conf
  /etc/network/interfaces
  /etc/ssl/certs
  /root/.ssh
TRANSFER-LIST

This change to works in dash, ash and bash.

I've made this change in our local repositories. Until I'm able to push them out publically, the workaround is:

1. Use dash instead of bash for /bin/sh
   OR
2. Make the change as specified above.

[dsiganos]

Hi Steve, I am not using fw.txt so I didn't investigate. I just thought I'd report it so you are aware. Your analysis makes sense. I am using Fedora and bash.

Removing the escape sequence and the double dashes fixes the problem. I.e. using
cat >>$fwul<<TRANSFER-LIST

This is what my fw.txt looks like after the fix, the format seems to have changed so I am not sure it is right. Copying here for you to check for yourself.

# e6430-?
# b9a6744e449baec282e1e3f880f91097  at91bs.bin  9280
# 304fa37de589b743844a5e8d7ce04d7b  u-boot.bin  358960
  6cf0d6ff16de114bcbce88f8cc20af09  kernel.bin  2091674
  cf16493a890e03d957233549449020ef  rootfs.bin  30670848
  flags -c
# transfer-list
  /etc/summit/profiles.conf
  /etc/network/interfaces
  /etc/ssl/certs
  /root/.ssh
# old format...
# http://e6430/wb/wb45n/20151230/at91bs.bin  b9a6744e449baec282e1e3f880f91097
# http://e6430/wb/wb45n/20151230/u-boot.bin  304fa37de589b743844a5e8d7ce04d7b
  http://e6430/wb/wb45n/20151230/kernel.bin  6cf0d6ff16de114bcbce88f8cc20af09
  http://e6430/wb/wb45n/20151230/rootfs.bin  cf16493a890e03d957233549449020ef

[derosier]

Yes, that is the correct format.

The first chunk is the stuff now used by fw_update to fetch the files. Note that the bootloaders are commented out by default (as updating the bootloaders is something you'd not likely want to do). "flags -c" turns on filesystem checking. The transfer-list section are the files to copy from the old rootfs to the new rootfs (ie your profile settings and such). And finally the "old format..." section is for compatibility with really old fw_updates (which incidentally we don't support using due to a bug that results in corrupted rootfs, so maybe we should just clean that out).