Incorrect results when reading from InputStream via ChannelSftp

[averemee-si]

Description

Incorrect results when reading binary binary file by chunks via ChannelSftp: after reading 63 chunks of 512 bytes, chunk #64 contains only 499 (512 -13) bytes instead of 512 requested, then again 63 "good" chunks with 512 bytes and chunk #128 again with only 499 bytes.
ChannelSftp.java contains various references with buffer adjustments of 13 bytes:

• private void _get(String src, OutputStream dst, SftpProgressMonitor monitor, int mode, long skip)

int request_len = buf.buffer.length - 13;

• public int read(byte[] d, int s, int len)

if (buf.buffer.length - 13 < len) {
            len = buf.buffer.length - 13;
        }

• etc

Testcase

Use any binary file greater than 32768 bytes and run

import java.io.IOException;
import java.io.InputStream;

import org.apache.log4j.BasicConfigurator;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;
import com.jcraft.jsch.Slf4jLogger;

public class JschTest {

	public static void main(String[] argv) {
		final int bs = 0x200;
		final byte[] ba = new byte[bs];
		final String fileName = "<FULL_PATH_TO_BINARY_FILE>";
		BasicConfigurator.configure();
		JSch.setLogger(new Slf4jLogger());

		try {
			InputStream is = null;
			int bytesRead;
			int iteration;
	
			JSch jsch = new JSch();
			Session session = jsch.getSession("nnnnnnnn", "test.local", 22);
			session.setConfig("StrictHostKeyChecking", "no");
			session.setDaemonThread(true);
			session.setPassword("XXXXXXXXXX");
			session.connect();

			ChannelSftp channelSftp = (ChannelSftp) session.openChannel("sftp");
			channelSftp.connect();

			is = channelSftp.get(fileName, null, 0L);
			bytesRead = bs;
			iteration = 0;
			do {
				iteration++;
				bytesRead = is.read(ba, 0, bs);
				if (bytesRead != -1 && bytesRead != bs) {
					System.out.println("sftp-> iteration=" + iteration + "\tbytesRead=" + bytesRead);
					break;
				}
			} while (bytesRead != -1);
			is.close();
			is = null;

		} catch (JSchException | SftpException | IOException  e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

	}
}

JSch log output

0 [main] INFO com.jcraft.jsch.JSch  - Connecting to XXXXXXXXXXXXX port 22
336 [main] INFO com.jcraft.jsch.JSch  - Connection established
505 [main] INFO com.jcraft.jsch.JSch  - Remote version string: SSH-2.0-OpenSSH_7.4
506 [main] INFO com.jcraft.jsch.JSch  - Local version string: SSH-2.0-JSCH_0.2.23
506 [main] INFO com.jcraft.jsch.JSch  - CheckCiphers: [email protected]
506 [main] INFO com.jcraft.jsch.JSch  - CheckKexes: mlkem768x25519-sha256,mlkem768nistp256-sha256,mlkem1024nistp384-sha384,sntrup761x25519-sha512,[email protected],curve25519-sha256,[email protected],curve448-sha512
576 [main] INFO com.jcraft.jsch.JSch  - mlkem768x25519-sha256 is not available.
577 [main] INFO com.jcraft.jsch.JSch  - mlkem768nistp256-sha256 is not available.
577 [main] INFO com.jcraft.jsch.JSch  - mlkem1024nistp384-sha384 is not available.
577 [main] INFO com.jcraft.jsch.JSch  - sntrup761x25519-sha512 is not available.
577 [main] INFO com.jcraft.jsch.JSch  - [email protected] is not available.
577 [main] DEBUG com.jcraft.jsch.JSch  - kex proposal before removing unavailable algos is: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
577 [main] DEBUG com.jcraft.jsch.JSch  - kex proposal after removing unavailable algos is: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
577 [main] INFO com.jcraft.jsch.JSch  - CheckSignatures: ssh-ed25519,ssh-ed448
578 [main] INFO com.jcraft.jsch.JSch  - ssh-ed25519 is not available.
578 [main] INFO com.jcraft.jsch.JSch  - ssh-ed448 is not available.
578 [main] DEBUG com.jcraft.jsch.JSch  - server_host_key proposal before removing unavailable algos is: ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
578 [main] DEBUG com.jcraft.jsch.JSch  - server_host_key proposal after removing unavailable algos is: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
578 [main] DEBUG com.jcraft.jsch.JSch  - server_host_key proposal before known_host reordering is: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
580 [main] DEBUG com.jcraft.jsch.JSch  - server_host_key proposal after known_host reordering is: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
580 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_KEXINIT sent
662 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_KEXINIT received
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: ciphers c2s: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: ciphers s2c: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: MACs c2s: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: MACs s2c: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: compression c2s: none,[email protected]
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: compression s2c: none,[email protected]
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: languages c2s: 
663 [main] INFO com.jcraft.jsch.JSch  - server proposal: languages s2c: 
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected]
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: host key algorithms: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: ciphers c2s: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: ciphers s2c: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: MACs c2s: [email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: MACs s2c: [email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: compression c2s: none
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: compression s2c: none
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: languages c2s: 
663 [main] INFO com.jcraft.jsch.JSch  - client proposal: languages s2c: 
665 [main] INFO com.jcraft.jsch.JSch  - kex: algorithm: curve25519-sha256
665 [main] INFO com.jcraft.jsch.JSch  - kex: host key algorithm: ecdsa-sha2-nistp256
665 [main] INFO com.jcraft.jsch.JSch  - kex: server->client cipher: aes128-ctr MAC: [email protected] compression: none
665 [main] INFO com.jcraft.jsch.JSch  - kex: client->server cipher: aes128-ctr MAC: [email protected] compression: none
666 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_KEX_ECDH_INIT sent
666 [main] INFO com.jcraft.jsch.JSch  - expecting SSH_MSG_KEX_ECDH_REPLY
837 [main] INFO com.jcraft.jsch.JSch  - ssh_ecdsa_verify: ecdsa-sha2-nistp256 signature true
838 [main] WARN com.jcraft.jsch.JSch  - Permanently added 'XXXXXXXXXXXXX' (ECDSA) to the list of known hosts.
838 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_NEWKEYS sent
838 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_NEWKEYS received
857 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_SERVICE_REQUEST sent
858 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_EXT_INFO received
858 [main] INFO com.jcraft.jsch.JSch  - server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
858 [main] INFO com.jcraft.jsch.JSch  - OpenSSH 7.4 detected: adding rsa-sha2-256 & rsa-sha2-512 to server-sig-algs
1018 [main] INFO com.jcraft.jsch.JSch  - SSH_MSG_SERVICE_ACCEPT received
1197 [main] INFO com.jcraft.jsch.JSch  - Authentications that can continue: gssapi-with-mic,publickey,keyboard-interactive,password
1197 [main] INFO com.jcraft.jsch.JSch  - Next authentication method: gssapi-with-mic
1411 [main] INFO com.jcraft.jsch.JSch  - Authentications that can continue: publickey,keyboard-interactive,password
1411 [main] INFO com.jcraft.jsch.JSch  - Next authentication method: publickey
1415 [main] INFO com.jcraft.jsch.JSch  - Authentications that can continue: password
1415 [main] INFO com.jcraft.jsch.JSch  - Next authentication method: password
1597 [main] INFO com.jcraft.jsch.JSch  - Authentication succeeded (password).