Copy-files-between-accounts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to copy files between accounts \n",
    "演示在Linux 下 ， 账号间拷贝文件的方式\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为什么无法直接用cp 复制其他用户账号下的文件呢 ？ 因为没有权限："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drwxr-xr-x 14 nscc-gz_jiangli nscc-gz 4096 May 20 15:35 /HOME/nscc-gz_jiangli/tmp\r\n"
     ]
    }
   ],
   "source": [
    "ls -ld ~/tmp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drwx------ 89 nscc-gz_jiangli nscc-gz_jiangli 12288 May 22 12:51 /HOME/nscc-gz_jiangli\r\n"
     ]
    }
   ],
   "source": [
    "ls -ld ~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drwxr-x--x 1800 root root 106496 May 23 14:05 /HOME\r\n"
     ]
    }
   ],
   "source": [
    "ls -ld /HOME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ls: cannot open directory /HOME: Permission denied\r\n"
     ]
    }
   ],
   "source": [
    "ls /HOME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bash: cd: /HOME/nscc_ts_1: Permission denied\r\n"
     ]
    }
   ],
   "source": [
    "cd /HOME/nscc_ts_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "cd /HOME/intel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drwxr-xr-x 19 root root 4096 Jun  8  2015 .\r\n"
     ]
    }
   ],
   "source": [
    "ls -ld ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从前面的演示中，我们可以看到，文件能否访问是受权限控制的。需要你有此文件的 r 权限（可读） ， 以及其所有父文件夹的可执行（访问）权限（x权限）。\n",
    "\n",
    "如果需要在不同的账号下共享文件，可以考虑以下几种方式 ："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. scp 命令  \n",
    "用 scp 命令 和  rsync 命令 在账号间进行账号拷贝是最好的方案之一。不过需要您有两个账号的key 文件。\n",
    "这两个命令都是通过 ssh 通道来进行拷贝 ， 命令用法也和 ssh 很像 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]\r\n",
      "           [-l limit] [-o ssh_option] [-P port] [-S program]\r\n",
      "           [[user@]host1:]file1 ... [[user@]host2:]file2\r\n"
     ]
    }
   ],
   "source": [
    "scp --help "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nscc-gz_kangyouzhong\r\n"
     ]
    }
   ],
   "source": [
    "ssh -i ~/keys/id_rsa.ky nscc-gz_kangyouzhong@ln3-gn0   whoami"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，通过这个KEY 文件可以成功访问 KY 的账号，那么，scp 命令也没问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "file.lj                                       100%    0     0.0KB/s   00:00    \r\n"
     ]
    }
   ],
   "source": [
    "scp -i ~/keys/id_rsa.ky  ~/file.lj  nscc-gz_kangyouzhong@ln3-gn0:~/."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到文件 ~/file.lj 顺利从我的账号传到了 ky 的账号 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "file.ky                                       100%    0     0.0KB/s   00:00    \r\n"
     ]
    }
   ],
   "source": [
    "scp -i ~/keys/id_rsa.ky    nscc-gz_kangyouzhong@ln3-gn0:~/file.ky  ~/."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样的办法也可以将数据从ky 的账号拷贝到我的账号下。\n",
    "\n",
    "但是， SCP 命令也有些问题：\n",
    "\n",
    "* 不支持断点续传\n",
    "\n",
    "* 对文件夹的传输支持不够好，可能会出现BUG 。\n",
    "\n",
    "那么替代的解决办法是： rsync 命令 。 当然，对于临时的需求，这种方式简单易用，挺不错的。\n",
    "\n",
    "## 2. rsync 命令 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rsync  version 3.0.6  protocol version 30\r\n",
      "Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.\r\n",
      "Web site: http://rsync.samba.org/\r\n",
      "Capabilities:\r\n",
      "    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,\r\n",
      "    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,\r\n",
      "    append, ACLs, xattrs, iconv, symtimes\r\n",
      "\r\n",
      "rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you\r\n",
      "are welcome to redistribute it under certain conditions.  See the GNU\r\n",
      "General Public Licence for details.\r\n",
      "\r\n",
      "rsync is a file transfer program capable of efficient remote update\r\n",
      "via a fast differencing algorithm.\r\n",
      "\r\n",
      "Usage: rsync [OPTION]... SRC [SRC]... DEST\r\n",
      "  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST\r\n",
      "  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST\r\n",
      "  or   rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST\r\n",
      "  or   rsync [OPTION]... [USER@]HOST:SRC [DEST]\r\n",
      "  or   rsync [OPTION]... [USER@]HOST::SRC [DEST]\r\n",
      "  or   rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]\r\n",
      "The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect\r\n",
      "to an rsync daemon, and require SRC or DEST to start with a module name.\r\n",
      "\r\n",
      "Options\r\n",
      " -v, --verbose               increase verbosity\r\n",
      " -q, --quiet                 suppress non-error messages\r\n",
      "     --no-motd               suppress daemon-mode MOTD (see manpage caveat)\r\n",
      " -c, --checksum              skip based on checksum, not mod-time & size\r\n",
      " -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)\r\n",
      "     --no-OPTION             turn off an implied OPTION (e.g. --no-D)\r\n",
      " -r, --recursive             recurse into directories\r\n",
      " -R, --relative              use relative path names\r\n",
      "     --no-implied-dirs       don't send implied dirs with --relative\r\n",
      " -b, --backup                make backups (see --suffix & --backup-dir)\r\n",
      "     --backup-dir=DIR        make backups into hierarchy based in DIR\r\n",
      "     --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)\r\n",
      " -u, --update                skip files that are newer on the receiver\r\n",
      "     --inplace               update destination files in-place (SEE MAN PAGE)\r\n",
      "     --append                append data onto shorter files\r\n",
      "     --append-verify         like --append, but with old data in file checksum\r\n",
      " -d, --dirs                  transfer directories without recursing\r\n",
      " -l, --links                 copy symlinks as symlinks\r\n",
      " -L, --copy-links            transform symlink into referent file/dir\r\n",
      "     --copy-unsafe-links     only \"unsafe\" symlinks are transformed\r\n",
      "     --safe-links            ignore symlinks that point outside the source tree\r\n",
      " -k, --copy-dirlinks         transform symlink to a dir into referent dir\r\n",
      " -K, --keep-dirlinks         treat symlinked dir on receiver as dir\r\n",
      " -H, --hard-links            preserve hard links\r\n",
      " -p, --perms                 preserve permissions\r\n",
      " -E, --executability         preserve the file's executability\r\n",
      "     --chmod=CHMOD           affect file and/or directory permissions\r\n",
      " -A, --acls                  preserve ACLs (implies --perms)\r\n",
      " -X, --xattrs                preserve extended attributes\r\n",
      " -o, --owner                 preserve owner (super-user only)\r\n",
      " -g, --group                 preserve group\r\n",
      "     --devices               preserve device files (super-user only)\r\n",
      "     --copy-devices          copy device contents as regular file\r\n",
      "     --specials              preserve special files\r\n",
      " -D                          same as --devices --specials\r\n",
      " -t, --times                 preserve modification times\r\n",
      " -O, --omit-dir-times        omit directories from --times\r\n",
      "     --super                 receiver attempts super-user activities\r\n",
      "     --fake-super            store/recover privileged attrs using xattrs\r\n",
      " -S, --sparse                handle sparse files efficiently\r\n",
      " -n, --dry-run               perform a trial run with no changes made\r\n",
      " -W, --whole-file            copy files whole (without delta-xfer algorithm)\r\n",
      " -x, --one-file-system       don't cross filesystem boundaries\r\n",
      " -B, --block-size=SIZE       force a fixed checksum block-size\r\n",
      " -e, --rsh=COMMAND           specify the remote shell to use\r\n",
      "     --rsync-path=PROGRAM    specify the rsync to run on the remote machine\r\n",
      "     --existing              skip creating new files on receiver\r\n",
      "     --ignore-existing       skip updating files that already exist on receiver\r\n",
      "     --remove-source-files   sender removes synchronized files (non-dirs)\r\n",
      "     --del                   an alias for --delete-during\r\n",
      "     --delete                delete extraneous files from destination dirs\r\n",
      "     --delete-before         receiver deletes before transfer, not during\r\n",
      "     --delete-during         receiver deletes during transfer (default)\r\n",
      "     --delete-delay          find deletions during, delete after\r\n",
      "     --delete-after          receiver deletes after transfer, not during\r\n",
      "     --delete-excluded       also delete excluded files from destination dirs\r\n",
      "     --ignore-errors         delete even if there are I/O errors\r\n",
      "     --force                 force deletion of directories even if not empty\r\n",
      "     --max-delete=NUM        don't delete more than NUM files\r\n",
      "     --max-size=SIZE         don't transfer any file larger than SIZE\r\n",
      "     --min-size=SIZE         don't transfer any file smaller than SIZE\r\n",
      "     --partial               keep partially transferred files\r\n",
      "     --partial-dir=DIR       put a partially transferred file into DIR\r\n",
      "     --delay-updates         put all updated files into place at transfer's end\r\n",
      " -m, --prune-empty-dirs      prune empty directory chains from the file-list\r\n",
      "     --numeric-ids           don't map uid/gid values by user/group name\r\n",
      "     --timeout=SECONDS       set I/O timeout in seconds\r\n",
      "     --contimeout=SECONDS    set daemon connection timeout in seconds\r\n",
      " -I, --ignore-times          don't skip files that match in size and mod-time\r\n",
      "     --size-only             skip files that match in size\r\n",
      "     --modify-window=NUM     compare mod-times with reduced accuracy\r\n",
      " -T, --temp-dir=DIR          create temporary files in directory DIR\r\n",
      " -y, --fuzzy                 find similar file for basis if no dest file\r\n",
      "     --compare-dest=DIR      also compare destination files relative to DIR\r\n",
      "     --copy-dest=DIR         ... and include copies of unchanged files\r\n",
      "     --link-dest=DIR         hardlink to files in DIR when unchanged\r\n",
      " -z, --compress              compress file data during the transfer\r\n",
      "     --compress-level=NUM    explicitly set compression level\r\n",
      "     --skip-compress=LIST    skip compressing files with a suffix in LIST\r\n",
      " -C, --cvs-exclude           auto-ignore files the same way CVS does\r\n",
      " -f, --filter=RULE           add a file-filtering RULE\r\n",
      " -F                          same as --filter='dir-merge /.rsync-filter'\r\n",
      "                             repeated: --filter='- .rsync-filter'\r\n",
      "     --exclude=PATTERN       exclude files matching PATTERN\r\n",
      "     --exclude-from=FILE     read exclude patterns from FILE\r\n",
      "     --include=PATTERN       don't exclude files matching PATTERN\r\n",
      "     --include-from=FILE     read include patterns from FILE\r\n",
      "     --files-from=FILE       read list of source-file names from FILE\r\n",
      " -0, --from0                 all *-from/filter files are delimited by 0s\r\n",
      " -s, --protect-args          no space-splitting; only wildcard special-chars\r\n",
      "     --address=ADDRESS       bind address for outgoing socket to daemon\r\n",
      "     --port=PORT             specify double-colon alternate port number\r\n",
      "     --sockopts=OPTIONS      specify custom TCP options\r\n",
      "     --blocking-io           use blocking I/O for the remote shell\r\n",
      "     --stats                 give some file-transfer stats\r\n",
      " -8, --8-bit-output          leave high-bit chars unescaped in output\r\n",
      " -h, --human-readable        output numbers in a human-readable format\r\n",
      "     --progress              show progress during transfer\r\n",
      " -P                          same as --partial --progress\r\n",
      " -i, --itemize-changes       output a change-summary for all updates\r\n",
      "     --out-format=FORMAT     output updates using the specified FORMAT\r\n",
      "     --log-file=FILE         log what we're doing to the specified FILE\r\n",
      "     --log-file-format=FMT   log updates using the specified FMT\r\n",
      "     --password-file=FILE    read daemon-access password from FILE\r\n",
      "     --list-only             list the files instead of copying them\r\n",
      "     --bwlimit=KBPS          limit I/O bandwidth; KBytes per second\r\n",
      "     --write-batch=FILE      write a batched update to FILE\r\n",
      "     --only-write-batch=FILE like --write-batch but w/o updating destination\r\n",
      "     --read-batch=FILE       read a batched update from FILE\r\n",
      "     --protocol=NUM          force an older protocol version to be used\r\n",
      "     --iconv=CONVERT_SPEC    request charset conversion of filenames\r\n",
      " -4, --ipv4                  prefer IPv4\r\n",
      " -6, --ipv6                  prefer IPv6\r\n",
      "     --version               print version number\r\n",
      "(-h) --help                  show this help (-h works with no other options)\r\n",
      "\r\n",
      "Use \"rsync --daemon --help\" to see the daemon-mode command-line options.\r\n",
      "Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation.\r\n",
      "See http://rsync.samba.org/ for updates, bug reports, and answers\r\n"
     ]
    }
   ],
   "source": [
    "rsync --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "rsync 是复杂而强大的工具。不过一般来说，下面这种用法可以解决大多数问题："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "rsync -e \"ssh -i /HOME/nscc-gz_jiangli/keys/id_rsa.ky\"   ~/file.lj  nscc-gz_kangyouzhong@ln3-gn0:~/.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 -e 里配置建立 ssh 连接需要的特殊参数，这里只是指定了 key 文件，有时候，你可能还需要指定其他的，如用 -p 指定端口。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 其他临时共享办法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "cp ~/file.lj /tmp/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "chmod 777 /tmp/file.lj"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如上例所示，将文件拷贝到一个公共文件夹下，并将权限放开，是一个比较简易的文件共享方式。\n",
    "\n",
    "不过您需要明确这种做法带来的问题：\n",
    "\n",
    "文件人人可以获取。\n",
    "\n",
    "此共享文件夹可能会被清理。\n",
    "\n",
    "如果此共享文件夹是在内存里（如tmp），需要在同一结点才能访问。并且如果文件太大会使系统出现故障。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}