diff options
| author | Ryo Nakamura <upa@haeena.net> | 2022-11-13 18:14:06 +0900 |
|---|---|---|
| committer | Ryo Nakamura <upa@haeena.net> | 2022-11-13 18:14:06 +0900 |
| commit | b8e204ae41d581a16520e29cf0099f10a68ad926 (patch) | |
| tree | df0f14eac5e2dda69d44add2e3f980f93320d28a /README.md | |
| parent | 613961b71dea603cb70515a285c3ae8111c23678 (diff) | |
update README
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 97 |
1 files changed, 59 insertions, 38 deletions
@@ -3,22 +3,23 @@ [](https://github.com/upa/mscp/actions/workflows/build-ubuntu.yml) [](https://github.com/upa/mscp/actions/workflows/build-macos.yml) [](https://github.com/upa/mscp/actions/workflows/test.yml) -`mscp`, a variant of `scp`, copies files over multiple ssh (sftp) +`mscp`, a variant of `scp`, copies files over multiple ssh (SFTP) sessions. Multiple threads in mscp transfer (1) multiple files simultaneously and (2) a large file in parallel. It may shorten the waiting time for transferring a lot of/large files over networks. -You can use `mscp` like `scp`, for example, `mscp example.com:srcfile -/tmp/dstfile`. Remote hosts only need to run `sshd` supporting the -SFTP subsystem, and you need to be able to ssh to the hosts (as -usual). +You can use `mscp` like `scp`, for example, `mscp +user@example.com:srcfile /tmp/dstfile`. Remote hosts only need to run +normal `sshd` supporting the SFTP subsystem, and you need to be able +to ssh to the hosts (as usual). `mscp` does not require anything else. + Differences from `scp` are: - remote glob on remote shell expansion is not supported. - remote to remote copy is not supported. - `-r` option is not needed. -- and any other differences I have not noticed and implemented... +- and any other differences I have not implemented and noticed... ## Install @@ -29,8 +30,10 @@ Differences from `scp` are: brew install upa/tap/mscp ``` -- Linux: Download a package for your environment from -[Releases page](https://github.com/upa/mscp/releases). +- Linux + +Download a package for your environment from [Releases +page](https://github.com/upa/mscp/releases). ## Build from source @@ -72,50 +75,30 @@ make install - Usage -```shell-session +```console $ mscp mscp v0.0.0: copy files over multiple ssh connections Usage: mscp [vqDCHdh] [-n nr_conns] [-s min_chunk_sz] [-S max_chunk_sz] - [-b sftp_buf_sz] [-B io_buf_sz] + [-b sftp_buf_sz] [-B io_buf_sz] [-a nr_ahead] [-l login_name] [-p port] [-i identity_file] [-c cipher_spec] source ... target - - -n NR_CONNECTIONS number of connections (default: half of # of cpu cores) - -s MIN_CHUNK_SIZE min chunk size (default: 64MB) - -S MAX_CHUNK_SIZE max chunk size (default: filesize / nr_conn) - -b SFTP_BUF_SIZE buf size for sftp_read/write (default 131072B) - -B IO_BUF_SIZE buf size for read/write (default 131072B) - Note that this value is derived from - qemu/block/ssh.c. need investigation... - -v increment verbose output level - -q disable output - -D dry run - - -l LOGIN_NAME login name - -p PORT port number - -i IDENTITY identity file for publickey authentication - -c CIPHER cipher spec, see `ssh -Q cipher` - -C enable compression on libssh - -H disable hostkey check - -d increment ssh debug output level - -h print this help ``` - Example: copy an 8GB file on tmpfs over a 100Gbps link - Two Intel Xeon Gold 6130 machines directly connected with Intel E810 100Gbps NICs. -```shell-session +```console $ mscp /tmp/test.img 10.0.0.1:/tmp/ -[===============================================================] 100% 8GB/8GB 3.02GB/s +[=====================================================] 100% 8GB/8GB 3.02GB/s ``` -- `-v` options increment verbose output level. +- `-v` option increments verbose output level. -```shell-session +```console $ mscp test 10.0.0.1: -[===============================================================] 100% 13B/13B 2.41KB/s +[=====================================================] 100% 13B/13B 2.41KB/s $ mscp -v test 10.0.0.1: file test/test.txt (local) -> ./test/test.txt (remote) 9B @@ -127,7 +110,7 @@ copy start: test/test2/2.txt copy done: test/1.txt copy done: test/test2/2.txt copy done: test/test.txt -[===============================================================] 100% 13B/13B 2.51KB/s +[=====================================================] 100% 13B/13B 2.51KB/s $ mscp -vv -n 4 test 10.0.0.1: connecting to 10.0.0.1 for checking destinations... @@ -144,8 +127,46 @@ copy start: test/test2/2.txt copy done: test/test.txt copy done: test/test2/2.txt copy done: test/1.txt -[===============================================================] 100% 13B/13B 3.27KB/s +[=====================================================] 100% 13B/13B 3.27KB/s +``` + +- Full usage + +```console +$ mscp -h +mscp v0.0.0: copy files over multiple ssh connections + +Usage: mscp [vqDCHdh] [-n nr_conns] + [-s min_chunk_sz] [-S max_chunk_sz] + [-b sftp_buf_sz] [-B io_buf_sz] [-a nr_ahead] + [-l login_name] [-p port] [-i identity_file] + [-c cipher_spec] source ... target + + -n NR_CONNECTIONS number of connections (default: half of # of cpu cores) + -s MIN_CHUNK_SIZE min chunk size (default: 64MB) + -S MAX_CHUNK_SIZE max chunk size (default: filesize / nr_conn) + + -b SFTP_BUF_SIZE buf size for sftp_read/write (default 131072B) + -B IO_BUF_SIZE buf size for read/write (default 131072B) + Note that the default value is derived from + qemu/block/ssh.c. need investigation... + -b and -B affect only local to remote copy + -a NR_AHEAD number of inflight SFTP read commands (default 16) + + -v increment verbose output level + -q disable output + -D dry run + + -l LOGIN_NAME login name + -p PORT port number + -i IDENTITY identity file for publickey authentication + -c CIPHER cipher spec, see `ssh -Q cipher` + -C enable compression on libssh + -H disable hostkey check + -d increment ssh debug output level + -h print this help ``` + Note: mscp is still under development, and the author is not -responsible for any accidents on mscp. +responsible for any accidents due to mscp. |