forked from kspalaiologos/xpar
-
Notifications
You must be signed in to change notification settings - Fork 0
/
xpar.1
174 lines (174 loc) · 5.66 KB
/
xpar.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
.TH XPAR 1
.SH NAME
xpar \- an error/erasure code system guarding data integrity
.SH SYNOPSIS
.ll +8
.B xpar
.RB [ " \-Je " / " \-Jd " ]
.RB [ " \-hfvqVc " ]
.RB [ " \-i/--interlacing\ # " ]
.RB [ " \--no-mmap\ " ]
[
.I "names \&..."
]
.ll -8
.br
.B xpar
.RB [ " \-Se " ]
.RB [ " \-hfvqV " ]
.RB [ " \--out-prefix\ # ", " \--dshards\ # ", " \--pshards\ # " ]
.RB [ " \--no-mmap\ " ]
[
.I "names \&..."
]
.br
.B xpar
.RB [ " \-Sd " ]
.RB [ " \-hfvqV " ]
[
.I "names \&..."
]
.SH DESCRIPTION
.PP
The
.B xpar
command
is the command-line interface to an error/erasure code system which
fortifies data and corrects errors in the data or reconstructs missing
shards.
.PP
.B xpar
in joint encoding mode (
.B \-Je
) generates a slightly inflated (by about 12%) parity-guarded
file from a given data file with the extension
.I .xpar
. Such a file can be recovered as long as no more than about 6.2% of the
data is corrupted. xpar internally uses a (255,223)-RS code over an 8-bit
Galois field. At most two filenames intertwined with flags are expected
as arguments.
.B xpar
will by default not overwrite existing files. If this behaviour is intended,
use the
.B \-f
option.
If no file names are specified,
.B xpar
will process data from the standard input and write it to the standard output,
refusing to output binary data to a terminal. If two files are specified,
the first one is used in place of standard input, and the second one is used
in place of standard output. If the -c flag is present,
.B xpar
will read from the specified file and output data to standard output instead.
Otherwise, if decoding,
.B xpar
will try to guess the decompressed filename by removing the
.I .xpar
extension. If not present, an error will be reported. If encoding
the output filename will be generated by appending the
.I .xpar
extension to the input filename.
.PP
In joint mode, the flag
.B \-i/\-\-interlacing
can be used to specify the interlacing factor. This does not affect the nominal
error correction capability (in the sense of a noisy memory-less channel model),
but can be used to increase the robustness against burst errors. The default
interlacing factor is 1 and the maximum correctible burst error length is
16 (as given by the Singleton bound). The interlacing factor of 2 transposes
65025-byte blocks of data and hence can correct burst errors of length 4080. The
interlacing factor of 3 transposes approx. 16MB blocks of data and can correct
burst errors of length approx. 1MB. In theory, with the interlacing factor of
one, up to 32 burst errors could be corrected - this problem is however NP-hard
and reduces to polynomial reconstruction (the basis of e.g. Shamir's secret
sharing scheme).
.PP
.B xpar
in sharded encoding mode (
.B \-Se
) generates a set of parity-guarded files from a given data file. The amount of
files generated is given by the sum of the mandatory parameters
.B \--dshards
and
.B \--pshards
- which specify the number of data and parity shards, respectively. The data
shards contain the file contents verbatim (
.B xpar\
uses a systematic code). The data is reconstructible as long as there are more
than
.B \--dshards
shards available.
.SH OPTIONS
.TP
.B \-S --sharded
Enable sharded processing mode.
.TP
.B \-J --joint
Enable joint processing mode.
.TP
.B \-e --encode
Enable encoding mode.
.TP
.B \-d --decode
Enable decoding mode.
.TP
.B \-h --help
Print a help message and exit.
.TP
.B \-f --force
Force operation. Ignore errors and try to recover from them, if possible.
Overwrite existing files.
.TP
.B \-v --verbose
Be verbose. Print additional information.
.TP
.B \-q --quiet
Be quiet. Suppress most output, except fatal errors.
.TP
.B \-V --version
Print version information of the package and exit.
.TP
.B \-c --stdout
Write resultant data to standard output. Joint mode only.
.TP
.B \-i --interlacing
Specify the interlacing factor. Joint mode only.
.TP
.B \--no-mmap
Disable memory mapping. Generally results in worse performance, as the fallback
method is to read the file in chunks using the standard I/O library.
.TP
.B \--out-prefix
Specify the prefix for the output files (shards). Sharded mode only.
.TP
.B \--dshards
Specify the number of data shards. Sharded mode only. Mandatory.
.TP
.B \--pshards
Specify the number of parity shards. Sharded mode only. Mandatory.
.SH REPORTING BUGS
Report bugs to: [email protected] or [email protected], or open a ticket
at https://github.com/kspalaiologos/xpar.
.SH COPYRIGHT NOTICE
Copyright \(co 2022-2024 Kamila Szewczyk
.PP
Thanks to: Phil Karn (KA9Q), for the original implementation of the Reed-Solomon
code, licensed under the GPLv2-or-later license. Marshall Lochbaum for noticing
an optimisation to the syndrome computaton code. Rich Felker for the
implementation of
.B getopt_long
in musl libc that is included in this package, licensed under the terms of the
Expat/MIT license.
.PP
Permission is granted to make and distribute verbatim copies of
this manual provided the copyright notice and this permission notice
are preserved on all copies. Permission is granted to process this file through troff and print the
results, provided the printed document carries copying permission
notice identical to this one except for the removal of this paragraph
(this paragraph not being relevant to the printed manual). Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the entire
resulting derived work is distributed under the terms of a permission
notice identical to this one. Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation approved
by the Foundation.