10564b1175
groff 1.23.0 added .MR to its -man macro package. The NEWS file states
that the inclusion of the macro "was prompted by its introduction to
Plan 9 from User Space's troff in August 2020." From d32deab it seems
that the name for Plan 9 from User Space's implementation was suggested
by groff maintainer G. Brandon Robinson.
Not sure if the intention was to make these definitions compatible, but
it would be nice if they were.
Currently, Plan 9 from User Space's .MR expects its second argument to
be parenthesized. groff's .MR does not. This results in extra
parentheses appearing in manual references when viewing Plan 9 from User
Space's manual pages on a system using groff.
409 lines
8.5 KiB
Groff
409 lines
8.5 KiB
Groff
.TH VENTI-FMT 8
|
|
.SH NAME
|
|
buildindex,
|
|
checkarenas,
|
|
checkindex,
|
|
conf,
|
|
fmtarenas,
|
|
fmtbloom,
|
|
fmtindex,
|
|
fmtisect,
|
|
syncindex \- prepare and maintain a venti server
|
|
.SH SYNOPSIS
|
|
.PP
|
|
.B venti/fmtarenas
|
|
[
|
|
.B -4Z
|
|
]
|
|
[
|
|
.B -a
|
|
.I arenasize
|
|
]
|
|
[
|
|
.B -b
|
|
.I blocksize
|
|
]
|
|
.I name
|
|
.I file
|
|
.PP
|
|
.B venti/fmtisect
|
|
[
|
|
.B -1Z
|
|
]
|
|
[
|
|
.B -b
|
|
.I blocksize
|
|
]
|
|
.I name
|
|
.I file
|
|
.PP
|
|
.B venti/fmtbloom
|
|
[
|
|
.B -n
|
|
.I nblocks
|
|
|
|
|
.B -N
|
|
.I nhash
|
|
]
|
|
[
|
|
.B -s
|
|
.I size
|
|
]
|
|
.I file
|
|
.PP
|
|
.B venti/fmtindex
|
|
[
|
|
.B -a
|
|
]
|
|
.I venti.conf
|
|
.PP
|
|
.B venti/conf
|
|
[
|
|
.B -w
|
|
]
|
|
.I partition
|
|
[
|
|
.I configfile
|
|
]
|
|
.if t .sp 0.5
|
|
.PP
|
|
.B venti/buildindex
|
|
[
|
|
.B -bd
|
|
] [
|
|
.B -i
|
|
.I isect
|
|
] ... [
|
|
.B -M
|
|
.I imemsize
|
|
]
|
|
.I venti.conf
|
|
.PP
|
|
.B venti/checkindex
|
|
[
|
|
.B -f
|
|
]
|
|
[
|
|
.B -B
|
|
.I blockcachesize
|
|
]
|
|
.I venti.conf
|
|
.I tmp
|
|
.PP
|
|
.B venti/checkarenas
|
|
[
|
|
.B -afv
|
|
]
|
|
.I file
|
|
.SH DESCRIPTION
|
|
These commands aid in the setup, maintenance, and debugging of
|
|
venti servers.
|
|
See
|
|
.MR venti 7
|
|
for an overview of the venti system and
|
|
.MR venti 8
|
|
for an overview of the data structures used by the venti server.
|
|
.PP
|
|
Note that the units for the various sizes in the following
|
|
commands can be specified by appending
|
|
.LR k ,
|
|
.LR m ,
|
|
or
|
|
.LR g
|
|
to indicate kilobytes, megabytes, or gigabytes respectively.
|
|
.SS Formatting
|
|
To prepare a server for its initial use, the arena partitions and
|
|
the index sections must be formatted individually, with
|
|
.I fmtarenas
|
|
and
|
|
.IR fmtisect .
|
|
Then the
|
|
collection of index sections must be combined into a venti
|
|
index with
|
|
.IR fmtindex .
|
|
.PP
|
|
.I Fmtarenas
|
|
formats the given
|
|
.IR file ,
|
|
typically a disk partition, into an arena partition.
|
|
The arenas in the partition are given names of the form
|
|
.IR name%d ,
|
|
where
|
|
.I %d
|
|
is replaced with a sequential number starting at 0.
|
|
.PP
|
|
Options to
|
|
.I fmtarenas
|
|
are:
|
|
.TP
|
|
.BI -a " arenasize
|
|
The arenas are of
|
|
.I arenasize
|
|
bytes. The default is
|
|
.BR 512M ,
|
|
which was selected to provide a balance
|
|
between the number of arenas and the ability to copy an arena to external
|
|
media such as recordable CDs and tapes.
|
|
.TP
|
|
.BI -b " blocksize
|
|
The size, in bytes, for read and write operations to the file.
|
|
The size is recorded in the file, and is used by applications that access the arenas.
|
|
The default is
|
|
.BR 8k .
|
|
.TP
|
|
.B -4
|
|
Create a `version 4' arena partition for backwards compatibility with old servers.
|
|
The default is version 5, used by the current venti server.
|
|
.TP
|
|
.B -Z
|
|
Do not zero the data sections of the arenas.
|
|
Using this option reduces the formatting time
|
|
but should only be used when it is known that the file was already zeroed.
|
|
(Version 4 only; version 5 sections are not and do not need to be zeroed.)
|
|
.PD
|
|
.PP
|
|
.I Fmtisect
|
|
formats the given
|
|
.IR file ,
|
|
typically a disk partition, as a venti index section with the specified
|
|
.IR name .
|
|
Each of the index sections in a venti configuration must have a unique name.
|
|
.PP
|
|
Options to
|
|
.I fmtisect
|
|
are:
|
|
.TP
|
|
.BI -b " bucketsize
|
|
The size of an index bucket, in bytes.
|
|
All the index sections within a index must have the same bucket size.
|
|
The default is
|
|
.BR 8k .
|
|
.TP
|
|
.B -1
|
|
Create a `version 1' index section for backwards compatibility with old servers.
|
|
The default is version 2, used by the current venti server.
|
|
.TP
|
|
.B -Z
|
|
Do not zero the index.
|
|
Using this option reduces the formatting time
|
|
but should only be used when it is known that the file was already zeroed.
|
|
(Version 1 only; version 2 sections are not and do not need to be zeroed.)
|
|
.PD
|
|
.PP
|
|
.I Fmtbloom
|
|
formats the given
|
|
.I file
|
|
as a Bloom filter
|
|
(see
|
|
.MR venti 7 ).
|
|
The options are:
|
|
.TF "\fL-s\fI size"
|
|
.PD
|
|
.TP
|
|
.BI -n " nblock \fR| " -N " nhash
|
|
The number of blocks expected to be indexed by the filter
|
|
or the number of hash functions to use.
|
|
If the
|
|
.B -n
|
|
option
|
|
is given, it is used, along with the total size of the filter,
|
|
to compute an appropriate
|
|
.IR nhash .
|
|
.TP
|
|
.BI -s " size
|
|
The size of the Bloom filter. The default is the total size of the file.
|
|
In either case,
|
|
.I size
|
|
is rounded down to a power of two.
|
|
.PD
|
|
.PP
|
|
The
|
|
.I file
|
|
argument in the commands above can be of the form
|
|
.IB file : lo - hi
|
|
to specify a range of the file.
|
|
.I Lo
|
|
and
|
|
.I hi
|
|
are specified in bytes but can have the usual
|
|
.BI k ,
|
|
.BI m ,
|
|
or
|
|
.B g
|
|
suffixes.
|
|
Either
|
|
.I lo
|
|
or
|
|
.I hi
|
|
may be omitted.
|
|
This notation eliminates the need to
|
|
partition raw disks on non-Plan 9 systems.
|
|
.PP
|
|
.I Fmtindex
|
|
reads the configuration file
|
|
.I venti.conf
|
|
and initializes the index sections to form a usable index structure.
|
|
The arena files and index sections must have previously been formatted
|
|
using
|
|
.I fmtarenas
|
|
and
|
|
.I fmtisect
|
|
respectively.
|
|
.PP
|
|
The function of a venti index is to map a SHA1 fingerprint to a location
|
|
in the data section of one of the arenas. The index is composed of
|
|
blocks, each of which contains the mapping for a fixed range of possible
|
|
fingerprint values.
|
|
.I Fmtindex
|
|
determines the mapping between SHA1 values and the blocks
|
|
of the collection of index sections. Once this mapping has been determined,
|
|
it cannot be changed without rebuilding the index.
|
|
The basic assumption in the current implementation is that the index
|
|
structure is sufficiently empty that individual blocks of the index will rarely
|
|
overflow. The total size of the index should be about 2% to 10% of
|
|
the total size of the arenas, but the exact percentage depends both on the
|
|
index block size and the compressed size of blocks stored.
|
|
See the discussion in
|
|
.MR venti 8
|
|
for more.
|
|
.PP
|
|
.I Fmtindex
|
|
also computes a mapping between a linear address space and
|
|
the data section of the collection of arenas. The
|
|
.B -a
|
|
option can be used to add additional arenas to an index.
|
|
To use this feature,
|
|
add the new arenas to
|
|
.I venti.conf
|
|
after the existing arenas and then run
|
|
.I fmtindex
|
|
.BR -a .
|
|
.PP
|
|
A copy of the above mappings is stored in the header for each of the index sections.
|
|
These copies enable
|
|
.I buildindex
|
|
to restore a single index section without rebuilding the entire index.
|
|
.PP
|
|
To make it easier to bootstrap servers, the configuration
|
|
file can be stored in otherwise empty space
|
|
at the beginning of any venti partitions using
|
|
.IR conf .
|
|
A partition so branded with a configuration file can
|
|
be used in place of a configuration file when invoking any
|
|
of the venti commands.
|
|
By default,
|
|
.I conf
|
|
prints the configuration stored in
|
|
.IR partition .
|
|
When invoked with the
|
|
.B -w
|
|
flag,
|
|
.I conf
|
|
reads a configuration file from
|
|
.I configfile
|
|
(or else standard input)
|
|
and stores it in
|
|
.IR partition .
|
|
.SS Checking and Rebuilding
|
|
.PP
|
|
.I Buildindex
|
|
populates the index for the Venti system described in
|
|
.IR venti.conf .
|
|
The index must have previously been formatted using
|
|
.IR fmtindex .
|
|
This command is typically used to build a new index for a Venti
|
|
system when the old index becomes too small, or to rebuild
|
|
an index after media failure.
|
|
Small errors in an index can usually be fixed with
|
|
.IR checkindex ,
|
|
but
|
|
.I checkindex
|
|
requires a large temporary workspace and
|
|
.I buildindex
|
|
does not.
|
|
.PP
|
|
Options to
|
|
.I buildindex
|
|
are:
|
|
.TF "\fL-M\fI imemsize"
|
|
.PD
|
|
.TP
|
|
.B -b
|
|
Reinitialise the Bloom filter, if any.
|
|
.TP
|
|
.B -d
|
|
`Dumb' mode; run all three passes.
|
|
.TP
|
|
.BI -i " isect
|
|
Only rebuild index section
|
|
.IR isect ;
|
|
may be repeated to rebuild multiple sections.
|
|
The name
|
|
.L none
|
|
is special and just reads the arenas.
|
|
.TP
|
|
.BI -M " imemsize
|
|
The amount of memory, in bytes, to use for caching raw disk accesses while running
|
|
.IR buildindex .
|
|
(This is not a property of the created index.)
|
|
The usual suffices apply.
|
|
The default is 256M.
|
|
.PD
|
|
.PP
|
|
.I Checkindex
|
|
examines the Venti index described in
|
|
.IR venti.conf .
|
|
The program detects various error conditions including:
|
|
blocks that are not indexed, index entries for blocks that do not exist,
|
|
and duplicate index entries.
|
|
If requested, an attempt can be made to fix errors that are found.
|
|
.PP
|
|
The
|
|
.I tmp
|
|
file, usually a disk partition, must be large enough to store a copy of the index.
|
|
This temporary space is used to perform a merge sort of index entries
|
|
generated by reading the arenas.
|
|
.PP
|
|
Options to
|
|
.I checkindex
|
|
are:
|
|
.TP
|
|
.BI -B " blockcachesize
|
|
The amount of memory, in bytes, to use for caching raw disk accesses while running
|
|
.IR checkindex .
|
|
The default is 8k.
|
|
.TP
|
|
.B -f
|
|
Attempt to fix any errors that are found.
|
|
.PD
|
|
.PP
|
|
.I Checkarenas
|
|
examines the Venti arenas contained in the given
|
|
.IR file .
|
|
The program detects various error conditions, and optionally attempts
|
|
to fix any errors that are found.
|
|
.PP
|
|
Options to
|
|
.I checkarenas
|
|
are:
|
|
.TP
|
|
.B -a
|
|
For each arena, scan the entire data section.
|
|
If this option is omitted, only the end section of
|
|
the arena is examined.
|
|
.TP
|
|
.B -f
|
|
Attempt to fix any errors that are found.
|
|
.TP
|
|
.B -v
|
|
Increase the verbosity of output.
|
|
.PD
|
|
.SH SOURCE
|
|
.B \*9/src/cmd/venti/srv
|
|
.SH SEE ALSO
|
|
.MR venti 7 ,
|
|
.MR venti 8
|
|
.SH BUGS
|
|
.I Buildindex
|
|
should allow an individual index section to be rebuilt.
|