cvsroot/TDB_FileX/TDB_FileX.pm

package TDB_FileX;

use common::sense;

use Exporter ();
use XSLoader ();

our @ISA = qw(Exporter);

# Items to export into callers namespace by default. Note: do not export
# names by default without a very good reason. Use EXPORT_OK instead.
# Do not simply export all your public functions/methods/constants.

our %EXPORT_TAGS = (
   flags => [qw(
      ALLOW_NESTING
      BIGENDIAN
      CLEAR_IF_FIRST
      CONVERT
      DEFAULT
      DISALLOW_NESTING
      INCOMPATIBLE_HASH
      INTERNAL
      MUTEX_LOCKING
      NOLOCK
      NOMMAP
      NOSYNC
      SEQNUM
      VOLATILE
   )],
   insert => [qw(
      INSERT
      MODIFY
      REPLACE
   )],
   error => [qw(
      SUCCESS
      ERR_CORRUPT
      ERR_EXISTS
      ERR_IO
      ERR_LOCK
      ERR_LOCK_TIMEOUT
      ERR_NOEXIST
      ERR_NOLOCK
      ERR_OOM
      ERR_EINVAL
      ERR_RDONLY
   )],
   debug => [qw(
      DEBUG_FATAL
      DEBUG_ERROR
      DEBUG_WARNING
      DEBUG_TRACE}
   )],
);

our @EXPORT_OK;

Exporter::export_ok_tags qw(flags insert error debug);

$EXPORT_TAGS{all} = \@EXPORT_OK;

our $VERSION = '0.97';

XSLoader::load __PACKAGE__, $VERSION;

1;
__END__

=head1 NAME

TDB_FileX - Perl access to the trivial database library

=head1 SYNOPSIS

  use TDB_FileX;

  # tie interface
  tie %hash, TDB_FileX => $filename,
     hash_size => 8000,
     mutex     => 1,
  ;
  $hash{key} = 'value';
  while (my ($k, $v) = each %hash) { print "$k -> $v\n" }

  # OO interface
  my $tdb = TDB_FileX->open ($filename, flags => TDB_FileX::CLEAR_IF_FIRST)
     or die $!;

  $tdb->store (key => 'value') or die $tdb->errorstr;
  $tdb->traverse (sub { print "$_[0] -> $_[1]\n" });

=head1 DESCRIPTION

TDB is a simple database similar to GDBM, but allows multiple simultaneous
writers. It's main drawback is the need to manually configure a hash table
size in advance - see the C<hash_size> option for C<open>.

TDB_FileX provides a simple C<tie> interface, similar to DB_File and
friends; and an object-oriented interface, which provides access to most
of the functions in the TDB library.

=head1 ERROR HANDLKING

The TDB C API is not well designed - among other things, error handling is
a bit erratic. Many functions that normally should just work are marked
with a [CROAK] - these will throw an exception on error, the exception
string being the error message. C<$!> is set to the numerical error value
in that case (and will stringify into the wrong OS-error string, so don't
do that!).

This is so that you can concentrate on the important parts, while there
are still no silent unexpected errors.

The other functions will not generally croak - check their description for
details on how errors are handled.

=head2 FUNCTIONS

=over 4

=item $tdb = tie %hash, TDB_FileX => $path[ , key => value...]

=item $tdb = TDB_FileX->open ($path[, key => value...])

TDB_FileX constructor (same as C<TIE>). Opens $path and returns a
TDB_FileX object. The same arguments may be passed to the C<tie>
function. On error, C<undef> is returned.

To open a tdb file that is created and used by another application
(maximising compatibility):

   my $tdb = TDB_FileX->open ($path, log_cb => sub { warn $_[1] })
      or die "$path: failed to open\n";

To open a tdb file with good performance for many thousands of large keys,
maybe not compatible to others:

   my $tdb = TDB_FileX->open ($path,
      hash_size => 10000,
      hash      => "xxh3",
      mutex     => 1,
      nocow     => 1,
      log_cb    => sub { warn $_[1] },
   ) or die "$path: failed to open\n";

A number of key-value pairs are accepted, with hopefully sensible defaults
for most of these (but you should consider at least setting C<hash_size>
if you want to store more than a thousand or so pairs, and C<mutex> if you
want to take advantage of much faster locking.

=over

=item tdb_flags => $flags (default: C<TDB_FileX::DEFAULT>)

A set of flags that influence the behaviour and format of the database.

See tdb_open(3) for the meanings and possible values (note that the
C<TDB_> prefix has to be removed, see the L<EXPORTS> section for a list.

=item open_flags => $flags (default: C<Fcntl::O_RDWR | Fcntl::O_CREAT>)

Standard open flags, as used in C<sysopen>.

=item mode => $mode (default: C<0666>)

Standard file open mode, as use din C<sysopen>. Only used when creating
the database file.

=item hash_size => $int (default: internal to libtdb, but normally C<131>)

The size of the internal hash table - only used when creating the
database. The default is usually very low and only good for a few thousand
keys. As a rule of thumb, it should be at least one percent of the number
of keys you plan to store, e.g. for C<800000> keys you should use around a
size of C<8000>.

Every hash entry is only 4 octets, os usually it isn't an issue to make the
hash table too large.

To give you an idea of the performance, I inserted about 600000 records,
3GB of data, into a TDB file, using different hash sizes.

    size time
     131 160s
     256  85s
    1024  12s
    4096   5s
    8192   4s

As you can see, the default hash table size for this case caused it to
use 40 times the time to insert than a larger hash table, the optimum
being around 75 keys per hash entry. But the databse easily was cached in
memory. If that is not the case, you might want to consider a hash table
that is larger than the number of keys you want to store - each hash slot
only uses 4 octets.

=item log_cb => $cb->($level, $msg)

Sets a code reference that is called with a message and a log level
(lower means more important, there are C<DEBUG_FATAL>, C<DEBUG_ERROR>,
C<DEBUG_WARNING> and C<DEBUG_TRACE}>). Unlike the "debug" in the name
might indicate, if you want to find out why, for instance, you could not
open a database, you need to use a logging callback.

=item hash => $hash (default: C<undef>)

Selects a hash function to use.

=over

=item C<"default"> or C<undef>

Use autodetection and use either C<"jenkins"> or the original tdb hash function.

=item C<"jenkins">

The jenkins hash function. Relatively fast, and recommended for modern tdb databases
that need to be interoperable between implementations.

=item C<"fnv1ax">

The FNV1-A hash with 32 bit post-mixing. Pretty good and verxy fast for
keys up to 10-20 octets.

=item C<"xxh3">

The XXH3 hash. Very good and fast especially for long keys.

=item C<1> .. C<4>

Specify one of up to four custom hash functions (see L<set_hash_function>).

=back

=item mutex => $bool (default: C<0>)

TDB can take advantage of fast interprocess mutexes, which can be
orders of magnitude faster than the syscall-based locking used by
default, but only works on the same machine.

Normally, you need to call C<TDB_FileX::runtime_check_for_robust_mutexes>
and set the C<MUTEX_LOCKING> flag if support is indicated.

This option, when enabled, enables C<MUTEX_LOCKING> if it is supported
by the platform - which involves a fork when opening the database. When
disabled, it will remove the flag.

It is recommended to keep this one, but enabling this changes the format
of the database, so might not be an option if interoperability with other
programs is required.

=item nocow => $bool (default: C<0>)

Set the no-copy-on-write flag I<iff> this flag is true, C<open_flags>
contains C<O_CREAT>, C<tdb_flags> do not contain C<INTERNAL> and this is
supported on the platform and filesystem. Copy-on-write filesystems have
to make a copy of every block written to, which can both be costly and can
cause massive fragmentation of the database file.

Setting the no-copy-on-write flag (same as C<chattr +C>) disables this,
usually at the expense of data protection (checksumming), reducing the
safety to the level of a normal filesystem such as ext4.

This is best effort and usually only takes effect when the
database is initially created. If it fails, TDB_FileX will simply
continue. Correctness should not be affected either way.

=back

There is no explicit close function. The database is closed implicitly
when there are no remaining references.

=item $tdb->store ($key, $value[, $flag=REPLACE]) [CROAK]

Store $value in the database with key w$key. The $flag defaults to
C<REPLACE>, but can also be C<INSERT> or C<MODIFY>, see tdb_store(3) for
details.

=item $tdb->append ($key, $value) [CROAK]

Appends $data to the data already stored for $key, or creates a new entry with it.

=item $tdb->fetch ($key)

Fetch the value associated with $key, or C<undef> if it is not found (or
on any error).

=item $tdb->delete ($key) [CROAK]

Delete the value associated with $key.

On failure, a perl false is returned. See L</$tdb->error> and
L</$tdb->errorstr> for the reason.

=item $bool = $tdb->exists ($key)

Return true if the $key is found, false otherwise.

=item $tdb->firstkey

Return the key of the first value in the database. Returns C<undef> on
failure or if there are no keys in the database. See tdb_firstkey(3)
for details.

=item $tdb->nextkey ($lastkey)

Return the next key in the database after $lastkey. Returns C<undef> on
failure or if there are no more keys in the database. See tdb_nextkey(3)
for details.

=item $tdb->error

Returns the current error state of the C<$tdb> object. See the list of
error codes given in L<EXPORTS>.

=item $tdb->errorstr

Returns a printable string that describes the error state of the
database.

=item $tdb->reopen [CROAK]

Closes and reopens the database. Required after a
L<fork|perlfunc/fork>, if both processes wish to use the database.

B<NB:> If C<reopen> fails, then it is unsafe to call any further methods
on C<$tdb>. Thus, the only way to find out I<why> C<reopen> failed is to
use a logging function.

=item TDB_FileX::reopen_all [CROAK]

Closes and reopens all open databases. See L</$tdb->reopen>.

B<NB:> If C<reopen_all> fails, there is no indication of I<which> C<$tdb>
objects failed or why. If you have to survive failures, you may wish to do
your own C<reopen> loop instead.

=item $tdb->traverse ($cb->($key, $data)) [CROAK]

Call $cb for each entry in the database. The callback should return false
to continue, or a true value to abort the traversal.

The callback is called with the key and value as arguments and should
return a false value if you wish to continue traversal, and a true value
if the traversal should be aborted.

C<traverse> returns the number of elements traversed. If $cb is C<undef>,
then this function simply counts the number of elements.

=item $tdb->traverse_read ($cb->($key, $data)) [CROAK]

Like C<traverse>, but only acquires a read lock.

=item $tdb->set_logging_function ($cb->($level, $msg))

Set the logging function to use when this database object encounters
errors.

$cb is called with the severity level (an integer) and the message (a
string).

=item $tdb->lockall [CROAK]

Lock an entire database with an exclusive write lock, returning false on
error. The purpose of this call is to avoid locking overhead for many
operations, but the database has to be unlocked manually when done.

=item $tdb->unlockall [CROAK]

Unlock an entire database previously locked with
L</$tdb->lockall>.

=item $tdb->lockall_read [CROAK]

Same as L</$tdb->lockall>, but uses a shared read lock instead of a writer lock.

=item $tdb->unlockall_read [CROAK]

Opposite of L</$tdb->lockall_read>.

=item $tdb->lockall_mark [CROAK]

=item $tdb->lockall_unmark [CROAK]

Please tlel me exactly what these do.

=item $tdb->lockall_nonblock [CROAK]

=item $tdb->lockall_read_nonblock [CROAK]

Please tell me what exactly these do.

=item $tdb->transaction_start [CROAK]

Starts a transaction - all operations will be queued, but not applied
to the database, until the transaction is either committed L</$tdb->transaction_commit>
or aborted/thrown away, with L</$tdb->transaction_cancel>.

=item $tdb->transaction_start_nonblock [CROAK]

Please tell me what this does.

=item $tdb->transaction_commit [CROAK]

Applies all changes in the transaction.

=item $tdb->transaction_cancel [CROAK]

Throws away all changes in the transaction.

=item $tdb->transaction_prepare_commit [CROAK]

Instead of calling L</$tdb->transaction_commit> you can do the commit in
two phases by calling this method before commit, which does the expensive
steps first.

=item $bool = $tdb->transaction_active

Returns true if a transaction is currently active.

=item $tdb->enable_seqnum

Enables sequence number generatiuon supporet for the database.

=item $seq = $tdb->get_seqnum

Returns the current sequence number. Internally, this is a 32 bit unsigned
integer, but the API converts it into a native integer, so the same
internal sequence number might be represented differently on different
machines.

=item $tdb->increment_seqnum_nonblock

Increments the sequence number. Note that the sequence number is also
incremented by TDB itself oon many operations.

=item $size = $tdb->hash_size

Returns the size of the hash table. The size cannot be changed other than
by recreating the database.

=item $octets = $tdb->map_size

Returns the current mmap size for the database.

=item $flags = $tdb->get_flags

Returns the flags for the database (the same as the C<tdb_flags> in C<open>).

=item $tdb->add_flags ($flag)

Tried to add flags to the database - yes, the parameter says C<$flag> (singular)
but the documentation and the code say I<flags> (plural).

=item $tdb->remove_flags ($flag)

Attempts to remove flags from the database.

=item $fileno = $tdb->fd

Returns the file descriptor (not file handle) for the underlying database file.

=item $path = $tdb->name

Returns the path used to open the database.

=item $tdb->wipe_all [CROAK]

Efficientlly deletes all entries in the database. This does not shrink the
file itself.

=item $tdb->repack [CROAK]

Tries to improve layout of the database by copying all items into a
temporary in-memory database, wiping the database, and copying all items
back. Yes, everything must fit into memory.

=item $bool = $tdb->check ($cb->($key, $value))

Does extensive checks on the database, optionally (if not C<undef>)
calling a check function for each pair, which must return true if the data
is valid.

Returns a boolean indicating whether the database was found healthy and
all the calls to the callback returned true.

=item $bool = $tdb->rescue ($cb->($key, $value))

Tries to recover some or all key-value pairs from a potentially damanged
database file. For each recovered pair it calls the given callback.

Returns a boolean indicating whether the database was found healthy and
all the calls to the callback returned true.

=item $bool = TDB_FileX::runtime_check_for_robust_mutexes

Tests whether robust mutexes are available for locking. This involves
forking the process, so it can be costly and problematic. This function
needs to be called before using the C<MUTEX_LOCKING> flag. But see the
C<mutex> parametrer to C<open> for an alternative.

=item $tdb->dump_all

Dump the records and freelist to STDOUT in an almost human readable
form.

=item $tdb->printfreelist

Dump the freelist to STDOUT.

=back

=head2 EXPORTS

Nothing constants are exported by default.

The tag C<:all> exports allpo of the constants.

Individually or with the tag C<:flags>:

  DEFAULT
  CLEAR_IF_FIRST
  INTERNAL
  NOLOCK
  NOMMAP
  CONVERT
  BIGENDIAN
  NOSYNC
  SEQNUM
  VOLATILE
  ALLOW_NESTING
  DISALLOW_NESTING
  INCOMPATIBLE_HASH
  MUTEX_LOCKING

Individually or with the tag C<:insert>:

  REPLACE
  INSERT
  MODIFY

Individually or with the tag C<:error>:

  SUCCESS
  ERR_CORRUPT
  ERR_IO
  ERR_LOCK
  ERR_OOM
  ERR_EXISTS
  ERR_NOLOCK
  ERR_LOCK_TIMEOUT
  ERR_NOEXIST
  ERR_EINVAL
  ERR_RDONLY

Individually or with the tag C<:debug>:

  DEBUG_FATAL
  DEBUG_ERROR
  DEBUG_WARNING
  DEBUG_TRACE

=head1 UNICODE HANDLING

TDB databases can only store octet strings. Unlike most other database
interfaces, TDB_FileX will safely handle Perl strings by downgrading
them. Perl will warn about strings that cnanot be downgraded.

=head1 DISK USAGE

TDB databses use 24 octets for every key-value pair, plus the octet size
of the key and sata, e.g. the pair "key" => "value" takes up 24+3+5 octets
on disk.

The database header is 168 octets (if I haven't miscounted).

Each hashtable entry is 4 octets, and one more than the hash size is
allocated, so the hahs table size is (hash_size + 1) * 4.

=head1 LIMITATIONS

=head2 Database Size

As far as I can see, TDB dastabases are limited to 4 GB.

=head2 Hash Functions

Hash functioons need to be set globally - they are limited to a maximum of
4, but this can be easily extended, but requires source code editing. This
is due to a limitation of the TDB C API.

=head2 No recoivery after failed C<reopoen_all>

There is no way to survive an error during C<reopen_all>.
Unfortunately this is a limitation in the TDB C API.

=head1 SEE ALSO

tdb(3), L<perltie>.

=head1 AUTHOR

Angus Lees, E<lt>gus@inodes.org>

Currently maintained by Marc A. Lehmann <schmorp@schmorp.de>
http://home.schmorp.de/

=cut
Revision:	1.7
Committed:	Fri May 2 19:41:38 2025 UTC (16 months, 3 weeks ago) by root
Branch:	MAIN
Changes since 1.6:	+4 -4 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	package TDB_FileX;
2
3			use common::sense;
4
5			use Exporter ();
6			use XSLoader ();
7
8			our @ISA = qw(Exporter);
9
10			# Items to export into callers namespace by default. Note: do not export
11			# names by default without a very good reason. Use EXPORT_OK instead.
12			# Do not simply export all your public functions/methods/constants.
13
14			our %EXPORT_TAGS = (
15			flags => [qw(
16			ALLOW_NESTING
17			BIGENDIAN
18			CLEAR_IF_FIRST
19			CONVERT
20			DEFAULT
21			DISALLOW_NESTING
22			INCOMPATIBLE_HASH
23			INTERNAL
24			MUTEX_LOCKING
25			NOLOCK
26			NOMMAP
27			NOSYNC
28			SEQNUM
29			VOLATILE
30			)],
31			insert => [qw(
32			INSERT
33			MODIFY
34			REPLACE
35			)],
36			error => [qw(
37			SUCCESS
38			ERR_CORRUPT
39			ERR_EXISTS
40			ERR_IO
41			ERR_LOCK
42			ERR_LOCK_TIMEOUT
43			ERR_NOEXIST
44			ERR_NOLOCK
45			ERR_OOM
46			ERR_EINVAL
47			ERR_RDONLY
48			)],
49			debug => [qw(
50			DEBUG_FATAL
51			DEBUG_ERROR
52			DEBUG_WARNING
53			DEBUG_TRACE}
54			)],
55			);
56
57			our @EXPORT_OK;
58
59			Exporter::export_ok_tags qw(flags insert error debug);
60
61			$EXPORT_TAGS{all} = \@EXPORT_OK;
62
63			our $VERSION = '0.97';
64
65			XSLoader::load __PACKAGE__, $VERSION;
66
67			1;
68			__END__
69
70			=head1 NAME
71
72			TDB_FileX - Perl access to the trivial database library
73
74			=head1 SYNOPSIS
75
76			use TDB_FileX;
77
78			# tie interface
79			tie %hash, TDB_FileX => $filename,
80			hash_size => 8000,
81			mutex => 1,
82			;
83			$hash{key} = 'value';
84			while (my ($k, $v) = each %hash) { print "$k -> $v\n" }
85
86			# OO interface
87			my $tdb = TDB_FileX->open ($filename, flags => TDB_FileX::CLEAR_IF_FIRST)
88			or die $!;
89
90			$tdb->store (key => 'value') or die $tdb->errorstr;
91			$tdb->traverse (sub { print "$_[0] -> $_[1]\n" });
92
93			=head1 DESCRIPTION
94
95			TDB is a simple database similar to GDBM, but allows multiple simultaneous
96			writers. It's main drawback is the need to manually configure a hash table
97			size in advance - see the C<hash_size> option for C<open>.
98
99			TDB_FileX provides a simple C<tie> interface, similar to DB_File and
100	root	1.3	friends; and an object-oriented interface, which provides access to most
101			of the functions in the TDB library.
102
103			=head1 ERROR HANDLKING
104
105			The TDB C API is not well designed - among other things, error handling is
106			a bit erratic. Many functions that normally should just work are marked
107			with a [CROAK] - these will throw an exception on error, the exception
108			string being the error message. C<$!> is set to the numerical error value
109			in that case (and will stringify into the wrong OS-error string, so don't
110			do that!).
111
112			This is so that you can concentrate on the important parts, while there
113			are still no silent unexpected errors.
114
115			The other functions will not generally croak - check their description for
116			details on how errors are handled.
117	root	1.1
118			=head2 FUNCTIONS
119
120			=over 4
121
122	root	1.3	=item $tdb = tie %hash, TDB_FileX => $path[ , key => value...]
123
124			=item $tdb = TDB_FileX->open ($path[, key => value...])
125
126			TDB_FileX constructor (same as C<TIE>). Opens $path and returns a
127			TDB_FileX object. The same arguments may be passed to the C<tie>
128			function. On error, C<undef> is returned.
129	root	1.1
130	root	1.3	To open a tdb file that is created and used by another application
131	root	1.6	(maximising compatibility):
132	root	1.3
133			my $tdb = TDB_FileX->open ($path, log_cb => sub { warn $_[1] })
134			or die "$path: failed to open\n";
135
136			To open a tdb file with good performance for many thousands of large keys,
137			maybe not compatible to others:
138
139			my $tdb = TDB_FileX->open ($path,
140			hash_size => 10000,
141			hash => "xxh3",
142			mutex => 1,
143	root	1.6	nocow => 1,
144	root	1.3	log_cb => sub { warn $_[1] },
145			) or die "$path: failed to open\n";
146	root	1.1
147			A number of key-value pairs are accepted, with hopefully sensible defaults
148			for most of these (but you should consider at least setting C<hash_size>
149	root	1.6	if you want to store more than a thousand or so pairs, and C<mutex> if you
150	root	1.1	want to take advantage of much faster locking.
151
152			=over
153
154			=item tdb_flags => $flags (default: C<TDB_FileX::DEFAULT>)
155
156			A set of flags that influence the behaviour and format of the database.
157
158			See tdb_open(3) for the meanings and possible values (note that the
159			C<TDB_> prefix has to be removed, see the L<EXPORTS> section for a list.
160
161			=item open_flags => $flags (default: C<Fcntl::O_RDWR \| Fcntl::O_CREAT>)
162
163			Standard open flags, as used in C<sysopen>.
164
165			=item mode => $mode (default: C<0666>)
166
167			Standard file open mode, as use din C<sysopen>. Only used when creating
168			the database file.
169
170			=item hash_size => $int (default: internal to libtdb, but normally C<131>)
171
172			The size of the internal hash table - only used when creating the
173			database. The default is usually very low and only good for a few thousand
174			keys. As a rule of thumb, it should be at least one percent of the number
175			of keys you plan to store, e.g. for C<800000> keys you should use around a
176			size of C<8000>.
177
178	root	1.5	Every hash entry is only 4 octets, os usually it isn't an issue to make the
179	root	1.2	hash table too large.
180
181			To give you an idea of the performance, I inserted about 600000 records,
182			3GB of data, into a TDB file, using different hash sizes.
183	root	1.1
184			size time
185			131 160s
186			256 85s
187			1024 12s
188			4096 5s
189			8192 4s
190
191	root	1.3	As you can see, the default hash table size for this case caused it to
192			use 40 times the time to insert than a larger hash table, the optimum
193	root	1.5	being around 75 keys per hash entry. But the databse easily was cached in
194	root	1.3	memory. If that is not the case, you might want to consider a hash table
195			that is larger than the number of keys you want to store - each hash slot
196			only uses 4 octets.
197	root	1.1
198			=item log_cb => $cb->($level, $msg)
199
200	root	1.2	Sets a code reference that is called with a message and a log level
201	root	1.1	(lower means more important, there are C<DEBUG_FATAL>, C<DEBUG_ERROR>,
202			C<DEBUG_WARNING> and C<DEBUG_TRACE}>). Unlike the "debug" in the name
203			might indicate, if you want to find out why, for instance, you could not
204			open a database, you need to use a logging callback.
205
206			=item hash => $hash (default: C<undef>)
207
208			Selects a hash function to use.
209
210			=over
211
212			=item C<"default"> or C<undef>
213
214			Use autodetection and use either C<"jenkins"> or the original tdb hash function.
215
216			=item C<"jenkins">
217
218			The jenkins hash function. Relatively fast, and recommended for modern tdb databases
219			that need to be interoperable between implementations.
220
221			=item C<"fnv1ax">
222
223			The FNV1-A hash with 32 bit post-mixing. Pretty good and verxy fast for
224			keys up to 10-20 octets.
225
226			=item C<"xxh3">
227
228			The XXH3 hash. Very good and fast especially for long keys.
229
230			=item C<1> .. C<4>
231
232			Specify one of up to four custom hash functions (see L<set_hash_function>).
233
234			=back
235
236			=item mutex => $bool (default: C<0>)
237
238	root	1.6	TDB can take advantage of fast interprocess mutexes, which can be
239			orders of magnitude faster than the syscall-based locking used by
240			default, but only works on the same machine.
241
242			Normally, you need to call C<TDB_FileX::runtime_check_for_robust_mutexes>
243			and set the C<MUTEX_LOCKING> flag if support is indicated.
244	root	1.1
245			This option, when enabled, enables C<MUTEX_LOCKING> if it is supported
246			by the platform - which involves a fork when opening the database. When
247			disabled, it will remove the flag.
248
249			It is recommended to keep this one, but enabling this changes the format
250			of the database, so might not be an option if interoperability with other
251			programs is required.
252
253	root	1.6	=item nocow => $bool (default: C<0>)
254
255			Set the no-copy-on-write flag I<iff> this flag is true, C<open_flags>
256	root	1.7	contains C<O_CREAT>, C<tdb_flags> do not contain C<INTERNAL> and this is
257			supported on the platform and filesystem. Copy-on-write filesystems have
258			to make a copy of every block written to, which can both be costly and can
259			cause massive fragmentation of the database file.
260	root	1.6
261			Setting the no-copy-on-write flag (same as C<chattr +C>) disables this,
262			usually at the expense of data protection (checksumming), reducing the
263			safety to the level of a normal filesystem such as ext4.
264
265			This is best effort and usually only takes effect when the
266			database is initially created. If it fails, TDB_FileX will simply
267			continue. Correctness should not be affected either way.
268
269	root	1.1	=back
270
271			There is no explicit close function. The database is closed implicitly
272			when there are no remaining references.
273
274	root	1.3	=item $tdb->store ($key, $value[, $flag=REPLACE]) [CROAK]
275	root	1.1
276			Store $value in the database with key w$key. The $flag defaults to
277			C<REPLACE>, but can also be C<INSERT> or C<MODIFY>, see tdb_store(3) for
278			details.
279
280	root	1.3	=item $tdb->append ($key, $value) [CROAK]
281
282			Appends $data to the data already stored for $key, or creates a new entry with it.
283	root	1.1
284			=item $tdb->fetch ($key)
285
286	root	1.3	Fetch the value associated with $key, or C<undef> if it is not found (or
287			on any error).
288	root	1.1
289	root	1.3	=item $tdb->delete ($key) [CROAK]
290	root	1.1
291			Delete the value associated with $key.
292
293			On failure, a perl false is returned. See L</$tdb->error> and
294			L</$tdb->errorstr> for the reason.
295
296	root	1.3	=item $bool = $tdb->exists ($key)
297	root	1.1
298			Return true if the $key is found, false otherwise.
299
300			=item $tdb->firstkey
301
302			Return the key of the first value in the database. Returns C<undef> on
303			failure or if there are no keys in the database. See tdb_firstkey(3)
304			for details.
305
306			=item $tdb->nextkey ($lastkey)
307
308			Return the next key in the database after $lastkey. Returns C<undef> on
309			failure or if there are no more keys in the database. See tdb_nextkey(3)
310			for details.
311
312			=item $tdb->error
313
314			Returns the current error state of the C<$tdb> object. See the list of
315			error codes given in L<EXPORTS>.
316
317			=item $tdb->errorstr
318
319			Returns a printable string that describes the error state of the
320			database.
321
322	root	1.3	=item $tdb->reopen [CROAK]
323	root	1.1
324			Closes and reopens the database. Required after a
325			L<fork\|perlfunc/fork>, if both processes wish to use the database.
326
327	root	1.3	B<NB:> If C<reopen> fails, then it is unsafe to call any further methods
328			on C<$tdb>. Thus, the only way to find out I<why> C<reopen> failed is to
329			use a logging function.
330	root	1.1
331	root	1.3	=item TDB_FileX::reopen_all [CROAK]
332	root	1.1
333			Closes and reopens all open databases. See L</$tdb->reopen>.
334
335	root	1.3	B<NB:> If C<reopen_all> fails, there is no indication of I<which> C<$tdb>
336			objects failed or why. If you have to survive failures, you may wish to do
337			your own C<reopen> loop instead.
338	root	1.1
339	root	1.4	=item $tdb->traverse ($cb->($key, $data)) [CROAK]
340	root	1.1
341	root	1.4	Call $cb for each entry in the database. The callback should return false
342			to continue, or a true value to abort the traversal.
343	root	1.1
344	root	1.3	The callback is called with the key and value as arguments and should
345	root	1.4	return a false value if you wish to continue traversal, and a true value
346			if the traversal should be aborted.
347	root	1.1
348	root	1.4	C<traverse> returns the number of elements traversed. If $cb is C<undef>,
349			then this function simply counts the number of elements.
350
351			=item $tdb->traverse_read ($cb->($key, $data)) [CROAK]
352
353			Like C<traverse>, but only acquires a read lock.
354	root	1.1
355	root	1.3	=item $tdb->set_logging_function ($cb->($level, $msg))
356	root	1.1
357			Set the logging function to use when this database object encounters
358	root	1.3	errors.
359	root	1.1
360	root	1.3	$cb is called with the severity level (an integer) and the message (a
361	root	1.1	string).
362
363	root	1.3	=item $tdb->lockall [CROAK]
364	root	1.1
365	root	1.2	Lock an entire database with an exclusive write lock, returning false on
366			error. The purpose of this call is to avoid locking overhead for many
367			operations, but the database has to be unlocked manually when done.
368	root	1.1
369	root	1.3	=item $tdb->unlockall [CROAK]
370	root	1.1
371			Unlock an entire database previously locked with
372			L</$tdb->lockall>.
373
374	root	1.3	=item $tdb->lockall_read [CROAK]
375	root	1.2
376			Same as L</$tdb->lockall>, but uses a shared read lock instead of a writer lock.
377
378	root	1.3	=item $tdb->unlockall_read [CROAK]
379	root	1.2
380			Opposite of L</$tdb->lockall_read>.
381
382	root	1.3	=item $tdb->lockall_mark [CROAK]
383	root	1.2
384	root	1.3	=item $tdb->lockall_unmark [CROAK]
385	root	1.2
386			Please tlel me exactly what these do.
387
388	root	1.3	=item $tdb->lockall_nonblock [CROAK]
389	root	1.2
390	root	1.3	=item $tdb->lockall_read_nonblock [CROAK]
391	root	1.2
392			Please tell me what exactly these do.
393
394	root	1.3	=item $tdb->transaction_start [CROAK]
395	root	1.2
396			Starts a transaction - all operations will be queued, but not applied
397			to the database, until the transaction is either committed L</$tdb->transaction_commit>
398			or aborted/thrown away, with L</$tdb->transaction_cancel>.
399
400	root	1.3	=item $tdb->transaction_start_nonblock [CROAK]
401	root	1.2
402			Please tell me what this does.
403
404	root	1.3	=item $tdb->transaction_commit [CROAK]
405	root	1.2
406			Applies all changes in the transaction.
407
408	root	1.3	=item $tdb->transaction_cancel [CROAK]
409	root	1.2
410			Throws away all changes in the transaction.
411
412	root	1.3	=item $tdb->transaction_prepare_commit [CROAK]
413	root	1.2
414			Instead of calling L</$tdb->transaction_commit> you can do the commit in
415			two phases by calling this method before commit, which does the expensive
416			steps first.
417
418			=item $bool = $tdb->transaction_active
419
420			Returns true if a transaction is currently active.
421
422			=item $tdb->enable_seqnum
423
424			Enables sequence number generatiuon supporet for the database.
425
426			=item $seq = $tdb->get_seqnum
427
428			Returns the current sequence number. Internally, this is a 32 bit unsigned
429			integer, but the API converts it into a native integer, so the same
430			internal sequence number might be represented differently on different
431			machines.
432
433			=item $tdb->increment_seqnum_nonblock
434
435			Increments the sequence number. Note that the sequence number is also
436			incremented by TDB itself oon many operations.
437
438			=item $size = $tdb->hash_size
439
440			Returns the size of the hash table. The size cannot be changed other than
441			by recreating the database.
442
443			=item $octets = $tdb->map_size
444
445			Returns the current mmap size for the database.
446
447			=item $flags = $tdb->get_flags
448
449			Returns the flags for the database (the same as the C<tdb_flags> in C<open>).
450
451			=item $tdb->add_flags ($flag)
452
453			Tried to add flags to the database - yes, the parameter says C<$flag> (singular)
454			but the documentation and the code say I<flags> (plural).
455
456			=item $tdb->remove_flags ($flag)
457
458			Attempts to remove flags from the database.
459
460			=item $fileno = $tdb->fd
461
462			Returns the file descriptor (not file handle) for the underlying database file.
463
464			=item $path = $tdb->name
465
466			Returns the path used to open the database.
467
468	root	1.3	=item $tdb->wipe_all [CROAK]
469	root	1.2
470			Efficientlly deletes all entries in the database. This does not shrink the
471			file itself.
472
473	root	1.3	=item $tdb->repack [CROAK]
474	root	1.2
475			Tries to improve layout of the database by copying all items into a
476			temporary in-memory database, wiping the database, and copying all items
477	root	1.3	back. Yes, everything must fit into memory.
478	root	1.2
479	root	1.4	=item $bool = $tdb->check ($cb->($key, $value))
480
481			Does extensive checks on the database, optionally (if not C<undef>)
482			calling a check function for each pair, which must return true if the data
483			is valid.
484
485			Returns a boolean indicating whether the database was found healthy and
486			all the calls to the callback returned true.
487
488			=item $bool = $tdb->rescue ($cb->($key, $value))
489
490			Tries to recover some or all key-value pairs from a potentially damanged
491			database file. For each recovered pair it calls the given callback.
492
493			Returns a boolean indicating whether the database was found healthy and
494			all the calls to the callback returned true.
495
496	root	1.2	=item $bool = TDB_FileX::runtime_check_for_robust_mutexes
497
498			Tests whether robust mutexes are available for locking. This involves
499			forking the process, so it can be costly and problematic. This function
500			needs to be called before using the C<MUTEX_LOCKING> flag. But see the
501			C<mutex> parametrer to C<open> for an alternative.
502
503	root	1.1	=item $tdb->dump_all
504
505			Dump the records and freelist to STDOUT in an almost human readable
506			form.
507
508			=item $tdb->printfreelist
509
510			Dump the freelist to STDOUT.
511
512			=back
513
514			=head2 EXPORTS
515
516			Nothing constants are exported by default.
517
518			The tag C<:all> exports allpo of the constants.
519
520			Individually or with the tag C<:flags>:
521
522			DEFAULT
523			CLEAR_IF_FIRST
524			INTERNAL
525			NOLOCK
526			NOMMAP
527			CONVERT
528			BIGENDIAN
529			NOSYNC
530			SEQNUM
531			VOLATILE
532			ALLOW_NESTING
533			DISALLOW_NESTING
534			INCOMPATIBLE_HASH
535			MUTEX_LOCKING
536
537			Individually or with the tag C<:insert>:
538
539			REPLACE
540			INSERT
541			MODIFY
542
543			Individually or with the tag C<:error>:
544
545			SUCCESS
546			ERR_CORRUPT
547			ERR_IO
548			ERR_LOCK
549			ERR_OOM
550			ERR_EXISTS
551			ERR_NOLOCK
552			ERR_LOCK_TIMEOUT
553			ERR_NOEXIST
554			ERR_EINVAL
555			ERR_RDONLY
556
557			Individually or with the tag C<:debug>:
558
559			DEBUG_FATAL
560			DEBUG_ERROR
561			DEBUG_WARNING
562			DEBUG_TRACE
563
564	root	1.2	=head1 UNICODE HANDLING
565
566			TDB databases can only store octet strings. Unlike most other database
567			interfaces, TDB_FileX will safely handle Perl strings by downgrading
568			them. Perl will warn about strings that cnanot be downgraded.
569
570			=head1 DISK USAGE
571
572			TDB databses use 24 octets for every key-value pair, plus the octet size
573			of the key and sata, e.g. the pair "key" => "value" takes up 24+3+5 octets
574			on disk.
575
576			The database header is 168 octets (if I haven't miscounted).
577
578			Each hashtable entry is 4 octets, and one more than the hash size is
579			allocated, so the hahs table size is (hash_size + 1) * 4.
580
581			=head1 LIMITATIONS
582
583			=head2 Database Size
584
585			As far as I can see, TDB dastabases are limited to 4 GB.
586
587			=head2 Hash Functions
588
589			Hash functioons need to be set globally - they are limited to a maximum of
590			4, but this can be easily extended, but requires source code editing. This
591			is due to a limitation of the TDB C API.
592
593			=head2 No recoivery after failed C<reopoen_all>
594	root	1.1
595			There is no way to survive an error during C<reopen_all>.
596			Unfortunately this is a limitation in the TDB C API.
597
598			=head1 SEE ALSO
599
600			tdb(3), L<perltie>.
601
602			=head1 AUTHOR
603
604			Angus Lees, E<lt>gus@inodes.org>
605
606			Currently maintained by Marc A. Lehmann <schmorp@schmorp.de>
607			http://home.schmorp.de/
608
609			=cut