cvsroot/TDB_FileX/TDB_FileX.pm

package TDB_FileX;

use common::sense;

use Exporter ();
use XSLoader ();

our @ISA = qw(Exporter);

# Items to export into callers namespace by default. Note: do not export
# names by default without a very good reason. Use EXPORT_OK instead.
# Do not simply export all your public functions/methods/constants.

our %EXPORT_TAGS = (
   flags => [qw(
      ALLOW_NESTING
      BIGENDIAN
      CLEAR_IF_FIRST
      CONVERT
      DEFAULT
      DISALLOW_NESTING
      INCOMPATIBLE_HASH
      INTERNAL
      MUTEX_LOCKING
      NOLOCK
      NOMMAP
      NOSYNC
      SEQNUM
      VOLATILE
   )],
   insert => [qw(
      INSERT
      MODIFY
      REPLACE
   )],
   error => [qw(
      SUCCESS
      ERR_CORRUPT
      ERR_EXISTS
      ERR_IO
      ERR_LOCK
      ERR_LOCK_TIMEOUT
      ERR_NOEXIST
      ERR_NOLOCK
      ERR_OOM
      ERR_EINVAL
      ERR_RDONLY
   )],
   debug => [qw(
      DEBUG_FATAL
      DEBUG_ERROR
      DEBUG_WARNING
      DEBUG_TRACE}
   )],
);

our @EXPORT_OK;

Exporter::export_ok_tags qw(flags insert error debug);

$EXPORT_TAGS{all} = \@EXPORT_OK;

our $VERSION = '0.97';

XSLoader::load __PACKAGE__, $VERSION;

1;
__END__

=head1 NAME

TDB_FileX - Perl access to the trivial database library

=head1 SYNOPSIS

  use TDB_FileX;

  # tie interface
  tie %hash, TDB_FileX => $filename,
     hash_size => 8000,
     mutex     => 1,
  ;
  $hash{key} = 'value';
  while (my ($k, $v) = each %hash) { print "$k -> $v\n" }

  # OO interface
  my $tdb = TDB_FileX->open ($filename, flags => TDB_FileX::CLEAR_IF_FIRST)
     or die $!;

  $tdb->store (key => 'value') or die $tdb->errorstr;
  $tdb->traverse (sub { print "$_[0] -> $_[1]\n" });

=head1 DESCRIPTION

TDB is a simple database similar to GDBM, but allows multiple simultaneous
writers. It's main drawback is the need to manually configure a hash table
size in advance - see the C<hash_size> option for C<open>.

TDB_FileX provides a simple C<tie> interface, similar to DB_File and
friends; and an object-oriented interface, which provides access to most
of the functions in the TDB library.

=head1 ERROR HANDLKING

The TDB C API is not well designed - among other things, error handling is
a bit erratic. Many functions that normally should just work are marked
with a [CROAK] - these will throw an exception on error, the exception
string being the error message. C<$!> is set to the numerical error value
in that case (and will stringify into the wrong OS-error string, so don't
do that!).

This is so that you can concentrate on the important parts, while there
are still no silent unexpected errors.

The other functions will not generally croak - check their description for
details on how errors are handled.

=head2 FUNCTIONS

=over 4

=item $tdb = tie %hash, TDB_FileX => $path[ , key => value...]

=item $tdb = TDB_FileX->open ($path[, key => value...])

TDB_FileX constructor (same as C<TIE>). Opens $path and returns a
TDB_FileX object. The same arguments may be passed to the C<tie>
function. On error, C<undef> is returned.

To open a tdb file that is created and used by another application
(maximize compatibility):

   my $tdb = TDB_FileX->open ($path, log_cb => sub { warn $_[1] })
      or die "$path: failed to open\n";

To open a tdb file with good performance for many thousands of large keys,
maybe not compatible to others:

   my $tdb = TDB_FileX->open ($path,
      hash_size => 10000,
      hash      => "xxh3",
      mutex     => 1,
      log_cb    => sub { warn $_[1] },
   ) or die "$path: failed to open\n";

A number of key-value pairs are accepted, with hopefully sensible defaults
for most of these (but you should consider at least setting C<hash_size>
if you want to store more than a thousand or so pairs, And C<mutex> if you
want to take advantage of much faster locking.

=over

=item tdb_flags => $flags (default: C<TDB_FileX::DEFAULT>)

A set of flags that influence the behaviour and format of the database.

See tdb_open(3) for the meanings and possible values (note that the
C<TDB_> prefix has to be removed, see the L<EXPORTS> section for a list.

=item open_flags => $flags (default: C<Fcntl::O_RDWR | Fcntl::O_CREAT>)

Standard open flags, as used in C<sysopen>.

=item mode => $mode (default: C<0666>)

Standard file open mode, as use din C<sysopen>. Only used when creating
the database file.

=item hash_size => $int (default: internal to libtdb, but normally C<131>)

The size of the internal hash table - only used when creating the
database. The default is usually very low and only good for a few thousand
keys. As a rule of thumb, it should be at least one percent of the number
of keys you plan to store, e.g. for C<800000> keys you should use around a
size of C<8000>.

Every hash entry is only 4 octets, os usually it isn't an issue to make the
hash table too large.

To give you an idea of the performance, I inserted about 600000 records,
3GB of data, into a TDB file, using different hash sizes.

    size time
     131 160s
     256  85s
    1024  12s
    4096   5s
    8192   4s

As you can see, the default hash table size for this case caused it to
use 40 times the time to insert than a larger hash table, the optimum
being around 75 keys per hash entry. But the databse easily was cached in
memory. If that is not the case, you might want to consider a hash table
that is larger than the number of keys you want to store - each hash slot
only uses 4 octets.

=item log_cb => $cb->($level, $msg)

Sets a code reference that is called with a message and a log level
(lower means more important, there are C<DEBUG_FATAL>, C<DEBUG_ERROR>,
C<DEBUG_WARNING> and C<DEBUG_TRACE}>). Unlike the "debug" in the name
might indicate, if you want to find out why, for instance, you could not
open a database, you need to use a logging callback.

=item hash => $hash (default: C<undef>)

Selects a hash function to use.

=over

=item C<"default"> or C<undef>

Use autodetection and use either C<"jenkins"> or the original tdb hash function.

=item C<"jenkins">

The jenkins hash function. Relatively fast, and recommended for modern tdb databases
that need to be interoperable between implementations.

=item C<"fnv1ax">

The FNV1-A hash with 32 bit post-mixing. Pretty good and verxy fast for
keys up to 10-20 octets.

=item C<"xxh3">

The XXH3 hash. Very good and fast especially for long keys.

=item C<1> .. C<4>

Specify one of up to four custom hash functions (see L<set_hash_function>).

=back

=item mutex => $bool (default: C<0>)

TDB can take advantage of fast interprocess mutexes, which
can be orders of magnitude faster than the syscall-based
locking used by default. Normally, you need to call
C<TDB_FileX::runtime_check_for_robust_mutexes> and set the
C<MUTEX_LOCKING> flag if support is indicated.

This option, when enabled, enables C<MUTEX_LOCKING> if it is supported
by the platform - which involves a fork when opening the database. When
disabled, it will remove the flag.

It is recommended to keep this one, but enabling this changes the format
of the database, so might not be an option if interoperability with other
programs is required.

=back

There is no explicit close function. The database is closed implicitly
when there are no remaining references.

=item $tdb->store ($key, $value[, $flag=REPLACE]) [CROAK]

Store $value in the database with key w$key. The $flag defaults to
C<REPLACE>, but can also be C<INSERT> or C<MODIFY>, see tdb_store(3) for
details.

=item $tdb->append ($key, $value) [CROAK]

Appends $data to the data already stored for $key, or creates a new entry with it.

=item $tdb->fetch ($key)

Fetch the value associated with $key, or C<undef> if it is not found (or
on any error).

=item $tdb->delete ($key) [CROAK]

Delete the value associated with $key.

On failure, a perl false is returned. See L</$tdb->error> and
L</$tdb->errorstr> for the reason.

=item $bool = $tdb->exists ($key)

Return true if the $key is found, false otherwise.

=item $tdb->firstkey

Return the key of the first value in the database. Returns C<undef> on
failure or if there are no keys in the database. See tdb_firstkey(3)
for details.

=item $tdb->nextkey ($lastkey)

Return the next key in the database after $lastkey. Returns C<undef> on
failure or if there are no more keys in the database. See tdb_nextkey(3)
for details.

=item $tdb->error

Returns the current error state of the C<$tdb> object. See the list of
error codes given in L<EXPORTS>.

=item $tdb->errorstr

Returns a printable string that describes the error state of the
database.

=item $tdb->reopen [CROAK]

Closes and reopens the database. Required after a
L<fork|perlfunc/fork>, if both processes wish to use the database.

B<NB:> If C<reopen> fails, then it is unsafe to call any further methods
on C<$tdb>. Thus, the only way to find out I<why> C<reopen> failed is to
use a logging function.

=item TDB_FileX::reopen_all [CROAK]

Closes and reopens all open databases. See L</$tdb->reopen>.

B<NB:> If C<reopen_all> fails, there is no indication of I<which> C<$tdb>
objects failed or why. If you have to survive failures, you may wish to do
your own C<reopen> loop instead.

=item $tdb->traverse ($cb->($key, $data)) [CROAK]

Call $cb for each entry in the database. The callback should return false
to continue, or a true value to abort the traversal.

The callback is called with the key and value as arguments and should
return a false value if you wish to continue traversal, and a true value
if the traversal should be aborted.

C<traverse> returns the number of elements traversed. If $cb is C<undef>,
then this function simply counts the number of elements.

=item $tdb->traverse_read ($cb->($key, $data)) [CROAK]

Like C<traverse>, but only acquires a read lock.

=item $tdb->set_logging_function ($cb->($level, $msg))

Set the logging function to use when this database object encounters
errors.

$cb is called with the severity level (an integer) and the message (a
string).

=item $tdb->lockall [CROAK]

Lock an entire database with an exclusive write lock, returning false on
error. The purpose of this call is to avoid locking overhead for many
operations, but the database has to be unlocked manually when done.

=item $tdb->unlockall [CROAK]

Unlock an entire database previously locked with
L</$tdb->lockall>.

=item $tdb->lockall_read [CROAK]

Same as L</$tdb->lockall>, but uses a shared read lock instead of a writer lock.

=item $tdb->unlockall_read [CROAK]

Opposite of L</$tdb->lockall_read>.

=item $tdb->lockall_mark [CROAK]

=item $tdb->lockall_unmark [CROAK]

Please tlel me exactly what these do.

=item $tdb->lockall_nonblock [CROAK]

=item $tdb->lockall_read_nonblock [CROAK]

Please tell me what exactly these do.

=item $tdb->transaction_start [CROAK]

Starts a transaction - all operations will be queued, but not applied
to the database, until the transaction is either committed L</$tdb->transaction_commit>
or aborted/thrown away, with L</$tdb->transaction_cancel>.

=item $tdb->transaction_start_nonblock [CROAK]

Please tell me what this does.

=item $tdb->transaction_commit [CROAK]

Applies all changes in the transaction.

=item $tdb->transaction_cancel [CROAK]

Throws away all changes in the transaction.

=item $tdb->transaction_prepare_commit [CROAK]

Instead of calling L</$tdb->transaction_commit> you can do the commit in
two phases by calling this method before commit, which does the expensive
steps first.

=item $bool = $tdb->transaction_active

Returns true if a transaction is currently active.

=item $tdb->enable_seqnum

Enables sequence number generatiuon supporet for the database.

=item $seq = $tdb->get_seqnum

Returns the current sequence number. Internally, this is a 32 bit unsigned
integer, but the API converts it into a native integer, so the same
internal sequence number might be represented differently on different
machines.

=item $tdb->increment_seqnum_nonblock

Increments the sequence number. Note that the sequence number is also
incremented by TDB itself oon many operations.

=item $size = $tdb->hash_size

Returns the size of the hash table. The size cannot be changed other than
by recreating the database.

=item $octets = $tdb->map_size

Returns the current mmap size for the database.

=item $flags = $tdb->get_flags

Returns the flags for the database (the same as the C<tdb_flags> in C<open>).

=item $tdb->add_flags ($flag)

Tried to add flags to the database - yes, the parameter says C<$flag> (singular)
but the documentation and the code say I<flags> (plural).

=item $tdb->remove_flags ($flag)

Attempts to remove flags from the database.

=item $fileno = $tdb->fd

Returns the file descriptor (not file handle) for the underlying database file.

=item $path = $tdb->name

Returns the path used to open the database.

=item $tdb->wipe_all [CROAK]

Efficientlly deletes all entries in the database. This does not shrink the
file itself.

=item $tdb->repack [CROAK]

Tries to improve layout of the database by copying all items into a
temporary in-memory database, wiping the database, and copying all items
back. Yes, everything must fit into memory.

=item $bool = $tdb->check ($cb->($key, $value))

Does extensive checks on the database, optionally (if not C<undef>)
calling a check function for each pair, which must return true if the data
is valid.

Returns a boolean indicating whether the database was found healthy and
all the calls to the callback returned true.

=item $bool = $tdb->rescue ($cb->($key, $value))

Tries to recover some or all key-value pairs from a potentially damanged
database file. For each recovered pair it calls the given callback.

Returns a boolean indicating whether the database was found healthy and
all the calls to the callback returned true.

=item $bool = TDB_FileX::runtime_check_for_robust_mutexes

Tests whether robust mutexes are available for locking. This involves
forking the process, so it can be costly and problematic. This function
needs to be called before using the C<MUTEX_LOCKING> flag. But see the
C<mutex> parametrer to C<open> for an alternative.

=item $tdb->dump_all

Dump the records and freelist to STDOUT in an almost human readable
form.

=item $tdb->printfreelist

Dump the freelist to STDOUT.

=back

=head2 EXPORTS

Nothing constants are exported by default.

The tag C<:all> exports allpo of the constants.

Individually or with the tag C<:flags>:

  DEFAULT
  CLEAR_IF_FIRST
  INTERNAL
  NOLOCK
  NOMMAP
  CONVERT
  BIGENDIAN
  NOSYNC
  SEQNUM
  VOLATILE
  ALLOW_NESTING
  DISALLOW_NESTING
  INCOMPATIBLE_HASH
  MUTEX_LOCKING

Individually or with the tag C<:insert>:

  REPLACE
  INSERT
  MODIFY

Individually or with the tag C<:error>:

  SUCCESS
  ERR_CORRUPT
  ERR_IO
  ERR_LOCK
  ERR_OOM
  ERR_EXISTS
  ERR_NOLOCK
  ERR_LOCK_TIMEOUT
  ERR_NOEXIST
  ERR_EINVAL
  ERR_RDONLY

Individually or with the tag C<:debug>:

  DEBUG_FATAL
  DEBUG_ERROR
  DEBUG_WARNING
  DEBUG_TRACE

=head1 UNICODE HANDLING

TDB databases can only store octet strings. Unlike most other database
interfaces, TDB_FileX will safely handle Perl strings by downgrading
them. Perl will warn about strings that cnanot be downgraded.

=head1 DISK USAGE

TDB databses use 24 octets for every key-value pair, plus the octet size
of the key and sata, e.g. the pair "key" => "value" takes up 24+3+5 octets
on disk.

The database header is 168 octets (if I haven't miscounted).

Each hashtable entry is 4 octets, and one more than the hash size is
allocated, so the hahs table size is (hash_size + 1) * 4.

=head1 LIMITATIONS

=head2 Database Size

As far as I can see, TDB dastabases are limited to 4 GB.

=head2 Hash Functions

Hash functioons need to be set globally - they are limited to a maximum of
4, but this can be easily extended, but requires source code editing. This
is due to a limitation of the TDB C API.

=head2 No recoivery after failed C<reopoen_all>

There is no way to survive an error during C<reopen_all>.
Unfortunately this is a limitation in the TDB C API.

=head1 SEE ALSO

tdb(3), L<perltie>.

=head1 AUTHOR

Angus Lees, E<lt>gus@inodes.org>

Currently maintained by Marc A. Lehmann <schmorp@schmorp.de>
http://home.schmorp.de/

=cut
Revision:	1.5
Committed:	Fri May 2 03:51:24 2025 UTC (16 months, 3 weeks ago) by root
Branch:	MAIN
Changes since 1.4:	+2 -2 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	package TDB_FileX;
2
3			use common::sense;
4
5			use Exporter ();
6			use XSLoader ();
7
8			our @ISA = qw(Exporter);
9
10			# Items to export into callers namespace by default. Note: do not export
11			# names by default without a very good reason. Use EXPORT_OK instead.
12			# Do not simply export all your public functions/methods/constants.
13
14			our %EXPORT_TAGS = (
15			flags => [qw(
16			ALLOW_NESTING
17			BIGENDIAN
18			CLEAR_IF_FIRST
19			CONVERT
20			DEFAULT
21			DISALLOW_NESTING
22			INCOMPATIBLE_HASH
23			INTERNAL
24			MUTEX_LOCKING
25			NOLOCK
26			NOMMAP
27			NOSYNC
28			SEQNUM
29			VOLATILE
30			)],
31			insert => [qw(
32			INSERT
33			MODIFY
34			REPLACE
35			)],
36			error => [qw(
37			SUCCESS
38			ERR_CORRUPT
39			ERR_EXISTS
40			ERR_IO
41			ERR_LOCK
42			ERR_LOCK_TIMEOUT
43			ERR_NOEXIST
44			ERR_NOLOCK
45			ERR_OOM
46			ERR_EINVAL
47			ERR_RDONLY
48			)],
49			debug => [qw(
50			DEBUG_FATAL
51			DEBUG_ERROR
52			DEBUG_WARNING
53			DEBUG_TRACE}
54			)],
55			);
56
57			our @EXPORT_OK;
58
59			Exporter::export_ok_tags qw(flags insert error debug);
60
61			$EXPORT_TAGS{all} = \@EXPORT_OK;
62
63			our $VERSION = '0.97';
64
65			XSLoader::load __PACKAGE__, $VERSION;
66
67			1;
68			__END__
69
70			=head1 NAME
71
72			TDB_FileX - Perl access to the trivial database library
73
74			=head1 SYNOPSIS
75
76			use TDB_FileX;
77
78			# tie interface
79			tie %hash, TDB_FileX => $filename,
80			hash_size => 8000,
81			mutex => 1,
82			;
83			$hash{key} = 'value';
84			while (my ($k, $v) = each %hash) { print "$k -> $v\n" }
85
86			# OO interface
87			my $tdb = TDB_FileX->open ($filename, flags => TDB_FileX::CLEAR_IF_FIRST)
88			or die $!;
89
90			$tdb->store (key => 'value') or die $tdb->errorstr;
91			$tdb->traverse (sub { print "$_[0] -> $_[1]\n" });
92
93			=head1 DESCRIPTION
94
95			TDB is a simple database similar to GDBM, but allows multiple simultaneous
96			writers. It's main drawback is the need to manually configure a hash table
97			size in advance - see the C<hash_size> option for C<open>.
98
99			TDB_FileX provides a simple C<tie> interface, similar to DB_File and
100	root	1.3	friends; and an object-oriented interface, which provides access to most
101			of the functions in the TDB library.
102
103			=head1 ERROR HANDLKING
104
105			The TDB C API is not well designed - among other things, error handling is
106			a bit erratic. Many functions that normally should just work are marked
107			with a [CROAK] - these will throw an exception on error, the exception
108			string being the error message. C<$!> is set to the numerical error value
109			in that case (and will stringify into the wrong OS-error string, so don't
110			do that!).
111
112			This is so that you can concentrate on the important parts, while there
113			are still no silent unexpected errors.
114
115			The other functions will not generally croak - check their description for
116			details on how errors are handled.
117	root	1.1
118			=head2 FUNCTIONS
119
120			=over 4
121
122	root	1.3	=item $tdb = tie %hash, TDB_FileX => $path[ , key => value...]
123
124			=item $tdb = TDB_FileX->open ($path[, key => value...])
125
126			TDB_FileX constructor (same as C<TIE>). Opens $path and returns a
127			TDB_FileX object. The same arguments may be passed to the C<tie>
128			function. On error, C<undef> is returned.
129	root	1.1
130	root	1.3	To open a tdb file that is created and used by another application
131			(maximize compatibility):
132
133			my $tdb = TDB_FileX->open ($path, log_cb => sub { warn $_[1] })
134			or die "$path: failed to open\n";
135
136			To open a tdb file with good performance for many thousands of large keys,
137			maybe not compatible to others:
138
139			my $tdb = TDB_FileX->open ($path,
140			hash_size => 10000,
141			hash => "xxh3",
142			mutex => 1,
143			log_cb => sub { warn $_[1] },
144			) or die "$path: failed to open\n";
145	root	1.1
146			A number of key-value pairs are accepted, with hopefully sensible defaults
147			for most of these (but you should consider at least setting C<hash_size>
148			if you want to store more than a thousand or so pairs, And C<mutex> if you
149			want to take advantage of much faster locking.
150
151			=over
152
153			=item tdb_flags => $flags (default: C<TDB_FileX::DEFAULT>)
154
155			A set of flags that influence the behaviour and format of the database.
156
157			See tdb_open(3) for the meanings and possible values (note that the
158			C<TDB_> prefix has to be removed, see the L<EXPORTS> section for a list.
159
160			=item open_flags => $flags (default: C<Fcntl::O_RDWR \| Fcntl::O_CREAT>)
161
162			Standard open flags, as used in C<sysopen>.
163
164			=item mode => $mode (default: C<0666>)
165
166			Standard file open mode, as use din C<sysopen>. Only used when creating
167			the database file.
168
169			=item hash_size => $int (default: internal to libtdb, but normally C<131>)
170
171			The size of the internal hash table - only used when creating the
172			database. The default is usually very low and only good for a few thousand
173			keys. As a rule of thumb, it should be at least one percent of the number
174			of keys you plan to store, e.g. for C<800000> keys you should use around a
175			size of C<8000>.
176
177	root	1.5	Every hash entry is only 4 octets, os usually it isn't an issue to make the
178	root	1.2	hash table too large.
179
180			To give you an idea of the performance, I inserted about 600000 records,
181			3GB of data, into a TDB file, using different hash sizes.
182	root	1.1
183			size time
184			131 160s
185			256 85s
186			1024 12s
187			4096 5s
188			8192 4s
189
190	root	1.3	As you can see, the default hash table size for this case caused it to
191			use 40 times the time to insert than a larger hash table, the optimum
192	root	1.5	being around 75 keys per hash entry. But the databse easily was cached in
193	root	1.3	memory. If that is not the case, you might want to consider a hash table
194			that is larger than the number of keys you want to store - each hash slot
195			only uses 4 octets.
196	root	1.1
197			=item log_cb => $cb->($level, $msg)
198
199	root	1.2	Sets a code reference that is called with a message and a log level
200	root	1.1	(lower means more important, there are C<DEBUG_FATAL>, C<DEBUG_ERROR>,
201			C<DEBUG_WARNING> and C<DEBUG_TRACE}>). Unlike the "debug" in the name
202			might indicate, if you want to find out why, for instance, you could not
203			open a database, you need to use a logging callback.
204
205			=item hash => $hash (default: C<undef>)
206
207			Selects a hash function to use.
208
209			=over
210
211			=item C<"default"> or C<undef>
212
213			Use autodetection and use either C<"jenkins"> or the original tdb hash function.
214
215			=item C<"jenkins">
216
217			The jenkins hash function. Relatively fast, and recommended for modern tdb databases
218			that need to be interoperable between implementations.
219
220			=item C<"fnv1ax">
221
222			The FNV1-A hash with 32 bit post-mixing. Pretty good and verxy fast for
223			keys up to 10-20 octets.
224
225			=item C<"xxh3">
226
227			The XXH3 hash. Very good and fast especially for long keys.
228
229			=item C<1> .. C<4>
230
231			Specify one of up to four custom hash functions (see L<set_hash_function>).
232
233			=back
234
235			=item mutex => $bool (default: C<0>)
236
237			TDB can take advantage of fast interprocess mutexes, which
238			can be orders of magnitude faster than the syscall-based
239			locking used by default. Normally, you need to call
240			C<TDB_FileX::runtime_check_for_robust_mutexes> and set the
241			C<MUTEX_LOCKING> flag if support is indicated.
242
243			This option, when enabled, enables C<MUTEX_LOCKING> if it is supported
244			by the platform - which involves a fork when opening the database. When
245			disabled, it will remove the flag.
246
247			It is recommended to keep this one, but enabling this changes the format
248			of the database, so might not be an option if interoperability with other
249			programs is required.
250
251			=back
252
253			There is no explicit close function. The database is closed implicitly
254			when there are no remaining references.
255
256	root	1.3	=item $tdb->store ($key, $value[, $flag=REPLACE]) [CROAK]
257	root	1.1
258			Store $value in the database with key w$key. The $flag defaults to
259			C<REPLACE>, but can also be C<INSERT> or C<MODIFY>, see tdb_store(3) for
260			details.
261
262	root	1.3	=item $tdb->append ($key, $value) [CROAK]
263
264			Appends $data to the data already stored for $key, or creates a new entry with it.
265	root	1.1
266			=item $tdb->fetch ($key)
267
268	root	1.3	Fetch the value associated with $key, or C<undef> if it is not found (or
269			on any error).
270	root	1.1
271	root	1.3	=item $tdb->delete ($key) [CROAK]
272	root	1.1
273			Delete the value associated with $key.
274
275			On failure, a perl false is returned. See L</$tdb->error> and
276			L</$tdb->errorstr> for the reason.
277
278	root	1.3	=item $bool = $tdb->exists ($key)
279	root	1.1
280			Return true if the $key is found, false otherwise.
281
282			=item $tdb->firstkey
283
284			Return the key of the first value in the database. Returns C<undef> on
285			failure or if there are no keys in the database. See tdb_firstkey(3)
286			for details.
287
288			=item $tdb->nextkey ($lastkey)
289
290			Return the next key in the database after $lastkey. Returns C<undef> on
291			failure or if there are no more keys in the database. See tdb_nextkey(3)
292			for details.
293
294			=item $tdb->error
295
296			Returns the current error state of the C<$tdb> object. See the list of
297			error codes given in L<EXPORTS>.
298
299			=item $tdb->errorstr
300
301			Returns a printable string that describes the error state of the
302			database.
303
304	root	1.3	=item $tdb->reopen [CROAK]
305	root	1.1
306			Closes and reopens the database. Required after a
307			L<fork\|perlfunc/fork>, if both processes wish to use the database.
308
309	root	1.3	B<NB:> If C<reopen> fails, then it is unsafe to call any further methods
310			on C<$tdb>. Thus, the only way to find out I<why> C<reopen> failed is to
311			use a logging function.
312	root	1.1
313	root	1.3	=item TDB_FileX::reopen_all [CROAK]
314	root	1.1
315			Closes and reopens all open databases. See L</$tdb->reopen>.
316
317	root	1.3	B<NB:> If C<reopen_all> fails, there is no indication of I<which> C<$tdb>
318			objects failed or why. If you have to survive failures, you may wish to do
319			your own C<reopen> loop instead.
320	root	1.1
321	root	1.4	=item $tdb->traverse ($cb->($key, $data)) [CROAK]
322	root	1.1
323	root	1.4	Call $cb for each entry in the database. The callback should return false
324			to continue, or a true value to abort the traversal.
325	root	1.1
326	root	1.3	The callback is called with the key and value as arguments and should
327	root	1.4	return a false value if you wish to continue traversal, and a true value
328			if the traversal should be aborted.
329	root	1.1
330	root	1.4	C<traverse> returns the number of elements traversed. If $cb is C<undef>,
331			then this function simply counts the number of elements.
332
333			=item $tdb->traverse_read ($cb->($key, $data)) [CROAK]
334
335			Like C<traverse>, but only acquires a read lock.
336	root	1.1
337	root	1.3	=item $tdb->set_logging_function ($cb->($level, $msg))
338	root	1.1
339			Set the logging function to use when this database object encounters
340	root	1.3	errors.
341	root	1.1
342	root	1.3	$cb is called with the severity level (an integer) and the message (a
343	root	1.1	string).
344
345	root	1.3	=item $tdb->lockall [CROAK]
346	root	1.1
347	root	1.2	Lock an entire database with an exclusive write lock, returning false on
348			error. The purpose of this call is to avoid locking overhead for many
349			operations, but the database has to be unlocked manually when done.
350	root	1.1
351	root	1.3	=item $tdb->unlockall [CROAK]
352	root	1.1
353			Unlock an entire database previously locked with
354			L</$tdb->lockall>.
355
356	root	1.3	=item $tdb->lockall_read [CROAK]
357	root	1.2
358			Same as L</$tdb->lockall>, but uses a shared read lock instead of a writer lock.
359
360	root	1.3	=item $tdb->unlockall_read [CROAK]
361	root	1.2
362			Opposite of L</$tdb->lockall_read>.
363
364	root	1.3	=item $tdb->lockall_mark [CROAK]
365	root	1.2
366	root	1.3	=item $tdb->lockall_unmark [CROAK]
367	root	1.2
368			Please tlel me exactly what these do.
369
370	root	1.3	=item $tdb->lockall_nonblock [CROAK]
371	root	1.2
372	root	1.3	=item $tdb->lockall_read_nonblock [CROAK]
373	root	1.2
374			Please tell me what exactly these do.
375
376	root	1.3	=item $tdb->transaction_start [CROAK]
377	root	1.2
378			Starts a transaction - all operations will be queued, but not applied
379			to the database, until the transaction is either committed L</$tdb->transaction_commit>
380			or aborted/thrown away, with L</$tdb->transaction_cancel>.
381
382	root	1.3	=item $tdb->transaction_start_nonblock [CROAK]
383	root	1.2
384			Please tell me what this does.
385
386	root	1.3	=item $tdb->transaction_commit [CROAK]
387	root	1.2
388			Applies all changes in the transaction.
389
390	root	1.3	=item $tdb->transaction_cancel [CROAK]
391	root	1.2
392			Throws away all changes in the transaction.
393
394	root	1.3	=item $tdb->transaction_prepare_commit [CROAK]
395	root	1.2
396			Instead of calling L</$tdb->transaction_commit> you can do the commit in
397			two phases by calling this method before commit, which does the expensive
398			steps first.
399
400			=item $bool = $tdb->transaction_active
401
402			Returns true if a transaction is currently active.
403
404			=item $tdb->enable_seqnum
405
406			Enables sequence number generatiuon supporet for the database.
407
408			=item $seq = $tdb->get_seqnum
409
410			Returns the current sequence number. Internally, this is a 32 bit unsigned
411			integer, but the API converts it into a native integer, so the same
412			internal sequence number might be represented differently on different
413			machines.
414
415			=item $tdb->increment_seqnum_nonblock
416
417			Increments the sequence number. Note that the sequence number is also
418			incremented by TDB itself oon many operations.
419
420			=item $size = $tdb->hash_size
421
422			Returns the size of the hash table. The size cannot be changed other than
423			by recreating the database.
424
425			=item $octets = $tdb->map_size
426
427			Returns the current mmap size for the database.
428
429			=item $flags = $tdb->get_flags
430
431			Returns the flags for the database (the same as the C<tdb_flags> in C<open>).
432
433			=item $tdb->add_flags ($flag)
434
435			Tried to add flags to the database - yes, the parameter says C<$flag> (singular)
436			but the documentation and the code say I<flags> (plural).
437
438			=item $tdb->remove_flags ($flag)
439
440			Attempts to remove flags from the database.
441
442			=item $fileno = $tdb->fd
443
444			Returns the file descriptor (not file handle) for the underlying database file.
445
446			=item $path = $tdb->name
447
448			Returns the path used to open the database.
449
450	root	1.3	=item $tdb->wipe_all [CROAK]
451	root	1.2
452			Efficientlly deletes all entries in the database. This does not shrink the
453			file itself.
454
455	root	1.3	=item $tdb->repack [CROAK]
456	root	1.2
457			Tries to improve layout of the database by copying all items into a
458			temporary in-memory database, wiping the database, and copying all items
459	root	1.3	back. Yes, everything must fit into memory.
460	root	1.2
461	root	1.4	=item $bool = $tdb->check ($cb->($key, $value))
462
463			Does extensive checks on the database, optionally (if not C<undef>)
464			calling a check function for each pair, which must return true if the data
465			is valid.
466
467			Returns a boolean indicating whether the database was found healthy and
468			all the calls to the callback returned true.
469
470			=item $bool = $tdb->rescue ($cb->($key, $value))
471
472			Tries to recover some or all key-value pairs from a potentially damanged
473			database file. For each recovered pair it calls the given callback.
474
475			Returns a boolean indicating whether the database was found healthy and
476			all the calls to the callback returned true.
477
478	root	1.2	=item $bool = TDB_FileX::runtime_check_for_robust_mutexes
479
480			Tests whether robust mutexes are available for locking. This involves
481			forking the process, so it can be costly and problematic. This function
482			needs to be called before using the C<MUTEX_LOCKING> flag. But see the
483			C<mutex> parametrer to C<open> for an alternative.
484
485	root	1.1	=item $tdb->dump_all
486
487			Dump the records and freelist to STDOUT in an almost human readable
488			form.
489
490			=item $tdb->printfreelist
491
492			Dump the freelist to STDOUT.
493
494			=back
495
496			=head2 EXPORTS
497
498			Nothing constants are exported by default.
499
500			The tag C<:all> exports allpo of the constants.
501
502			Individually or with the tag C<:flags>:
503
504			DEFAULT
505			CLEAR_IF_FIRST
506			INTERNAL
507			NOLOCK
508			NOMMAP
509			CONVERT
510			BIGENDIAN
511			NOSYNC
512			SEQNUM
513			VOLATILE
514			ALLOW_NESTING
515			DISALLOW_NESTING
516			INCOMPATIBLE_HASH
517			MUTEX_LOCKING
518
519			Individually or with the tag C<:insert>:
520
521			REPLACE
522			INSERT
523			MODIFY
524
525			Individually or with the tag C<:error>:
526
527			SUCCESS
528			ERR_CORRUPT
529			ERR_IO
530			ERR_LOCK
531			ERR_OOM
532			ERR_EXISTS
533			ERR_NOLOCK
534			ERR_LOCK_TIMEOUT
535			ERR_NOEXIST
536			ERR_EINVAL
537			ERR_RDONLY
538
539			Individually or with the tag C<:debug>:
540
541			DEBUG_FATAL
542			DEBUG_ERROR
543			DEBUG_WARNING
544			DEBUG_TRACE
545
546	root	1.2	=head1 UNICODE HANDLING
547
548			TDB databases can only store octet strings. Unlike most other database
549			interfaces, TDB_FileX will safely handle Perl strings by downgrading
550			them. Perl will warn about strings that cnanot be downgraded.
551
552			=head1 DISK USAGE
553
554			TDB databses use 24 octets for every key-value pair, plus the octet size
555			of the key and sata, e.g. the pair "key" => "value" takes up 24+3+5 octets
556			on disk.
557
558			The database header is 168 octets (if I haven't miscounted).
559
560			Each hashtable entry is 4 octets, and one more than the hash size is
561			allocated, so the hahs table size is (hash_size + 1) * 4.
562
563			=head1 LIMITATIONS
564
565			=head2 Database Size
566
567			As far as I can see, TDB dastabases are limited to 4 GB.
568
569			=head2 Hash Functions
570
571			Hash functioons need to be set globally - they are limited to a maximum of
572			4, but this can be easily extended, but requires source code editing. This
573			is due to a limitation of the TDB C API.
574
575			=head2 No recoivery after failed C<reopoen_all>
576	root	1.1
577			There is no way to survive an error during C<reopen_all>.
578			Unfortunately this is a limitation in the TDB C API.
579
580			=head1 SEE ALSO
581
582			tdb(3), L<perltie>.
583
584			=head1 AUTHOR
585
586			Angus Lees, E<lt>gus@inodes.org>
587
588			Currently maintained by Marc A. Lehmann <schmorp@schmorp.de>
589			http://home.schmorp.de/
590
591			=cut