| 1 |
package TDB_FileX; |
| 2 |
|
| 3 |
use common::sense; |
| 4 |
|
| 5 |
use Exporter (); |
| 6 |
use XSLoader (); |
| 7 |
|
| 8 |
our @ISA = qw(Exporter); |
| 9 |
|
| 10 |
# Items to export into callers namespace by default. Note: do not export |
| 11 |
# names by default without a very good reason. Use EXPORT_OK instead. |
| 12 |
# Do not simply export all your public functions/methods/constants. |
| 13 |
|
| 14 |
our %EXPORT_TAGS = ( |
| 15 |
flags => [qw( |
| 16 |
ALLOW_NESTING |
| 17 |
BIGENDIAN |
| 18 |
CLEAR_IF_FIRST |
| 19 |
CONVERT |
| 20 |
DEFAULT |
| 21 |
DISALLOW_NESTING |
| 22 |
INCOMPATIBLE_HASH |
| 23 |
INTERNAL |
| 24 |
MUTEX_LOCKING |
| 25 |
NOLOCK |
| 26 |
NOMMAP |
| 27 |
NOSYNC |
| 28 |
SEQNUM |
| 29 |
VOLATILE |
| 30 |
)], |
| 31 |
insert => [qw( |
| 32 |
INSERT |
| 33 |
MODIFY |
| 34 |
REPLACE |
| 35 |
)], |
| 36 |
error => [qw( |
| 37 |
SUCCESS |
| 38 |
ERR_CORRUPT |
| 39 |
ERR_EXISTS |
| 40 |
ERR_IO |
| 41 |
ERR_LOCK |
| 42 |
ERR_LOCK_TIMEOUT |
| 43 |
ERR_NOEXIST |
| 44 |
ERR_NOLOCK |
| 45 |
ERR_OOM |
| 46 |
ERR_EINVAL |
| 47 |
ERR_RDONLY |
| 48 |
)], |
| 49 |
debug => [qw( |
| 50 |
DEBUG_FATAL |
| 51 |
DEBUG_ERROR |
| 52 |
DEBUG_WARNING |
| 53 |
DEBUG_TRACE} |
| 54 |
)], |
| 55 |
); |
| 56 |
|
| 57 |
our @EXPORT_OK; |
| 58 |
|
| 59 |
Exporter::export_ok_tags qw(flags insert error debug); |
| 60 |
|
| 61 |
$EXPORT_TAGS{all} = \@EXPORT_OK; |
| 62 |
|
| 63 |
our $VERSION = '0.97'; |
| 64 |
|
| 65 |
XSLoader::load __PACKAGE__, $VERSION; |
| 66 |
|
| 67 |
1; |
| 68 |
__END__ |
| 69 |
|
| 70 |
=head1 NAME |
| 71 |
|
| 72 |
TDB_FileX - Perl access to the trivial database library |
| 73 |
|
| 74 |
=head1 SYNOPSIS |
| 75 |
|
| 76 |
use TDB_FileX; |
| 77 |
|
| 78 |
# tie interface |
| 79 |
tie %hash, TDB_FileX => $filename, |
| 80 |
hash_size => 8000, |
| 81 |
mutex => 1, |
| 82 |
; |
| 83 |
$hash{key} = 'value'; |
| 84 |
while (my ($k, $v) = each %hash) { print "$k -> $v\n" } |
| 85 |
|
| 86 |
# OO interface |
| 87 |
my $tdb = TDB_FileX->open ($filename, flags => TDB_FileX::CLEAR_IF_FIRST) |
| 88 |
or die $!; |
| 89 |
|
| 90 |
$tdb->store (key => 'value') or die $tdb->errorstr; |
| 91 |
$tdb->traverse (sub { print "$_[0] -> $_[1]\n" }); |
| 92 |
|
| 93 |
=head1 DESCRIPTION |
| 94 |
|
| 95 |
TDB is a simple database similar to GDBM, but allows multiple simultaneous |
| 96 |
writers. It's main drawback is the need to manually configure a hash table |
| 97 |
size in advance - see the C<hash_size> option for C<open>. |
| 98 |
|
| 99 |
TDB_FileX provides a simple C<tie> interface, similar to DB_File and |
| 100 |
friends; and an object-oriented interface, which provides access to most |
| 101 |
of the functions in the TDB library. |
| 102 |
|
| 103 |
=head1 ERROR HANDLKING |
| 104 |
|
| 105 |
The TDB C API is not well designed - among other things, error handling is |
| 106 |
a bit erratic. Many functions that normally should just work are marked |
| 107 |
with a [CROAK] - these will throw an exception on error, the exception |
| 108 |
string being the error message. C<$!> is set to the numerical error value |
| 109 |
in that case (and will stringify into the wrong OS-error string, so don't |
| 110 |
do that!). |
| 111 |
|
| 112 |
This is so that you can concentrate on the important parts, while there |
| 113 |
are still no silent unexpected errors. |
| 114 |
|
| 115 |
The other functions will not generally croak - check their description for |
| 116 |
details on how errors are handled. |
| 117 |
|
| 118 |
=head2 FUNCTIONS |
| 119 |
|
| 120 |
=over 4 |
| 121 |
|
| 122 |
=item $tdb = tie %hash, TDB_FileX => $path[ , key => value...] |
| 123 |
|
| 124 |
=item $tdb = TDB_FileX->open ($path[, key => value...]) |
| 125 |
|
| 126 |
TDB_FileX constructor (same as C<TIE>). Opens $path and returns a |
| 127 |
TDB_FileX object. The same arguments may be passed to the C<tie> |
| 128 |
function. On error, C<undef> is returned. |
| 129 |
|
| 130 |
You should consider specifying at least C<log_cb> and C<hash_size> (and |
| 131 |
possibly C<mutex>), everything else has sensible defaults. |
| 132 |
|
| 133 |
To open a tdb file that is created and used by another application |
| 134 |
(maximising compatibility): |
| 135 |
|
| 136 |
my $tdb = TDB_FileX->open ($path, log_cb => sub { warn $_[1] }) |
| 137 |
or die "$path: failed to open\n"; |
| 138 |
|
| 139 |
To successfully open another database, you might have to duplicate some of |
| 140 |
the settings, e.g. whether mutex locking is used or the hash function. The |
| 141 |
C<log_cb> output usually will tell you what's wrong. |
| 142 |
|
| 143 |
To open a tdb file with good performance for many thousands of large keys, |
| 144 |
maybe not compatible to other programs: |
| 145 |
|
| 146 |
my $tdb = TDB_FileX->open ($path, |
| 147 |
hash_size => 10000, |
| 148 |
hash => "xxh3", |
| 149 |
mutex => 1, |
| 150 |
nocow => 1, |
| 151 |
log_cb => sub { warn $_[1] }, |
| 152 |
) or die "$path: failed to open\n"; |
| 153 |
|
| 154 |
The following key-value pairs are understood: |
| 155 |
|
| 156 |
=over |
| 157 |
|
| 158 |
=item tdb_flags => $flags (default: C<TDB_FileX::DEFAULT>) |
| 159 |
|
| 160 |
A set of flags that influence the behaviour and format of the database. |
| 161 |
|
| 162 |
=over |
| 163 |
|
| 164 |
=item C<DEFAULT> |
| 165 |
|
| 166 |
Same as C<0> - no flags. |
| 167 |
|
| 168 |
=item C<CLEAR_IF_FIRST> |
| 169 |
|
| 170 |
If this is the first open, wipe the db. |
| 171 |
|
| 172 |
=item C<INTERNAL> |
| 173 |
|
| 174 |
In-memory database only, path will be ignored. |
| 175 |
|
| 176 |
=item C<NOLOCK> |
| 177 |
|
| 178 |
Don't do any locking. |
| 179 |
|
| 180 |
=item C<NOMMAP> |
| 181 |
|
| 182 |
Don't use mmap. |
| 183 |
|
| 184 |
=item C<NOSYNC> |
| 185 |
|
| 186 |
Don't use synchronous transactions. |
| 187 |
|
| 188 |
=item C<SEQNUM> |
| 189 |
|
| 190 |
Maintain a sequence number. |
| 191 |
|
| 192 |
=item C<VOLATILE> |
| 193 |
|
| 194 |
Activate the per-hashchain freelist, default 5. |
| 195 |
|
| 196 |
=item C<ALLOW_NESTING> |
| 197 |
|
| 198 |
Allow transactions to nest. |
| 199 |
|
| 200 |
=item C<DISALLOW_NESTING> |
| 201 |
|
| 202 |
Disallow transactions to nest. |
| 203 |
|
| 204 |
=item C<INCOMPATIBLE_HASH> |
| 205 |
|
| 206 |
Better default hash functioa, but can't be opened by tdb < 1.2.6. |
| 207 |
|
| 208 |
=item C<MUTEX_LOCKING> |
| 209 |
|
| 210 |
Optimized locking using robust mutexes if supported, |
| 211 |
|
| 212 |
=back |
| 213 |
|
| 214 |
=item open_flags => $flags (default: C<Fcntl::O_RDWR | Fcntl::O_CREAT>) |
| 215 |
|
| 216 |
Standard open flags, as used in C<sysopen>. |
| 217 |
|
| 218 |
=item mode => $mode (default: C<0666>) |
| 219 |
|
| 220 |
Standard file open mode, as use din C<sysopen>. Only used when creating |
| 221 |
the database file. |
| 222 |
|
| 223 |
=item hash_size => $int (default: internal to libtdb, but normally C<131>) |
| 224 |
|
| 225 |
The size of the internal hash table - only used when creating the |
| 226 |
database. The default is usually very low and only good for a few thousand |
| 227 |
keys. As a rule of thumb, it should be at least one percent of the number |
| 228 |
of keys you plan to store, e.g. for C<800000> keys you should use around a |
| 229 |
size of C<8000>. |
| 230 |
|
| 231 |
Every hash entry is only 4 octets, os usually it isn't an issue to make the |
| 232 |
hash table too large. |
| 233 |
|
| 234 |
To give you an idea of the performance, I inserted about 600000 records, |
| 235 |
3GB of data, into a TDB file, using different hash sizes. |
| 236 |
|
| 237 |
size time |
| 238 |
131 160s |
| 239 |
256 85s |
| 240 |
1024 12s |
| 241 |
4096 5s |
| 242 |
8192 4s |
| 243 |
|
| 244 |
As you can see, the default hash table size for this case caused it to |
| 245 |
use 40 times the time to insert than a larger hash table, the optimum |
| 246 |
being around 75 keys per hash entry. But the databse easily was cached in |
| 247 |
memory. If that is not the case, you might want to consider a hash table |
| 248 |
that is larger than the number of keys you want to store - each hash slot |
| 249 |
only uses 4 octets. |
| 250 |
|
| 251 |
=item log_cb => $cb->($level, $msg) |
| 252 |
|
| 253 |
Sets a code reference that is called with a message and a log level |
| 254 |
(lower means more important, there are C<DEBUG_FATAL>, C<DEBUG_ERROR>, |
| 255 |
C<DEBUG_WARNING> and C<DEBUG_TRACE}>). Unlike the "debug" in the name |
| 256 |
might indicate, if you want to find out why, for instance, you could not |
| 257 |
open a database, you need to use a logging callback. |
| 258 |
|
| 259 |
=item hash => $hash (default: C<undef>) |
| 260 |
|
| 261 |
Selects a hash function to use. |
| 262 |
|
| 263 |
=over |
| 264 |
|
| 265 |
=item C<"default"> or C<undef> |
| 266 |
|
| 267 |
Use autodetection and use either C<"jenkins"> or the original tdb hash function. |
| 268 |
|
| 269 |
=item C<"jenkins"> |
| 270 |
|
| 271 |
The jenkins hash function. Relatively fast, and recommended for modern tdb databases |
| 272 |
that need to be interoperable between implementations. |
| 273 |
|
| 274 |
=item C<"fnv1ax"> |
| 275 |
|
| 276 |
The FNV1-A hash with 32 bit post-mixing. Pretty good and very fast for |
| 277 |
keys up to 10-20 octets. |
| 278 |
|
| 279 |
=item C<"xxh3"> |
| 280 |
|
| 281 |
The XXH3 hash. Very good and fast especially for long keys. |
| 282 |
|
| 283 |
=item C<1> .. C<4> |
| 284 |
|
| 285 |
Specify one of up to four custom hash functions (see L<set_hash_function>). |
| 286 |
|
| 287 |
=back |
| 288 |
|
| 289 |
=item mutex => $bool (default: C<0>) |
| 290 |
|
| 291 |
TDB can take advantage of fast interprocess mutexes, which can be |
| 292 |
orders of magnitude faster than the syscall-based locking used by |
| 293 |
default, but only works on the same machine. |
| 294 |
|
| 295 |
Normally, you need to call C<TDB_FileX::runtime_check_for_robust_mutexes> |
| 296 |
and set the C<MUTEX_LOCKING> flag if support is indicated. |
| 297 |
|
| 298 |
This option, when enabled, enables C<MUTEX_LOCKING> if it is supported |
| 299 |
by the platform - which involves a fork when opening the database. When |
| 300 |
disabled, it will remove the flag. |
| 301 |
|
| 302 |
It is recommended to keep this one, but enabling this changes the format |
| 303 |
of the database, so might not be an option if interoperability with other |
| 304 |
programs is required. |
| 305 |
|
| 306 |
=item nocow => $bool (default: C<0>) |
| 307 |
|
| 308 |
Set the no-copy-on-write flag I<iff> this flag is true, C<open_flags> |
| 309 |
contains C<O_CREAT>, C<tdb_flags> do not contain C<INTERNAL> and this is |
| 310 |
supported on the platform and filesystem. Copy-on-write filesystems have |
| 311 |
to make a copy of every block written to, which can both be costly and can |
| 312 |
cause massive fragmentation of the database file. |
| 313 |
|
| 314 |
Setting the no-copy-on-write flag (same as C<chattr +C>) disables this, |
| 315 |
usually at the expense of data protection (checksumming), reducing the |
| 316 |
safety to the level of a normal filesystem such as ext4. |
| 317 |
|
| 318 |
This is best effort and usually only takes effect when the |
| 319 |
database is initially created. If it fails, TDB_FileX will simply |
| 320 |
continue. Correctness should not be affected either way. |
| 321 |
|
| 322 |
=back |
| 323 |
|
| 324 |
There is no explicit close function. The database is closed implicitly |
| 325 |
when there are no remaining references. |
| 326 |
|
| 327 |
=item $tdb->store ($key, $value[, $flag=REPLACE]) [CROAK] |
| 328 |
|
| 329 |
Store $value in the database with key w$key. The $flag defaults to |
| 330 |
C<REPLACE>, but can also be C<INSERT> or C<MODIFY>, see tdb_store(3) for |
| 331 |
details. |
| 332 |
|
| 333 |
=item $tdb->append ($key, $value) [CROAK] |
| 334 |
|
| 335 |
Appends $data to the data already stored for $key, or creates a new entry with it. |
| 336 |
|
| 337 |
=item $data = $tdb->fetch ($key) [CROAK] |
| 338 |
|
| 339 |
Fetch the value associated with $key, or C<undef> if it is not found (or |
| 340 |
on any error). |
| 341 |
|
| 342 |
=item $tdb->delete ($key) [CROAK] |
| 343 |
|
| 344 |
Delete the value associated with $key. |
| 345 |
|
| 346 |
=item $bool = $tdb->exists ($key) |
| 347 |
|
| 348 |
Return true if the $key is found, false otherwise. |
| 349 |
|
| 350 |
=item $key = $tdb->firstkey |
| 351 |
|
| 352 |
Return the key of the first value in the database. Returns C<undef> on |
| 353 |
failure or if there are no keys in the database. See tdb_firstkey(3) |
| 354 |
for details. |
| 355 |
|
| 356 |
=item $key = $tdb->nextkey ($lastkey) |
| 357 |
|
| 358 |
Return the next key in the database after $lastkey. Returns C<undef> on |
| 359 |
failure or if there are no more keys in the database. See tdb_nextkey(3) |
| 360 |
for details. |
| 361 |
|
| 362 |
=item $code = $tdb->error |
| 363 |
|
| 364 |
Returns the current error state of the C<$tdb> object. See the list of |
| 365 |
error codes given in L<EXPORTS>. |
| 366 |
|
| 367 |
=item $mesage = $tdb->errorstr |
| 368 |
|
| 369 |
Returns a printable string that describes the error state of the |
| 370 |
database. |
| 371 |
|
| 372 |
=item $tdb->reopen [CROAK] |
| 373 |
|
| 374 |
Closes and reopens the database. Required after a |
| 375 |
L<fork|perlfunc/fork>, if both processes wish to use the database. |
| 376 |
|
| 377 |
B<NB:> If C<reopen> fails, then it is unsafe to call any further methods |
| 378 |
on C<$tdb>. Thus, the only way to find out I<why> C<reopen> failed is to |
| 379 |
use a logging function. |
| 380 |
|
| 381 |
=item TDB_FileX::reopen_all [CROAK] |
| 382 |
|
| 383 |
Closes and reopens all open databases. See C<reopen>. |
| 384 |
|
| 385 |
B<NB:> If C<reopen_all> fails, there is no indication of I<which> C<$tdb> |
| 386 |
objects failed or why. If you have to survive failures, you may wish to do |
| 387 |
your own C<reopen> loop instead. |
| 388 |
|
| 389 |
=item $tdb->traverse ($cb->($key, $data)) [CROAK] |
| 390 |
|
| 391 |
Call $cb for each entry in the database. The callback should return false |
| 392 |
to continue, or a true value to abort the traversal. |
| 393 |
|
| 394 |
The callback is called with the key and value as arguments and should |
| 395 |
return a false value if you wish to continue traversal, and a true value |
| 396 |
if the traversal should be aborted. |
| 397 |
|
| 398 |
C<traverse> returns the number of elements traversed. If $cb is C<undef>, |
| 399 |
then this function simply counts the number of elements. |
| 400 |
|
| 401 |
=item $tdb->traverse_read ($cb->($key, $data)) [CROAK] |
| 402 |
|
| 403 |
Like C<traverse>, but only acquires a read lock. |
| 404 |
|
| 405 |
=item $tdb->set_logging_function ($cb->($level, $msg)) |
| 406 |
|
| 407 |
Set the logging function to use when this database object encounters |
| 408 |
errors. |
| 409 |
|
| 410 |
$cb is called with the severity level (an integer) and the message (a |
| 411 |
string). |
| 412 |
|
| 413 |
=item $tdb->lockall [CROAK] |
| 414 |
|
| 415 |
Lock an entire database with an exclusive write lock, returning false on |
| 416 |
error. The purpose of this call is to avoid locking overhead for many |
| 417 |
operations, but the database has to be unlocked manually when done. |
| 418 |
|
| 419 |
=item $tdb->unlockall [CROAK] |
| 420 |
|
| 421 |
Unlock an entire database previously locked with |
| 422 |
C<lockall>. |
| 423 |
|
| 424 |
=item $tdb->lockall_read [CROAK] |
| 425 |
|
| 426 |
Same as C<lockall>, but uses a shared read lock instead of a writer lock. |
| 427 |
|
| 428 |
=item $tdb->unlockall_read [CROAK] |
| 429 |
|
| 430 |
Opposite of C<lockall_read>. |
| 431 |
|
| 432 |
=item $tdb->lockall_mark [CROAK] |
| 433 |
|
| 434 |
=item $tdb->lockall_unmark [CROAK] |
| 435 |
|
| 436 |
These apparently mark and unmark locks internally, but do not actually do |
| 437 |
locking. Probably you should not use this, but feel free to tell me when |
| 438 |
these are useful. |
| 439 |
|
| 440 |
=item $tdb->lockall_nonblock [CROAK] |
| 441 |
|
| 442 |
=item $success = $tdb->lockall_read_nonblock [CROAK] |
| 443 |
|
| 444 |
Try to lock, but instead of waiting, fail if the lock could not |
| 445 |
be acquired. Returns true if the lock could be acquired, false |
| 446 |
otherwise. Croaks on all other errors. |
| 447 |
|
| 448 |
=item $tdb->transaction_start [CROAK] |
| 449 |
|
| 450 |
Starts a transaction - all operations will be queued, but not applied |
| 451 |
to the database, until the transaction is either committed C<transaction_commit> |
| 452 |
or aborted/thrown away, with C<transaction_cancel>. |
| 453 |
|
| 454 |
=item $tdb->transaction_start_nonblock [CROAK] |
| 455 |
|
| 456 |
Tries to start a transaction, but instead of waiting, fail if the lock |
| 457 |
could not be acquired. Returns true if the transaction could be started, |
| 458 |
false otherwise. Croaks on all other errors. |
| 459 |
|
| 460 |
Please tell me what this does. |
| 461 |
|
| 462 |
=item $tdb->transaction_commit [CROAK] |
| 463 |
|
| 464 |
Applies all changes in the transaction. |
| 465 |
|
| 466 |
=item $tdb->transaction_cancel [CROAK] |
| 467 |
|
| 468 |
Throws away all changes in the transaction. |
| 469 |
|
| 470 |
=item $tdb->transaction_prepare_commit [CROAK] |
| 471 |
|
| 472 |
Instead of calling C<transaction_commit> you can do the commit in |
| 473 |
two phases by calling this method before commit, which does the expensive |
| 474 |
steps first. |
| 475 |
|
| 476 |
=item $bool = $tdb->transaction_active |
| 477 |
|
| 478 |
Returns true if a transaction is currently active. |
| 479 |
|
| 480 |
=item $tdb->enable_seqnum |
| 481 |
|
| 482 |
Enables sequence number generatiuon supporet for the database. |
| 483 |
|
| 484 |
=item $seq = $tdb->get_seqnum |
| 485 |
|
| 486 |
Returns the current sequence number. Internally, this is a 32 bit unsigned |
| 487 |
integer, but the API converts it into a native integer, so the same |
| 488 |
internal sequence number might be represented differently on different |
| 489 |
machines. |
| 490 |
|
| 491 |
=item $tdb->increment_seqnum_nonblock |
| 492 |
|
| 493 |
Increments the sequence number. Note that the sequence number is also |
| 494 |
incremented by TDB itself oon many operations. |
| 495 |
|
| 496 |
=item $size = $tdb->hash_size |
| 497 |
|
| 498 |
Returns the size of the hash table. The size cannot be changed other than |
| 499 |
by recreating the database. |
| 500 |
|
| 501 |
=item $octets = $tdb->map_size |
| 502 |
|
| 503 |
Returns the current mmap size for the database. |
| 504 |
|
| 505 |
=item $flags = $tdb->get_flags |
| 506 |
|
| 507 |
Returns the flags for the database (the same as the C<tdb_flags> in C<open>). |
| 508 |
|
| 509 |
=item $tdb->add_flags ($flag) |
| 510 |
|
| 511 |
Tried to add flags to the database - yes, the parameter says C<$flag> (singular) |
| 512 |
but the documentation and the code say I<flags> (plural). |
| 513 |
|
| 514 |
=item $tdb->remove_flags ($flag) |
| 515 |
|
| 516 |
Attempts to remove flags from the database. |
| 517 |
|
| 518 |
=item $fileno = $tdb->fd |
| 519 |
|
| 520 |
Returns the file descriptor (not file handle) for the underlying database file. |
| 521 |
|
| 522 |
=item $path = $tdb->name |
| 523 |
|
| 524 |
Returns the path used to open the database. |
| 525 |
|
| 526 |
=item $tdb->wipe_all [CROAK] |
| 527 |
|
| 528 |
Efficientlly deletes all entries in the database. This does not shrink the |
| 529 |
file itself. |
| 530 |
|
| 531 |
=item $tdb->repack [CROAK] |
| 532 |
|
| 533 |
Tries to improve layout of the database by copying all items into a |
| 534 |
temporary in-memory database, wiping the database, and copying all items |
| 535 |
back. Yes, everything must fit into memory. |
| 536 |
|
| 537 |
=item $bool = $tdb->check ($cb->($key, $value)) |
| 538 |
|
| 539 |
Does extensive checks on the database, optionally (if not C<undef>) |
| 540 |
calling a check function for each pair, which must return true if the data |
| 541 |
is valid. |
| 542 |
|
| 543 |
Returns a boolean indicating whether the database was found healthy and |
| 544 |
all the calls to the callback returned true. |
| 545 |
|
| 546 |
=item $bool = $tdb->rescue ($cb->($key, $value)) |
| 547 |
|
| 548 |
Tries to recover some or all key-value pairs from a potentially damanged |
| 549 |
database file. For each recovered pair it calls the given callback. |
| 550 |
|
| 551 |
Returns a boolean indicating whether the database was found healthy and |
| 552 |
all the calls to the callback returned true. |
| 553 |
|
| 554 |
=item $bool = TDB_FileX::runtime_check_for_robust_mutexes |
| 555 |
|
| 556 |
Tests whether robust mutexes are available for locking. This involves |
| 557 |
forking the process, so it can be costly and problematic. This function |
| 558 |
needs to be called before using the C<MUTEX_LOCKING> flag. But see the |
| 559 |
C<mutex> parametrer to C<open> for an alternative. |
| 560 |
|
| 561 |
=item $tdb->dump_all |
| 562 |
|
| 563 |
Dump the records and freelist to STDOUT in an almost human readable |
| 564 |
form. |
| 565 |
|
| 566 |
=item $summary = $tdb->summary |
| 567 |
|
| 568 |
Return a textual summary of the database. The format isn't documented, |
| 569 |
but for some random databas,e I got this output: |
| 570 |
|
| 571 |
Size of file/data: 325001216/238324906 |
| 572 |
Header offset/logical size: 4001792/320999424 |
| 573 |
Number of records: 117209 |
| 574 |
Incompatible hash: no |
| 575 |
Active/supported feature flags: 0x00000001/0x00000001 |
| 576 |
Robust mutexes locking: yes |
| 577 |
Smallest/average/largest keys: 11/36/102 |
| 578 |
Smallest/average/largest data: 10/1997/2921430 |
| 579 |
Smallest/average/largest padding: 9/530/749374 |
| 580 |
Number of dead records: 0 |
| 581 |
Smallest/average/largest dead records: 0/0/0 |
| 582 |
Number of free records: 1126 |
| 583 |
Smallest/average/largest free records: 12/15312/14681136 |
| 584 |
Number of hash chains: 100000 |
| 585 |
Smallest/average/largest hash chains: 0/1/8 |
| 586 |
Number of uncoalesced records: 0 |
| 587 |
Smallest/average/largest uncoalesced runs: 0/0/0 |
| 588 |
Percentage keys/data/padding/free/dead/rechdrs&tailers/hashes: 1/72/19/5/0/1/0 |
| 589 |
|
| 590 |
=item $octets = $tdb->freelist_size |
| 591 |
|
| 592 |
Returns the total number of free (unused) octets in the file. |
| 593 |
|
| 594 |
=item $tdb->printfreelist |
| 595 |
|
| 596 |
Dump the freelist to STDOUT. |
| 597 |
|
| 598 |
=back |
| 599 |
|
| 600 |
=head2 EXPORTS |
| 601 |
|
| 602 |
Nothing constants are exported by default. |
| 603 |
|
| 604 |
The tag C<:all> exports allpo of the constants. |
| 605 |
|
| 606 |
Individually or with the tag C<:flags>: |
| 607 |
|
| 608 |
DEFAULT |
| 609 |
CLEAR_IF_FIRST |
| 610 |
INTERNAL |
| 611 |
NOLOCK |
| 612 |
NOMMAP |
| 613 |
CONVERT |
| 614 |
BIGENDIAN |
| 615 |
NOSYNC |
| 616 |
SEQNUM |
| 617 |
VOLATILE |
| 618 |
ALLOW_NESTING |
| 619 |
DISALLOW_NESTING |
| 620 |
INCOMPATIBLE_HASH |
| 621 |
MUTEX_LOCKING |
| 622 |
|
| 623 |
Individually or with the tag C<:insert>: |
| 624 |
|
| 625 |
REPLACE |
| 626 |
INSERT |
| 627 |
MODIFY |
| 628 |
|
| 629 |
Individually or with the tag C<:error>: |
| 630 |
|
| 631 |
SUCCESS |
| 632 |
ERR_CORRUPT |
| 633 |
ERR_IO |
| 634 |
ERR_LOCK |
| 635 |
ERR_OOM |
| 636 |
ERR_EXISTS |
| 637 |
ERR_NOLOCK |
| 638 |
ERR_LOCK_TIMEOUT |
| 639 |
ERR_NOEXIST |
| 640 |
ERR_EINVAL |
| 641 |
ERR_RDONLY |
| 642 |
|
| 643 |
Individually or with the tag C<:debug>: |
| 644 |
|
| 645 |
DEBUG_FATAL |
| 646 |
DEBUG_ERROR |
| 647 |
DEBUG_WARNING |
| 648 |
DEBUG_TRACE |
| 649 |
|
| 650 |
=head1 UNICODE HANDLING |
| 651 |
|
| 652 |
TDB databases can only store octet strings. Unlike most other database |
| 653 |
interfaces, TDB_FileX will safely handle Perl strings by downgrading |
| 654 |
them. Perl will warn about strings that cnanot be downgraded. |
| 655 |
|
| 656 |
=head1 DISK USAGE |
| 657 |
|
| 658 |
TDB databses use 24 octets for every key-value pair, plus the octet size |
| 659 |
of the key and sata, e.g. the pair "key" => "value" takes up 24+3+5 octets |
| 660 |
on disk. |
| 661 |
|
| 662 |
The database header is 168 octets (if I haven't miscounted). |
| 663 |
|
| 664 |
Each hashtable entry is 4 octets, and one more than the hash size is |
| 665 |
allocated, so the hash table size is (hash_size + 1) * 4. |
| 666 |
|
| 667 |
=head1 LIMITATIONS |
| 668 |
|
| 669 |
=head2 Database Size |
| 670 |
|
| 671 |
As far as I can see, TDB dastabases are limited to 4 GB. |
| 672 |
|
| 673 |
=head2 Hash Functions |
| 674 |
|
| 675 |
Hash functioons need to be set globally - they are limited to a maximum of |
| 676 |
4, but this can be easily extended, but requires source code editing. This |
| 677 |
is due to a limitation of the TDB C API. |
| 678 |
|
| 679 |
=head2 No recoivery after failed C<reopoen_all> |
| 680 |
|
| 681 |
There is no way to survive an error during C<reopen_all>. |
| 682 |
Unfortunately this is a limitation in the TDB C API. |
| 683 |
|
| 684 |
=head1 SEE ALSO |
| 685 |
|
| 686 |
tdb(3), L<perltie>. |
| 687 |
|
| 688 |
=head1 AUTHOR |
| 689 |
|
| 690 |
Angus Lees, E<lt>gus@inodes.org> |
| 691 |
|
| 692 |
Currently maintained by Marc A. Lehmann <schmorp@schmorp.de> |
| 693 |
http://home.schmorp.de/ |
| 694 |
|
| 695 |
=cut |
