ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/README
(Generate patch)

Comparing BDB/README (file contents):
Revision 1.3 by root, Tue Dec 4 10:14:34 2007 UTC vs.
Revision 1.5 by root, Wed Dec 12 01:20:54 2007 UTC

1NAME 1NAME
2 BDB - Asynchronous Berkeley DB access 2 BDB - Asynchronous Berkeley DB access
3 3
4SYNOPSIS 4SYNOPSIS
5 use BDB; 5 use BDB;
6
7 my $env = db_env_create;
8
9 mkdir "bdtest", 0700;
10 db_env_open
11 $env,
12 "bdtest",
13 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL
14 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
15 0600;
16
17 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
18
19 my $db = db_create $env;
20 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE
21 | BDB::READ_UNCOMMITTED, 0600;
22 db_put $db, undef, "key", "data", 0, sub {
23 db_del $db, undef, "key";
24 };
25 db_sync $db;
26
27 # automatic result processing with AnyEvent:
28 our $FH; open $FH, "<&=" . BDB::poll_fileno;
29 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
30
31 # automatic result processing with EV:
32 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
33
34 # with Glib:
35 add_watch Glib::IO BDB::poll_fileno,
36 in => sub { BDB::poll_cb; 1 };
37
38 # or simply flush manually
39 BDB::flush;
6 40
7DESCRIPTION 41DESCRIPTION
8 See the BerkeleyDB documentation 42 See the BerkeleyDB documentation
9 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht 43 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht
10 ml>). The BDB API is very similar to the C API (the translation has been 44 ml>). The BDB API is very similar to the C API (the translation has been
53 either do nothing or result in a runtime error). 87 either do nothing or result in a runtime error).
54 88
55 BERKELEYDB FUNCTIONS 89 BERKELEYDB FUNCTIONS
56 All of these are functions. The create functions simply return a new 90 All of these are functions. The create functions simply return a new
57 object and never block. All the remaining functions all take an optional 91 object and never block. All the remaining functions all take an optional
58 callback as last argument. If it is missing, then the fucntion will be 92 callback as last argument. If it is missing, then the function will be
59 executed synchronously. 93 executed synchronously. In both cases, $! will reflect the return value
94 of the function.
60 95
61 BDB functions that cannot block (mostly functions that manipulate 96 BDB functions that cannot block (mostly functions that manipulate
62 settings) are method calls on the relevant objects, so the rule of thumb 97 settings) are method calls on the relevant objects, so the rule of thumb
63 is: if its a method, its not blocking, if its a function, it takes a 98 is: if its a method, its not blocking, if its a function, it takes a
64 callback as last argument. 99 callback as last argument.
98 133
99 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef) 134 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
100 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE 135 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
101 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 136 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
102 flags: DB_NOSYNC 137 flags: DB_NOSYNC
138 db_upgrade (DB *db, octetstring file, U32 flags = 0, SV *callback = &PL_sv_undef)
103 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef) 139 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
104 flags: FREELIST_ONLY FREE_SPACE 140 flags: FREELIST_ONLY FREE_SPACE
105 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 141 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
106 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef) 142 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
107 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 143 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
193 $int = $env->set_lk_max_lockers (U32 max) 229 $int = $env->set_lk_max_lockers (U32 max)
194 $int = $env->set_lk_max_locks (U32 max) 230 $int = $env->set_lk_max_locks (U32 max)
195 $int = $env->set_lk_max_objects (U32 max) 231 $int = $env->set_lk_max_objects (U32 max)
196 $int = $env->set_lg_bsize (U32 max) 232 $int = $env->set_lg_bsize (U32 max)
197 $int = $env->set_lg_max (U32 max) 233 $int = $env->set_lg_max (U32 max)
234 $int = $env->mutex_set_increment (U32 increment)
235 $int = $env->mutex_set_tas_spins (U32 tas_spins)
236 $int = $env->mutex_set_max (U32 max)
237 $int = $env->mutex_set_align (U32 align)
198 238
199 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 239 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
200 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 240 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
201 241
202 Example: 242 Example:
322 db_sequence_open $seq, undef, "seq", BDB::CREATE; 362 db_sequence_open $seq, undef, "seq", BDB::CREATE;
323 db_sequence_get $seq, undef, 1, my $value; 363 db_sequence_get $seq, undef, 1, my $value;
324 364
325 SUPPORT FUNCTIONS 365 SUPPORT FUNCTIONS
326 EVENT PROCESSING AND EVENT LOOP INTEGRATION 366 EVENT PROCESSING AND EVENT LOOP INTEGRATION
367 $msg = BDB::strerror [$errno]
368 Returns the string corresponding to the given errno value. If no
369 argument is given, use $!.
370
327 $fileno = BDB::poll_fileno 371 $fileno = BDB::poll_fileno
328 Return the *request result pipe file descriptor*. This filehandle 372 Return the *request result pipe file descriptor*. This filehandle
329 must be polled for reading by some mechanism outside this module 373 must be polled for reading by some mechanism outside this module
330 (e.g. Event or select, see below or the SYNOPSIS). If the pipe 374 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
331 becomes readable you have to call "poll_cb" to check the results. 375 becomes readable you have to call "poll_cb" to check the results.
367 interactiveness when perl is not fast enough to process all requests 411 interactiveness when perl is not fast enough to process all requests
368 in time. 412 in time.
369 413
370 For interactive programs, values such as 0.01 to 0.1 should be fine. 414 For interactive programs, values such as 0.01 to 0.1 should be fine.
371 415
372 Example: Install an Event watcher that automatically calls 416 Example: Install an EV watcher that automatically calls BDB::poll_cb
373 BDB::poll_cb with low priority, to ensure that other parts of the 417 with low priority, to ensure that other parts of the program get the
374 program get the CPU sometimes even under high AIO load. 418 CPU sometimes even under high load.
375 419
376 # try not to spend much more than 0.1s in poll_cb 420 # try not to spend much more than 0.1s in poll_cb
377 BDB::max_poll_time 0.1; 421 BDB::max_poll_time 0.1;
378 422
379 # use a low priority so other tasks have priority 423 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
380 Event->io (fd => BDB::poll_fileno,
381 poll => 'r', nice => 1,
382 cb => &BDB::poll_cb);
383 424
384 BDB::poll_wait 425 BDB::poll_wait
385 If there are any outstanding requests and none of them in the result 426 If there are any outstanding requests and none of them in the result
386 phase, wait till the result filehandle becomes ready for reading 427 phase, wait till the result filehandle becomes ready for reading
387 (simply does a "select" on the filehandle. This is useful if you 428 (simply does a "select" on the filehandle. This is useful if you
396 equivalent to: 437 equivalent to:
397 438
398 BDB::poll_wait, BDB::poll_cb 439 BDB::poll_wait, BDB::poll_cb
399 440
400 BDB::flush 441 BDB::flush
401 Wait till all outstanding AIO requests have been handled. 442 Wait till all outstanding BDB requests have been handled.
402 443
403 Strictly equivalent to: 444 Strictly equivalent to:
404 445
405 BDB::poll_wait, BDB::poll_cb 446 BDB::poll_wait, BDB::poll_cb
406 while BDB::nreqs; 447 while BDB::nreqs;
407 448
408 CONTROLLING THE NUMBER OF THREADS 449 CONTROLLING THE NUMBER OF THREADS
409 BDB::min_parallel $nthreads 450 BDB::min_parallel $nthreads
410 Set the minimum number of AIO threads to $nthreads. The current 451 Set the minimum number of BDB threads to $nthreads. The current
411 default is 8, which means eight asynchronous operations can execute 452 default is 8, which means eight asynchronous operations can execute
412 concurrently at any one time (the number of outstanding requests, 453 concurrently at any one time (the number of outstanding requests,
413 however, is unlimited). 454 however, is unlimited).
414 455
415 BDB starts threads only on demand, when an AIO request is queued and 456 BDB starts threads only on demand, when an BDB request is queued and
416 no free thread exists. Please note that queueing up a hundred 457 no free thread exists. Please note that queueing up a hundred
417 requests can create demand for a hundred threads, even if it turns 458 requests can create demand for a hundred threads, even if it turns
418 out that everything is in the cache and could have been processed 459 out that everything is in the cache and could have been processed
419 faster by a single thread. 460 faster by a single thread.
420 461
426 Under most circumstances you don't need to call this function, as 467 Under most circumstances you don't need to call this function, as
427 the module selects a default that is suitable for low to moderate 468 the module selects a default that is suitable for low to moderate
428 load. 469 load.
429 470
430 BDB::max_parallel $nthreads 471 BDB::max_parallel $nthreads
431 Sets the maximum number of AIO threads to $nthreads. If more than 472 Sets the maximum number of BDB threads to $nthreads. If more than
432 the specified number of threads are currently running, this function 473 the specified number of threads are currently running, this function
433 kills them. This function blocks until the limit is reached. 474 kills them. This function blocks until the limit is reached.
434 475
435 While $nthreads are zero, aio requests get queued but not executed 476 While $nthreads are zero, aio requests get queued but not executed
436 until the number of threads has been increased again. 477 until the number of threads has been increased again.
509 (executed, but not yet processed by poll_cb). 550 (executed, but not yet processed by poll_cb).
510 551
511 FORK BEHAVIOUR 552 FORK BEHAVIOUR
512 This module should do "the right thing" when the process using it forks: 553 This module should do "the right thing" when the process using it forks:
513 554
514 Before the fork, IO::AIO enters a quiescent state where no requests can 555 Before the fork, BDB enters a quiescent state where no requests can be
515 be added in other threads and no results will be processed. After the 556 added in other threads and no results will be processed. After the fork
516 fork the parent simply leaves the quiescent state and continues 557 the parent simply leaves the quiescent state and continues
517 request/result processing, while the child frees the request/result 558 request/result processing, while the child frees the request/result
518 queue (so that the requests started before the fork will only be handled 559 queue (so that the requests started before the fork will only be handled
519 in the parent). Threads will be started on demand until the limit set in 560 in the parent). Threads will be started on demand until the limit set in
520 the parent process has been reached again. 561 the parent process has been reached again.
521 562
522 In short: the parent will, after a short pause, continue as if fork had 563 In short: the parent will, after a short pause, continue as if fork had
523 not been called, while the child will act as if IO::AIO has not been 564 not been called, while the child will act as if BDB has not been used
524 used yet. 565 yet.
566
567 Win32 note: there is no fork on win32, and perls emulation of it is too
568 broken to be supported, so do not use BDB in a windows pseudo-fork,
569 better yet, switch to a more capable platform.
525 570
526 MEMORY USAGE 571 MEMORY USAGE
527 Per-request usage: 572 Per-request usage:
528 573
529 Each aio request uses - depending on your architecture - around 100-200 574 Each aio request uses - depending on your architecture - around 100-200
548 with an operating system error or DB_LOCK_NOTGRANTED, the internal 593 with an operating system error or DB_LOCK_NOTGRANTED, the internal
549 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>, 594 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
550 above. 595 above.
551 596
552SEE ALSO 597SEE ALSO
553 Coro::AIO. 598 Coro::BDB, IO::AIO.
554 599
555AUTHOR 600AUTHOR
556 Marc Lehmann <schmorp@schmorp.de> 601 Marc Lehmann <schmorp@schmorp.de>
557 http://home.schmorp.de/ 602 http://home.schmorp.de/
558 603

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines