ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/README
(Generate patch)

Comparing BDB/README (file contents):
Revision 1.3 by root, Tue Dec 4 10:14:34 2007 UTC vs.
Revision 1.9 by root, Sat May 10 20:23:19 2008 UTC

1NAME 1NAME
2 BDB - Asynchronous Berkeley DB access 2 BDB - Asynchronous Berkeley DB access
3 3
4SYNOPSIS 4SYNOPSIS
5 use BDB; 5 use BDB;
6
7 my $env = db_env_create;
8
9 mkdir "bdtest", 0700;
10 db_env_open
11 $env,
12 "bdtest",
13 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL
14 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
15 0600;
16
17 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
18
19 my $db = db_create $env;
20 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE
21 | BDB::READ_UNCOMMITTED, 0600;
22 db_put $db, undef, "key", "data", 0, sub {
23 db_del $db, undef, "key";
24 };
25 db_sync $db;
26
27 # when you also use Coro, management is easy:
28 use Coro::BDB;
29
30 # automatic event loop intergration with AnyEvent:
31 use AnyEvent::BDB;
32
33 # automatic result processing with EV:
34 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
35
36 # with Glib:
37 add_watch Glib::IO BDB::poll_fileno,
38 in => sub { BDB::poll_cb; 1 };
39
40 # or simply flush manually
41 BDB::flush;
6 42
7DESCRIPTION 43DESCRIPTION
8 See the BerkeleyDB documentation 44 See the BerkeleyDB documentation
9 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht 45 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht
10 ml>). The BDB API is very similar to the C API (the translation has been 46 ml>). The BDB API is very similar to the C API (the translation has been
50 Request has reached the end of its lifetime and holds no resources 86 Request has reached the end of its lifetime and holds no resources
51 anymore (except possibly for the Perl object, but its connection to 87 anymore (except possibly for the Perl object, but its connection to
52 the actual aio request is severed and calling its methods will 88 the actual aio request is severed and calling its methods will
53 either do nothing or result in a runtime error). 89 either do nothing or result in a runtime error).
54 90
91 WIN32 FILENAMES/DATABASE NAME MESS
92 Perl on Win32 supports only ASCII filenames (the reason is that it
93 abuses an internal flag to store wether a filename is Unicode or ANSI,
94 but that flag is used for somethign else in the perl core, so there is
95 no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
96 module tries to work around this issue by assuming that the filename is
97 an ANSI filename and BDB was built for unicode support.
98
55 BERKELEYDB FUNCTIONS 99 BERKELEYDB FUNCTIONS
56 All of these are functions. The create functions simply return a new 100 All of these are functions. The create functions simply return a new
57 object and never block. All the remaining functions all take an optional 101 object and never block. All the remaining functions take an optional
58 callback as last argument. If it is missing, then the fucntion will be 102 callback as last argument. If it is missing, then the function will be
59 executed synchronously. 103 executed synchronously. In both cases, $! will reflect the return value
104 of the function.
60 105
61 BDB functions that cannot block (mostly functions that manipulate 106 BDB functions that cannot block (mostly functions that manipulate
62 settings) are method calls on the relevant objects, so the rule of thumb 107 settings) are method calls on the relevant objects, so the rule of thumb
63 is: if its a method, its not blocking, if its a function, it takes a 108 is: if it's a method, it's not blocking, if it's a function, it takes a
64 callback as last argument. 109 callback as last argument.
65 110
66 In the following, $int signifies an integer return value, "octetstring" 111 In the following, $int signifies an integer return value, "octetstring"
67 is a "binary string" (i.e. a perl string with no character indices 112 is a "binary string" (i.e. a perl string with no character indices
68 >255), "U32" is an unsigned 32 bit integer, "int" is some integer, "NV" 113 >255), "U32" is an unsigned 32 bit integer, "int" is some integer, "NV"
98 143
99 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef) 144 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
100 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE 145 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
101 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 146 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
102 flags: DB_NOSYNC 147 flags: DB_NOSYNC
148 db_upgrade (DB *db, octetstring file, U32 flags = 0, SV *callback = &PL_sv_undef)
103 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef) 149 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
104 flags: FREELIST_ONLY FREE_SPACE 150 flags: FREELIST_ONLY FREE_SPACE
105 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 151 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
106 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef) 152 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
107 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 153 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
193 $int = $env->set_lk_max_lockers (U32 max) 239 $int = $env->set_lk_max_lockers (U32 max)
194 $int = $env->set_lk_max_locks (U32 max) 240 $int = $env->set_lk_max_locks (U32 max)
195 $int = $env->set_lk_max_objects (U32 max) 241 $int = $env->set_lk_max_objects (U32 max)
196 $int = $env->set_lg_bsize (U32 max) 242 $int = $env->set_lg_bsize (U32 max)
197 $int = $env->set_lg_max (U32 max) 243 $int = $env->set_lg_max (U32 max)
244 $int = $env->mutex_set_increment (U32 increment)
245 $int = $env->mutex_set_tas_spins (U32 tas_spins)
246 $int = $env->mutex_set_max (U32 max)
247 $int = $env->mutex_set_align (U32 align)
198 248
199 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 249 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
200 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 250 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
201 251
202 Example: 252 Example:
289 DESTROY (DBC_ornull *dbc) 339 DESTROY (DBC_ornull *dbc)
290 CODE: 340 CODE:
291 if (dbc) 341 if (dbc)
292 dbc->c_close (dbc); 342 dbc->c_close (dbc);
293 343
344 $int = $cursor->set_priority ($priority = PRIORITY_*)
345
294 Example: 346 Example:
295 my $c = $db->cursor; 347 my $c = $db->cursor;
296 348
297 for (;;) { 349 for (;;) {
298 db_c_get $c, my $key, my $data, BDB::NEXT; 350 db_c_get $c, my $key, my $data, BDB::NEXT;
316 flags: SEQ_DEC SEQ_INC SEQ_WRAP 368 flags: SEQ_DEC SEQ_INC SEQ_WRAP
317 $int = $seq->set_range (db_seq_t min, db_seq_t max) 369 $int = $seq->set_range (db_seq_t min, db_seq_t max)
318 370
319 Example: 371 Example:
320 my $seq = $db->sequence; 372 my $seq = $db->sequence;
321 373
322 db_sequence_open $seq, undef, "seq", BDB::CREATE; 374 db_sequence_open $seq, undef, "seq", BDB::CREATE;
323 db_sequence_get $seq, undef, 1, my $value; 375 db_sequence_get $seq, undef, 1, my $value;
324 376
325 SUPPORT FUNCTIONS 377 SUPPORT FUNCTIONS
326 EVENT PROCESSING AND EVENT LOOP INTEGRATION 378 EVENT PROCESSING AND EVENT LOOP INTEGRATION
379 $msg = BDB::strerror [$errno]
380 Returns the string corresponding to the given errno value. If no
381 argument is given, use $!.
382
383 Note that the BDB module also patches the $! variable directly, so
384 you should be able to get a bdb error string by simply stringifying
385 $!.
386
327 $fileno = BDB::poll_fileno 387 $fileno = BDB::poll_fileno
328 Return the *request result pipe file descriptor*. This filehandle 388 Return the *request result pipe file descriptor*. This filehandle
329 must be polled for reading by some mechanism outside this module 389 must be polled for reading by some mechanism outside this module
330 (e.g. Event or select, see below or the SYNOPSIS). If the pipe 390 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
331 becomes readable you have to call "poll_cb" to check the results. 391 becomes readable you have to call "poll_cb" to check the results.
367 interactiveness when perl is not fast enough to process all requests 427 interactiveness when perl is not fast enough to process all requests
368 in time. 428 in time.
369 429
370 For interactive programs, values such as 0.01 to 0.1 should be fine. 430 For interactive programs, values such as 0.01 to 0.1 should be fine.
371 431
372 Example: Install an Event watcher that automatically calls 432 Example: Install an EV watcher that automatically calls BDB::poll_cb
373 BDB::poll_cb with low priority, to ensure that other parts of the 433 with low priority, to ensure that other parts of the program get the
374 program get the CPU sometimes even under high AIO load. 434 CPU sometimes even under high load.
375 435
376 # try not to spend much more than 0.1s in poll_cb 436 # try not to spend much more than 0.1s in poll_cb
377 BDB::max_poll_time 0.1; 437 BDB::max_poll_time 0.1;
378 438
379 # use a low priority so other tasks have priority 439 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
380 Event->io (fd => BDB::poll_fileno,
381 poll => 'r', nice => 1,
382 cb => &BDB::poll_cb);
383 440
384 BDB::poll_wait 441 BDB::poll_wait
385 If there are any outstanding requests and none of them in the result 442 If there are any outstanding requests and none of them in the result
386 phase, wait till the result filehandle becomes ready for reading 443 phase, wait till the result filehandle becomes ready for reading
387 (simply does a "select" on the filehandle. This is useful if you 444 (simply does a "select" on the filehandle. This is useful if you
396 equivalent to: 453 equivalent to:
397 454
398 BDB::poll_wait, BDB::poll_cb 455 BDB::poll_wait, BDB::poll_cb
399 456
400 BDB::flush 457 BDB::flush
401 Wait till all outstanding AIO requests have been handled. 458 Wait till all outstanding BDB requests have been handled.
402 459
403 Strictly equivalent to: 460 Strictly equivalent to:
404 461
405 BDB::poll_wait, BDB::poll_cb 462 BDB::poll_wait, BDB::poll_cb
406 while BDB::nreqs; 463 while BDB::nreqs;
407 464
408 CONTROLLING THE NUMBER OF THREADS 465 CONTROLLING THE NUMBER OF THREADS
409 BDB::min_parallel $nthreads 466 BDB::min_parallel $nthreads
410 Set the minimum number of AIO threads to $nthreads. The current 467 Set the minimum number of BDB threads to $nthreads. The current
411 default is 8, which means eight asynchronous operations can execute 468 default is 8, which means eight asynchronous operations can execute
412 concurrently at any one time (the number of outstanding requests, 469 concurrently at any one time (the number of outstanding requests,
413 however, is unlimited). 470 however, is unlimited).
414 471
415 BDB starts threads only on demand, when an AIO request is queued and 472 BDB starts threads only on demand, when an BDB request is queued and
416 no free thread exists. Please note that queueing up a hundred 473 no free thread exists. Please note that queueing up a hundred
417 requests can create demand for a hundred threads, even if it turns 474 requests can create demand for a hundred threads, even if it turns
418 out that everything is in the cache and could have been processed 475 out that everything is in the cache and could have been processed
419 faster by a single thread. 476 faster by a single thread.
420 477
426 Under most circumstances you don't need to call this function, as 483 Under most circumstances you don't need to call this function, as
427 the module selects a default that is suitable for low to moderate 484 the module selects a default that is suitable for low to moderate
428 load. 485 load.
429 486
430 BDB::max_parallel $nthreads 487 BDB::max_parallel $nthreads
431 Sets the maximum number of AIO threads to $nthreads. If more than 488 Sets the maximum number of BDB threads to $nthreads. If more than
432 the specified number of threads are currently running, this function 489 the specified number of threads are currently running, this function
433 kills them. This function blocks until the limit is reached. 490 kills them. This function blocks until the limit is reached.
434 491
435 While $nthreads are zero, aio requests get queued but not executed 492 While $nthreads are zero, aio requests get queued but not executed
436 until the number of threads has been increased again. 493 until the number of threads has been increased again.
509 (executed, but not yet processed by poll_cb). 566 (executed, but not yet processed by poll_cb).
510 567
511 FORK BEHAVIOUR 568 FORK BEHAVIOUR
512 This module should do "the right thing" when the process using it forks: 569 This module should do "the right thing" when the process using it forks:
513 570
514 Before the fork, IO::AIO enters a quiescent state where no requests can 571 Before the fork, BDB enters a quiescent state where no requests can be
515 be added in other threads and no results will be processed. After the 572 added in other threads and no results will be processed. After the fork
516 fork the parent simply leaves the quiescent state and continues 573 the parent simply leaves the quiescent state and continues
517 request/result processing, while the child frees the request/result 574 request/result processing, while the child frees the request/result
518 queue (so that the requests started before the fork will only be handled 575 queue (so that the requests started before the fork will only be handled
519 in the parent). Threads will be started on demand until the limit set in 576 in the parent). Threads will be started on demand until the limit set in
520 the parent process has been reached again. 577 the parent process has been reached again.
521 578
522 In short: the parent will, after a short pause, continue as if fork had 579 In short: the parent will, after a short pause, continue as if fork had
523 not been called, while the child will act as if IO::AIO has not been 580 not been called, while the child will act as if BDB has not been used
524 used yet. 581 yet.
582
583 Win32 note: there is no fork on win32, and perls emulation of it is too
584 broken to be supported, so do not use BDB in a windows pseudo-fork,
585 better yet, switch to a more capable platform.
525 586
526 MEMORY USAGE 587 MEMORY USAGE
527 Per-request usage: 588 Per-request usage:
528 589
529 Each aio request uses - depending on your architecture - around 100-200 590 Each aio request uses - depending on your architecture - around 100-200
548 with an operating system error or DB_LOCK_NOTGRANTED, the internal 609 with an operating system error or DB_LOCK_NOTGRANTED, the internal
549 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>, 610 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
550 above. 611 above.
551 612
552SEE ALSO 613SEE ALSO
553 Coro::AIO. 614 AnyEvent::BDB (event loop integration), Coro::BDB (more natural syntax),
615 IO::AIO (nice to have).
554 616
555AUTHOR 617AUTHOR
556 Marc Lehmann <schmorp@schmorp.de> 618 Marc Lehmann <schmorp@schmorp.de>
557 http://home.schmorp.de/ 619 http://home.schmorp.de/
558 620

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines