ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/BDB.pm
(Generate patch)

Comparing BDB/BDB.pm (file contents):
Revision 1.17 by root, Tue Dec 4 10:13:50 2007 UTC vs.
Revision 1.35 by root, Sun Mar 30 08:01:58 2008 UTC

3BDB - Asynchronous Berkeley DB access 3BDB - Asynchronous Berkeley DB access
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use BDB; 7 use BDB;
8
9 my $env = db_env_create;
10
11 mkdir "bdtest", 0700;
12 db_env_open
13 $env,
14 "bdtest",
15 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL
16 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
17 0600;
18
19 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
20
21 my $db = db_create $env;
22 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE
23 | BDB::READ_UNCOMMITTED, 0600;
24 db_put $db, undef, "key", "data", 0, sub {
25 db_del $db, undef, "key";
26 };
27 db_sync $db;
28
29 # when you also use Coro, management is easy:
30 use Coro::BDB;
31
32 # automatic result processing with AnyEvent:
33 our $FH; open $FH, "<&=" . BDB::poll_fileno;
34 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
35
36 # automatic result processing with EV:
37 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
38
39 # with Glib:
40 add_watch Glib::IO BDB::poll_fileno,
41 in => sub { BDB::poll_cb; 1 };
42
43 # or simply flush manually
44 BDB::flush;
45
8 46
9=head1 DESCRIPTION 47=head1 DESCRIPTION
10 48
11See the BerkeleyDB documentation (L<http://www.oracle.com/technology/documentation/berkeley-db/db/index.html>). 49See the BerkeleyDB documentation (L<http://www.oracle.com/technology/documentation/berkeley-db/db/index.html>).
12The BDB API is very similar to the C API (the translation has been very faithful). 50The BDB API is very similar to the C API (the translation has been very faithful).
72use strict 'vars'; 110use strict 'vars';
73 111
74use base 'Exporter'; 112use base 'Exporter';
75 113
76BEGIN { 114BEGIN {
77 our $VERSION = '1.2'; 115 our $VERSION = '1.45';
78 116
79 our @BDB_REQ = qw( 117 our @BDB_REQ = qw(
80 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect 118 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect
81 db_env_memp_sync db_env_memp_trickle 119 db_env_memp_sync db_env_memp_trickle
82 db_open db_close db_compact db_sync db_put db_get db_pget db_del db_key_range 120 db_open db_close db_compact db_sync db_upgrade
121 db_put db_get db_pget db_del db_key_range
83 db_txn_commit db_txn_abort db_txn_finish 122 db_txn_commit db_txn_abort db_txn_finish
84 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del 123 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del
85 db_sequence_open db_sequence_close 124 db_sequence_open db_sequence_close
86 db_sequence_get db_sequence_remove 125 db_sequence_get db_sequence_remove
87 ); 126 );
95 134
96 require XSLoader; 135 require XSLoader;
97 XSLoader::load ("BDB", $VERSION); 136 XSLoader::load ("BDB", $VERSION);
98} 137}
99 138
139=head2 WIN32 FILENAMES/DATABASE NAME MESS
140
141Perl on Win32 supports only ASCII filenames (the reason is that it abuses
142an internal flag to store wether a filename is Unicode or ANSI, but that
143flag is used for somethign else in the perl core, so there is no way to
144detect wether a filename is ANSI or Unicode-encoded). The BDB module
145tries to work around this issue by assuming that the filename is an ANSI
146filename and BDB was built for unicode support.
147
100=head2 BERKELEYDB FUNCTIONS 148=head2 BERKELEYDB FUNCTIONS
101 149
102All of these are functions. The create functions simply return a new 150All of these are functions. The create functions simply return a new
103object and never block. All the remaining functions all take an optional 151object and never block. All the remaining functions take an optional
104callback as last argument. If it is missing, then the fucntion will be 152callback as last argument. If it is missing, then the function will be
105executed synchronously. 153executed synchronously. In both cases, C<$!> will reflect the return value
154of the function.
106 155
107BDB functions that cannot block (mostly functions that manipulate 156BDB functions that cannot block (mostly functions that manipulate
108settings) are method calls on the relevant objects, so the rule of thumb 157settings) are method calls on the relevant objects, so the rule of thumb
109is: if its a method, its not blocking, if its a function, it takes a 158is: if it's a method, it's not blocking, if it's a function, it takes a
110callback as last argument. 159callback as last argument.
111 160
112In the following, C<$int> signifies an integer return value, 161In the following, C<$int> signifies an integer return value,
113C<octetstring> is a "binary string" (i.e. a perl string with no character 162C<octetstring> is a "binary string" (i.e. a perl string with no character
114indices >255), C<U32> is an unsigned 32 bit integer, C<int> is some 163indices >255), C<U32> is an unsigned 32 bit integer, C<int> is some
145 194
146 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef) 195 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
147 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE 196 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
148 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 197 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
149 flags: DB_NOSYNC 198 flags: DB_NOSYNC
199 db_upgrade (DB *db, octetstring file, U32 flags = 0, SV *callback = &PL_sv_undef)
150 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef) 200 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
151 flags: FREELIST_ONLY FREE_SPACE 201 flags: FREELIST_ONLY FREE_SPACE
152 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 202 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
153 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef) 203 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
154 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 204 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
242 $int = $env->set_lk_max_lockers (U32 max) 292 $int = $env->set_lk_max_lockers (U32 max)
243 $int = $env->set_lk_max_locks (U32 max) 293 $int = $env->set_lk_max_locks (U32 max)
244 $int = $env->set_lk_max_objects (U32 max) 294 $int = $env->set_lk_max_objects (U32 max)
245 $int = $env->set_lg_bsize (U32 max) 295 $int = $env->set_lg_bsize (U32 max)
246 $int = $env->set_lg_max (U32 max) 296 $int = $env->set_lg_max (U32 max)
297 $int = $env->mutex_set_increment (U32 increment)
298 $int = $env->mutex_set_tas_spins (U32 tas_spins)
299 $int = $env->mutex_set_max (U32 max)
300 $int = $env->mutex_set_align (U32 align)
247 301
248 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 302 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
249 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 303 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
250 304
251=head4 Example: 305=head4 Example:
346 DESTROY (DBC_ornull *dbc) 400 DESTROY (DBC_ornull *dbc)
347 CODE: 401 CODE:
348 if (dbc) 402 if (dbc)
349 dbc->c_close (dbc); 403 dbc->c_close (dbc);
350 404
405 $int = $cursor->set_priority ($priority = PRIORITY_*)
406
351=head4 Example: 407=head4 Example:
352 408
353 my $c = $db->cursor; 409 my $c = $db->cursor;
354 410
355 for (;;) { 411 for (;;) {
388 444
389=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION 445=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
390 446
391=over 4 447=over 4
392 448
449=item $msg = BDB::strerror [$errno]
450
451Returns the string corresponding to the given errno value. If no argument
452is given, use C<$!>.
453
454Note that the BDB module also patches the C<$!> variable directly, so you
455should be able to get a bdb error string by simply stringifying C<$!>.
456
393=item $fileno = BDB::poll_fileno 457=item $fileno = BDB::poll_fileno
394 458
395Return the I<request result pipe file descriptor>. This filehandle must be 459Return the I<request result pipe file descriptor>. This filehandle must be
396polled for reading by some mechanism outside this module (e.g. Event or 460polled for reading by some mechanism outside this module (e.g. Event or
397select, see below or the SYNOPSIS). If the pipe becomes readable you have 461select, see below or the SYNOPSIS). If the pipe becomes readable you have
435interactiveness when perl is not fast enough to process all requests in 499interactiveness when perl is not fast enough to process all requests in
436time. 500time.
437 501
438For interactive programs, values such as C<0.01> to C<0.1> should be fine. 502For interactive programs, values such as C<0.01> to C<0.1> should be fine.
439 503
440Example: Install an Event watcher that automatically calls 504Example: Install an EV watcher that automatically calls
441BDB::poll_cb with low priority, to ensure that other parts of the 505BDB::poll_cb with low priority, to ensure that other parts of the
442program get the CPU sometimes even under high AIO load. 506program get the CPU sometimes even under high load.
443 507
444 # try not to spend much more than 0.1s in poll_cb 508 # try not to spend much more than 0.1s in poll_cb
445 BDB::max_poll_time 0.1; 509 BDB::max_poll_time 0.1;
446 510
447 # use a low priority so other tasks have priority 511 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
448 Event->io (fd => BDB::poll_fileno,
449 poll => 'r', nice => 1,
450 cb => &BDB::poll_cb);
451 512
452=item BDB::poll_wait 513=item BDB::poll_wait
453 514
454If there are any outstanding requests and none of them in the result 515If there are any outstanding requests and none of them in the result
455phase, wait till the result filehandle becomes ready for reading (simply 516phase, wait till the result filehandle becomes ready for reading (simply
467 528
468 BDB::poll_wait, BDB::poll_cb 529 BDB::poll_wait, BDB::poll_cb
469 530
470=item BDB::flush 531=item BDB::flush
471 532
472Wait till all outstanding AIO requests have been handled. 533Wait till all outstanding BDB requests have been handled.
473 534
474Strictly equivalent to: 535Strictly equivalent to:
475 536
476 BDB::poll_wait, BDB::poll_cb 537 BDB::poll_wait, BDB::poll_cb
477 while BDB::nreqs; 538 while BDB::nreqs;
482 543
483=over 4 544=over 4
484 545
485=item BDB::min_parallel $nthreads 546=item BDB::min_parallel $nthreads
486 547
487Set the minimum number of AIO threads to C<$nthreads>. The current 548Set the minimum number of BDB threads to C<$nthreads>. The current
488default is C<8>, which means eight asynchronous operations can execute 549default is C<8>, which means eight asynchronous operations can execute
489concurrently at any one time (the number of outstanding requests, 550concurrently at any one time (the number of outstanding requests,
490however, is unlimited). 551however, is unlimited).
491 552
492BDB starts threads only on demand, when an AIO request is queued and 553BDB starts threads only on demand, when an BDB request is queued and
493no free thread exists. Please note that queueing up a hundred requests can 554no free thread exists. Please note that queueing up a hundred requests can
494create demand for a hundred threads, even if it turns out that everything 555create demand for a hundred threads, even if it turns out that everything
495is in the cache and could have been processed faster by a single thread. 556is in the cache and could have been processed faster by a single thread.
496 557
497It is recommended to keep the number of threads relatively low, as some 558It is recommended to keep the number of threads relatively low, as some
502Under most circumstances you don't need to call this function, as the 563Under most circumstances you don't need to call this function, as the
503module selects a default that is suitable for low to moderate load. 564module selects a default that is suitable for low to moderate load.
504 565
505=item BDB::max_parallel $nthreads 566=item BDB::max_parallel $nthreads
506 567
507Sets the maximum number of AIO threads to C<$nthreads>. If more than the 568Sets the maximum number of BDB threads to C<$nthreads>. If more than the
508specified number of threads are currently running, this function kills 569specified number of threads are currently running, this function kills
509them. This function blocks until the limit is reached. 570them. This function blocks until the limit is reached.
510 571
511While C<$nthreads> are zero, aio requests get queued but not executed 572While C<$nthreads> are zero, aio requests get queued but not executed
512until the number of threads has been increased again. 573until the number of threads has been increased again.
615 676
616=head2 FORK BEHAVIOUR 677=head2 FORK BEHAVIOUR
617 678
618This module should do "the right thing" when the process using it forks: 679This module should do "the right thing" when the process using it forks:
619 680
620Before the fork, IO::AIO enters a quiescent state where no requests 681Before the fork, BDB enters a quiescent state where no requests
621can be added in other threads and no results will be processed. After 682can be added in other threads and no results will be processed. After
622the fork the parent simply leaves the quiescent state and continues 683the fork the parent simply leaves the quiescent state and continues
623request/result processing, while the child frees the request/result queue 684request/result processing, while the child frees the request/result queue
624(so that the requests started before the fork will only be handled in the 685(so that the requests started before the fork will only be handled in the
625parent). Threads will be started on demand until the limit set in the 686parent). Threads will be started on demand until the limit set in the
626parent process has been reached again. 687parent process has been reached again.
627 688
628In short: the parent will, after a short pause, continue as if fork had 689In short: the parent will, after a short pause, continue as if fork had
629not been called, while the child will act as if IO::AIO has not been used 690not been called, while the child will act as if BDB has not been used
630yet. 691yet.
692
693Win32 note: there is no fork on win32, and perls emulation of it is too
694broken to be supported, so do not use BDB in a windows pseudo-fork, better
695yet, switch to a more capable platform.
631 696
632=head2 MEMORY USAGE 697=head2 MEMORY USAGE
633 698
634Per-request usage: 699Per-request usage:
635 700
657 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>, 722 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
658 above. 723 above.
659 724
660=head1 SEE ALSO 725=head1 SEE ALSO
661 726
662L<Coro::AIO>. 727L<Coro::BDB>, L<IO::AIO>.
663 728
664=head1 AUTHOR 729=head1 AUTHOR
665 730
666 Marc Lehmann <schmorp@schmorp.de> 731 Marc Lehmann <schmorp@schmorp.de>
667 http://home.schmorp.de/ 732 http://home.schmorp.de/

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines