ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/BDB.pm
(Generate patch)

Comparing BDB/BDB.pm (file contents):
Revision 1.14 by root, Thu Sep 13 12:29:49 2007 UTC vs.
Revision 1.18 by root, Tue Dec 4 11:07:39 2007 UTC

72use strict 'vars'; 72use strict 'vars';
73 73
74use base 'Exporter'; 74use base 'Exporter';
75 75
76BEGIN { 76BEGIN {
77 our $VERSION = '1.0'; 77 our $VERSION = '1.2';
78 78
79 our @BDB_REQ = qw( 79 our @BDB_REQ = qw(
80 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect 80 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect
81 db_env_memp_sync db_env_memp_trickle 81 db_env_memp_sync db_env_memp_trickle
82 db_open db_close db_compact db_sync db_put db_get db_pget db_del db_key_range 82 db_open db_close db_compact db_sync db_put db_get db_pget db_del db_key_range
83 db_txn_commit db_txn_abort 83 db_txn_commit db_txn_abort db_txn_finish
84 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del 84 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del
85 db_sequence_open db_sequence_close 85 db_sequence_open db_sequence_close
86 db_sequence_get db_sequence_remove 86 db_sequence_get db_sequence_remove
87 ); 87 );
88 our @EXPORT = (@BDB_REQ, qw(dbreq_pri dbreq_nice db_env_create db_create)); 88 our @EXPORT = (@BDB_REQ, qw(dbreq_pri dbreq_nice db_env_create db_create));
177 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef) 177 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef)
178 flags: TXN_NOSYNC 178 flags: TXN_NOSYNC
179 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef) 179 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
180 flags: TXN_NOSYNC 180 flags: TXN_NOSYNC
181 181
182=head4 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
183
184This is not actually a Berkeley DB function but a BDB module
185extension. The background for this exytension is: It is very annoying to
186have to check every single BDB function for error returns and provide a
187codepath out of your transaction. While the BDB module still makes this
188possible, it contains the following extensions:
189
190When a transaction-protected function returns any operating system
191error (errno > 0), BDB will set the C<TXN_DEADLOCK> flag on the
192transaction. This flag is also set by Berkeley DB functions themselves
193when an operation fails with LOCK_DEADLOCK, and it causes all further
194operations on that transaction (including C<db_txn_commit>) to fail.
195
196The C<db_txn_finish> request will look at this flag, and, if it is set,
197will automatically call C<db_txn_abort> (setting errno to C<LOCK_DEADLOCK>
198if it isn't set to something else yet). If it isn't set, it will call
199C<db_txn_commit> and return the error normally.
200
201How to use this? Easy: just write your transaction normally:
202
203 my $txn = $db_env->txn_begin;
204 db_get $db, $txn, "key", my $data;
205 db_put $db, $txn, "key", $data + 1 unless $! == BDB::NOTFOUND;
206 db_txn_finish $txn;
207 die "transaction failed" if $!;
208
209That is, handle only the expected errors. If something unexpected happens
210(EIO, LOCK_NOTGRANTED or a deadlock in either db_get or db_put), then the remaining
211requests (db_put in this case) will simply be skipped (they will fail with
212LOCK_DEADLOCK) and the transaction will be aborted.
213
214You can use the C<< $txn->failed >> method to check wether a transaction
215has failed in this way and abort further processing (excluding
216C<db_txn_finish>).
217
182=head3 DB_ENV/database environment methods 218=head3 DB_ENV/database environment methods
183 219
184Methods available on DB_ENV/$env handles: 220Methods available on DB_ENV/$env handles:
185 221
186 DESTROY (DB_ENV_ornull *env) 222 DESTROY (DB_ENV_ornull *env)
196 $int = $env->set_flags (U32 flags, int onoff) 232 $int = $env->set_flags (U32 flags, int onoff)
197 $env->set_errfile (FILE *errfile = 0) 233 $env->set_errfile (FILE *errfile = 0)
198 $env->set_msgfile (FILE *msgfile = 0) 234 $env->set_msgfile (FILE *msgfile = 0)
199 $int = $env->set_verbose (U32 which, int onoff = 1) 235 $int = $env->set_verbose (U32 which, int onoff = 1)
200 $int = $env->set_encrypt (const char *password, U32 flags = 0) 236 $int = $env->set_encrypt (const char *password, U32 flags = 0)
201 $int = $env->set_timeout (NV timeout, U32 flags) 237 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
202 $int = $env->set_mp_max_openfd (int maxopenfd); 238 $int = $env->set_mp_max_openfd (int maxopenfd);
203 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep); 239 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep);
204 $int = $env->set_mp_mmapsize (int mmapsize_mb) 240 $int = $env->set_mp_mmapsize (int mmapsize_mb)
205 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT) 241 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT)
206 $int = $env->set_lk_max_lockers (U32 max) 242 $int = $env->set_lk_max_lockers (U32 max)
294 DESTROY (DB_TXN_ornull *txn) 330 DESTROY (DB_TXN_ornull *txn)
295 CODE: 331 CODE:
296 if (txn) 332 if (txn)
297 txn->abort (txn); 333 txn->abort (txn);
298 334
299 $int = $txn->set_timeout (NV timeout, U32 flags) 335 $int = $txn->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
300 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT 336 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
337
338 $bool = $txn->failed
339 # see db_txn_finish documentation, above
301 340
302 341
303=head3 DBC/cursor methods 342=head3 DBC/cursor methods
304 343
305Methods available on DBC/$dbc handles: 344Methods available on DBC/$dbc handles:
396interactiveness when perl is not fast enough to process all requests in 435interactiveness when perl is not fast enough to process all requests in
397time. 436time.
398 437
399For interactive programs, values such as C<0.01> to C<0.1> should be fine. 438For interactive programs, values such as C<0.01> to C<0.1> should be fine.
400 439
401Example: Install an Event watcher that automatically calls 440Example: Install an EV watcher that automatically calls
402BDB::poll_cb with low priority, to ensure that other parts of the 441BDB::poll_cb with low priority, to ensure that other parts of the
403program get the CPU sometimes even under high AIO load. 442program get the CPU sometimes even under high load.
404 443
405 # try not to spend much more than 0.1s in poll_cb 444 # try not to spend much more than 0.1s in poll_cb
406 BDB::max_poll_time 0.1; 445 BDB::max_poll_time 0.1;
407 446
408 # use a low priority so other tasks have priority 447 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb);
409 Event->io (fd => BDB::poll_fileno,
410 poll => 'r', nice => 1,
411 cb => &BDB::poll_cb);
412 448
413=item BDB::poll_wait 449=item BDB::poll_wait
414 450
415If there are any outstanding requests and none of them in the result 451If there are any outstanding requests and none of them in the result
416phase, wait till the result filehandle becomes ready for reading (simply 452phase, wait till the result filehandle becomes ready for reading (simply
428 464
429 BDB::poll_wait, BDB::poll_cb 465 BDB::poll_wait, BDB::poll_cb
430 466
431=item BDB::flush 467=item BDB::flush
432 468
433Wait till all outstanding AIO requests have been handled. 469Wait till all outstanding BDB requests have been handled.
434 470
435Strictly equivalent to: 471Strictly equivalent to:
436 472
437 BDB::poll_wait, BDB::poll_cb 473 BDB::poll_wait, BDB::poll_cb
438 while BDB::nreqs; 474 while BDB::nreqs;
443 479
444=over 4 480=over 4
445 481
446=item BDB::min_parallel $nthreads 482=item BDB::min_parallel $nthreads
447 483
448Set the minimum number of AIO threads to C<$nthreads>. The current 484Set the minimum number of BDB threads to C<$nthreads>. The current
449default is C<8>, which means eight asynchronous operations can execute 485default is C<8>, which means eight asynchronous operations can execute
450concurrently at any one time (the number of outstanding requests, 486concurrently at any one time (the number of outstanding requests,
451however, is unlimited). 487however, is unlimited).
452 488
453BDB starts threads only on demand, when an AIO request is queued and 489BDB starts threads only on demand, when an BDB request is queued and
454no free thread exists. Please note that queueing up a hundred requests can 490no free thread exists. Please note that queueing up a hundred requests can
455create demand for a hundred threads, even if it turns out that everything 491create demand for a hundred threads, even if it turns out that everything
456is in the cache and could have been processed faster by a single thread. 492is in the cache and could have been processed faster by a single thread.
457 493
458It is recommended to keep the number of threads relatively low, as some 494It is recommended to keep the number of threads relatively low, as some
463Under most circumstances you don't need to call this function, as the 499Under most circumstances you don't need to call this function, as the
464module selects a default that is suitable for low to moderate load. 500module selects a default that is suitable for low to moderate load.
465 501
466=item BDB::max_parallel $nthreads 502=item BDB::max_parallel $nthreads
467 503
468Sets the maximum number of AIO threads to C<$nthreads>. If more than the 504Sets the maximum number of BDB threads to C<$nthreads>. If more than the
469specified number of threads are currently running, this function kills 505specified number of threads are currently running, this function kills
470them. This function blocks until the limit is reached. 506them. This function blocks until the limit is reached.
471 507
472While C<$nthreads> are zero, aio requests get queued but not executed 508While C<$nthreads> are zero, aio requests get queued but not executed
473until the number of threads has been increased again. 509until the number of threads has been increased again.
576 612
577=head2 FORK BEHAVIOUR 613=head2 FORK BEHAVIOUR
578 614
579This module should do "the right thing" when the process using it forks: 615This module should do "the right thing" when the process using it forks:
580 616
581Before the fork, IO::AIO enters a quiescent state where no requests 617Before the fork, BDB enters a quiescent state where no requests
582can be added in other threads and no results will be processed. After 618can be added in other threads and no results will be processed. After
583the fork the parent simply leaves the quiescent state and continues 619the fork the parent simply leaves the quiescent state and continues
584request/result processing, while the child frees the request/result queue 620request/result processing, while the child frees the request/result queue
585(so that the requests started before the fork will only be handled in the 621(so that the requests started before the fork will only be handled in the
586parent). Threads will be started on demand until the limit set in the 622parent). Threads will be started on demand until the limit set in the
587parent process has been reached again. 623parent process has been reached again.
588 624
589In short: the parent will, after a short pause, continue as if fork had 625In short: the parent will, after a short pause, continue as if fork had
590not been called, while the child will act as if IO::AIO has not been used 626not been called, while the child will act as if BDB has not been used
591yet. 627yet.
592 628
593=head2 MEMORY USAGE 629=head2 MEMORY USAGE
594 630
595Per-request usage: 631Per-request usage:
609temporary buffers, and each thread requires a stack and other data 645temporary buffers, and each thread requires a stack and other data
610structures (usually around 16k-128k, depending on the OS). 646structures (usually around 16k-128k, depending on the OS).
611 647
612=head1 KNOWN BUGS 648=head1 KNOWN BUGS
613 649
614Known bugs will be fixed in the next release. 650Known bugs will be fixed in the next release, except:
651
652 If you use a transaction in any request, and the request returns
653 with an operating system error or DB_LOCK_NOTGRANTED, the internal
654 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
655 above.
615 656
616=head1 SEE ALSO 657=head1 SEE ALSO
617 658
618L<Coro::AIO>. 659L<Coro::BDB>, L<IO::AIO>.
619 660
620=head1 AUTHOR 661=head1 AUTHOR
621 662
622 Marc Lehmann <schmorp@schmorp.de> 663 Marc Lehmann <schmorp@schmorp.de>
623 http://home.schmorp.de/ 664 http://home.schmorp.de/

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines