ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/BDB.pm
(Generate patch)

Comparing BDB/BDB.pm (file contents):
Revision 1.51 by root, Mon Sep 29 02:59:08 2008 UTC vs.
Revision 1.55 by root, Sun Jan 4 10:48:15 2009 UTC

111use base 'Exporter'; 111use base 'Exporter';
112 112
113our $VERSION; 113our $VERSION;
114 114
115BEGIN { 115BEGIN {
116 $VERSION = '1.801'; 116 $VERSION = '1.82';
117 117
118 our @BDB_REQ = qw( 118 our @BDB_REQ = qw(
119 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect 119 db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect
120 db_env_memp_sync db_env_memp_trickle db_env_dbrename db_env_dbremove 120 db_env_memp_sync db_env_memp_trickle db_env_dbrename db_env_dbremove
121 db_env_log_archive
121 db_open db_close db_compact db_sync db_upgrade 122 db_open db_close db_compact db_sync db_upgrade
122 db_put db_exists db_get db_pget db_del db_key_range 123 db_put db_exists db_get db_pget db_del db_key_range
123 db_txn_commit db_txn_abort db_txn_finish 124 db_txn_commit db_txn_abort db_txn_finish
124 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del 125 db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del
125 db_sequence_open db_sequence_close 126 db_sequence_open db_sequence_close
135 136
136 require XSLoader; 137 require XSLoader;
137 XSLoader::load ("BDB", $VERSION); 138 XSLoader::load ("BDB", $VERSION);
138} 139}
139 140
140=head2 WIN32 FILENAMES/DATABASE NAME MESS
141
142Perl on Win32 supports only ASCII filenames (the reason is that it abuses
143an internal flag to store wether a filename is Unicode or ANSI, but that
144flag is used for somethign else in the perl core, so there is no way to
145detect wether a filename is ANSI or Unicode-encoded). The BDB module
146tries to work around this issue by assuming that the filename is an ANSI
147filename and BDB was built for unicode support.
148
149=head2 BERKELEYDB FUNCTIONS 141=head1 BERKELEYDB FUNCTIONS
150 142
151All of these are functions. The create functions simply return a new 143All of these are functions. The create functions simply return a new
152object and never block. All the remaining functions take an optional 144object and never block. All the remaining functions take an optional
153callback as last argument. If it is missing, then the function will be 145callback as last argument. If it is missing, then the function will be
154executed synchronously. In both cases, C<$!> will reflect the return value 146executed synchronously. In both cases, C<$!> will reflect the return value
193 185
194 db_env_txn_checkpoint $db_env, 0, 0, 0, undef; 186 db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
195 db_env_txn_checkpoint $db_env, 0, 0, 0; 187 db_env_txn_checkpoint $db_env, 0, 0, 0;
196 db_env_txn_checkpoint $db_env, 0; 188 db_env_txn_checkpoint $db_env, 0;
197 189
198=head3 BDB functions 190=head2 BDB functions
199 191
200Functions in the BDB namespace, exported by default: 192Functions in the BDB namespace, exported by default:
201 193
202 $env = db_env_create (U32 env_flags = 0) 194 $env = db_env_create (U32 env_flags = 0)
203 flags: RPCCLIENT 195 flags: RPCCLIENT
211 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST 203 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
212 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef) 204 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
213 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef) 205 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
214 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef) 206 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef)
215 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef) 207 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef)
208 db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = &PL_sv_undef)
216 209
217 $db = db_create (DB_ENV *env = 0, U32 flags = 0) 210 $db = db_create (DB_ENV *env = 0, U32 flags = 0)
218 flags: XA_CREATE 211 flags: XA_CREATE
219 212
220 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef) 213 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
253 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef) 246 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef)
254 flags: TXN_NOSYNC 247 flags: TXN_NOSYNC
255 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef) 248 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
256 flags: TXN_NOSYNC 249 flags: TXN_NOSYNC
257 250
258=head4 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef) 251=head3 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
259 252
260This is not actually a Berkeley DB function but a BDB module 253This is not actually a Berkeley DB function but a BDB module
261extension. The background for this exytension is: It is very annoying to 254extension. The background for this exytension is: It is very annoying to
262have to check every single BDB function for error returns and provide a 255have to check every single BDB function for error returns and provide a
263codepath out of your transaction. While the BDB module still makes this 256codepath out of your transaction. While the BDB module still makes this
289 282
290You can use the C<< $txn->failed >> method to check wether a transaction 283You can use the C<< $txn->failed >> method to check wether a transaction
291has failed in this way and abort further processing (excluding 284has failed in this way and abort further processing (excluding
292C<db_txn_finish>). 285C<db_txn_finish>).
293 286
287
294=head3 DB_ENV/database environment methods 288=head2 DB_ENV/database environment methods
295 289
296Methods available on DB_ENV/$env handles: 290Methods available on DB_ENV/$env handles:
297 291
298 DESTROY (DB_ENV_ornull *env) 292 DESTROY (DB_ENV_ornull *env)
299 CODE: 293 CODE:
329 323
330 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 324 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
331 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 325 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
332 $txn = $env->cdsgroup_begin; (v4.5) 326 $txn = $env->cdsgroup_begin; (v4.5)
333 327
334=head4 Example: 328=head3 Example:
335 329
336 use AnyEvent; 330 use AnyEvent;
337 use BDB; 331 use BDB;
338 332
339 our $FH; open $FH, "<&=" . BDB::poll_fileno; 333 our $FH; open $FH, "<&=" . BDB::poll_fileno;
351 0600; 345 0600;
352 346
353 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1); 347 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
354 348
355 349
356=head3 DB/database methods 350=head2 DB/database methods
357 351
358Methods available on DB/$db handles: 352Methods available on DB/$db handles:
359 353
360 DESTROY (DB_ornull *db) 354 DESTROY (DB_ornull *db)
361 CODE: 355 CODE:
387 381
388 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0) 382 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
389 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT 383 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
390 $seq = $db->sequence (U32 flags = 0) 384 $seq = $db->sequence (U32 flags = 0)
391 385
392=head4 Example: 386=head3 Example:
393 387
394 my $db = db_create $env; 388 my $db = db_create $env;
395 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600; 389 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600;
396 390
397 for (1..1000) { 391 for (1..1000) {
404 db_del $db, undef, "key $_" for 1..1000; 398 db_del $db, undef, "key $_" for 1..1000;
405 399
406 db_sync $db; 400 db_sync $db;
407 401
408 402
409=head3 DB_TXN/transaction methods 403=head2 DB_TXN/transaction methods
410 404
411Methods available on DB_TXN/$txn handles: 405Methods available on DB_TXN/$txn handles:
412 406
413 DESTROY (DB_TXN_ornull *txn) 407 DESTROY (DB_TXN_ornull *txn)
414 CODE: 408 CODE:
420 414
421 $bool = $txn->failed 415 $bool = $txn->failed
422 # see db_txn_finish documentation, above 416 # see db_txn_finish documentation, above
423 417
424 418
425=head3 DBC/cursor methods 419=head2 DBC/cursor methods
426 420
427Methods available on DBC/$dbc handles: 421Methods available on DBC/$dbc handles:
428 422
429 DESTROY (DBC_ornull *dbc) 423 DESTROY (DBC_ornull *dbc)
430 CODE: 424 CODE:
431 if (dbc) 425 if (dbc)
432 dbc->c_close (dbc); 426 dbc->c_close (dbc);
433 427
434 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6) 428 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
435 429
436=head4 Example: 430=head3 Example:
437 431
438 my $c = $db->cursor; 432 my $c = $db->cursor;
439 433
440 for (;;) { 434 for (;;) {
441 db_c_get $c, my $key, my $data, BDB::NEXT; 435 db_c_get $c, my $key, my $data, BDB::NEXT;
444 } 438 }
445 439
446 db_c_close $c; 440 db_c_close $c;
447 441
448 442
449=head3 DB_SEQUENCE/sequence methods 443=head2 DB_SEQUENCE/sequence methods
450 444
451Methods available on DB_SEQUENCE/$seq handles: 445Methods available on DB_SEQUENCE/$seq handles:
452 446
453 DESTROY (DB_SEQUENCE_ornull *seq) 447 DESTROY (DB_SEQUENCE_ornull *seq)
454 CODE: 448 CODE:
459 $int = $seq->set_cachesize (U32 size) 453 $int = $seq->set_cachesize (U32 size)
460 $int = $seq->set_flags (U32 flags) 454 $int = $seq->set_flags (U32 flags)
461 flags: SEQ_DEC SEQ_INC SEQ_WRAP 455 flags: SEQ_DEC SEQ_INC SEQ_WRAP
462 $int = $seq->set_range (db_seq_t min, db_seq_t max) 456 $int = $seq->set_range (db_seq_t min, db_seq_t max)
463 457
464=head4 Example: 458=head3 Example:
465 459
466 my $seq = $db->sequence; 460 my $seq = $db->sequence;
467 461
468 db_sequence_open $seq, undef, "seq", BDB::CREATE; 462 db_sequence_open $seq, undef, "seq", BDB::CREATE;
469 db_sequence_get $seq, undef, 1, my $value; 463 db_sequence_get $seq, undef, 1, my $value;
470 464
471 465
472=head2 SUPPORT FUNCTIONS 466=head1 SUPPORT FUNCTIONS
473 467
474=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION 468=head2 EVENT PROCESSING AND EVENT LOOP INTEGRATION
475 469
476=over 4 470=over 4
477 471
478=item $msg = BDB::strerror [$errno] 472=item $msg = BDB::strerror [$errno]
479 473
566 BDB::poll_wait, BDB::poll_cb 560 BDB::poll_wait, BDB::poll_cb
567 while BDB::nreqs; 561 while BDB::nreqs;
568 562
569=back 563=back
570 564
571=head3 VERSION CHECKING 565=head2 VERSION CHECKING
572 566
573BerkeleyDB comes in various versions, many of them have minor 567BerkeleyDB comes in various versions, many of them have minor
574incompatibilities. This means that traditional "at least version x.x" 568incompatibilities. This means that traditional "at least version x.x"
575checks are often not sufficient. 569checks are often not sufficient.
576 570
632 626
633 VERSION_v 627 VERSION_v
634 } 628 }
635} 629}
636 630
637=head3 CONTROLLING THE NUMBER OF THREADS 631=head2 CONTROLLING THE NUMBER OF THREADS
638 632
639=over 4 633=over 4
640 634
641=item BDB::min_parallel $nthreads 635=item BDB::min_parallel $nthreads
642 636
735execute callback-less BDB requests in the foreground thread, setting C<$!> 729execute callback-less BDB requests in the foreground thread, setting C<$!>
736to the return value, without polling for other events. 730to the return value, without polling for other events.
737 731
738=back 732=back
739 733
740=head3 STATISTICAL INFORMATION 734=head2 STATISTICAL INFORMATION
741 735
742=over 4 736=over 4
743 737
744=item BDB::nreqs 738=item BDB::nreqs
745 739
771 765
772END { flush } 766END { flush }
773 767
7741; 7681;
775 769
770=head1 COMMON PITFALLS
771
772=head2 Unexpected Crashes
773
774Remember that, by default, BDB will execute requests in parallel, in
775somewhat random order. That means that it is easy to run a C<db_get>
776request on thesa me database as a concurrent C<db_close> request, leading
777to a crash, silent data corruption, eventually the next world war on
778terrorism.
779
780If you only ever use foreground requests (without a callback), this will
781not be an issue.
782
783=head2 Unexpected Freezes or Deadlocks
784
785Remember that, by default, BDB will execute requests in parallel, which
786easily leads to deadlocks (even concurrent put's on the same database can
787deadlock).
788
789You either need to run deadlock detection (and handle the resulting
790errors), or make sure only one process ever updates the database, ine one
791thread, e.g. by using only foreground requests (without a callback).
792
776=head2 FORK BEHAVIOUR 793=head1 FORK BEHAVIOUR
777 794
778This module should do "the right thing" when the process using it forks: 795This module should do "the right thing" when the process using it forks:
779 796
780Before the fork, BDB enters a quiescent state where no requests 797Before the fork, BDB enters a quiescent state where no requests
781can be added in other threads and no results will be processed. After 798can be added in other threads and no results will be processed. After
791 808
792Win32 note: there is no fork on win32, and perls emulation of it is too 809Win32 note: there is no fork on win32, and perls emulation of it is too
793broken to be supported, so do not use BDB in a windows pseudo-fork, better 810broken to be supported, so do not use BDB in a windows pseudo-fork, better
794yet, switch to a more capable platform. 811yet, switch to a more capable platform.
795 812
796=head2 MEMORY USAGE 813=head1 MEMORY USAGE
797 814
798Per-request usage: 815Per-request usage:
799 816
800Each aio request uses - depending on your architecture - around 100-200 817Each aio request uses - depending on your architecture - around 100-200
801bytes of memory. In addition, stat requests need a stat buffer (possibly 818bytes of memory. In addition, stat requests need a stat buffer (possibly
810 827
811In the execution phase, some aio requests require more memory for 828In the execution phase, some aio requests require more memory for
812temporary buffers, and each thread requires a stack and other data 829temporary buffers, and each thread requires a stack and other data
813structures (usually around 16k-128k, depending on the OS). 830structures (usually around 16k-128k, depending on the OS).
814 831
832=head1 WIN32 FILENAMES/DATABASE NAME MESS
833
834Perl on Win32 supports only ASCII filenames (the reason is that it abuses
835an internal flag to store wether a filename is Unicode or ANSI, but that
836flag is used for somethign else in the perl core, so there is no way to
837detect wether a filename is ANSI or Unicode-encoded). The BDB module
838tries to work around this issue by assuming that the filename is an ANSI
839filename and BDB was built for unicode support.
840
815=head1 KNOWN BUGS 841=head1 KNOWN BUGS
816 842
817Known bugs will be fixed in the next release, except: 843Known bugs will be fixed in the next release, except:
818 844
819 If you use a transaction in any request, and the request returns 845 If you use a transaction in any request, and the request returns

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines