ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/README
(Generate patch)

Comparing BDB/README (file contents):
Revision 1.4 by root, Fri Dec 7 13:39:04 2007 UTC vs.
Revision 1.11 by root, Thu Jul 17 12:09:56 2008 UTC

… …
22 db_put $db, undef, "key", "data", 0, sub { 22 db_put $db, undef, "key", "data", 0, sub {
23 db_del $db, undef, "key"; 23 db_del $db, undef, "key";
24 }; 24 };
25 db_sync $db; 25 db_sync $db;
26 26
27 # automatic result processing with AnyEvent: 27 # when you also use Coro, management is easy:
28 our $FH; open $FH, "<&=" . BDB::poll_fileno; 28 use Coro::BDB;
29 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb); 29
30 # automatic event loop intergration with AnyEvent:
31 use AnyEvent::BDB;
30 32
31 # automatic result processing with EV: 33 # automatic result processing with EV:
32 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb; 34 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
33 35
34 # with Glib: 36 # with Glib:
… …
84 Request has reached the end of its lifetime and holds no resources 86 Request has reached the end of its lifetime and holds no resources
85 anymore (except possibly for the Perl object, but its connection to 87 anymore (except possibly for the Perl object, but its connection to
86 the actual aio request is severed and calling its methods will 88 the actual aio request is severed and calling its methods will
87 either do nothing or result in a runtime error). 89 either do nothing or result in a runtime error).
88 90
91 WIN32 FILENAMES/DATABASE NAME MESS
92 Perl on Win32 supports only ASCII filenames (the reason is that it
93 abuses an internal flag to store wether a filename is Unicode or ANSI,
94 but that flag is used for somethign else in the perl core, so there is
95 no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
96 module tries to work around this issue by assuming that the filename is
97 an ANSI filename and BDB was built for unicode support.
98
89 BERKELEYDB FUNCTIONS 99 BERKELEYDB FUNCTIONS
90 All of these are functions. The create functions simply return a new 100 All of these are functions. The create functions simply return a new
91 object and never block. All the remaining functions all take an optional 101 object and never block. All the remaining functions take an optional
92 callback as last argument. If it is missing, then the fucntion will be 102 callback as last argument. If it is missing, then the function will be
93 executed synchronously. 103 executed synchronously. In both cases, $! will reflect the return value
104 of the function.
94 105
95 BDB functions that cannot block (mostly functions that manipulate 106 BDB functions that cannot block (mostly functions that manipulate
96 settings) are method calls on the relevant objects, so the rule of thumb 107 settings) are method calls on the relevant objects, so the rule of thumb
97 is: if its a method, its not blocking, if its a function, it takes a 108 is: if it's a method, it's not blocking, if it's a function, it takes a
98 callback as last argument. 109 callback as last argument.
99 110
100 In the following, $int signifies an integer return value, "octetstring" 111 In the following, $int signifies an integer return value, "bdb_filename"
101 is a "binary string" (i.e. a perl string with no character indices 112 is a "filename" (octets on unix, madness on windows), "U32" is an
102 >255), "U32" is an unsigned 32 bit integer, "int" is some integer, "NV" 113 unsigned 32 bit integer, "int" is some integer, "NV" is a floating point
103 is a floating point value. 114 value.
104 115
105 The "SV *" types are generic perl scalars (for input and output of data 116 Most "SV *" types are generic perl scalars (for input and output of data
106 values), and the "SV *callback" is the optional callback function to 117 values).
107 call when the request is completed.
108 118
109 The various "DB_ENV" etc. arguments are handles return by 119 The various "DB_ENV" etc. arguments are handles return by
110 "db_env_create", "db_create", "txn_begin" and so on. If they have an 120 "db_env_create", "db_create", "txn_begin" and so on. If they have an
111 appended "_ornull" this means they are optional and you can pass "undef" 121 appended "_ornull" this means they are optional and you can pass "undef"
112 for them, resulting a NULL pointer on the C level. 122 for them, resulting a NULL pointer on the C level.
113 123
124 The "SV *callback" is the optional callback function to call when the
125 request is completed. This last callback argument is special: the
126 callback is simply the last argument passed. If there are "optional"
127 arguments before the callback they can be left out. The callback itself
128 can be left out or specified as "undef", in which case the function will
129 be executed synchronously.
130
131 For example, "db_env_txn_checkpoint" usually is called with all integer
132 arguments zero. These can be left out, so all of these specify a call to
133 "DB_ENV->txn_checkpoint", to be executed asynchronously with a callback
134 to be called:
135
136 db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
137 db_env_txn_checkpoint $db_env, 0, 0, sub { };
138 db_env_txn_checkpoint $db_env, sub { };
139
140 While these all specify a call to "DB_ENV->txn_checkpoint" to be
141 executed synchronously:
142
143 db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
144 db_env_txn_checkpoint $db_env, 0, 0, 0;
145 db_env_txn_checkpoint $db_env, 0;
146
114 BDB functions 147 BDB functions
115 Functions in the BDB namespace, exported by default: 148 Functions in the BDB namespace, exported by default:
116 149
117 $env = db_env_create (U32 env_flags = 0) 150 $env = db_env_create (U32 env_flags = 0)
118 flags: RPCCLIENT 151 flags: RPCCLIENT
119 152
120 db_env_open (DB_ENV *env, octetstring db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef) 153 db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef)
121 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM 154 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
122 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = &PL_sv_undef) 155 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = &PL_sv_undef)
123 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = &PL_sv_undef) 156 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
124 flags: FORCE 157 flags: FORCE
125 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = &PL_sv_undef) 158 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = &PL_sv_undef)
126 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST 159 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
127 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef) 160 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
128 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef) 161 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
162 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef)
163 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef)
129 164
130 $db = db_create (DB_ENV *env = 0, U32 flags = 0) 165 $db = db_create (DB_ENV *env = 0, U32 flags = 0)
131 flags: XA_CREATE 166 flags: XA_CREATE
132 167
133 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef) 168 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
134 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE 169 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
135 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 170 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
136 flags: DB_NOSYNC 171 flags: DB_NOSYNC
172 db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = &PL_sv_undef)
137 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef) 173 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
138 flags: FREELIST_ONLY FREE_SPACE 174 flags: FREELIST_ONLY FREE_SPACE
139 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef) 175 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
140 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef) 176 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
141 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 177 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
142 flags: APPEND NODUPDATA NOOVERWRITE 178 flags: APPEND NODUPDATA NOOVERWRITE
179 db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
143 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 180 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
144 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW 181 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
145 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef) 182 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
146 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW 183 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
147 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef) 184 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
… …
212 $int = $env->set_data_dir (const char *dir) 249 $int = $env->set_data_dir (const char *dir)
213 $int = $env->set_tmp_dir (const char *dir) 250 $int = $env->set_tmp_dir (const char *dir)
214 $int = $env->set_lg_dir (const char *dir) 251 $int = $env->set_lg_dir (const char *dir)
215 $int = $env->set_shm_key (long shm_key) 252 $int = $env->set_shm_key (long shm_key)
216 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0) 253 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
217 $int = $env->set_flags (U32 flags, int onoff) 254 $int = $env->set_flags (U32 flags, int onoff = 1)
255 $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
256 $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
218 $env->set_errfile (FILE *errfile = 0) 257 $env->set_errfile (FILE *errfile = 0)
219 $env->set_msgfile (FILE *msgfile = 0) 258 $env->set_msgfile (FILE *msgfile = 0)
220 $int = $env->set_verbose (U32 which, int onoff = 1) 259 $int = $env->set_verbose (U32 which, int onoff = 1)
221 $int = $env->set_encrypt (const char *password, U32 flags = 0) 260 $int = $env->set_encrypt (const char *password, U32 flags = 0)
222 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT) 261 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
… …
234 $int = $env->mutex_set_max (U32 max) 273 $int = $env->mutex_set_max (U32 max)
235 $int = $env->mutex_set_align (U32 align) 274 $int = $env->mutex_set_align (U32 align)
236 275
237 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 276 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
238 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 277 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
278 $txn = $env->cdsgroup_begin; (v4.5)
239 279
240 Example: 280 Example:
241 use AnyEvent; 281 use AnyEvent;
242 use BDB; 282 use BDB;
243 283
… …
327 DESTROY (DBC_ornull *dbc) 367 DESTROY (DBC_ornull *dbc)
328 CODE: 368 CODE:
329 if (dbc) 369 if (dbc)
330 dbc->c_close (dbc); 370 dbc->c_close (dbc);
331 371
372 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
373
332 Example: 374 Example:
333 my $c = $db->cursor; 375 my $c = $db->cursor;
334 376
335 for (;;) { 377 for (;;) {
336 db_c_get $c, my $key, my $data, BDB::NEXT; 378 db_c_get $c, my $key, my $data, BDB::NEXT;
… …
354 flags: SEQ_DEC SEQ_INC SEQ_WRAP 396 flags: SEQ_DEC SEQ_INC SEQ_WRAP
355 $int = $seq->set_range (db_seq_t min, db_seq_t max) 397 $int = $seq->set_range (db_seq_t min, db_seq_t max)
356 398
357 Example: 399 Example:
358 my $seq = $db->sequence; 400 my $seq = $db->sequence;
359 401
360 db_sequence_open $seq, undef, "seq", BDB::CREATE; 402 db_sequence_open $seq, undef, "seq", BDB::CREATE;
361 db_sequence_get $seq, undef, 1, my $value; 403 db_sequence_get $seq, undef, 1, my $value;
362 404
363 SUPPORT FUNCTIONS 405 SUPPORT FUNCTIONS
364 EVENT PROCESSING AND EVENT LOOP INTEGRATION 406 EVENT PROCESSING AND EVENT LOOP INTEGRATION
407 $msg = BDB::strerror [$errno]
408 Returns the string corresponding to the given errno value. If no
409 argument is given, use $!.
410
411 Note that the BDB module also patches the $! variable directly, so
412 you should be able to get a bdb error string by simply stringifying
413 $!.
414
365 $fileno = BDB::poll_fileno 415 $fileno = BDB::poll_fileno
366 Return the *request result pipe file descriptor*. This filehandle 416 Return the *request result pipe file descriptor*. This filehandle
367 must be polled for reading by some mechanism outside this module 417 must be polled for reading by some mechanism outside this module
368 (e.g. Event or select, see below or the SYNOPSIS). If the pipe 418 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
369 becomes readable you have to call "poll_cb" to check the results. 419 becomes readable you have to call "poll_cb" to check the results.
… …
438 Strictly equivalent to: 488 Strictly equivalent to:
439 489
440 BDB::poll_wait, BDB::poll_cb 490 BDB::poll_wait, BDB::poll_cb
441 while BDB::nreqs; 491 while BDB::nreqs;
442 492
493 VERSION CHECKING
494 BerkeleyDB comes in various versions, many of them have minor
495 incompatibilities. This means that traditional "at least version x.x"
496 checks are often not sufficient.
497
498 Example: set the log_autoremove option in a way compatible with <v.47
499 and v4.7. Note the use of & on the constants to avoid triggering a
500 compiletime bug when the symbol isn't available.
501
502 $DB_ENV->set_flags (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
503 $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
504
505 BDB::VERSION
506 The "BDB::VERSION" function, when called without arguments, returns
507 the Berkeley DB version as a v-string (usually with 3 components).
508 You should use "lt" and "ge" operators exclusively to make
509 comparisons.
510
511 Example: check for at least version 4.7.
512
513 BDB::VERSION ge v4.7 or die;
514
515 BDB::VERSION min-version
516 Returns true if the BDB version is at least the given version
517 (specified as a v-string), false otherwise.
518
519 Example: check for at least version 4.5.
520
521 BDB::VERSION v4.7 or die;
522
523 BDB::VERSION min-version, max-version
524 Returns true of the BDB version is at least version "min-version"
525 (specify "undef" or "v0" for any minimum version) and less then
526 "max-version".
527
528 Example: check wether version is strictly less then v4.7.
529
530 BDB::VERSION v0, v4.7
531 or die "version 4.7 is not yet supported";
532
443 CONTROLLING THE NUMBER OF THREADS 533 CONTROLLING THE NUMBER OF THREADS
444 BDB::min_parallel $nthreads 534 BDB::min_parallel $nthreads
445 Set the minimum number of BDB threads to $nthreads. The current 535 Set the minimum number of BDB threads to $nthreads. The current
446 default is 8, which means eight asynchronous operations can execute 536 default is 8, which means eight asynchronous operations can execute
447 concurrently at any one time (the number of outstanding requests, 537 concurrently at any one time (the number of outstanding requests,
… …
510 (with large values). 600 (with large values).
511 601
512 BDB::set_sync_prepare $cb 602 BDB::set_sync_prepare $cb
513 Sets a callback that is called whenever a request is created without 603 Sets a callback that is called whenever a request is created without
514 an explicit callback. It has to return two code references. The 604 an explicit callback. It has to return two code references. The
515 first is used as the request callback, and the second is called to 605 first is used as the request callback (it should save the return
516 wait until the first callback has been called. The default 606 status), and the second is called to wait until the first callback
607 has been called (it must set $! to the return status).
608
609 This mechanism can be used to include BDB into other event
610 mechanisms, such as AnyEvent::BDB or Coro::BDB.
611
517 implementation works like this: 612 The default implementation works like this:
518 613
519 sub { 614 sub {
520 my $status; 615 my $status;
521 ( 616 (
522 sub { $status = $! }, 617 sub { $status = $! },
523 sub { BDB::poll while !defined $status; $! = $status }, 618 sub { BDB::poll while !defined $status; $! = $status },
524 ) 619 )
525 } 620 }
526 621
622 It simply blocks the process till the request has finished and then
623 sets $! to the return value. This means that if you don't use a
624 callback, BDB will simply fall back to synchronous operations.
625
527 STATISTICAL INFORMATION 626 STATISTICAL INFORMATION
528 BDB::nreqs 627 BDB::nreqs
529 Returns the number of requests currently in the ready, execute or 628 Returns the number of requests currently in the ready, execute or
530 pending states (i.e. for which their callback has not been invoked 629 pending states (i.e. for which their callback has not been invoked
531 yet). 630 yet).
… …
556 655
557 In short: the parent will, after a short pause, continue as if fork had 656 In short: the parent will, after a short pause, continue as if fork had
558 not been called, while the child will act as if BDB has not been used 657 not been called, while the child will act as if BDB has not been used
559 yet. 658 yet.
560 659
660 Win32 note: there is no fork on win32, and perls emulation of it is too
661 broken to be supported, so do not use BDB in a windows pseudo-fork,
662 better yet, switch to a more capable platform.
663
561 MEMORY USAGE 664 MEMORY USAGE
562 Per-request usage: 665 Per-request usage:
563 666
564 Each aio request uses - depending on your architecture - around 100-200 667 Each aio request uses - depending on your architecture - around 100-200
565 bytes of memory. In addition, stat requests need a stat buffer (possibly 668 bytes of memory. In addition, stat requests need a stat buffer (possibly
… …
583 with an operating system error or DB_LOCK_NOTGRANTED, the internal 686 with an operating system error or DB_LOCK_NOTGRANTED, the internal
584 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>, 687 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
585 above. 688 above.
586 689
587SEE ALSO 690SEE ALSO
588 Coro::BDB, IO::AIO. 691 AnyEvent::BDB (event loop integration), Coro::BDB (more natural syntax),
692 IO::AIO (nice to have).
589 693
590AUTHOR 694AUTHOR
591 Marc Lehmann <schmorp@schmorp.de> 695 Marc Lehmann <schmorp@schmorp.de>
592 http://home.schmorp.de/ 696 http://home.schmorp.de/
593 697

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines