[ViewVC] Diff of: cvs/BDB/BDB.pm

Comparing BDB/BDB.pm (file contents):
Revision 1.39 by root, Tue Jul 8 08:25:31 2008 UTC vs.
Revision 1.53 by root, Tue Oct 21 02:21:25 2008 UTC

 no warnings;
 use strict 'vars';
 use base 'Exporter';
+our $VERSION;
 BEGIN {
-   our $VERSION = '1.6';
+   $VERSION = '1.81';
    our @BDB_REQ = qw(
       db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect
       db_env_memp_sync db_env_memp_trickle db_env_dbrename db_env_dbremove
+      db_env_log_archive
       db_open db_close db_compact db_sync db_upgrade
-      db_put db_get db_pget db_del db_key_range
+      db_put db_exists db_get db_pget db_del db_key_range
       db_txn_commit db_txn_abort db_txn_finish
       db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del
       db_sequence_open db_sequence_close
       db_sequence_get db_sequence_remove
    );
    require XSLoader;
    XSLoader::load ("BDB", $VERSION);
 }
-=head2 WIN32 FILENAMES/DATABASE NAME MESS
-Perl on Win32 supports only ASCII filenames (the reason is that it abuses
-an internal flag to store wether a filename is Unicode or ANSI, but that
-flag is used for somethign else in the perl core, so there is no way to
-detect wether a filename is ANSI or Unicode-encoded). The BDB module
-tries to work around this issue by assuming that the filename is an ANSI
-filename and BDB was built for unicode support.
-=head2 BERKELEYDB FUNCTIONS
+=head1 BERKELEYDB FUNCTIONS
 All of these are functions. The create functions simply return a new
 object and never block. All the remaining functions take an optional
 callback as last argument. If it is missing, then the function will be
 executed synchronously. In both cases, C<$!> will reflect the return value
 In the following, C<$int> signifies an integer return value,
 C<bdb_filename> is a "filename" (octets on unix, madness on windows),
 C<U32> is an unsigned 32 bit integer, C<int> is some integer, C<NV> is a
 floating point value.
-The C<SV *> types are generic perl scalars (for input and output of data
+Most C<SV *> types are generic perl scalars (for input and output of data
-values), and the C<SV *callback> is the optional callback function to call
+values).
-when the request is completed.
 The various C<DB_ENV> etc. arguments are handles return by
 C<db_env_create>, C<db_create>, C<txn_begin> and so on. If they have an
 appended C<_ornull> this means they are optional and you can pass C<undef>
 for them, resulting a NULL pointer on the C level.
+The C<SV *callback> is the optional callback function to call when the
+request is completed. This last callback argument is special: the callback
+is simply the last argument passed. If there are "optional" arguments
+before the callback they can be left out. The callback itself can be left
+out or specified as C<undef>, in which case the function will be executed
+synchronously.
+For example, C<db_env_txn_checkpoint> usually is called with all integer
+arguments zero. These can be left out, so all of these specify a call
+to C<< DB_ENV->txn_checkpoint >>, to be executed asynchronously with a
+callback to be called:
+   db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
+   db_env_txn_checkpoint $db_env, 0, 0, sub { };
+   db_env_txn_checkpoint $db_env, sub { };
+While these all specify a call to C<< DB_ENV->txn_checkpoint >> to be
+executed synchronously:
+   db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
+   db_env_txn_checkpoint $db_env, 0, 0, 0;
+   db_env_txn_checkpoint $db_env, 0;
-=head3 BDB functions
+=head2 BDB functions
 Functions in the BDB namespace, exported by default:
    $env = db_env_create (U32 env_flags = 0)
       flags: RPCCLIENT
       atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
    db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
    db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
    db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef)
    db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef)
+   db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = &PL_sv_undef)
    $db = db_create (DB_ENV *env = 0, U32 flags = 0)
       flags: XA_CREATE
    db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
       flags: FREELIST_ONLY FREE_SPACE
    db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
    db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
    db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
       flags: APPEND NODUPDATA NOOVERWRITE
+   db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
    db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
       flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
    db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
       flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
    db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
    db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef)
       flags: TXN_NOSYNC
    db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
       flags: TXN_NOSYNC
-=head4 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
+=head3 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
 This is not actually a Berkeley DB function but a BDB module
 extension. The background for this exytension is: It is very annoying to
 have to check every single BDB function for error returns and provide a
 codepath out of your transaction. While the BDB module still makes this
 You can use the C<< $txn->failed >> method to check wether a transaction
 has failed in this way and abort further processing (excluding
 C<db_txn_finish>).
-=head3 DB_ENV/database environment methods
+=head2 DB_ENV/database environment methods
 Methods available on DB_ENV/$env handles:
    DESTROY (DB_ENV_ornull *env)
            CODE:
    $int = $env->set_tmp_dir (const char *dir)
    $int = $env->set_lg_dir (const char *dir)
    $int = $env->set_shm_key (long shm_key)
    $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
    $int = $env->set_flags (U32 flags, int onoff = 1)
-   $int = $env->log_set_config (U32 flags, int onoff = 1) [v4.7]
+   $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
-   $int = $env->set_intermediate_dir_mode (const char *modestring) [v4.7]
+   $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
    $env->set_errfile (FILE *errfile = 0)
    $env->set_msgfile (FILE *msgfile = 0)
    $int = $env->set_verbose (U32 which, int onoff = 1)
    $int = $env->set_encrypt (const char *password, U32 flags = 0)
    $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
    $int = $env->mutex_set_max (U32 max)
    $int = $env->mutex_set_align (U32 align)
    $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
       flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
+   $txn = $env->cdsgroup_begin; (v4.5)
-=head4 Example:
+=head3 Example:
    use AnyEvent;
    use BDB;
    our $FH; open $FH, "<&=" . BDB::poll_fileno;
       0600;
    $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
-=head3 DB/database methods
+=head2 DB/database methods
 Methods available on DB/$db handles:
    DESTROY (DB_ornull *db)
            CODE:
    $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
       flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
    $seq = $db->sequence (U32 flags = 0)
-=head4 Example:
+=head3 Example:
    my $db = db_create $env;
    db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600;
    for (1..1000) {
    db_del $db, undef, "key $_" for 1..1000;
    db_sync $db;
-=head3 DB_TXN/transaction methods
+=head2 DB_TXN/transaction methods
 Methods available on DB_TXN/$txn handles:
    DESTROY (DB_TXN_ornull *txn)
            CODE:
    $bool = $txn->failed
    # see db_txn_finish documentation, above
-=head3 DBC/cursor methods
+=head2 DBC/cursor methods
 Methods available on DBC/$dbc handles:
    DESTROY (DBC_ornull *dbc)
            CODE:
            if (dbc)
              dbc->c_close (dbc);
-   $int = $cursor->set_priority ($priority = PRIORITY_*)
+   $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
-=head4 Example:
+=head3 Example:
    my $c = $db->cursor;
    for (;;) {
       db_c_get $c, my $key, my $data, BDB::NEXT;
    }
    db_c_close $c;
-=head3 DB_SEQUENCE/sequence methods
+=head2 DB_SEQUENCE/sequence methods
 Methods available on DB_SEQUENCE/$seq handles:
    DESTROY (DB_SEQUENCE_ornull *seq)
            CODE:
    $int = $seq->set_cachesize (U32 size)
    $int = $seq->set_flags (U32 flags)
       flags: SEQ_DEC SEQ_INC SEQ_WRAP
    $int = $seq->set_range (db_seq_t min, db_seq_t max)
-=head4 Example:
+=head3 Example:
    my $seq = $db->sequence;
    db_sequence_open $seq, undef, "seq", BDB::CREATE;
    db_sequence_get $seq, undef, 1, my $value;
-=head2 SUPPORT FUNCTIONS
+=head1 SUPPORT FUNCTIONS
-=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
+=head2 EVENT PROCESSING AND EVENT LOOP INTEGRATION
 =over 4
 =item $msg = BDB::strerror [$errno]
    BDB::poll_wait, BDB::poll_cb
       while BDB::nreqs;
 =back
-=head3 VERSION CHECKING
+=head2 VERSION CHECKING
 BerkeleyDB comes in various versions, many of them have minor
 incompatibilities. This means that traditional "at least version x.x"
 checks are often not sufficient.
+Example: set the log_autoremove option in a way compatible with <v4.7 and
+v4.7. Note the use of & on the constants to avoid triggering a compiletime
+bug when the symbol isn't available.
+   $DB_ENV->set_flags      (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
+   $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
 =over 4
 =item BDB::VERSION
 The C<BDB::VERSION> function, when called without arguments, returns the
 =back
 =cut
 sub VERSION {
+   # I was dumb enough to override the VERSION method here, so let's try
+   # to fix it up.
+   if ($_[0] eq __PACKAGE__) {
+      $VERSION
+   } else {
-   if (@_ > 0) {
+      if (@_ > 0) {
-      return undef if VERSION_v lt $_[0];
+         return undef if VERSION_v lt $_[0];
-      if (@_ > 1) {
+         if (@_ > 1) {
-         return undef if VERSION_v ge $_[1];
+            return undef if VERSION_v ge $_[1];
+         }
       }
+      VERSION_v
    }
-   VERSION_v
 }
-=head3 CONTROLLING THE NUMBER OF THREADS
+=head2 CONTROLLING THE NUMBER OF THREADS
 =over 4
 =item BDB::min_parallel $nthreads
 You can still queue as many requests as you want. Therefore,
 C<max_oustsanding> is mainly useful in simple scripts (with low values) or
 as a stop gap to shield against fatal memory overflow (with large values).
-=item BDB::set_sync_prepare $cb
+=item $old_cb = BDB::set_sync_prepare $cb
 Sets a callback that is called whenever a request is created without an
 explicit callback. It has to return two code references. The first is used
 as the request callback (it should save the return status), and the second
 is called to wait until the first callback has been called (it must set
 C<$!> to the return status).
 This mechanism can be used to include BDB into other event mechanisms,
-such as L<AnyEvent::BDB> or L<Coro::BDB>.
+such as L<Coro::BDB>.
-The default implementation works like this:
+To allow other, callback-based, events to be executed while callback-less
+ones are run, you could use this sync prepare function:
    sub {
       my $status;
       (
          sub { $status = $! },
          sub { BDB::poll while !defined $status; $! = $status },
       )
    }
-It simply blocks the process till the request has finished and then sets
+It works by polling for results till the request has finished and then
-C<$!> to the return value. This means that if you don't use a callback,
+sets C<$!> to the return value. This means that if you don't use a
-BDB will simply fall back to synchronous operations.
+callback, BDB would simply fall back to synchronous operations.
+By default, or if the sync prepare function is set to C<undef>, is to
+execute callback-less BDB requests in the foreground thread, setting C<$!>
+to the return value, without polling for other events.
 =back
-=head3 STATISTICAL INFORMATION
+=head2 STATISTICAL INFORMATION
 =over 4
 =item BDB::nreqs
 =back
 =cut
-set_sync_prepare {
+set_sync_prepare (undef);
-   my $status;
-   (
-      sub {
-         $status = $!;
-      },
-      sub {
-         BDB::poll while !defined $status;
-         $! = $status;
-      },
-   )
-};
 min_parallel 8;
 END { flush }
 1;
+=head1 COMMON PITFALLS
+=head2 Unexpected Crashes
+Remember that, by default, BDB will execute requests in parallel, in
+somewhat random order. That means that it is easy to run a C<db_get>
+request on thesa me database as a concurrent C<db_close> request, leading
+to a crash, silent data corruption, eventually the next world war on
+terrorism.
+If you only ever use foreground requests (without a callback), this will
+not be an issue.
+=head2 Unexpected Freezes or Deadlocks
+Remember that, by default, BDB will execute requests in parallel, which
+easily leads to deadlocks (even concurrent put's on the same database can
+deadlock).
+You either need to run deadlock detection (and handle the resulting
+errors), or make sure only one process ever updates the database, ine one
+thread, e.g. by using only foreground requests (without a callback).
-=head2 FORK BEHAVIOUR
+=head1 FORK BEHAVIOUR
 This module should do "the right thing" when the process using it forks:
 Before the fork, BDB enters a quiescent state where no requests
 can be added in other threads and no results will be processed. After
 Win32 note: there is no fork on win32, and perls emulation of it is too
 broken to be supported, so do not use BDB in a windows pseudo-fork, better
 yet, switch to a more capable platform.
-=head2 MEMORY USAGE
+=head1 MEMORY USAGE
 Per-request usage:
 Each aio request uses - depending on your architecture - around 100-200
 bytes of memory. In addition, stat requests need a stat buffer (possibly
 In the execution phase, some aio requests require more memory for
 temporary buffers, and each thread requires a stack and other data
 structures (usually around 16k-128k, depending on the OS).
+=head1 WIN32 FILENAMES/DATABASE NAME MESS
+Perl on Win32 supports only ASCII filenames (the reason is that it abuses
+an internal flag to store wether a filename is Unicode or ANSI, but that
+flag is used for somethign else in the perl core, so there is no way to
+detect wether a filename is ANSI or Unicode-encoded). The BDB module
+tries to work around this issue by assuming that the filename is an ANSI
+filename and BDB was built for unicode support.
 =head1 KNOWN BUGS
 Known bugs will be fixed in the next release, except:
    If you use a transaction in any request, and the request returns

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing BDB/BDB.pm (file contents): Revision 1.39 by root, Tue Jul 8 08:25:31 2008 UTC vs. Revision 1.53 by root, Tue Oct 21 02:21:25 2008 UTC

Diff Legend

Comparing BDB/BDB.pm (file contents):
Revision 1.39 by root, Tue Jul 8 08:25:31 2008 UTC vs.
Revision 1.53 by root, Tue Oct 21 02:21:25 2008 UTC