[ViewVC] Diff of: cvs/BDB/README

Comparing BDB/README (file contents):
Revision 1.10 by root, Tue Jul 8 08:35:12 2008 UTC vs.
Revision 1.18 by root, Thu Jan 18 16:45:27 2018 UTC

      db_sync $db;
      # when you also use Coro, management is easy:
      use Coro::BDB;
-     # automatic event loop intergration with AnyEvent:
+     # automatic event loop integration with AnyEvent:
      use AnyEvent::BDB;
      # automatic result processing with EV:
      my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
         Request has reached the end of its lifetime and holds no resources
         anymore (except possibly for the Perl object, but its connection to
         the actual aio request is severed and calling its methods will
         either do nothing or result in a runtime error).
-  WIN32 FILENAMES/DATABASE NAME MESS
-    Perl on Win32 supports only ASCII filenames (the reason is that it
-    abuses an internal flag to store wether a filename is Unicode or ANSI,
-    but that flag is used for somethign else in the perl core, so there is
-    no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
-    module tries to work around this issue by assuming that the filename is
-    an ANSI filename and BDB was built for unicode support.
-  BERKELEYDB FUNCTIONS
+BERKELEYDB FUNCTIONS
     All of these are functions. The create functions simply return a new
     object and never block. All the remaining functions take an optional
     callback as last argument. If it is missing, then the function will be
     executed synchronously. In both cases, $! will reflect the return value
     of the function.
     In the following, $int signifies an integer return value, "bdb_filename"
     is a "filename" (octets on unix, madness on windows), "U32" is an
     unsigned 32 bit integer, "int" is some integer, "NV" is a floating point
     value.
-    The "SV *" types are generic perl scalars (for input and output of data
+    Most "SV *" types are generic perl scalars (for input and output of data
-    values), and the "SV *callback" is the optional callback function to
+    values).
-    call when the request is completed.
     The various "DB_ENV" etc. arguments are handles return by
     "db_env_create", "db_create", "txn_begin" and so on. If they have an
     appended "_ornull" this means they are optional and you can pass "undef"
     for them, resulting a NULL pointer on the C level.
+    The "SV *callback" is the optional callback function to call when the
+    request is completed. This last callback argument is special: the
+    callback is simply the last argument passed. If there are "optional"
+    arguments before the callback they can be left out. The callback itself
+    can be left out or specified as "undef", in which case the function will
+    be executed synchronously.
+    For example, "db_env_txn_checkpoint" usually is called with all integer
+    arguments zero. These can be left out, so all of these specify a call to
+    "DB_ENV->txn_checkpoint", to be executed asynchronously with a callback
+    to be called:
+       db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
+       db_env_txn_checkpoint $db_env, 0, 0, sub { };
+       db_env_txn_checkpoint $db_env, sub { };
+    While these all specify a call to "DB_ENV->txn_checkpoint" to be
+    executed synchronously:
+       db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
+       db_env_txn_checkpoint $db_env, 0, 0, 0;
+       db_env_txn_checkpoint $db_env, 0;
-   BDB functions
+  BDB functions
     Functions in the BDB namespace, exported by default:
        $env = db_env_create (U32 env_flags = 0)
           flags: RPCCLIENT
-       db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef)
+       db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = 0)
           open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
-       db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = 0)
-       db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = 0)
           flags: FORCE
-       db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = &PL_sv_undef)
+       db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = 0)
           atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
-       db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
+       db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = 0)
-       db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
+       db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = 0)
-       db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = 0)
-       db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = 0)
+       db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = 0)
+       db_env_lsn_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0)
+       db_env_fileid_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0)
        $db = db_create (DB_ENV *env = 0, U32 flags = 0)
           flags: XA_CREATE
-       db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
+       db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = 0)
           flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
-       db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_close (DB *db, U32 flags = 0, SV *callback = 0)
           flags: DB_NOSYNC
+       db_verify (DB *db, bdb_filename file, bdb_filename database = 0, SV *dummy = 0, U32 flags = 0, SV *callback = 0)
-       db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = 0)
-       db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
+       db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = 0)
           flags: FREELIST_ONLY FREE_SPACE
-       db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_sync (DB *db, U32 flags = 0, SV *callback = 0)
-       db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = 0)
-       db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
           flags: APPEND NODUPDATA NOOVERWRITE
+       db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
-       db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
           flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
-       db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
           flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
-       db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0)
-       db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
           flags: TXN_NOSYNC TXN_SYNC
-       db_txn_abort (DB_TXN *txn, SV *callback = &PL_sv_undef)
+       db_txn_abort (DB_TXN *txn, SV *callback = 0)
-       db_c_close (DBC *dbc, SV *callback = &PL_sv_undef)
+       db_c_close (DBC *dbc, SV *callback = 0)
-       db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = 0)
-       db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
           flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA
-       db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0)
           flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW
-       db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0)
-       db_c_del (DBC *dbc, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_c_del (DBC *dbc, U32 flags = 0, SV *callback = 0)
-       db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = 0)
           flags: CREATE EXCL
-       db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = 0)
-       db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef)
+       db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = 0)
           flags: TXN_NOSYNC
-       db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = 0)
           flags: TXN_NOSYNC
-   db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
+   db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = 0)
     This is not actually a Berkeley DB function but a BDB module extension.
     The background for this exytension is: It is very annoying to have to
     check every single BDB function for error returns and provide a codepath
     out of your transaction. While the BDB module still makes this possible,
     it contains the following extensions:
     You can use the "$txn->failed" method to check wether a transaction has
     failed in this way and abort further processing (excluding
     "db_txn_finish").
-   DB_ENV/database environment methods
+  DB_ENV/database environment methods
     Methods available on DB_ENV/$env handles:
        DESTROY (DB_ENV_ornull *env)
                CODE:
                if (env)
        $int = $env->set_tmp_dir (const char *dir)
        $int = $env->set_lg_dir (const char *dir)
        $int = $env->set_shm_key (long shm_key)
        $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
        $int = $env->set_flags (U32 flags, int onoff = 1)
-       $int = $env->log_set_config (U32 flags, int onoff = 1) [v4.7]
+       $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
-       $int = $env->set_intermediate_dir_mode (const char *modestring) [v4.7]
+       $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
        $env->set_errfile (FILE *errfile = 0)
        $env->set_msgfile (FILE *msgfile = 0)
        $int = $env->set_verbose (U32 which, int onoff = 1)
        $int = $env->set_encrypt (const char *password, U32 flags = 0)
        $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
        $int = $env->mutex_set_max (U32 max)
        $int = $env->mutex_set_align (U32 align)
        $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
           flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
+       $txn = $env->cdsgroup_begin; (v4.5)
    Example:
        use AnyEvent;
        use BDB;
           BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
           0600;
        $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
-   DB/database methods
+  DB/database methods
     Methods available on DB/$db handles:
        DESTROY (DB_ornull *db)
                CODE:
                if (db)
        db_del $db, undef, "key $_" for 1..1000;
        db_sync $db;
-   DB_TXN/transaction methods
+  DB_TXN/transaction methods
     Methods available on DB_TXN/$txn handles:
        DESTROY (DB_TXN_ornull *txn)
                CODE:
                if (txn)
           flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
        $bool = $txn->failed
        # see db_txn_finish documentation, above
-   DBC/cursor methods
+  DBC/cursor methods
     Methods available on DBC/$dbc handles:
        DESTROY (DBC_ornull *dbc)
                CODE:
                if (dbc)
                  dbc->c_close (dbc);
-       $int = $cursor->set_priority ($priority = PRIORITY_*)
+       $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
    Example:
        my $c = $db->cursor;
        for (;;) {
           last if $!;
        }
        db_c_close $c;
-   DB_SEQUENCE/sequence methods
+  DB_SEQUENCE/sequence methods
     Methods available on DB_SEQUENCE/$seq handles:
        DESTROY (DB_SEQUENCE_ornull *seq)
                CODE:
                if (seq)
           flags: SEQ_DEC SEQ_INC SEQ_WRAP
        $int = $seq->set_range (db_seq_t min, db_seq_t max)
    Example:
        my $seq = $db->sequence;
-   db_sequence_open $seq, undef, "seq", BDB::CREATE;
+       db_sequence_open $seq, undef, "seq", BDB::CREATE;
        db_sequence_get $seq, undef, 1, my $value;
-  SUPPORT FUNCTIONS
+SUPPORT FUNCTIONS
-   EVENT PROCESSING AND EVENT LOOP INTEGRATION
+  EVENT PROCESSING AND EVENT LOOP INTEGRATION
     $msg = BDB::strerror [$errno]
         Returns the string corresponding to the given errno value. If no
         argument is given, use $!.
         Note that the BDB module also patches the $! variable directly, so
         Strictly equivalent to:
            BDB::poll_wait, BDB::poll_cb
               while BDB::nreqs;
-   VERSION CHECKING
+  VERSION CHECKING
     BerkeleyDB comes in various versions, many of them have minor
     incompatibilities. This means that traditional "at least version x.x"
     checks are often not sufficient.
-    Example: set the log_autoremove option in a way compatible with <v.47
+    Example: set the log_autoremove option in a way compatible with <v4.7
     and v4.7. Note the use of & on the constants to avoid triggering a
     compiletime bug when the symbol isn't available.
        $DB_ENV->set_flags      (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
        $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
         Example: check wether version is strictly less then v4.7.
            BDB::VERSION v0, v4.7
               or die "version 4.7 is not yet supported";
-   CONTROLLING THE NUMBER OF THREADS
+  CONTROLLING THE NUMBER OF THREADS
     BDB::min_parallel $nthreads
         Set the minimum number of BDB threads to $nthreads. The current
         default is 8, which means eight asynchronous operations can execute
         concurrently at any one time (the number of outstanding requests,
         however, is unlimited).
         You can still queue as many requests as you want. Therefore,
         "max_oustsanding" is mainly useful in simple scripts (with low
         values) or as a stop gap to shield against fatal memory overflow
         (with large values).
-    BDB::set_sync_prepare $cb
+    $old_cb = BDB::set_sync_prepare $cb
         Sets a callback that is called whenever a request is created without
         an explicit callback. It has to return two code references. The
         first is used as the request callback (it should save the return
         status), and the second is called to wait until the first callback
         has been called (it must set $! to the return status).
         This mechanism can be used to include BDB into other event
-        mechanisms, such as AnyEvent::BDB or Coro::BDB.
+        mechanisms, such as Coro::BDB.
-        The default implementation works like this:
+        To allow other, callback-based, events to be executed while
+        callback-less ones are run, you could use this sync prepare
+        function:
            sub {
               my $status;
               (
                  sub { $status = $! },
                  sub { BDB::poll while !defined $status; $! = $status },
               )
            }
-        It simply blocks the process till the request has finished and then
+        It works by polling for results till the request has finished and
-        sets $! to the return value. This means that if you don't use a
+        then sets $! to the return value. This means that if you don't use a
-        callback, BDB will simply fall back to synchronous operations.
+        callback, BDB would simply fall back to synchronous operations.
+        By default, or if the sync prepare function is set to "undef", is to
+        execute callback-less BDB requests in the foreground thread, setting
+        $! to the return value, without polling for other events.
-   STATISTICAL INFORMATION
+  STATISTICAL INFORMATION
     BDB::nreqs
         Returns the number of requests currently in the ready, execute or
         pending states (i.e. for which their callback has not been invoked
         yet).
     BDB::npending
         Returns the number of requests currently in the pending state
         (executed, but not yet processed by poll_cb).
+COMMON PITFALLS
+  Unexpected Crashes
+    Remember that, by default, BDB will execute requests in parallel, in
+    somewhat random order. That means that it is easy to run a "db_get"
+    request on the same database as a concurrent "db_close" request, leading
+    to a crash, silent data corruption, eventually the next world war on
+    terrorism.
+    If you only ever use foreground requests (without a callback), this will
+    not be an issue (unless you use threads).
+  Unexpected Freezes or Deadlocks
+    Remember that, by default, BDB will execute requests in parallel, which
+    easily leads to deadlocks (even concurrent put's on the same database
+    can deadlock).
+    You either need to run deadlock detection (and handle the resulting
+    errors), or make sure only one process ever updates the database, ine
+    one thread, e.g. by using only foreground requests (without a callback).
-  FORK BEHAVIOUR
+FORK BEHAVIOUR
     This module should do "the right thing" when the process using it forks:
     Before the fork, BDB enters a quiescent state where no requests can be
     added in other threads and no results will be processed. After the fork
     the parent simply leaves the quiescent state and continues
     Win32 note: there is no fork on win32, and perls emulation of it is too
     broken to be supported, so do not use BDB in a windows pseudo-fork,
     better yet, switch to a more capable platform.
-  MEMORY USAGE
+MEMORY USAGE
     Per-request usage:
     Each aio request uses - depending on your architecture - around 100-200
     bytes of memory. In addition, stat requests need a stat buffer (possibly
     a few hundred bytes), readdir requires a result buffer and so on. Perl
     In the execution phase, some aio requests require more memory for
     temporary buffers, and each thread requires a stack and other data
     structures (usually around 16k-128k, depending on the OS).
+WIN32 FILENAMES/DATABASE NAME MESS
+    Perl on Win32 supports only ASCII filenames (the reason is that it
+    abuses an internal flag to store wether a filename is Unicode or ANSI,
+    but that flag is used for somethign else in the perl core, so there is
+    no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
+    module tries to work around this issue by assuming that the filename is
+    an ANSI filename and BDB was built for unicode support.
 KNOWN BUGS
     Known bugs will be fixed in the next release, except:
        If you use a transaction in any request, and the request returns
        with an operating system error or DB_LOCK_NOTGRANTED, the internal

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing BDB/README (file contents): Revision 1.10 by root, Tue Jul 8 08:35:12 2008 UTC vs. Revision 1.18 by root, Thu Jan 18 16:45:27 2018 UTC

Diff Legend

Comparing BDB/README (file contents):
Revision 1.10 by root, Tue Jul 8 08:35:12 2008 UTC vs.
Revision 1.18 by root, Thu Jan 18 16:45:27 2018 UTC