[ViewVC] Diff of: cvs/BDB/README

Comparing BDB/README (file contents):
Revision 1.7 by root, Sun Mar 30 06:29:07 2008 UTC vs.
Revision 1.14 by root, Tue Oct 21 02:21:25 2008 UTC

      db_sync $db;
      # when you also use Coro, management is easy:
      use Coro::BDB;
-     # automatic result processing with AnyEvent:
+     # automatic event loop intergration with AnyEvent:
-     our $FH; open $FH, "<&=" . BDB::poll_fileno;
+     use AnyEvent::BDB;
-     our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
      # automatic result processing with EV:
      my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb;
      # with Glib:
         Request has reached the end of its lifetime and holds no resources
         anymore (except possibly for the Perl object, but its connection to
         the actual aio request is severed and calling its methods will
         either do nothing or result in a runtime error).
-  FILENAMES/DATABASE NAMES
-    The BDB expects "binary" filenames (octet strings) for pathnames on
-    POSIX systems, and "unicode" filenames (strings with characters
-    potentially >255) on Win32 (expecting a Unicode win32 build - win32 is a
-    total mess).
-  BERKELEYDB FUNCTIONS
+BERKELEYDB FUNCTIONS
     All of these are functions. The create functions simply return a new
     object and never block. All the remaining functions take an optional
     callback as last argument. If it is missing, then the function will be
     executed synchronously. In both cases, $! will reflect the return value
     of the function.
     BDB functions that cannot block (mostly functions that manipulate
     settings) are method calls on the relevant objects, so the rule of thumb
     is: if it's a method, it's not blocking, if it's a function, it takes a
     callback as last argument.
-    In the following, $int signifies an integer return value, "octetstring"
+    In the following, $int signifies an integer return value, "bdb_filename"
-    is a "binary string" (i.e. a perl string with no character indices
+    is a "filename" (octets on unix, madness on windows), "U32" is an
-    >255), "U32" is an unsigned 32 bit integer, "int" is some integer, "NV"
+    unsigned 32 bit integer, "int" is some integer, "NV" is a floating point
-    is a floating point value.
+    value.
-    The "SV *" types are generic perl scalars (for input and output of data
+    Most "SV *" types are generic perl scalars (for input and output of data
-    values), and the "SV *callback" is the optional callback function to
+    values).
-    call when the request is completed.
     The various "DB_ENV" etc. arguments are handles return by
     "db_env_create", "db_create", "txn_begin" and so on. If they have an
     appended "_ornull" this means they are optional and you can pass "undef"
     for them, resulting a NULL pointer on the C level.
+    The "SV *callback" is the optional callback function to call when the
+    request is completed. This last callback argument is special: the
+    callback is simply the last argument passed. If there are "optional"
+    arguments before the callback they can be left out. The callback itself
+    can be left out or specified as "undef", in which case the function will
+    be executed synchronously.
+    For example, "db_env_txn_checkpoint" usually is called with all integer
+    arguments zero. These can be left out, so all of these specify a call to
+    "DB_ENV->txn_checkpoint", to be executed asynchronously with a callback
+    to be called:
+       db_env_txn_checkpoint $db_env, 0, 0, 0, sub { };
+       db_env_txn_checkpoint $db_env, 0, 0, sub { };
+       db_env_txn_checkpoint $db_env, sub { };
+    While these all specify a call to "DB_ENV->txn_checkpoint" to be
+    executed synchronously:
+       db_env_txn_checkpoint $db_env, 0, 0, 0, undef;
+       db_env_txn_checkpoint $db_env, 0, 0, 0;
+       db_env_txn_checkpoint $db_env, 0;
-   BDB functions
+  BDB functions
     Functions in the BDB namespace, exported by default:
        $env = db_env_create (U32 env_flags = 0)
           flags: RPCCLIENT
-       db_env_open (DB_ENV *env, octetstring db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef)
+       db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef)
           open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
        db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = &PL_sv_undef)
        db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
           flags: FORCE
        db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = &PL_sv_undef)
           atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
        db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
        db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
+       db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = &PL_sv_undef)
        $db = db_create (DB_ENV *env = 0, U32 flags = 0)
           flags: XA_CREATE
-       db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
+       db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
           flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
        db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
           flags: DB_NOSYNC
-       db_upgrade (DB *db, octetstring file, U32 flags = 0, SV *callback = &PL_sv_undef)
+       db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = &PL_sv_undef)
        db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
           flags: FREELIST_ONLY FREE_SPACE
        db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
        db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
        db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
           flags: APPEND NODUPDATA NOOVERWRITE
+       db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6)
        db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
           flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
        db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
           flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
        db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
     You can use the "$txn->failed" method to check wether a transaction has
     failed in this way and abort further processing (excluding
     "db_txn_finish").
-   DB_ENV/database environment methods
+  DB_ENV/database environment methods
     Methods available on DB_ENV/$env handles:
        DESTROY (DB_ENV_ornull *env)
                CODE:
                if (env)
        $int = $env->set_data_dir (const char *dir)
        $int = $env->set_tmp_dir (const char *dir)
        $int = $env->set_lg_dir (const char *dir)
        $int = $env->set_shm_key (long shm_key)
        $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
-       $int = $env->set_flags (U32 flags, int onoff)
+       $int = $env->set_flags (U32 flags, int onoff = 1)
+       $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7)
+       $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7)
        $env->set_errfile (FILE *errfile = 0)
        $env->set_msgfile (FILE *msgfile = 0)
        $int = $env->set_verbose (U32 which, int onoff = 1)
        $int = $env->set_encrypt (const char *password, U32 flags = 0)
        $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
        $int = $env->mutex_set_max (U32 max)
        $int = $env->mutex_set_align (U32 align)
        $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
           flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
+       $txn = $env->cdsgroup_begin; (v4.5)
    Example:
        use AnyEvent;
        use BDB;
           BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
           0600;
        $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
-   DB/database methods
+  DB/database methods
     Methods available on DB/$db handles:
        DESTROY (DB_ornull *db)
                CODE:
                if (db)
        db_del $db, undef, "key $_" for 1..1000;
        db_sync $db;
-   DB_TXN/transaction methods
+  DB_TXN/transaction methods
     Methods available on DB_TXN/$txn handles:
        DESTROY (DB_TXN_ornull *txn)
                CODE:
                if (txn)
           flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
        $bool = $txn->failed
        # see db_txn_finish documentation, above
-   DBC/cursor methods
+  DBC/cursor methods
     Methods available on DBC/$dbc handles:
        DESTROY (DBC_ornull *dbc)
                CODE:
                if (dbc)
                  dbc->c_close (dbc);
-       $int = $cursor->set_priority ($priority = PRIORITY_*)
+       $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6)
    Example:
        my $c = $db->cursor;
        for (;;) {
           last if $!;
        }
        db_c_close $c;
-   DB_SEQUENCE/sequence methods
+  DB_SEQUENCE/sequence methods
     Methods available on DB_SEQUENCE/$seq handles:
        DESTROY (DB_SEQUENCE_ornull *seq)
                CODE:
                if (seq)
        my $seq = $db->sequence;
    db_sequence_open $seq, undef, "seq", BDB::CREATE;
        db_sequence_get $seq, undef, 1, my $value;
-  SUPPORT FUNCTIONS
+SUPPORT FUNCTIONS
-   EVENT PROCESSING AND EVENT LOOP INTEGRATION
+  EVENT PROCESSING AND EVENT LOOP INTEGRATION
     $msg = BDB::strerror [$errno]
         Returns the string corresponding to the given errno value. If no
         argument is given, use $!.
         Note that the BDB module also patches the $! variable directly, so
         Strictly equivalent to:
            BDB::poll_wait, BDB::poll_cb
               while BDB::nreqs;
+  VERSION CHECKING
+    BerkeleyDB comes in various versions, many of them have minor
+    incompatibilities. This means that traditional "at least version x.x"
+    checks are often not sufficient.
+    Example: set the log_autoremove option in a way compatible with <v4.7
+    and v4.7. Note the use of & on the constants to avoid triggering a
+    compiletime bug when the symbol isn't available.
+       $DB_ENV->set_flags      (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7;
+       $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7;
+    BDB::VERSION
+        The "BDB::VERSION" function, when called without arguments, returns
+        the Berkeley DB version as a v-string (usually with 3 components).
+        You should use "lt" and "ge" operators exclusively to make
+        comparisons.
+        Example: check for at least version 4.7.
+           BDB::VERSION ge v4.7 or die;
+    BDB::VERSION min-version
+        Returns true if the BDB version is at least the given version
+        (specified as a v-string), false otherwise.
+        Example: check for at least version 4.5.
+           BDB::VERSION v4.7 or die;
+    BDB::VERSION min-version, max-version
+        Returns true of the BDB version is at least version "min-version"
+        (specify "undef" or "v0" for any minimum version) and less then
+        "max-version".
+        Example: check wether version is strictly less then v4.7.
+           BDB::VERSION v0, v4.7
+              or die "version 4.7 is not yet supported";
-   CONTROLLING THE NUMBER OF THREADS
+  CONTROLLING THE NUMBER OF THREADS
     BDB::min_parallel $nthreads
         Set the minimum number of BDB threads to $nthreads. The current
         default is 8, which means eight asynchronous operations can execute
         concurrently at any one time (the number of outstanding requests,
         however, is unlimited).
         You can still queue as many requests as you want. Therefore,
         "max_oustsanding" is mainly useful in simple scripts (with low
         values) or as a stop gap to shield against fatal memory overflow
         (with large values).
-    BDB::set_sync_prepare $cb
+    $old_cb = BDB::set_sync_prepare $cb
         Sets a callback that is called whenever a request is created without
         an explicit callback. It has to return two code references. The
-        first is used as the request callback, and the second is called to
+        first is used as the request callback (it should save the return
-        wait until the first callback has been called. The default
+        status), and the second is called to wait until the first callback
-        implementation works like this:
+        has been called (it must set $! to the return status).
+        This mechanism can be used to include BDB into other event
+        mechanisms, such as Coro::BDB.
+        To allow other, callback-based, events to be executed while
+        callback-less ones are run, you could use this sync prepare
+        function:
            sub {
               my $status;
               (
                  sub { $status = $! },
                  sub { BDB::poll while !defined $status; $! = $status },
               )
            }
+        It works by polling for results till the request has finished and
+        then sets $! to the return value. This means that if you don't use a
+        callback, BDB would simply fall back to synchronous operations.
+        By default, or if the sync prepare function is set to "undef", is to
+        execute callback-less BDB requests in the foreground thread, setting
+        $! to the return value, without polling for other events.
-   STATISTICAL INFORMATION
+  STATISTICAL INFORMATION
     BDB::nreqs
         Returns the number of requests currently in the ready, execute or
         pending states (i.e. for which their callback has not been invoked
         yet).
     BDB::npending
         Returns the number of requests currently in the pending state
         (executed, but not yet processed by poll_cb).
+COMMON PITFALLS
+  Unexpected Crashes
+    Remember that, by default, BDB will execute requests in parallel, in
+    somewhat random order. That means that it is easy to run a "db_get"
+    request on thesa me database as a concurrent "db_close" request, leading
+    to a crash, silent data corruption, eventually the next world war on
+    terrorism.
+    If you only ever use foreground requests (without a callback), this will
+    not be an issue.
+  Unexpected Freezes or Deadlocks
+    Remember that, by default, BDB will execute requests in parallel, which
+    easily leads to deadlocks (even concurrent put's on the same database
+    can deadlock).
+    You either need to run deadlock detection (and handle the resulting
+    errors), or make sure only one process ever updates the database, ine
+    one thread, e.g. by using only foreground requests (without a callback).
-  FORK BEHAVIOUR
+FORK BEHAVIOUR
     This module should do "the right thing" when the process using it forks:
     Before the fork, BDB enters a quiescent state where no requests can be
     added in other threads and no results will be processed. After the fork
     the parent simply leaves the quiescent state and continues
     Win32 note: there is no fork on win32, and perls emulation of it is too
     broken to be supported, so do not use BDB in a windows pseudo-fork,
     better yet, switch to a more capable platform.
-  MEMORY USAGE
+MEMORY USAGE
     Per-request usage:
     Each aio request uses - depending on your architecture - around 100-200
     bytes of memory. In addition, stat requests need a stat buffer (possibly
     a few hundred bytes), readdir requires a result buffer and so on. Perl
     In the execution phase, some aio requests require more memory for
     temporary buffers, and each thread requires a stack and other data
     structures (usually around 16k-128k, depending on the OS).
+WIN32 FILENAMES/DATABASE NAME MESS
+    Perl on Win32 supports only ASCII filenames (the reason is that it
+    abuses an internal flag to store wether a filename is Unicode or ANSI,
+    but that flag is used for somethign else in the perl core, so there is
+    no way to detect wether a filename is ANSI or Unicode-encoded). The BDB
+    module tries to work around this issue by assuming that the filename is
+    an ANSI filename and BDB was built for unicode support.
 KNOWN BUGS
     Known bugs will be fixed in the next release, except:
        If you use a transaction in any request, and the request returns
        with an operating system error or DB_LOCK_NOTGRANTED, the internal
        TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
        above.
 SEE ALSO
-    Coro::BDB, IO::AIO.
+    AnyEvent::BDB (event loop integration), Coro::BDB (more natural syntax),
+    IO::AIO (nice to have).
 AUTHOR
      Marc Lehmann <schmorp@schmorp.de>
      http://home.schmorp.de/

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing BDB/README (file contents): Revision 1.7 by root, Sun Mar 30 06:29:07 2008 UTC vs. Revision 1.14 by root, Tue Oct 21 02:21:25 2008 UTC

Diff Legend

Comparing BDB/README (file contents):
Revision 1.7 by root, Sun Mar 30 06:29:07 2008 UTC vs.
Revision 1.14 by root, Tue Oct 21 02:21:25 2008 UTC