[ViewVC] Diff of: cvs/BDB/BDB.pm

Comparing BDB/BDB.pm (file contents):
Revision 1.2 by root, Mon Feb 5 20:21:38 2007 UTC vs.
Revision 1.14 by root, Thu Sep 13 12:29:49 2007 UTC

…		…
6		6
7	use BDB;	7	use BDB;
8		8
9	=head1 DESCRIPTION	9	=head1 DESCRIPTION
10		10
11	=head2 EXAMPLE	11	See the BerkeleyDB documentation (L<http://www.oracle.com/technology/documentation/berkeley-db/db/index.html>).
		12	The BDB API is very similar to the C API (the translation has been very faithful).
		13
		14	See also the example sections in the document below and possibly the eg/
		15	subdirectory of the BDB distribution. Last not least see the IO::AIO
		16	documentation, as that module uses almost the same asynchronous request
		17	model as this module.
		18
		19	I know this is woefully inadequate documentation. Send a patch!
		20
12		21
13	=head1 REQUEST ANATOMY AND LIFETIME	22	=head1 REQUEST ANATOMY AND LIFETIME
14		23
15	Every request method creates a request. which is a C data structure not	24	Every request method creates a request. which is a C data structure not
16	directly visible to Perl.	25	directly visible to Perl.
…		…
63	use strict 'vars';	72	use strict 'vars';
64		73
65	use base 'Exporter';	74	use base 'Exporter';
66		75
67	BEGIN {	76	BEGIN {
68	our $VERSION = '0.1';	77	our $VERSION = '1.0';
69		78
70	our @BDB_REQ = qw();	79	our @BDB_REQ = qw(
		80	db_env_open db_env_close db_env_txn_checkpoint db_env_lock_detect
		81	db_env_memp_sync db_env_memp_trickle
		82	db_open db_close db_compact db_sync db_put db_get db_pget db_del db_key_range
		83	db_txn_commit db_txn_abort
		84	db_c_close db_c_count db_c_put db_c_get db_c_pget db_c_del
		85	db_sequence_open db_sequence_close
		86	db_sequence_get db_sequence_remove
		87	);
		88	our @EXPORT = (@BDB_REQ, qw(dbreq_pri dbreq_nice db_env_create db_create));
		89	our @EXPORT_OK = qw(
71	our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush	90	poll_fileno poll_cb poll_wait flush
72	min_parallel max_parallel max_idle	91	min_parallel max_parallel max_idle
73	nreqs nready npending nthreads	92	nreqs nready npending nthreads
74	max_poll_time max_poll_reqs);	93	max_poll_time max_poll_reqs
		94	);
75		95
76	require XSLoader;	96	require XSLoader;
77	XSLoader::load ("BDB", $VERSION);	97	XSLoader::load ("BDB", $VERSION);
78	}	98	}
		99
		100	=head2 BERKELEYDB FUNCTIONS
		101
		102	All of these are functions. The create functions simply return a new
		103	object and never block. All the remaining functions all take an optional
		104	callback as last argument. If it is missing, then the fucntion will be
		105	executed synchronously.
		106
		107	BDB functions that cannot block (mostly functions that manipulate
		108	settings) are method calls on the relevant objects, so the rule of thumb
		109	is: if its a method, its not blocking, if its a function, it takes a
		110	callback as last argument.
		111
		112	In the following, C<$int> signifies an integer return value,
		113	C<octetstring> is a "binary string" (i.e. a perl string with no character
		114	indices >255), C<U32> is an unsigned 32 bit integer, C<int> is some
		115	integer, C<NV> is a floating point value.
		116
		117	The C<SV *> types are generic perl scalars (for input and output of data
		118	values), and the C<SV *callback> is the optional callback function to call
		119	when the request is completed.
		120
		121	The various C<DB_ENV> etc. arguments are handles return by
		122	C<db_env_create>, C<db_create>, C<txn_begin> and so on. If they have an
		123	appended C<_ornull> this means they are optional and you can pass C<undef>
		124	for them, resulting a NULL pointer on the C level.
		125
		126	=head3 BDB functions
		127
		128	Functions in the BDB namespace, exported by default:
		129
		130	$env = db_env_create (U32 env_flags = 0)
		131	flags: RPCCLIENT
		132
		133	db_env_open (DB_ENV env, octetstring db_home, U32 open_flags, int mode, SV callback = &PL_sv_undef)
		134	open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
		135	db_env_close (DB_ENV env, U32 flags = 0, SV callback = &PL_sv_undef)
		136	db_env_txn_checkpoint (DB_ENV env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV callback = &PL_sv_undef)
		137	flags: FORCE
		138	db_env_lock_detect (DB_ENV env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV dummy = 0, SV *callback = &PL_sv_undef)
		139	atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
		140	db_env_memp_sync (DB_ENV env, SV dummy = 0, SV *callback = &PL_sv_undef)
		141	db_env_memp_trickle (DB_ENV env, int percent, SV dummy = 0, SV *callback = &PL_sv_undef)
		142
		143	$db = db_create (DB_ENV *env = 0, U32 flags = 0)
		144	flags: XA_CREATE
		145
		146	db_open (DB db, DB_TXN_ornull txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
		147	flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
		148	db_close (DB db, U32 flags = 0, SV callback = &PL_sv_undef)
		149	flags: DB_NOSYNC
		150	db_compact (DB db, DB_TXN_ornull txn = 0, SV start = 0, SV stop = 0, SV unused1 = 0, U32 flags = DB_FREE_SPACE, SV unused2 = 0, SV *callback = &PL_sv_undef)
		151	flags: FREELIST_ONLY FREE_SPACE
		152	db_sync (DB db, U32 flags = 0, SV callback = &PL_sv_undef)
		153	db_key_range (DB db, DB_TXN_ornull txn, SV key, SV key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
		154	db_put (DB db, DB_TXN_ornull txn, SV key, SV data, U32 flags = 0, SV *callback = &PL_sv_undef)
		155	flags: APPEND NODUPDATA NOOVERWRITE
		156	db_get (DB db, DB_TXN_ornull txn, SV key, SV data, U32 flags = 0, SV *callback = &PL_sv_undef)
		157	flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
		158	db_pget (DB db, DB_TXN_ornull txn, SV key, SV pkey, SV data, U32 flags = 0, SV callback = &PL_sv_undef)
		159	flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
		160	db_del (DB db, DB_TXN_ornull txn, SV key, U32 flags = 0, SV callback = &PL_sv_undef)
		161	db_txn_commit (DB_TXN txn, U32 flags = 0, SV callback = &PL_sv_undef)
		162	flags: TXN_NOSYNC TXN_SYNC
		163	db_txn_abort (DB_TXN txn, SV callback = &PL_sv_undef)
		164
		165	db_c_close (DBC dbc, SV callback = &PL_sv_undef)
		166	db_c_count (DBC dbc, SV count, U32 flags = 0, SV *callback = &PL_sv_undef)
		167	db_c_put (DBC dbc, SV key, SV data, U32 flags = 0, SV callback = &PL_sv_undef)
		168	flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA
		169	db_c_get (DBC dbc, SV key, SV data, U32 flags = 0, SV callback = &PL_sv_undef)
		170	flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW
		171	db_c_pget (DBC dbc, SV key, SV pkey, SV data, U32 flags = 0, SV *callback = &PL_sv_undef)
		172	db_c_del (DBC dbc, U32 flags = 0, SV callback = &PL_sv_undef)
		173
		174	db_sequence_open (DB_SEQUENCE seq, DB_TXN_ornull txnid, SV key, U32 flags = 0, SV callback = &PL_sv_undef)
		175	flags: CREATE EXCL
		176	db_sequence_close (DB_SEQUENCE seq, U32 flags = 0, SV callback = &PL_sv_undef)
		177	db_sequence_get (DB_SEQUENCE seq, DB_TXN_ornull txnid, int delta, SV seq_value, U32 flags = DB_TXN_NOSYNC, SV callback = &PL_sv_undef)
		178	flags: TXN_NOSYNC
		179	db_sequence_remove (DB_SEQUENCE seq, DB_TXN_ornull txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
		180	flags: TXN_NOSYNC
		181
		182	=head3 DB_ENV/database environment methods
		183
		184	Methods available on DB_ENV/$env handles:
		185
		186	DESTROY (DB_ENV_ornull *env)
		187	CODE:
		188	if (env)
		189	env->close (env, 0);
		190
		191	$int = $env->set_data_dir (const char *dir)
		192	$int = $env->set_tmp_dir (const char *dir)
		193	$int = $env->set_lg_dir (const char *dir)
		194	$int = $env->set_shm_key (long shm_key)
		195	$int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
		196	$int = $env->set_flags (U32 flags, int onoff)
		197	$env->set_errfile (FILE *errfile = 0)
		198	$env->set_msgfile (FILE *msgfile = 0)
		199	$int = $env->set_verbose (U32 which, int onoff = 1)
		200	$int = $env->set_encrypt (const char *password, U32 flags = 0)
		201	$int = $env->set_timeout (NV timeout, U32 flags)
		202	$int = $env->set_mp_max_openfd (int maxopenfd);
		203	$int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep);
		204	$int = $env->set_mp_mmapsize (int mmapsize_mb)
		205	$int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT)
		206	$int = $env->set_lk_max_lockers (U32 max)
		207	$int = $env->set_lk_max_locks (U32 max)
		208	$int = $env->set_lk_max_objects (U32 max)
		209	$int = $env->set_lg_bsize (U32 max)
		210	$int = $env->set_lg_max (U32 max)
		211
		212	$txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
		213	flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
		214
		215	=head4 Example:
		216
		217	use AnyEvent;
		218	use BDB;
		219
		220	our $FH; open $FH, "<&=" . BDB::poll_fileno;
		221	our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
		222
		223	BDB::min_parallel 8;
		224
		225	my $env = db_env_create;
		226
		227	mkdir "bdtest", 0700;
		228	db_env_open
		229	$env,
		230	"bdtest",
		231	BDB::INIT_LOCK \| BDB::INIT_LOG \| BDB::INIT_MPOOL \| BDB::INIT_TXN \| BDB::RECOVER \| BDB::USE_ENVIRON \| BDB::CREATE,
		232	0600;
		233
		234	$env->set_flags (BDB::AUTO_COMMIT \| BDB::TXN_NOSYNC, 1);
		235
		236
		237	=head3 DB/database methods
		238
		239	Methods available on DB/$db handles:
		240
		241	DESTROY (DB_ornull *db)
		242	CODE:
		243	if (db)
		244	{
		245	SV env = (SV )db->app_private;
		246	db->close (db, 0);
		247	SvREFCNT_dec (env);
		248	}
		249
		250	$int = $db->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
		251	$int = $db->set_flags (U32 flags)
		252	flags: CHKSUM ENCRYPT TXN_NOT_DURABLE
		253	Btree: DUP DUPSORT RECNUM REVSPLITOFF
		254	Hash: DUP DUPSORT
		255	Queue: INORDER
		256	Recno: RENUMBER SNAPSHOT
		257
		258	$int = $db->set_encrypt (const char *password, U32 flags)
		259	$int = $db->set_lorder (int lorder)
		260	$int = $db->set_bt_minkey (U32 minkey)
		261	$int = $db->set_re_delim (int delim)
		262	$int = $db->set_re_pad (int re_pad)
		263	$int = $db->set_re_source (char *source)
		264	$int = $db->set_re_len (U32 re_len)
		265	$int = $db->set_h_ffactor (U32 h_ffactor)
		266	$int = $db->set_h_nelem (U32 h_nelem)
		267	$int = $db->set_q_extentsize (U32 extentsize)
		268
		269	$dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
		270	flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
		271	$seq = $db->sequence (U32 flags = 0)
		272
		273	=head4 Example:
		274
		275	my $db = db_create $env;
		276	db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT \| BDB::CREATE \| BDB::READ_UNCOMMITTED, 0600;
		277
		278	for (1..1000) {
		279	db_put $db, undef, "key $_", "data $_";
		280
		281	db_key_range $db, undef, "key $_", my $keyrange;
		282	my ($lt, $eq, $gt) = @$keyrange;
		283	}
		284
		285	db_del $db, undef, "key $_" for 1..1000;
		286
		287	db_sync $db;
		288
		289
		290	=head3 DB_TXN/transaction methods
		291
		292	Methods available on DB_TXN/$txn handles:
		293
		294	DESTROY (DB_TXN_ornull *txn)
		295	CODE:
		296	if (txn)
		297	txn->abort (txn);
		298
		299	$int = $txn->set_timeout (NV timeout, U32 flags)
		300	flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
		301
		302
		303	=head3 DBC/cursor methods
		304
		305	Methods available on DBC/$dbc handles:
		306
		307	DESTROY (DBC_ornull *dbc)
		308	CODE:
		309	if (dbc)
		310	dbc->c_close (dbc);
		311
		312	=head4 Example:
		313
		314	my $c = $db->cursor;
		315
		316	for (;;) {
		317	db_c_get $c, my $key, my $data, BDB::NEXT;
		318	warn "<$!,$key,$data>";
		319	last if $!;
		320	}
		321
		322	db_c_close $c;
		323
		324
		325	=head3 DB_SEQUENCE/sequence methods
		326
		327	Methods available on DB_SEQUENCE/$seq handles:
		328
		329	DESTROY (DB_SEQUENCE_ornull *seq)
		330	CODE:
		331	if (seq)
		332	seq->close (seq, 0);
		333
		334	$int = $seq->initial_value (db_seq_t value)
		335	$int = $seq->set_cachesize (U32 size)
		336	$int = $seq->set_flags (U32 flags)
		337	flags: SEQ_DEC SEQ_INC SEQ_WRAP
		338	$int = $seq->set_range (db_seq_t min, db_seq_t max)
		339
		340	=head4 Example:
		341
		342	my $seq = $db->sequence;
		343
		344	db_sequence_open $seq, undef, "seq", BDB::CREATE;
		345	db_sequence_get $seq, undef, 1, my $value;
		346
79		347
80	=head2 SUPPORT FUNCTIONS	348	=head2 SUPPORT FUNCTIONS
81		349
82	=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION	350	=head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
83		351
…		…
167	Strictly equivalent to:	435	Strictly equivalent to:
168		436
169	BDB::poll_wait, BDB::poll_cb	437	BDB::poll_wait, BDB::poll_cb
170	while BDB::nreqs;	438	while BDB::nreqs;
171		439
		440	=back
		441
172	=head3 CONTROLLING THE NUMBER OF THREADS	442	=head3 CONTROLLING THE NUMBER OF THREADS
		443
		444	=over 4
173		445
174	=item BDB::min_parallel $nthreads	446	=item BDB::min_parallel $nthreads
175		447
176	Set the minimum number of AIO threads to C<$nthreads>. The current	448	Set the minimum number of AIO threads to C<$nthreads>. The current
177	default is C<8>, which means eight asynchronous operations can execute	449	default is C<8>, which means eight asynchronous operations can execute
…		…
236		508
237	You can still queue as many requests as you want. Therefore,	509	You can still queue as many requests as you want. Therefore,
238	C<max_oustsanding> is mainly useful in simple scripts (with low values) or	510	C<max_oustsanding> is mainly useful in simple scripts (with low values) or
239	as a stop gap to shield against fatal memory overflow (with large values).	511	as a stop gap to shield against fatal memory overflow (with large values).
240		512
		513	=item BDB::set_sync_prepare $cb
		514
		515	Sets a callback that is called whenever a request is created without an
		516	explicit callback. It has to return two code references. The first is used
		517	as the request callback, and the second is called to wait until the first
		518	callback has been called. The default implementation works like this:
		519
		520	sub {
		521	my $status;
		522	(
		523	sub { $status = $! },
		524	sub { BDB::poll while !defined $status; $! = $status },
		525	)
		526	}
		527
		528	=back
		529
241	=head3 STATISTICAL INFORMATION	530	=head3 STATISTICAL INFORMATION
		531
		532	=over 4
242		533
243	=item BDB::nreqs	534	=item BDB::nreqs
244		535
245	Returns the number of requests currently in the ready, execute or pending	536	Returns the number of requests currently in the ready, execute or pending
246	states (i.e. for which their callback has not been invoked yet).	537	states (i.e. for which their callback has not been invoked yet).
…		…
261	but not yet processed by poll_cb).	552	but not yet processed by poll_cb).
262		553
263	=back	554	=back
264		555
265	=cut	556	=cut
		557
		558	set_sync_prepare {
		559	my $status;
		560	(
		561	sub {
		562	$status = $!;
		563	},
		564	sub {
		565	BDB::poll while !defined $status;
		566	$! = $status;
		567	},
		568	)
		569	};
266		570
267	min_parallel 8;	571	min_parallel 8;
268		572
269	END { flush }	573	END { flush }
270		574
…		…
294	bytes of memory. In addition, stat requests need a stat buffer (possibly	598	bytes of memory. In addition, stat requests need a stat buffer (possibly
295	a few hundred bytes), readdir requires a result buffer and so on. Perl	599	a few hundred bytes), readdir requires a result buffer and so on. Perl
296	scalars and other data passed into aio requests will also be locked and	600	scalars and other data passed into aio requests will also be locked and
297	will consume memory till the request has entered the done state.	601	will consume memory till the request has entered the done state.
298		602
299	This is now awfully much, so queuing lots of requests is not usually a	603	This is not awfully much, so queuing lots of requests is not usually a
300	problem.	604	problem.
301		605
302	Per-thread usage:	606	Per-thread usage:
303		607
304	In the execution phase, some aio requests require more memory for	608	In the execution phase, some aio requests require more memory for

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing BDB/BDB.pm (file contents): Revision 1.2 by root, Mon Feb 5 20:21:38 2007 UTC vs. Revision 1.14 by root, Thu Sep 13 12:29:49 2007 UTC

Diff Legend

Comparing BDB/BDB.pm (file contents):
Revision 1.2 by root, Mon Feb 5 20:21:38 2007 UTC vs.
Revision 1.14 by root, Thu Sep 13 12:29:49 2007 UTC