ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/BDB/README
(Generate patch)

Comparing BDB/README (file contents):
Revision 1.1 by root, Mon Feb 5 18:40:55 2007 UTC vs.
Revision 1.3 by root, Tue Dec 4 10:14:34 2007 UTC

1NAME 1NAME
2 BDB::AIO - Asynchronous Berkeley DB access 2 BDB - Asynchronous Berkeley DB access
3 3
4SYNOPSIS 4SYNOPSIS
5 use BDB::AIO; 5 use BDB;
6 6
7DESCRIPTION 7DESCRIPTION
8 EXAMPLE 8 See the BerkeleyDB documentation
9 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht
10 ml>). The BDB API is very similar to the C API (the translation has been
11 very faithful).
12
13 See also the example sections in the document below and possibly the eg/
14 subdirectory of the BDB distribution. Last not least see the IO::AIO
15 documentation, as that module uses almost the same asynchronous request
16 model as this module.
17
18 I know this is woefully inadequate documentation. Send a patch!
19
9REQUEST ANATOMY AND LIFETIME 20REQUEST ANATOMY AND LIFETIME
10 Every request method creates a request. which is a C data structure not 21 Every request method creates a request. which is a C data structure not
11 directly visible to Perl. 22 directly visible to Perl.
12 23
13 During their existance, bdb requests travel through the following 24 During their existance, bdb requests travel through the following
… …
39 Request has reached the end of its lifetime and holds no resources 50 Request has reached the end of its lifetime and holds no resources
40 anymore (except possibly for the Perl object, but its connection to 51 anymore (except possibly for the Perl object, but its connection to
41 the actual aio request is severed and calling its methods will 52 the actual aio request is severed and calling its methods will
42 either do nothing or result in a runtime error). 53 either do nothing or result in a runtime error).
43 54
55 BERKELEYDB FUNCTIONS
56 All of these are functions. The create functions simply return a new
57 object and never block. All the remaining functions all take an optional
58 callback as last argument. If it is missing, then the fucntion will be
59 executed synchronously.
60
61 BDB functions that cannot block (mostly functions that manipulate
62 settings) are method calls on the relevant objects, so the rule of thumb
63 is: if its a method, its not blocking, if its a function, it takes a
64 callback as last argument.
65
66 In the following, $int signifies an integer return value, "octetstring"
67 is a "binary string" (i.e. a perl string with no character indices
68 >255), "U32" is an unsigned 32 bit integer, "int" is some integer, "NV"
69 is a floating point value.
70
71 The "SV *" types are generic perl scalars (for input and output of data
72 values), and the "SV *callback" is the optional callback function to
73 call when the request is completed.
74
75 The various "DB_ENV" etc. arguments are handles return by
76 "db_env_create", "db_create", "txn_begin" and so on. If they have an
77 appended "_ornull" this means they are optional and you can pass "undef"
78 for them, resulting a NULL pointer on the C level.
79
80 BDB functions
81 Functions in the BDB namespace, exported by default:
82
83 $env = db_env_create (U32 env_flags = 0)
84 flags: RPCCLIENT
85
86 db_env_open (DB_ENV *env, octetstring db_home, U32 open_flags, int mode, SV *callback = &PL_sv_undef)
87 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM
88 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = &PL_sv_undef)
89 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
90 flags: FORCE
91 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = &PL_sv_undef)
92 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST
93 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = &PL_sv_undef)
94 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = &PL_sv_undef)
95
96 $db = db_create (DB_ENV *env = 0, U32 flags = 0)
97 flags: XA_CREATE
98
99 db_open (DB *db, DB_TXN_ornull *txnid, octetstring file, octetstring database, int type, U32 flags, int mode, SV *callback = &PL_sv_undef)
100 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE
101 db_close (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
102 flags: DB_NOSYNC
103 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = &PL_sv_undef)
104 flags: FREELIST_ONLY FREE_SPACE
105 db_sync (DB *db, U32 flags = 0, SV *callback = &PL_sv_undef)
106 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = &PL_sv_undef)
107 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
108 flags: APPEND NODUPDATA NOOVERWRITE
109 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
110 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
111 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
112 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW
113 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
114 db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
115 flags: TXN_NOSYNC TXN_SYNC
116 db_txn_abort (DB_TXN *txn, SV *callback = &PL_sv_undef)
117
118 db_c_close (DBC *dbc, SV *callback = &PL_sv_undef)
119 db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = &PL_sv_undef)
120 db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
121 flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA
122 db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
123 flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW
124 db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = &PL_sv_undef)
125 db_c_del (DBC *dbc, U32 flags = 0, SV *callback = &PL_sv_undef)
126
127 db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = &PL_sv_undef)
128 flags: CREATE EXCL
129 db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = &PL_sv_undef)
130 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = &PL_sv_undef)
131 flags: TXN_NOSYNC
132 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = &PL_sv_undef)
133 flags: TXN_NOSYNC
134
135 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = &PL_sv_undef)
136 This is not actually a Berkeley DB function but a BDB module extension.
137 The background for this exytension is: It is very annoying to have to
138 check every single BDB function for error returns and provide a codepath
139 out of your transaction. While the BDB module still makes this possible,
140 it contains the following extensions:
141
142 When a transaction-protected function returns any operating system error
143 (errno > 0), BDB will set the "TXN_DEADLOCK" flag on the transaction.
144 This flag is also set by Berkeley DB functions themselves when an
145 operation fails with LOCK_DEADLOCK, and it causes all further operations
146 on that transaction (including "db_txn_commit") to fail.
147
148 The "db_txn_finish" request will look at this flag, and, if it is set,
149 will automatically call "db_txn_abort" (setting errno to "LOCK_DEADLOCK"
150 if it isn't set to something else yet). If it isn't set, it will call
151 "db_txn_commit" and return the error normally.
152
153 How to use this? Easy: just write your transaction normally:
154
155 my $txn = $db_env->txn_begin;
156 db_get $db, $txn, "key", my $data;
157 db_put $db, $txn, "key", $data + 1 unless $! == BDB::NOTFOUND;
158 db_txn_finish $txn;
159 die "transaction failed" if $!;
160
161 That is, handle only the expected errors. If something unexpected
162 happens (EIO, LOCK_NOTGRANTED or a deadlock in either db_get or db_put),
163 then the remaining requests (db_put in this case) will simply be skipped
164 (they will fail with LOCK_DEADLOCK) and the transaction will be aborted.
165
166 You can use the "$txn->failed" method to check wether a transaction has
167 failed in this way and abort further processing (excluding
168 "db_txn_finish").
169
170 DB_ENV/database environment methods
171 Methods available on DB_ENV/$env handles:
172
173 DESTROY (DB_ENV_ornull *env)
174 CODE:
175 if (env)
176 env->close (env, 0);
177
178 $int = $env->set_data_dir (const char *dir)
179 $int = $env->set_tmp_dir (const char *dir)
180 $int = $env->set_lg_dir (const char *dir)
181 $int = $env->set_shm_key (long shm_key)
182 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
183 $int = $env->set_flags (U32 flags, int onoff)
184 $env->set_errfile (FILE *errfile = 0)
185 $env->set_msgfile (FILE *msgfile = 0)
186 $int = $env->set_verbose (U32 which, int onoff = 1)
187 $int = $env->set_encrypt (const char *password, U32 flags = 0)
188 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
189 $int = $env->set_mp_max_openfd (int maxopenfd);
190 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep);
191 $int = $env->set_mp_mmapsize (int mmapsize_mb)
192 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT)
193 $int = $env->set_lk_max_lockers (U32 max)
194 $int = $env->set_lk_max_locks (U32 max)
195 $int = $env->set_lk_max_objects (U32 max)
196 $int = $env->set_lg_bsize (U32 max)
197 $int = $env->set_lg_max (U32 max)
198
199 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0)
200 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC
201
202 Example:
203 use AnyEvent;
204 use BDB;
205
206 our $FH; open $FH, "<&=" . BDB::poll_fileno;
207 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb);
208
209 BDB::min_parallel 8;
210
211 my $env = db_env_create;
212
213 mkdir "bdtest", 0700;
214 db_env_open
215 $env,
216 "bdtest",
217 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE,
218 0600;
219
220 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1);
221
222 DB/database methods
223 Methods available on DB/$db handles:
224
225 DESTROY (DB_ornull *db)
226 CODE:
227 if (db)
228 {
229 SV *env = (SV *)db->app_private;
230 db->close (db, 0);
231 SvREFCNT_dec (env);
232 }
233
234 $int = $db->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0)
235 $int = $db->set_flags (U32 flags)
236 flags: CHKSUM ENCRYPT TXN_NOT_DURABLE
237 Btree: DUP DUPSORT RECNUM REVSPLITOFF
238 Hash: DUP DUPSORT
239 Queue: INORDER
240 Recno: RENUMBER SNAPSHOT
241
242 $int = $db->set_encrypt (const char *password, U32 flags)
243 $int = $db->set_lorder (int lorder)
244 $int = $db->set_bt_minkey (U32 minkey)
245 $int = $db->set_re_delim (int delim)
246 $int = $db->set_re_pad (int re_pad)
247 $int = $db->set_re_source (char *source)
248 $int = $db->set_re_len (U32 re_len)
249 $int = $db->set_h_ffactor (U32 h_ffactor)
250 $int = $db->set_h_nelem (U32 h_nelem)
251 $int = $db->set_q_extentsize (U32 extentsize)
252
253 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0)
254 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT
255 $seq = $db->sequence (U32 flags = 0)
256
257 Example:
258 my $db = db_create $env;
259 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600;
260
261 for (1..1000) {
262 db_put $db, undef, "key $_", "data $_";
263
264 db_key_range $db, undef, "key $_", my $keyrange;
265 my ($lt, $eq, $gt) = @$keyrange;
266 }
267
268 db_del $db, undef, "key $_" for 1..1000;
269
270 db_sync $db;
271
272 DB_TXN/transaction methods
273 Methods available on DB_TXN/$txn handles:
274
275 DESTROY (DB_TXN_ornull *txn)
276 CODE:
277 if (txn)
278 txn->abort (txn);
279
280 $int = $txn->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT)
281 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT
282
283 $bool = $txn->failed
284 # see db_txn_finish documentation, above
285
286 DBC/cursor methods
287 Methods available on DBC/$dbc handles:
288
289 DESTROY (DBC_ornull *dbc)
290 CODE:
291 if (dbc)
292 dbc->c_close (dbc);
293
294 Example:
295 my $c = $db->cursor;
296
297 for (;;) {
298 db_c_get $c, my $key, my $data, BDB::NEXT;
299 warn "<$!,$key,$data>";
300 last if $!;
301 }
302
303 db_c_close $c;
304
305 DB_SEQUENCE/sequence methods
306 Methods available on DB_SEQUENCE/$seq handles:
307
308 DESTROY (DB_SEQUENCE_ornull *seq)
309 CODE:
310 if (seq)
311 seq->close (seq, 0);
312
313 $int = $seq->initial_value (db_seq_t value)
314 $int = $seq->set_cachesize (U32 size)
315 $int = $seq->set_flags (U32 flags)
316 flags: SEQ_DEC SEQ_INC SEQ_WRAP
317 $int = $seq->set_range (db_seq_t min, db_seq_t max)
318
319 Example:
320 my $seq = $db->sequence;
321
322 db_sequence_open $seq, undef, "seq", BDB::CREATE;
323 db_sequence_get $seq, undef, 1, my $value;
324
44 SUPPORT FUNCTIONS 325 SUPPORT FUNCTIONS
45 EVENT PROCESSING AND EVENT LOOP INTEGRATION 326 EVENT PROCESSING AND EVENT LOOP INTEGRATION
46 $fileno = BDB::AIO::poll_fileno 327 $fileno = BDB::poll_fileno
47 Return the *request result pipe file descriptor*. This filehandle 328 Return the *request result pipe file descriptor*. This filehandle
48 must be polled for reading by some mechanism outside this module 329 must be polled for reading by some mechanism outside this module
49 (e.g. Event or select, see below or the SYNOPSIS). If the pipe 330 (e.g. Event or select, see below or the SYNOPSIS). If the pipe
50 becomes readable you have to call "poll_cb" to check the results. 331 becomes readable you have to call "poll_cb" to check the results.
51 332
52 See "poll_cb" for an example. 333 See "poll_cb" for an example.
53 334
54 BDB::AIO::poll_cb 335 BDB::poll_cb
55 Process some outstanding events on the result pipe. You have to call 336 Process some outstanding events on the result pipe. You have to call
56 this regularly. Returns the number of events processed. Returns 337 this regularly. Returns the number of events processed. Returns
57 immediately when no events are outstanding. The amount of events 338 immediately when no events are outstanding. The amount of events
58 processed depends on the settings of "BDB::AIO::max_poll_req" and 339 processed depends on the settings of "BDB::max_poll_req" and
59 "BDB::AIO::max_poll_time". 340 "BDB::max_poll_time".
60 341
61 If not all requests were processed for whatever reason, the 342 If not all requests were processed for whatever reason, the
62 filehandle will still be ready when "poll_cb" returns. 343 filehandle will still be ready when "poll_cb" returns.
63 344
64 Example: Install an Event watcher that automatically calls 345 Example: Install an Event watcher that automatically calls
65 BDB::AIO::poll_cb with high priority: 346 BDB::poll_cb with high priority:
66 347
67 Event->io (fd => BDB::AIO::poll_fileno, 348 Event->io (fd => BDB::poll_fileno,
68 poll => 'r', async => 1, 349 poll => 'r', async => 1,
69 cb => \&BDB::AIO::poll_cb); 350 cb => \&BDB::poll_cb);
70 351
71 BDB::AIO::max_poll_reqs $nreqs 352 BDB::max_poll_reqs $nreqs
72 BDB::AIO::max_poll_time $seconds 353 BDB::max_poll_time $seconds
73 These set the maximum number of requests (default 0, meaning 354 These set the maximum number of requests (default 0, meaning
74 infinity) that are being processed by "BDB::AIO::poll_cb" in one 355 infinity) that are being processed by "BDB::poll_cb" in one call,
75 call, respectively the maximum amount of time (default 0, meaning 356 respectively the maximum amount of time (default 0, meaning
76 infinity) spent in "BDB::AIO::poll_cb" to process requests (more 357 infinity) spent in "BDB::poll_cb" to process requests (more
77 correctly the mininum amount of time "poll_cb" is allowed to use). 358 correctly the mininum amount of time "poll_cb" is allowed to use).
78 359
79 Setting "max_poll_time" to a non-zero value creates an overhead of 360 Setting "max_poll_time" to a non-zero value creates an overhead of
80 one syscall per request processed, which is not normally a problem 361 one syscall per request processed, which is not normally a problem
81 unless your callbacks are really really fast or your OS is really 362 unless your callbacks are really really fast or your OS is really
… …
87 in time. 368 in time.
88 369
89 For interactive programs, values such as 0.01 to 0.1 should be fine. 370 For interactive programs, values such as 0.01 to 0.1 should be fine.
90 371
91 Example: Install an Event watcher that automatically calls 372 Example: Install an Event watcher that automatically calls
92 BDB::AIO::poll_cb with low priority, to ensure that other parts of 373 BDB::poll_cb with low priority, to ensure that other parts of the
93 the program get the CPU sometimes even under high AIO load. 374 program get the CPU sometimes even under high AIO load.
94 375
95 # try not to spend much more than 0.1s in poll_cb 376 # try not to spend much more than 0.1s in poll_cb
96 BDB::AIO::max_poll_time 0.1; 377 BDB::max_poll_time 0.1;
97 378
98 # use a low priority so other tasks have priority 379 # use a low priority so other tasks have priority
99 Event->io (fd => BDB::AIO::poll_fileno, 380 Event->io (fd => BDB::poll_fileno,
100 poll => 'r', nice => 1, 381 poll => 'r', nice => 1,
101 cb => &BDB::AIO::poll_cb); 382 cb => &BDB::poll_cb);
102 383
103 BDB::AIO::poll_wait 384 BDB::poll_wait
104 If there are any outstanding requests and none of them in the result 385 If there are any outstanding requests and none of them in the result
105 phase, wait till the result filehandle becomes ready for reading 386 phase, wait till the result filehandle becomes ready for reading
106 (simply does a "select" on the filehandle. This is useful if you 387 (simply does a "select" on the filehandle. This is useful if you
107 want to synchronously wait for some requests to finish). 388 want to synchronously wait for some requests to finish).
108 389
109 See "nreqs" for an example. 390 See "nreqs" for an example.
110 391
111 BDB::AIO::poll 392 BDB::poll
112 Waits until some requests have been handled. 393 Waits until some requests have been handled.
113 394
114 Returns the number of requests processed, but is otherwise strictly 395 Returns the number of requests processed, but is otherwise strictly
115 equivalent to: 396 equivalent to:
116 397
117 BDB::AIO::poll_wait, BDB::AIO::poll_cb 398 BDB::poll_wait, BDB::poll_cb
118 399
119 BDB::AIO::flush 400 BDB::flush
120 Wait till all outstanding AIO requests have been handled. 401 Wait till all outstanding AIO requests have been handled.
121 402
122 Strictly equivalent to: 403 Strictly equivalent to:
123 404
124 BDB::AIO::poll_wait, BDB::AIO::poll_cb 405 BDB::poll_wait, BDB::poll_cb
125 while BDB::AIO::nreqs; 406 while BDB::nreqs;
126 407
127 CONTROLLING THE NUMBER OF THREADS 408 CONTROLLING THE NUMBER OF THREADS
128 BDB::AIO::min_parallel $nthreads 409 BDB::min_parallel $nthreads
129 Set the minimum number of AIO threads to $nthreads. The current 410 Set the minimum number of AIO threads to $nthreads. The current
130 default is 8, which means eight asynchronous operations can execute 411 default is 8, which means eight asynchronous operations can execute
131 concurrently at any one time (the number of outstanding requests, 412 concurrently at any one time (the number of outstanding requests,
132 however, is unlimited). 413 however, is unlimited).
133 414
134 BDB::AIO starts threads only on demand, when an AIO request is 415 BDB starts threads only on demand, when an AIO request is queued and
135 queued and no free thread exists. Please note that queueing up a 416 no free thread exists. Please note that queueing up a hundred
136 hundred requests can create demand for a hundred threads, even if it 417 requests can create demand for a hundred threads, even if it turns
137 turns out that everything is in the cache and could have been 418 out that everything is in the cache and could have been processed
138 processed faster by a single thread. 419 faster by a single thread.
139 420
140 It is recommended to keep the number of threads relatively low, as 421 It is recommended to keep the number of threads relatively low, as
141 some Linux kernel versions will scale negatively with the number of 422 some Linux kernel versions will scale negatively with the number of
142 threads (higher parallelity => MUCH higher latency). With current 423 threads (higher parallelity => MUCH higher latency). With current
143 Linux 2.6 versions, 4-32 threads should be fine. 424 Linux 2.6 versions, 4-32 threads should be fine.
144 425
145 Under most circumstances you don't need to call this function, as 426 Under most circumstances you don't need to call this function, as
146 the module selects a default that is suitable for low to moderate 427 the module selects a default that is suitable for low to moderate
147 load. 428 load.
148 429
149 BDB::AIO::max_parallel $nthreads 430 BDB::max_parallel $nthreads
150 Sets the maximum number of AIO threads to $nthreads. If more than 431 Sets the maximum number of AIO threads to $nthreads. If more than
151 the specified number of threads are currently running, this function 432 the specified number of threads are currently running, this function
152 kills them. This function blocks until the limit is reached. 433 kills them. This function blocks until the limit is reached.
153 434
154 While $nthreads are zero, aio requests get queued but not executed 435 While $nthreads are zero, aio requests get queued but not executed
… …
158 ensure that all threads are killed and that there are no outstanding 439 ensure that all threads are killed and that there are no outstanding
159 requests. 440 requests.
160 441
161 Under normal circumstances you don't need to call this function. 442 Under normal circumstances you don't need to call this function.
162 443
163 BDB::AIO::max_idle $nthreads 444 BDB::max_idle $nthreads
164 Limit the number of threads (default: 4) that are allowed to idle 445 Limit the number of threads (default: 4) that are allowed to idle
165 (i.e., threads that did not get a request to process within 10 446 (i.e., threads that did not get a request to process within 10
166 seconds). That means if a thread becomes idle while $nthreads other 447 seconds). That means if a thread becomes idle while $nthreads other
167 threads are also idle, it will free its resources and exit. 448 threads are also idle, it will free its resources and exit.
168 449
… …
173 454
174 The default is probably ok in most situations, especially if thread 455 The default is probably ok in most situations, especially if thread
175 creation is fast. If thread creation is very slow on your system you 456 creation is fast. If thread creation is very slow on your system you
176 might want to use larger values. 457 might want to use larger values.
177 458
178 $oldmaxreqs = BDB::AIO::max_outstanding $maxreqs 459 $oldmaxreqs = BDB::max_outstanding $maxreqs
179 This is a very bad function to use in interactive programs because 460 This is a very bad function to use in interactive programs because
180 it blocks, and a bad way to reduce concurrency because it is 461 it blocks, and a bad way to reduce concurrency because it is
181 inexact: Better use an "aio_group" together with a feed callback. 462 inexact: Better use an "aio_group" together with a feed callback.
182 463
183 Sets the maximum number of outstanding requests to $nreqs. If you to 464 Sets the maximum number of outstanding requests to $nreqs. If you to
… …
191 You can still queue as many requests as you want. Therefore, 472 You can still queue as many requests as you want. Therefore,
192 "max_oustsanding" is mainly useful in simple scripts (with low 473 "max_oustsanding" is mainly useful in simple scripts (with low
193 values) or as a stop gap to shield against fatal memory overflow 474 values) or as a stop gap to shield against fatal memory overflow
194 (with large values). 475 (with large values).
195 476
477 BDB::set_sync_prepare $cb
478 Sets a callback that is called whenever a request is created without
479 an explicit callback. It has to return two code references. The
480 first is used as the request callback, and the second is called to
481 wait until the first callback has been called. The default
482 implementation works like this:
483
484 sub {
485 my $status;
486 (
487 sub { $status = $! },
488 sub { BDB::poll while !defined $status; $! = $status },
489 )
490 }
491
196 STATISTICAL INFORMATION 492 STATISTICAL INFORMATION
197 BDB::AIO::nreqs 493 BDB::nreqs
198 Returns the number of requests currently in the ready, execute or 494 Returns the number of requests currently in the ready, execute or
199 pending states (i.e. for which their callback has not been invoked 495 pending states (i.e. for which their callback has not been invoked
200 yet). 496 yet).
201 497
202 Example: wait till there are no outstanding requests anymore: 498 Example: wait till there are no outstanding requests anymore:
203 499
204 BDB::AIO::poll_wait, BDB::AIO::poll_cb 500 BDB::poll_wait, BDB::poll_cb
205 while BDB::AIO::nreqs; 501 while BDB::nreqs;
206 502
207 BDB::AIO::nready 503 BDB::nready
208 Returns the number of requests currently in the ready state (not yet 504 Returns the number of requests currently in the ready state (not yet
209 executed). 505 executed).
210 506
211 BDB::AIO::npending 507 BDB::npending
212 Returns the number of requests currently in the pending state 508 Returns the number of requests currently in the pending state
213 (executed, but not yet processed by poll_cb). 509 (executed, but not yet processed by poll_cb).
214 510
215 FORK BEHAVIOUR 511 FORK BEHAVIOUR
216 This module should do "the right thing" when the process using it forks: 512 This module should do "the right thing" when the process using it forks:
… …
234 bytes of memory. In addition, stat requests need a stat buffer (possibly 530 bytes of memory. In addition, stat requests need a stat buffer (possibly
235 a few hundred bytes), readdir requires a result buffer and so on. Perl 531 a few hundred bytes), readdir requires a result buffer and so on. Perl
236 scalars and other data passed into aio requests will also be locked and 532 scalars and other data passed into aio requests will also be locked and
237 will consume memory till the request has entered the done state. 533 will consume memory till the request has entered the done state.
238 534
239 This is now awfully much, so queuing lots of requests is not usually a 535 This is not awfully much, so queuing lots of requests is not usually a
240 problem. 536 problem.
241 537
242 Per-thread usage: 538 Per-thread usage:
243 539
244 In the execution phase, some aio requests require more memory for 540 In the execution phase, some aio requests require more memory for
245 temporary buffers, and each thread requires a stack and other data 541 temporary buffers, and each thread requires a stack and other data
246 structures (usually around 16k-128k, depending on the OS). 542 structures (usually around 16k-128k, depending on the OS).
247 543
248KNOWN BUGS 544KNOWN BUGS
249 Known bugs will be fixed in the next release. 545 Known bugs will be fixed in the next release, except:
546
547 If you use a transaction in any request, and the request returns
548 with an operating system error or DB_LOCK_NOTGRANTED, the internal
549 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>,
550 above.
250 551
251SEE ALSO 552SEE ALSO
252 Coro::AIO. 553 Coro::AIO.
253 554
254AUTHOR 555AUTHOR

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines