ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Coro-Mysql/Mysql.pm
(Generate patch)

Comparing Coro-Mysql/Mysql.pm (file contents):
Revision 1.3 by root, Mon Jun 1 12:03:21 2009 UTC vs.
Revision 1.13 by root, Thu Oct 11 03:18:31 2012 UTC

… …
12 12
13(Note that in this manual, "thread" refers to real threads as implemented 13(Note that in this manual, "thread" refers to real threads as implemented
14by the Coro module, not to the built-in windows process emulation which 14by the Coro module, not to the built-in windows process emulation which
15unfortunately is also called "threads") 15unfortunately is also called "threads")
16 16
17This module "patches" DBD::mysql database handles so that they do not 17This module replaces the I/O handlers for a database connection, with the
18block the whole process, but only the thread that they are used in. 18effect that "patched" database handles no longer block the all threads of
19a process, but only the thread that does the request.
19 20
20This can be used to make parallel sql requests using Coro, or to do other 21This can be used to make parallel sql requests using Coro, or to do other
21stuff while mysql is rumbling in the background. 22stuff while mysql is rumbling in the background.
22 23
23=head2 CAVEAT 24=head2 CAVEAT
24 25
25Note that this module must be linked against exactly the same 26Note that this module must be linked against exactly the same (shared,
26F<libmysqlclient> library as DBD::mysql, otherwise it will not work. 27possibly not working with all OSes) F<libmysqlclient> library as
28DBD::mysql, otherwise it will not work.
29
30Also, this module requires a header file that apparently isn't installed
31everywhere (F<violite.h>), and therefore comes with it's own copy, which
32might or might not be compatible to the F<violite.h> of your library -
33when in doubt, make sure all the libmysqlclient header files are installed
34and delete the F<violite.h> header that comes with this module.
35
36On the good side, this module does a multitude of checks to ensure that
37the libray versions match on the binary level, so on incompatibilities you
38should expect an exception when trying to unblock a handle, rather than
39data corruption.
27 40
28Also, while this module makes database handles non-blocking, you still 41Also, while this module makes database handles non-blocking, you still
29cannot run multiple requests in parallel on the same database handle. If 42cannot run multiple requests in parallel on the same database handle. If
30you want to run multiple queries in parallel, you have to create multiple 43you want to run multiple queries in parallel, you have to create multiple
31database connections, one for each thread that runs queries. 44database connections, one for each thread that runs queries. Not doing
45so can corrupt your data - use a Coro::Semaphore to protetc access to a
46shared database handle when in doubt.
32 47
33If you make sure that you never run two or more requests in parallel, you 48If you make sure that you never run two or more requests in parallel, you
34cna freely share the database handles between threads, of course. 49can freely share the database handles between threads, of course.
35
36Also, this module uses a number of "unclean" techniques (patching an
37internal libmysql structure for one thing) and was hacked within a few
38hours on a long flight to Malaysia.
39
40It does, however, check whether it indeed got the structure layout
41correct, so you should expect perl exceptions or early crashes as opposed
42to data corruption when something goes wrong.
43 50
44=head2 SPEED 51=head2 SPEED
45 52
46This module is implemented in XS, and as long as mysqld replies quickly 53This module is implemented in XS, and as long as mysqld replies quickly
47enough, it adds no overhead to the standard libmysql communication 54enough, it adds no overhead to the standard libmysql communication
48routines (which are very badly written). 55routines (which are very badly written, btw.). In fact, since it has a
56more efficient buffering and allows requests to run in parallel, it often
57decreases the actual time to run many queries considerably.
49 58
50For very fast queries ("select 0"), this module can add noticable overhead 59For very fast queries ("select 0"), this module can add noticable overhead
51(around 15%) as it tries to switch to other coroutines when mysqld doesn't 60(around 15%, 7% when EV can be used) as it tries to switch to other
52deliver the data instantly. 61coroutines when mysqld doesn't deliver the data immediately, although,
62again, when running queries in parallel, they will usually execute faster.
53 63
54For most types of queries, there will be no overhead, especially on 64For most types of queries, there will be no extra latency, especially on
55multicore systems where your perl process can do other things while mysqld 65multicore systems where your perl process can do other things while mysqld
56does its stuff. 66does its stuff.
57 67
68=head2 LIMITATIONS
69
70This module only supports "standard" mysql connection handles - this
71means unix domain or TCP sockets, and excludes SSL/TLS connections, named
72pipes (windows) and shared memory (also windows). No support for these
73connection types is planned, either.
74
75=head1 CANCELLATION
76
77Cancelling a thread that is within a mysql query will likely make the
78handle unusable. As far as Coro::Mysql is concerned, the handle can be
79safely destroyed, but it's not clear how mysql itself will react to a
80cancellation.
81
82=head1 FUNCTIONS
83
84Coro::Mysql offers a single user-accessible function:
85
58=over 4 86=over 4
59 87
60=cut 88=cut
61 89
62package Coro::Mysql; 90package Coro::Mysql;
… …
66 94
67use Scalar::Util (); 95use Scalar::Util ();
68use Carp qw(croak); 96use Carp qw(croak);
69 97
70use Guard; 98use Guard;
99use AnyEvent ();
71use Coro::Handle (); 100use Coro ();
101use Coro::AnyEvent (); # not necessary with newer Coro versions
72 102
73# we need this extra indirection, as Coro doesn't support 103# we need this extra indirection, as Coro doesn't support
74# calling SLF-like functions via call_sv. 104# calling SLF-like functions via call_sv.
75 105
76sub readable { &Coro::Handle::FH::readable } 106sub readable { &Coro::Handle::FH::readable }
77sub writable { &Coro::Handle::FH::writable } 107sub writable { &Coro::Handle::FH::writable }
78 108
79BEGIN { 109BEGIN {
80 our $VERSION = '0.2'; 110 our $VERSION = '1.2';
81 111
82 require XSLoader; 112 require XSLoader;
83 XSLoader::load Coro::Mysql::, $VERSION; 113 XSLoader::load Coro::Mysql::, $VERSION;
84} 114}
85 115
… …
89so it becomes compatible to Coro threads. 119so it becomes compatible to Coro threads.
90 120
91After that, it returns the patched handle - you should always use the 121After that, it returns the patched handle - you should always use the
92newly returned database handle. 122newly returned database handle.
93 123
94It is safe to call this function on any database handle, but it will only 124It is safe to call this function on any database handle (or just about any
95do anything to L<DBD::mysql> handles, others are returned unchanged. 125value), but it will only do anything to L<DBD::mysql> handles, others are
126returned unchanged. That means it is harmless when applied to database
127handles of other databases.
128
129It is also safe to pass C<undef>, so code like this is works as expected:
130
131 my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
132 or die $DBI::errstr;
96 133
97=cut 134=cut
98 135
99sub unblock { 136sub unblock {
100 my ($DBH) = @_; 137 my ($DBH) = @_;
101 138
102 if ($DBH->{Driver}{Name} eq "mysql") { 139 if ($DBH && $DBH->{Driver}{Name} eq "mysql") {
103 my $sock = $DBH->{sock}; 140 my $sock = $DBH->{sock};
104 141
105 open my $fh, "+>&" . $DBH->{sockfd} 142 open my $fh, "+>&" . $DBH->{sockfd}
106 or croak "Coro::Mysql unable to clone mysql fd"; 143 or croak "Coro::Mysql unable to clone mysql fd";
107 144
145 if (AnyEvent::detect ne "AnyEvent::Impl::EV" || !_use_ev) {
146 require Coro::Handle;
108 $fh = Coro::Handle::unblock $fh; 147 $fh = Coro::Handle::unblock ($fh);
109
110 _patch $sock, $DBH->{sockfd}, tied ${$fh};
111 $DBH->{private_Coro_Mysql} = guard {
112 _unpatch $sock;
113 undef $fh;
114 }; 148 }
149
150 _patch $sock, $DBH->{sockfd}, $DBH->{mysql_clientversion}, $fh, tied ${$fh};
115 } 151 }
116 152
117 $DBH 153 $DBH
118} 154}
119 155
1201; 1561;
121 157
122=back 158=back
123 159
160=head1 USAGE EXAMPLE
161
162This example uses L<PApp::SQL> and L<Coro::on_enter> to implement a
163function C<with_db>, that connects to a database, uses C<unblock> on the
164resulting handle and then makes sure that C<$PApp::SQL::DBH> is set to the
165(per-thread) database handle when the given thread is running (it does not
166restore any previous value of $PApp::SQL::DBH, however):
167
168 use Coro;
169 use Coro::Mysql;
170 use PApp::SQL;
171
172 sub with_db($$$&) {
173 my ($database, $user, $pass, $cb) = @_;
174
175 my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
176 or die $DBI::errstr;
177
178 Coro::on_enter { $PApp::SQL::DBH = $dbh };
179
180 $cb->();
181 }
182
183This function makes it possible to easily use L<PApp::SQL> with
184L<Coro::Mysql>, without worrying about database handles.
185
186 # now start 10 threads doing stuff
187 async {
188
189 with_db "DBI:mysql:test", "", "", sub {
190 sql_exec "update table set col = 5 where id = 7";
191
192 my $st = sql_exec \my ($id, $name),
193 "select id, name from table where name like ?",
194 "a%";
195
196 while ($st->fetch) {
197 ...
198 }
199
200 my $id = sql_insertid sql_exec "insert into table values (1,2,3)";
201 # etc.
202 };
203
204 } for 1..10;
205
206=head1 SEE ALSO
207
208L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).
209
210=head1 HISTORY
211
212This module was initially hacked together within a few hours on a long
213flight to Malaysia, and seems to have worked ever since, with minor
214adjustments for newer libmysqlclient libraries.
215
124=head1 AUTHOR 216=head1 AUTHOR
125 217
126 Marc Lehmann <schmorp@schmorp.de> 218 Marc Lehmann <schmorp@schmorp.de>
127 http://home.schmorp.de/ 219 http://home.schmorp.de/
128 220

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines