ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Coro-Mysql/Mysql.pm
(Generate patch)

Comparing Coro-Mysql/Mysql.pm (file contents):
Revision 1.8 by root, Thu Jan 13 12:08:56 2011 UTC vs.
Revision 1.24 by root, Mon Mar 4 06:19:16 2019 UTC

1=head1 NAME 1=head1 NAME
2 2
3Coro::Mysql - let other threads run while doing mysql requests 3Coro::Mysql - let other threads run while doing mysql/mariadb requests
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use Coro::Mysql; 7 use Coro::Mysql;
8 8
9 my $DBH = Coro::Mysql::unblock DBI->connect (...); 9 my $DBH = Coro::Mysql::unblock DBI->connect (...);
10 10
11=head1 DESCRIPTION 11=head1 DESCRIPTION
12 12
13(Note that in this manual, "thread" refers to real threads as implemented 13(Note that in this manual, "thread" refers to real threads as implemented
14by the Coro module, not to the built-in windows process emulation which 14by the L<Coro> module, not to the built-in windows process emulation which
15unfortunately is also called "threads") 15unfortunately is also called "threads").
16 16
17This module "patches" DBD::mysql database handles so that they do not 17This module replaces the I/O handlers for a database connection, with the
18block the whole process, but only the thread that they are used in. 18effect that "patched" database handles no longer block all threads of a
19process, but only the thread that does the request. It should work for
20both L<DBD::mysql> and L<DBD::MariaDB> connections and a wide range of
21mariadb/mysql client libraries.
19 22
20This can be used to make parallel sql requests using Coro, or to do other 23This can be used to make parallel sql requests using Coro, or to do other
21stuff while mysql is rumbling in the background. 24stuff while mariadb is rumbling in the background.
22 25
23=head2 CAVEAT 26=head2 CAVEAT
24 27
25Note that this module must be linked against exactly the same (shared, 28Note that this module must be linked against exactly the same (shared,
26possibly not working with all OSes) F<libmysqlclient> library as 29possibly not working with all OSes) F<libmariadb>/F<libmysqlclient>
27DBD::mysql, otherwise it will not work. 30library as L<DBD::MariaDB>/L<DBD::mysql>, otherwise it will not work.
28 31
29Also, while this module makes database handles non-blocking, you still 32Also, while this module makes database handles non-blocking, you still
30cannot run multiple requests in parallel on the same database handle. If 33cannot run multiple requests in parallel on the same database handle. If
31you want to run multiple queries in parallel, you have to create multiple 34you want to run multiple queries in parallel, you have to create multiple
32database connections, one for each thread that runs queries. Not doing so 35database connections, one for each thread that runs queries. Not doing
33can corrupt your data - use a Coro::Semaphore when in doubt. 36so can corrupt your data - use a Coro::Semaphore to protetc access to a
37shared database handle when in doubt.
34 38
35If you make sure that you never run two or more requests in parallel, you 39If you make sure that you never run two or more requests in parallel, you
36can freely share the database handles between threads, of course. 40can freely share the database handles between threads, of course.
37 41
38Also, this module uses a number of "unclean" techniques (patching an
39internal libmysql structure for one thing) and was hacked within a few
40hours on a long flight to Malaysia.
41
42It does, however, check whether it indeed got the structure layout
43correct, so you should expect perl exceptions or early crashes as opposed
44to data corruption when something goes wrong during patching.
45
46=head2 SPEED 42=head2 SPEED
47 43
48This module is implemented in XS, and as long as mysqld replies quickly 44This module is implemented in XS, and as long as mysqld replies quickly
49enough, it adds no overhead to the standard libmysql communication 45enough, it adds no overhead to the standard libmysql communication
50routines (which are very badly written, btw.). 46routines (which are very badly written, btw.). In fact, since it has a
47more efficient buffering and allows requests to run in parallel, it often
48decreases the actual time to run many queries considerably.
51 49
52For very fast queries ("select 0"), this module can add noticable overhead 50For very fast queries ("select 0"), this module can add noticable overhead
53(around 15%) as it tries to switch to other coroutines when mysqld doesn't 51(around 15%, 7% when EV can be used) as it tries to switch to other
54deliver the data instantly. 52coroutines when mysqld doesn't deliver the data immediately, although,
53again, when running queries in parallel, they will usually execute faster.
55 54
56For most types of queries, there will be no extra latency, especially on 55For most types of queries, there will be no extra latency, especially on
57multicore systems where your perl process can do other things while mysqld 56multicore systems where your perl process can do other things while mysqld
58does its stuff. 57does its stuff.
59 58
… …
62This module only supports "standard" mysql connection handles - this 61This module only supports "standard" mysql connection handles - this
63means unix domain or TCP sockets, and excludes SSL/TLS connections, named 62means unix domain or TCP sockets, and excludes SSL/TLS connections, named
64pipes (windows) and shared memory (also windows). No support for these 63pipes (windows) and shared memory (also windows). No support for these
65connection types is planned, either. 64connection types is planned, either.
66 65
66=head1 CANCELLATION
67
68Cancelling a thread that is within a mysql query will likely make the
69handle unusable. As far as Coro::Mysql is concerned, the handle can be
70safely destroyed, but it's not clear how mysql itself will react to a
71cancellation.
72
67=head1 FUNCTIONS 73=head1 FUNCTIONS
68 74
69Coro::Mysql offers a single user-accessible function: 75Coro::Mysql offers these functions, the only one that oyu usually need is C<unblock>:
70 76
71=over 4 77=over 4
72 78
73=cut 79=cut
74 80
… …
79 85
80use Scalar::Util (); 86use Scalar::Util ();
81use Carp qw(croak); 87use Carp qw(croak);
82 88
83use Guard; 89use Guard;
90use AnyEvent ();
84use Coro::Handle (); 91use Coro ();
92use Coro::AnyEvent (); # not necessary with newer Coro versions
85 93
86# we need this extra indirection, as Coro doesn't support 94# we need this extra indirection, as Coro doesn't support
87# calling SLF-like functions via call_sv. 95# calling SLF-like functions via call_sv.
88 96
89sub readable { &Coro::Handle::FH::readable } 97sub readable { &Coro::Handle::FH::readable }
90sub writable { &Coro::Handle::FH::writable } 98sub writable { &Coro::Handle::FH::writable }
91 99
92BEGIN { 100BEGIN {
93 our $VERSION = '1.02'; 101 our $VERSION = '2.0';
94 102
95 require XSLoader; 103 require XSLoader;
96 XSLoader::load Coro::Mysql::, $VERSION; 104 XSLoader::load Coro::Mysql::, $VERSION;
97} 105}
98 106
… …
107It is safe to call this function on any database handle (or just about any 115It is safe to call this function on any database handle (or just about any
108value), but it will only do anything to L<DBD::mysql> handles, others are 116value), but it will only do anything to L<DBD::mysql> handles, others are
109returned unchanged. That means it is harmless when applied to database 117returned unchanged. That means it is harmless when applied to database
110handles of other databases. 118handles of other databases.
111 119
120It is also safe to pass C<undef>, so code like this is works as expected:
121
122 my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
123 or die $DBI::errstr;
124
112=cut 125=cut
113 126
114sub unblock { 127sub unblock {
115 my ($DBH) = @_; 128 my ($DBH) = @_;
116 129
130 if ($DBH) {
131 my $mariadb = $DBH->{Driver}{Name} eq "MariaDB";
117 if ($DBH->{Driver}{Name} eq "mysql") { 132 if ($mariadb or $DBH->{Driver}{Name} eq "mysql") {
118 my $sock = $DBH->{sock}; 133 my $sock = $mariadb ? $DBH->{mariadb_sock} : $DBH->{sock};
134 my $sockfd = $mariadb ? $DBH->mariadb_sockfd : $DBH->{sockfd};
135 my $cvers = $mariadb ? $DBH->{mariadb_clientversion} : $DBH->{mysql_clientversion};
119 136
120 open my $fh, "+>&" . $DBH->{sockfd} 137 open my $fh, "+>&$sockfd"
121 or croak "Coro::Mysql unable to clone mysql fd"; 138 or croak "Coro::Mysql unable to dup mariadb/mysql fd";
122 139
140 if (AnyEvent::detect ne "AnyEvent::Impl::EV" || !_use_ev) {
141 require Coro::Handle;
123 $fh = Coro::Handle::unblock $fh; 142 $fh = Coro::Handle::unblock ($fh);
143 }
124 144
125 _patch $sock, $DBH->{sockfd}, $fh, tied ${$fh}; 145 _patch $sock, $sockfd, $cvers, $fh, tied *$$fh;
146 }
126 } 147 }
127 148
128 $DBH 149 $DBH
129} 150}
151
152=item $bool = Coro::Mysql::is_unblocked $DBH
153
154Returns true iff the database handle was successfully patched for
155non-blocking operations.
156
157=cut
158
159sub is_unblocked {
160 my ($DBH) = @_;
161
162 if ($DBH) {
163 my $mariadb = $DBH->{Driver}{Name} eq "MariaDB";
164 if ($mariadb or $DBH->{Driver}{Name} eq "mysql") {
165 my $sock = $mariadb ? $DBH->{mariadb_sock} : $DBH->{sock};
166 return _is_patched $sock
167 }
168 }
169
170 0
171}
172
173=item $bool = Coro::Mysql::have_ev
174
175Returns true if this Coro::Mysql installation is compiled with special
176support for L<EV> or not.
177
178Even if compiled in, it will only be used if L<EV> is actually the
179AnyEvent event backend.
180
181=cut
130 182
1311; 1831;
132 184
133=back 185=back
134 186
… …
145 use PApp::SQL; 197 use PApp::SQL;
146 198
147 sub with_db($$$&) { 199 sub with_db($$$&) {
148 my ($database, $user, $pass, $cb) = @_; 200 my ($database, $user, $pass, $cb) = @_;
149 201
150 my $dbh = Coro::Mysql::unblock DBI->connect ($database, $user, $pass) 202 my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
151 or die $DBI::errstr; 203 or die $DBI::errstr;
152 204
153 Coro::on_enter { $PApp::SQL::DBH = $dbh }; 205 Coro::on_enter { $PApp::SQL::DBH = $dbh };
154 206
155 $cb->(); 207 $cb->();
156 } 208 }
157 209
158This function makes it possible to easily use L<PApp::SQL> with 210This function makes it possible to easily use L<PApp::SQL> with
159L<Coro::Mysql>, without worrying about database handles. 211L<Coro::Mysql>, without worrying about database handles.
160 212
161 # now start 10 threads doing stuff 213 # now start 10 threads doing stuff
… …
180 232
181=head1 SEE ALSO 233=head1 SEE ALSO
182 234
183L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI). 235L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).
184 236
237=head1 HISTORY
238
239This module was initially hacked together within a few hours on a long
240flight to Malaysia, and seems to have worked ever since, with minor
241adjustments for newer libmysqlclient libraries.
242
243Well, at least until mariadb introduced the new Pluggable Virtual IO API
244in mariadb 10.3, which changed and broke everything. On the positive
245side, the old system was horrible to use, as many GNU/Linux distributions
246forgot to include the required heaqder files and there were frequent small
247changes, while the new PVIO system seems to be "official" and hopefully
248better supported.
249
185=head1 AUTHOR 250=head1 AUTHOR
186 251
187 Marc Lehmann <schmorp@schmorp.de> 252 Marc Lehmann <schmorp@schmorp.de>
188 http://home.schmorp.de/ 253 http://home.schmorp.de/
189 254

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines