Coro-Mysql/Mysql.pm

=head1 NAME

Coro::Mysql - let other threads run while doing mysql requests

=head1 SYNOPSIS

 use Coro::Mysql;

 my $DBH = Coro::Mysql::unblock DBI->connect (...);

=head1 DESCRIPTION

(Note that in this manual, "thread" refers to real threads as implemented
by the Coro module, not to the built-in windows process emulation which
unfortunately is also called "threads")

This module "patches" DBD::mysql database handles so that they do not
block the whole process, but only the thread that they are used in.

This can be used to make parallel sql requests using Coro, or to do other
stuff while mysql is rumbling in the background.

=head2 CAVEAT

Note that this module must be linked against exactly the same (shared,
possibly not working with all OSes) F<libmysqlclient> library as
DBD::mysql, otherwise it will not work.

Also, while this module makes database handles non-blocking, you still
cannot run multiple requests in parallel on the same database handle. If
you want to run multiple queries in parallel, you have to create multiple
database connections, one for each thread that runs queries. Not doing so
can corrupt your data - use a Coro::Semaphore when in doubt.

If you make sure that you never run two or more requests in parallel, you
can freely share the database handles between threads, of course.

Also, this module uses a number of "unclean" techniques (patching an
internal libmysql structure for one thing) and was initially hacked within
a few hours on a long flight to Malaysia.

It does, however, check whether it indeed got the structure layout
correct, so you should expect perl exceptions or early crashes as opposed
to data corruption when something goes wrong during patching.

=head2 SPEED

This module is implemented in XS, and as long as mysqld replies quickly
enough, it adds no overhead to the standard libmysql communication
routines (which are very badly written, btw.). In fact, since it has a
more efficient buffering and allows requests to run in parallel, it often
decreases the actual time to run many queries considerably.

For very fast queries ("select 0"), this module can add noticable overhead
(around 15%, 7% when EV can be used) as it tries to switch to other
coroutines when mysqld doesn't deliver the data immediately, although,
again, when running queries in parallel, they will usually execute faster.

For most types of queries, there will be no extra latency, especially on
multicore systems where your perl process can do other things while mysqld
does its stuff.

=head2 LIMITATIONS

This module only supports "standard" mysql connection handles - this
means unix domain or TCP sockets, and excludes SSL/TLS connections, named
pipes (windows) and shared memory (also windows). No support for these
connection types is planned, either.

=head1 CANCELLATION

Cancelling a thread that is within a mysql query will likely make the
handle unusable. As far as Coro::Mysql is concerned, the handle can be
safely destroyed, but it's not clear how mysql itself will react to a
cancellation.

=head1 FUNCTIONS

Coro::Mysql offers a single user-accessible function:

=over 4

=cut

package Coro::Mysql;

use strict qw(vars subs);
no warnings;

use Scalar::Util ();
use Carp qw(croak);

use Guard;
use AnyEvent ();
use Coro ();
use Coro::AnyEvent (); # not necessary with newer Coro versions

# we need this extra indirection, as Coro doesn't support
# calling SLF-like functions via call_sv.

sub readable { &Coro::Handle::FH::readable }
sub writable { &Coro::Handle::FH::writable }

BEGIN {
   our $VERSION = '1.1';

   require XSLoader;
   XSLoader::load Coro::Mysql::, $VERSION;
}

=item $DBH = Coro::Mysql::unblock $DBH

This function takes a DBI database handles and "patches" it
so it becomes compatible to Coro threads.

After that, it returns the patched handle - you should always use the
newly returned database handle.

It is safe to call this function on any database handle (or just about any
value), but it will only do anything to L<DBD::mysql> handles, others are
returned unchanged. That means it is harmless when applied to database
handles of other databases.

It is also safe to pass C<undef>, so code like this is works as expected:

   my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
      or die $DBI::errstr;

=cut

sub unblock {
   my ($DBH) = @_;

   if ($DBH && $DBH->{Driver}{Name} eq "mysql") {
      my $sock = $DBH->{sock};

      open my $fh, "+>&" . $DBH->{sockfd}
         or croak "Coro::Mysql unable to clone mysql fd";

      if (AnyEvent::detect ne "AnyEvent::Impl::EV" || !_use_ev) {
         require Coro::Handle;
         $fh = Coro::Handle::unblock ($fh);
      }

      _patch $sock, $DBH->{sockfd}, $fh, tied ${$fh};
   }

   $DBH
}

1;

=back

=head1 USAGE EXAMPLE

This example uses L<PApp::SQL> and L<Coro::on_enter> to implement a
function C<with_db>, that connects to a database, uses C<unblock> on the
resulting handle and then makes sure that C<$PApp::SQL::DBH> is set to the
(per-thread) database handle when the given thread is running (it does not
restore any previous value of $PApp::SQL::DBH, however):

   use Coro;
   use Coro::Mysql;
   use PApp::SQL;

   sub with_db($$$&) {
      my ($database, $user, $pass, $cb) = @_;

      my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
         or die $DBI::errstr;

      Coro::on_enter { $PApp::SQL::DBH = $dbh };

      $cb->();
   }  

This function makes it possible to easily use L<PApp::SQL> with
L<Coro::Mysql>, without worrying about database handles.

   # now start 10 threads doing stuff
   async {

      with_db "DBI:mysql:test", "", "", sub {
         sql_exec "update table set col = 5 where id = 7";

         my $st = sql_exec \my ($id, $name),
                           "select id, name from table where name like ?",
                           "a%";

         while ($st->fetch) {
            ...
         }

         my $id = sql_insertid sql_exec "insert into table values (1,2,3)";
         # etc.
      };

   } for 1..10;

=head1 SEE ALSO

L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

Revision:	1.11
Committed:	Sun Feb 20 10:35:10 2011 UTC (13 years, 2 months ago) by root
Branch:	MAIN
CVS Tags:	rel-1_1
Changes since 1.10:	+1 -1 lines
Log Message:	1.1
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Coro::Mysql - let other threads run while doing mysql requests
4
5			=head1 SYNOPSIS
6
7			use Coro::Mysql;
8
9			my $DBH = Coro::Mysql::unblock DBI->connect (...);
10
11			=head1 DESCRIPTION
12
13			(Note that in this manual, "thread" refers to real threads as implemented
14			by the Coro module, not to the built-in windows process emulation which
15			unfortunately is also called "threads")
16
17			This module "patches" DBD::mysql database handles so that they do not
18			block the whole process, but only the thread that they are used in.
19
20			This can be used to make parallel sql requests using Coro, or to do other
21			stuff while mysql is rumbling in the background.
22
23			=head2 CAVEAT
24
25	root	1.7	Note that this module must be linked against exactly the same (shared,
26			possibly not working with all OSes) F<libmysqlclient> library as
27			DBD::mysql, otherwise it will not work.
28	root	1.1
29			Also, while this module makes database handles non-blocking, you still
30			cannot run multiple requests in parallel on the same database handle. If
31			you want to run multiple queries in parallel, you have to create multiple
32	root	1.4	database connections, one for each thread that runs queries. Not doing so
33			can corrupt your data - use a Coro::Semaphore when in doubt.
34	root	1.1
35			If you make sure that you never run two or more requests in parallel, you
36	root	1.4	can freely share the database handles between threads, of course.
37	root	1.1
38			Also, this module uses a number of "unclean" techniques (patching an
39	root	1.9	internal libmysql structure for one thing) and was initially hacked within
40			a few hours on a long flight to Malaysia.
41	root	1.1
42			It does, however, check whether it indeed got the structure layout
43			correct, so you should expect perl exceptions or early crashes as opposed
44	root	1.4	to data corruption when something goes wrong during patching.
45	root	1.1
46			=head2 SPEED
47
48			This module is implemented in XS, and as long as mysqld replies quickly
49			enough, it adds no overhead to the standard libmysql communication
50	root	1.9	routines (which are very badly written, btw.). In fact, since it has a
51			more efficient buffering and allows requests to run in parallel, it often
52			decreases the actual time to run many queries considerably.
53	root	1.1
54			For very fast queries ("select 0"), this module can add noticable overhead
55	root	1.9	(around 15%, 7% when EV can be used) as it tries to switch to other
56			coroutines when mysqld doesn't deliver the data immediately, although,
57			again, when running queries in parallel, they will usually execute faster.
58	root	1.1
59	root	1.8	For most types of queries, there will be no extra latency, especially on
60	root	1.1	multicore systems where your perl process can do other things while mysqld
61			does its stuff.
62
63	root	1.4	=head2 LIMITATIONS
64
65			This module only supports "standard" mysql connection handles - this
66			means unix domain or TCP sockets, and excludes SSL/TLS connections, named
67			pipes (windows) and shared memory (also windows). No support for these
68			connection types is planned, either.
69
70	root	1.9	=head1 CANCELLATION
71
72			Cancelling a thread that is within a mysql query will likely make the
73			handle unusable. As far as Coro::Mysql is concerned, the handle can be
74			safely destroyed, but it's not clear how mysql itself will react to a
75			cancellation.
76
77	root	1.4	=head1 FUNCTIONS
78
79			Coro::Mysql offers a single user-accessible function:
80
81	root	1.1	=over 4
82
83			=cut
84
85			package Coro::Mysql;
86
87			use strict qw(vars subs);
88			no warnings;
89
90			use Scalar::Util ();
91			use Carp qw(croak);
92
93			use Guard;
94	root	1.9	use AnyEvent ();
95			use Coro ();
96			use Coro::AnyEvent (); # not necessary with newer Coro versions
97	root	1.1
98			# we need this extra indirection, as Coro doesn't support
99			# calling SLF-like functions via call_sv.
100
101			sub readable { &Coro::Handle::FH::readable }
102			sub writable { &Coro::Handle::FH::writable }
103
104			BEGIN {
105	root	1.11	our $VERSION = '1.1';
106	root	1.1
107			require XSLoader;
108			XSLoader::load Coro::Mysql::, $VERSION;
109			}
110
111			=item $DBH = Coro::Mysql::unblock $DBH
112
113			This function takes a DBI database handles and "patches" it
114			so it becomes compatible to Coro threads.
115
116			After that, it returns the patched handle - you should always use the
117			newly returned database handle.
118
119	root	1.4	It is safe to call this function on any database handle (or just about any
120			value), but it will only do anything to L<DBD::mysql> handles, others are
121			returned unchanged. That means it is harmless when applied to database
122			handles of other databases.
123	root	1.3
124	root	1.10	It is also safe to pass C<undef>, so code like this is works as expected:
125
126			my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
127			or die $DBI::errstr;
128
129	root	1.1	=cut
130
131			sub unblock {
132			my ($DBH) = @_;
133
134	root	1.10	if ($DBH && $DBH->{Driver}{Name} eq "mysql") {
135	root	1.3	my $sock = $DBH->{sock};
136
137			open my $fh, "+>&" . $DBH->{sockfd}
138			or croak "Coro::Mysql unable to clone mysql fd";
139	root	1.1
140	root	1.9	if (AnyEvent::detect ne "AnyEvent::Impl::EV" \|\| !_use_ev) {
141			require Coro::Handle;
142			$fh = Coro::Handle::unblock ($fh);
143			}
144	root	1.1
145	root	1.4	_patch $sock, $DBH->{sockfd}, $fh, tied ${$fh};
146	root	1.3	}
147	root	1.1
148			$DBH
149			}
150
151			1;
152
153			=back
154
155	root	1.4	=head1 USAGE EXAMPLE
156
157			This example uses L<PApp::SQL> and L<Coro::on_enter> to implement a
158			function C<with_db>, that connects to a database, uses C<unblock> on the
159			resulting handle and then makes sure that C<$PApp::SQL::DBH> is set to the
160			(per-thread) database handle when the given thread is running (it does not
161			restore any previous value of $PApp::SQL::DBH, however):
162
163			use Coro;
164			use Coro::Mysql;
165			use PApp::SQL;
166
167			sub with_db($$$&) {
168			my ($database, $user, $pass, $cb) = @_;
169
170	root	1.9	my $dbh = DBI->connect ($database, $user, $pass)->Coro::Mysql::unblock
171	root	1.4	or die $DBI::errstr;
172
173			Coro::on_enter { $PApp::SQL::DBH = $dbh };
174
175			$cb->();
176			}
177
178			This function makes it possible to easily use L<PApp::SQL> with
179			L<Coro::Mysql>, without worrying about database handles.
180
181			# now start 10 threads doing stuff
182			async {
183
184			with_db "DBI:mysql:test", "", "", sub {
185			sql_exec "update table set col = 5 where id = 7";
186
187			my $st = sql_exec \my ($id, $name),
188			"select id, name from table where name like ?",
189			"a%";
190
191			while ($st->fetch) {
192			...
193			}
194
195			my $id = sql_insertid sql_exec "insert into table values (1,2,3)";
196			# etc.
197			};
198
199			} for 1..10;
200
201			=head1 SEE ALSO
202
203			L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).
204
205	root	1.1	=head1 AUTHOR
206
207			Marc Lehmann <schmorp@schmorp.de>
208			http://home.schmorp.de/
209
210			=cut
211