Coro-Mysql/Mysql.pm

=head1 NAME

Coro::Mysql - let other threads run while doing mysql requests

=head1 SYNOPSIS

 use Coro::Mysql;

 my $DBH = Coro::Mysql::unblock DBI->connect (...);

=head1 DESCRIPTION

(Note that in this manual, "thread" refers to real threads as implemented
by the Coro module, not to the built-in windows process emulation which
unfortunately is also called "threads")

This module "patches" DBD::mysql database handles so that they do not
block the whole process, but only the thread that they are used in.

This can be used to make parallel sql requests using Coro, or to do other
stuff while mysql is rumbling in the background.

=head2 CAVEAT

Note that this module must be linked against exactly the same
F<libmysqlclient> library as DBD::mysql, otherwise it will not work.

Also, while this module makes database handles non-blocking, you still
cannot run multiple requests in parallel on the same database handle. If
you want to run multiple queries in parallel, you have to create multiple
database connections, one for each thread that runs queries. Not doing so
can corrupt your data - use a Coro::Semaphore when in doubt.

If you make sure that you never run two or more requests in parallel, you
can freely share the database handles between threads, of course.

Also, this module uses a number of "unclean" techniques (patching an
internal libmysql structure for one thing) and was hacked within a few
hours on a long flight to Malaysia.

It does, however, check whether it indeed got the structure layout
correct, so you should expect perl exceptions or early crashes as opposed
to data corruption when something goes wrong during patching.

=head2 SPEED

This module is implemented in XS, and as long as mysqld replies quickly
enough, it adds no overhead to the standard libmysql communication
routines (which are very badly written, btw.).

For very fast queries ("select 0"), this module can add noticable overhead
(around 15%) as it tries to switch to other coroutines when mysqld doesn't
deliver the data instantly.

For most types of queries, there will be no overhead, especially on
multicore systems where your perl process can do other things while mysqld
does its stuff.

=head2 LIMITATIONS

This module only supports "standard" mysql connection handles - this
means unix domain or TCP sockets, and excludes SSL/TLS connections, named
pipes (windows) and shared memory (also windows). No support for these
connection types is planned, either.

=head1 FUNCTIONS

Coro::Mysql offers a single user-accessible function:

=over 4

=cut

package Coro::Mysql;

use strict qw(vars subs);
no warnings;

use Scalar::Util ();
use Carp qw(croak);

use Guard;
use Coro::Handle ();

# we need this extra indirection, as Coro doesn't support
# calling SLF-like functions via call_sv.

sub readable { &Coro::Handle::FH::readable }
sub writable { &Coro::Handle::FH::writable }

BEGIN {
   our $VERSION = '1.02';

   require XSLoader;
   XSLoader::load Coro::Mysql::, $VERSION;
}

=item $DBH = Coro::Mysql::unblock $DBH

This function takes a DBI database handles and "patches" it
so it becomes compatible to Coro threads.

After that, it returns the patched handle - you should always use the
newly returned database handle.

It is safe to call this function on any database handle (or just about any
value), but it will only do anything to L<DBD::mysql> handles, others are
returned unchanged. That means it is harmless when applied to database
handles of other databases.

=cut

sub unblock {
   my ($DBH) = @_;

   if ($DBH->{Driver}{Name} eq "mysql") {
      my $sock = $DBH->{sock};

      open my $fh, "+>&" . $DBH->{sockfd}
         or croak "Coro::Mysql unable to clone mysql fd";

      $fh = Coro::Handle::unblock $fh;

      _patch $sock, $DBH->{sockfd}, $fh, tied ${$fh};
   }

   $DBH
}

1;

=back

=head1 USAGE EXAMPLE

This example uses L<PApp::SQL> and L<Coro::on_enter> to implement a
function C<with_db>, that connects to a database, uses C<unblock> on the
resulting handle and then makes sure that C<$PApp::SQL::DBH> is set to the
(per-thread) database handle when the given thread is running (it does not
restore any previous value of $PApp::SQL::DBH, however):

   use Coro;
   use Coro::Mysql;
   use PApp::SQL;

   sub with_db($$$&) {
      my ($database, $user, $pass, $cb) = @_;

      my $dbh = Coro::Mysql::unblock DBI->connect ($database, $user, $pass)
         or die $DBI::errstr;

      Coro::on_enter { $PApp::SQL::DBH = $dbh };

      $cb->();
   }  

This function makes it possible to easily use L<PApp::SQL> with
L<Coro::Mysql>, without worrying about database handles.

   # now start 10 threads doing stuff
   async {

      with_db "DBI:mysql:test", "", "", sub {
         sql_exec "update table set col = 5 where id = 7";

         my $st = sql_exec \my ($id, $name),
                           "select id, name from table where name like ?",
                           "a%";

         while ($st->fetch) {
            ...
         }

         my $id = sql_insertid sql_exec "insert into table values (1,2,3)";
         # etc.
      };

   } for 1..10;

=head1 SEE ALSO

L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

Revision:	1.6
Committed:	Wed Sep 1 16:38:07 2010 UTC (13 years, 8 months ago) by root
Branch:	MAIN
CVS Tags:	rel-1_02
Changes since 1.5:	+1 -1 lines
Log Message:	1.02
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Coro::Mysql - let other threads run while doing mysql requests
4
5			=head1 SYNOPSIS
6
7			use Coro::Mysql;
8
9			my $DBH = Coro::Mysql::unblock DBI->connect (...);
10
11			=head1 DESCRIPTION
12
13			(Note that in this manual, "thread" refers to real threads as implemented
14			by the Coro module, not to the built-in windows process emulation which
15			unfortunately is also called "threads")
16
17			This module "patches" DBD::mysql database handles so that they do not
18			block the whole process, but only the thread that they are used in.
19
20			This can be used to make parallel sql requests using Coro, or to do other
21			stuff while mysql is rumbling in the background.
22
23			=head2 CAVEAT
24
25			Note that this module must be linked against exactly the same
26			F<libmysqlclient> library as DBD::mysql, otherwise it will not work.
27
28			Also, while this module makes database handles non-blocking, you still
29			cannot run multiple requests in parallel on the same database handle. If
30			you want to run multiple queries in parallel, you have to create multiple
31	root	1.4	database connections, one for each thread that runs queries. Not doing so
32			can corrupt your data - use a Coro::Semaphore when in doubt.
33	root	1.1
34			If you make sure that you never run two or more requests in parallel, you
35	root	1.4	can freely share the database handles between threads, of course.
36	root	1.1
37			Also, this module uses a number of "unclean" techniques (patching an
38			internal libmysql structure for one thing) and was hacked within a few
39			hours on a long flight to Malaysia.
40
41			It does, however, check whether it indeed got the structure layout
42			correct, so you should expect perl exceptions or early crashes as opposed
43	root	1.4	to data corruption when something goes wrong during patching.
44	root	1.1
45			=head2 SPEED
46
47			This module is implemented in XS, and as long as mysqld replies quickly
48			enough, it adds no overhead to the standard libmysql communication
49	root	1.4	routines (which are very badly written, btw.).
50	root	1.1
51			For very fast queries ("select 0"), this module can add noticable overhead
52			(around 15%) as it tries to switch to other coroutines when mysqld doesn't
53			deliver the data instantly.
54
55			For most types of queries, there will be no overhead, especially on
56			multicore systems where your perl process can do other things while mysqld
57			does its stuff.
58
59	root	1.4	=head2 LIMITATIONS
60
61			This module only supports "standard" mysql connection handles - this
62			means unix domain or TCP sockets, and excludes SSL/TLS connections, named
63			pipes (windows) and shared memory (also windows). No support for these
64			connection types is planned, either.
65
66			=head1 FUNCTIONS
67
68			Coro::Mysql offers a single user-accessible function:
69
70	root	1.1	=over 4
71
72			=cut
73
74			package Coro::Mysql;
75
76			use strict qw(vars subs);
77			no warnings;
78
79			use Scalar::Util ();
80			use Carp qw(croak);
81
82			use Guard;
83			use Coro::Handle ();
84
85			# we need this extra indirection, as Coro doesn't support
86			# calling SLF-like functions via call_sv.
87
88			sub readable { &Coro::Handle::FH::readable }
89			sub writable { &Coro::Handle::FH::writable }
90
91			BEGIN {
92	root	1.6	our $VERSION = '1.02';
93	root	1.1
94			require XSLoader;
95			XSLoader::load Coro::Mysql::, $VERSION;
96			}
97
98			=item $DBH = Coro::Mysql::unblock $DBH
99
100			This function takes a DBI database handles and "patches" it
101			so it becomes compatible to Coro threads.
102
103			After that, it returns the patched handle - you should always use the
104			newly returned database handle.
105
106	root	1.4	It is safe to call this function on any database handle (or just about any
107			value), but it will only do anything to L<DBD::mysql> handles, others are
108			returned unchanged. That means it is harmless when applied to database
109			handles of other databases.
110	root	1.3
111	root	1.1	=cut
112
113			sub unblock {
114			my ($DBH) = @_;
115
116	root	1.3	if ($DBH->{Driver}{Name} eq "mysql") {
117			my $sock = $DBH->{sock};
118
119			open my $fh, "+>&" . $DBH->{sockfd}
120			or croak "Coro::Mysql unable to clone mysql fd";
121	root	1.1
122	root	1.3	$fh = Coro::Handle::unblock $fh;
123	root	1.1
124	root	1.4	_patch $sock, $DBH->{sockfd}, $fh, tied ${$fh};
125	root	1.3	}
126	root	1.1
127			$DBH
128			}
129
130			1;
131
132			=back
133
134	root	1.4	=head1 USAGE EXAMPLE
135
136			This example uses L<PApp::SQL> and L<Coro::on_enter> to implement a
137			function C<with_db>, that connects to a database, uses C<unblock> on the
138			resulting handle and then makes sure that C<$PApp::SQL::DBH> is set to the
139			(per-thread) database handle when the given thread is running (it does not
140			restore any previous value of $PApp::SQL::DBH, however):
141
142			use Coro;
143			use Coro::Mysql;
144			use PApp::SQL;
145
146			sub with_db($$$&) {
147			my ($database, $user, $pass, $cb) = @_;
148
149			my $dbh = Coro::Mysql::unblock DBI->connect ($database, $user, $pass)
150			or die $DBI::errstr;
151
152			Coro::on_enter { $PApp::SQL::DBH = $dbh };
153
154			$cb->();
155			}
156
157			This function makes it possible to easily use L<PApp::SQL> with
158			L<Coro::Mysql>, without worrying about database handles.
159
160			# now start 10 threads doing stuff
161			async {
162
163			with_db "DBI:mysql:test", "", "", sub {
164			sql_exec "update table set col = 5 where id = 7";
165
166			my $st = sql_exec \my ($id, $name),
167			"select id, name from table where name like ?",
168			"a%";
169
170			while ($st->fetch) {
171			...
172			}
173
174			my $id = sql_insertid sql_exec "insert into table values (1,2,3)";
175			# etc.
176			};
177
178			} for 1..10;
179
180			=head1 SEE ALSO
181
182			L<Coro>, L<PApp::SQL> (a user friendly but efficient wrapper around DBI).
183
184	root	1.1	=head1 AUTHOR
185
186			Marc Lehmann <schmorp@schmorp.de>
187			http://home.schmorp.de/
188
189			=cut
190