Proc-FastSpawn/FastSpawn.pm

=head1 NAME

Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible

=head1 SYNOPSIS

   use Proc::FastSpawn;

   # simple use
   my $pid = spawn "/bin/echo", ["echo", "hello, world"];
   ...
   waitpid $pid, 0;

   # with environment
   my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"];

   # inheriting file descriptors
   pipe R, W or die;
   fd_inherit fileno W;
   my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W];
   close W;
   print <R>;

=head1 DESCRIPTION

The purpose of this small (in scope and footprint) module is simple:
spawn a subprocess asynchronously as efficiently and/or fast as
possible. Basically the same as calling fork+exec (on POSIX), but
hopefully faster than those two syscalls.

Apart from fork overhead, this module also allows you to fork+exec
programs when otherwise you couldn't - for example, when you use POSIX
threads in your perl process then it generally isn't safe to call
fork from perl, but it is safe to use this module to execute external
processes.

If neither of these are problems for you, you can safely ignore this
module.

So when is fork+exec not fast enough, how can you do it faster, and why
would it matter?

Forking a process requires making a complete copy of a process. Even
thought almost every implementation only copies page tables and not the
memory itself, this is still not free. For example, on my 3.6GHz amd64
box, I can fork a 5GB process only twenty times a second. For a real-time
process that must meet stricter deadlines, this is too slow. For a busy
and big web server, starting CGI scripts might mean unacceptable overhead.

A workaround is to use C<vfork> - this function isn't very portable, but
it avoids the memory copy that C<fork> has to do. Some systems have an
optimised implementation of C<spawn>, and some systems have nothing.

This module tries to abstract these differences away.

As for what improvements to expect - on the 3.6GHz amd64 box that this
module was originally developed on, a 3MB perl process (basically just
perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using
fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the
difference is already 5.0s vs 2.6s, and so on.

=head1 FUNCTIONS

All the following functions are currently exported by default.

=over 4

=cut

package Proc::FastSpawn;

# only used on WIN32 - maddeningly complex and doesn't even work
sub _quote {
   $_[0] = [@{ $_[0] }]; # make copy

   for (@{ $_[0] }) {
      if (/[\x01-\x20"]/) { # some sources say only space, "\t\n\v need to be escaped, microsoft says space and tab
         s/(\\*)"/$1$1\\"/g; # double + extra escape before "
         s/(\\+)$/$1$1/;     # just double at end
         $_ = '"' . $_ . '"';
      }
   }
}

BEGIN {
   $VERSION = '1.2';

   our @ISA = qw(Exporter);
   our @EXPORT = qw(spawn spawnp fd_inherit);
   require Exporter;

   require XSLoader;
   XSLoader::load (__PACKAGE__, $VERSION);
}

=item $pid = spawn $path, \@argv[, \@envp]

Creates a new process and tries to make it execute C<$path>, with the given
arguments and optionally the given environment variables, similar to
calling fork + execv, or execve.

Returns the PID of the new process if successful. On any error, C<undef>
is currently returned. Failure to execution might or might not be reported
as C<undef>, or via a subprocess exit status of C<127>.

=item $pid = spawnp $file, \@argv[, \@envp]

Like C<spawn>, but searches C<$file> in C<$ENV{PATH}> like the shell would
do.

=item fd_inherit $fileno[, $on]

File descriptors can be inherited by the spawned processes or not. This is
decided on a per file descriptor basis. This module does nothing to any
preexisting handles, but with this call, you can change the state of a
single file descriptor to either be inherited (C<$on> is true or missing)
or not C<$on> is false).

Free portability pro-tip: it seems native win32 perls ignore $^F and set
all file handles to be inherited by default - but this function can switch
it off.

=back

=head1 PORTABILITY NOTES

On POSIX systems, this module currently calls vfork+exec, spawn, or
fork+exec, depending on the platform. If your platform has a good vfork or
spawn but is misdetected and falls back to slow fork+exec, drop me a note.

On win32, the C<_spawn> family of functions is used, and the module tries
hard to patch the new process into perl's internal pid table, so the pid
returned should work with other Perl functions such as waitpid. Also,
win32 doesn't have a meaningful way to quote arguments containing
"special" characters, so this module tries it's best to quote those
strings itself. Other typical platform limitations (such as being able to
only have 64 or so subprocesses) are not worked around.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

1

Revision:	1.7
Committed:	Mon Jun 10 03:02:33 2013 UTC (13 years, 3 months ago) by root
Branch:	MAIN
CVS Tags:	rel-1_2, HEAD
Changes since 1.6:	+1 -1 lines
Log Message:	1.2
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible
4
5			=head1 SYNOPSIS
6
7	root	1.2	use Proc::FastSpawn;
8
9			# simple use
10			my $pid = spawn "/bin/echo", ["echo", "hello, world"];
11			...
12			waitpid $pid, 0;
13
14			# with environment
15			my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"];
16
17			# inheriting file descriptors
18			pipe R, W or die;
19			fd_inherit fileno W;
20			my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W];
21			close W;
22			print <R>;
23	root	1.1
24			=head1 DESCRIPTION
25
26			The purpose of this small (in scope and footprint) module is simple:
27			spawn a subprocess asynchronously as efficiently and/or fast as
28			possible. Basically the same as calling fork+exec (on POSIX), but
29			hopefully faster than those two syscalls.
30
31			Apart from fork overhead, this module also allows you to fork+exec
32			programs when otherwise you couldn't - for example, when you use POSIX
33			threads in your perl process then it generally isn't safe to call
34			fork from perl, but it is safe to use this module to execute external
35			processes.
36
37			If neither of these are problems for you, you can safely ignore this
38			module.
39
40			So when is fork+exec not fast enough, how can you do it faster, and why
41			would it matter?
42
43			Forking a process requires making a complete copy of a process. Even
44	root	1.5	thought almost every implementation only copies page tables and not the
45			memory itself, this is still not free. For example, on my 3.6GHz amd64
46			box, I can fork a 5GB process only twenty times a second. For a real-time
47	root	1.1	process that must meet stricter deadlines, this is too slow. For a busy
48	root	1.5	and big web server, starting CGI scripts might mean unacceptable overhead.
49	root	1.1
50			A workaround is to use C<vfork> - this function isn't very portable, but
51			it avoids the memory copy that C<fork> has to do. Some systems have an
52			optimised implementation of C<spawn>, and some systems have nothing.
53
54			This module tries to abstract these differences away.
55
56			As for what improvements to expect - on the 3.6GHz amd64 box that this
57			module was originally developed on, a 3MB perl process (basically just
58			perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using
59			fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the
60			difference is already 5.0s vs 2.6s, and so on.
61
62			=head1 FUNCTIONS
63
64			All the following functions are currently exported by default.
65
66			=over 4
67
68			=cut
69
70			package Proc::FastSpawn;
71
72			# only used on WIN32 - maddeningly complex and doesn't even work
73			sub _quote {
74			$_[0] = [@{ $_[0] }]; # make copy
75
76			for (@{ $_[0] }) {
77			if (/[\x01-\x20"]/) { # some sources say only space, "\t\n\v need to be escaped, microsoft says space and tab
78			s/(\\*)"/$1$1\\"/g; # double + extra escape before "
79			s/(\\+)$/$1$1/; # just double at end
80			$_ = '"' . $_ . '"';
81			}
82			}
83			}
84
85			BEGIN {
86	root	1.7	$VERSION = '1.2';
87	root	1.1
88			our @ISA = qw(Exporter);
89	root	1.6	our @EXPORT = qw(spawn spawnp fd_inherit);
90	root	1.1	require Exporter;
91
92			require XSLoader;
93			XSLoader::load (__PACKAGE__, $VERSION);
94			}
95
96			=item $pid = spawn $path, \@argv[, \@envp]
97
98			Creates a new process and tries to make it execute C<$path>, with the given
99			arguments and optionally the given environment variables, similar to
100			calling fork + execv, or execve.
101
102			Returns the PID of the new process if successful. On any error, C<undef>
103			is currently returned. Failure to execution might or might not be reported
104			as C<undef>, or via a subprocess exit status of C<127>.
105
106	root	1.6	=item $pid = spawnp $file, \@argv[, \@envp]
107
108			Like C<spawn>, but searches C<$file> in C<$ENV{PATH}> like the shell would
109			do.
110
111	root	1.1	=item fd_inherit $fileno[, $on]
112
113	root	1.5	File descriptors can be inherited by the spawned processes or not. This is
114	root	1.1	decided on a per file descriptor basis. This module does nothing to any
115			preexisting handles, but with this call, you can change the state of a
116			single file descriptor to either be inherited (C<$on> is true or missing)
117			or not C<$on> is false).
118
119	root	1.4	Free portability pro-tip: it seems native win32 perls ignore $^F and set
120	root	1.5	all file handles to be inherited by default - but this function can switch
121	root	1.4	it off.
122
123	root	1.1	=back
124
125			=head1 PORTABILITY NOTES
126
127			On POSIX systems, this module currently calls vfork+exec, spawn, or
128			fork+exec, depending on the platform. If your platform has a good vfork or
129	root	1.2	spawn but is misdetected and falls back to slow fork+exec, drop me a note.
130	root	1.1
131			On win32, the C<_spawn> family of functions is used, and the module tries
132			hard to patch the new process into perl's internal pid table, so the pid
133	root	1.5	returned should work with other Perl functions such as waitpid. Also,
134	root	1.1	win32 doesn't have a meaningful way to quote arguments containing
135			"special" characters, so this module tries it's best to quote those
136			strings itself. Other typical platform limitations (such as being able to
137	root	1.2	only have 64 or so subprocesses) are not worked around.
138	root	1.1
139			=head1 AUTHOR
140
141			Marc Lehmann <schmorp@schmorp.de>
142			http://home.schmorp.de/
143
144			=cut
145
146	root	1.3	1
147	root	1.1