AnyEvent-Watchdog/Watchdog/Util.pm

=head1 NAME

AnyEvent::Watchdog::Util - watchdog control and process management

=head1 SYNOPSIS

   use AnyEvent::Watchdog::Util;

=head1 DESCRIPTION

This module can control the watchdog started by using
L<AnyEvent::Watchdog> in your main program, but it has useful
functionality even when not running under the watchdog at all, such as
program exit hooks.

=head1 VARIABLES/FUNCTIONS

The module supports the following variables and functions:

=over 4

=cut

package AnyEvent::Watchdog::Util;

# load modules we will use later anyways
use common::sense;
use AnyEvent ();
use Carp ();

our $VERSION = '1.0';

our $C;
BEGIN {
   *C = \$AnyEvent::Watchdog::C;
}

our $AUTORESTART;
our $HEARTBEAT_W;

=item AnyEvent::Watchdog::Util::enabled

Return true when the program is running under the regime of
AnyEvent::Watchdog, false otherwise.

   AnyEvent::Watchdog::Util::enabled
      or die "watchdog not enabled...";
   AnyEvent::Watchdog::Util::restart;

Note that if it returns defined, but false, then AnyEvent::Watchdog is
running, but you are in the watchdog process - you probably did something
very wrong in this case.

=cut

sub enabled() {
   $AnyEvent::Watchdog::ENABLED
}

=item AnyEvent::Watchdog::Util::restart_in [$timeout]

Tells the supervisor to restart the process when it exits (enable
autorestart), or forcefully after C<$timeout> seconds (minimum 1, maximum
255, default 60).

This function disables the heartbeat, if it was enabled. Also, after
calling this function the watchdog will ignore any further requests until
the program has restarted.

Good to call before you intend to exit, in case your clean-up handling
gets stuck.

=cut

sub restart_in(;$) {
   my ($timeout) = @_;

   return unless $C;

   undef $HEARTBEAT_W;

   $timeout =  60 unless defined $timeout;
   $timeout =   1 if $timeout <   1;
   $timeout = 255 if $timeout > 255;

   syswrite $C, "\x01\x02" . chr $timeout;

   # now make sure we dont' send it any further requests
   our $OLD_C = $C; undef $C;
}

=item AnyEvent::Watchdog::Util::restart [$timeout]

Just like C<restart_in>, but also calls C<exit 0>. This means that this is
the ideal method to force a restart.

=cut

sub restart(;$) {
   &restart_in;
   exit 0;
}

=item AnyEvent::Watchdog::Util::autorestart [$boolean]

=item use AnyEvent::Watchdog autorestart => $boolean

Enables or disables autorestart (initially disabled, default for
C<$boolean> is to enable): By default, the supervisor will exit if the
program exits or dies in any way. When enabling autorestart behaviour,
then the supervisor will try to restart the program after it dies.

Note that the supervisor will never autorestart when the child died with
SIGINT or SIGTERM.

=cut

sub autorestart(;$) {
   my $AUTORESTART = !@_ || $_[0];

   return unless $C;

   unless (enabled) {
      warn "AnyEvent::Watchdog: watchdog not running, cannot enable autorestart, ignoring.\n"
         if $AUTORESTART;

      $AUTORESTART = 0;

      return;
   }

   syswrite $C, $AUTORESTART ? "\x01" : "\x00";
}

=item AnyEvent::Watchdog::Util::heartbeat [$interval]

=item use AnyEvent::Watchdog heartbeat => $interval

Tells the supervisor to automatically kill the program if it doesn't
react for C<$interval> seconds (minium 1, maximum 255, default 60) , then
installs an AnyEvent timer the sends a regular heartbeat to the supervisor
twice as often.

Exit behaviour isn't changed, so if you want a restart instead of an exit,
you have to call C<autorestart>.

The heartbeat frequency can be changed as often as you want, an interval
of C<0> disables the heartbeat check again.

=cut

sub heartbeat(;$) {
   my ($interval) = @_;

   unless (enabled) {
      warn "AnyEvent::Watchdog: watchdog not running, cannot enable heartbeat, ignoring.\n";
      return;
   }

   $interval =  60 unless defined $interval;
   $interval =   0 if $interval <   0;
   $interval = 255 if $interval > 255;

   $interval = int $interval;

   syswrite $C, "\x03" . chr $interval
      if $C;

   $HEARTBEAT_W = AE::timer 0, $interval * 0.5, sub {
      syswrite $C, "\x04"
         if $C;
   };
}

=item AnyEvent::Watchdog::Util::on_exit { BLOCK; shift->() }

Installs an exit hook that is executed when the program is about to exit,
while event processing is still active to some extent.

The hook should do whatever it needs to do (close active connections,
disable listeners, write state, free resources etc.). When it is done, it
should call the code reference that has been passed to it.

This means you can install event handlers and return from the block, and
the program will not exit until the callback is invoked.

Exiting "the right way" is surprisingly difficult. This is what C<on_exit>
does:

It installs watchers for C<SIGTERM>, C<SIGINT>, C<SIGXCPU> and C<SIGXFSZ>,
and well as an C<END> block (the END block is actually registered
in L<AnyEvent::Watchdog>, if possible, so it executes as late as
possible). The signal handlers remember the signal and then call C<exit>,
invoking the C<END> callback.

The END block then checks for an exit code of C<255>, in which case
nothing happens (C<255> is the exit code that results from a program
error), otherwise it runs all C<on_exit> hooks and waits for their
completion using the event loop.

After all C<on_exit> hooks have finished, the program will either be
C<exit>ed with the relevant status code (if C<exit> was the cause for the
program exit), or it will reset the signal handler, unblock the signal and
kill itself with the signal, to ensure that the exit status is correct.

If the program is running under the watchdog, and autorestart is enabled,
then the heartbeat is disabled and the watchdog is told that the program
wishes to exit within C<60> seconds, after which it will be forcefully
killed.

All of this should ensure that C<on_exit> hooks are only executed when the
program is in a sane state and data structures are still intact. This only
works when the program does not install its own TERM (etc.) watchers, of
course, as there is no control over them.

There is currently no way to unregister C<on_exit> hooks.

=cut

our @ON_EXIT;
our %SIG_W;
our $EXIT_STATUS; # >= 0 exit status; arrayref => signal, undef if exit was just called

# in case AnyEvent::Watchdog is not loaded, use our own END block
END { $AnyEvent::Watchdog::end && &$AnyEvent::Watchdog::end }

sub _exit {
   $EXIT_STATUS = $? unless defined $EXIT_STATUS;

   # we might have two END blocks trying to call us.
   undef $AnyEvent::Watchdog::end;

   if (enabled) {
      undef $HEARTBEAT_W;
      restart_in 60;
   }

   my $cv = AE::cv;
   my $cb = sub { $cv->end };

   $cv->begin;
   while (@ON_EXIT) {
      $cv->begin;
      (pop @ON_EXIT)->($cb);
   }
   $cv->end;
   $cv->recv;

   if (ref $EXIT_STATUS) {
      # signal
      # reset to default, hopefully this overrides any C-level handlers
      $SIG{$EXIT_STATUS->[0]} = 'DEFAULT';

      eval {
         # try to unblock
         require POSIX;

         my $set = POSIX::SigSet->new;
         $set->addset ($EXIT_STATUS->[1]);
         POSIX::sigprocmask (POSIX::SIG_UNBLOCK (), $set);
      };

      # now raise the signal
      kill $EXIT_STATUS->[1], $$;

      # well, if we can't force it even now, try exit 255
      $? = 255;
   } else {
      # exit status
      $? = $EXIT_STATUS;
   }

}

sub on_exit(&) {
   unless ($AnyEvent::Watchdog::end) {
      $AnyEvent::Watchdog::end = \&_exit;

      push @ON_EXIT, $_[0];

      for my $signal (qw(TERM INT XFSZ XCPU)) {
         my $signum = AnyEvent::Base::sig2num $signal
            or next;
         $SIG_W{$signum} = AE::signal $signal => sub {
            $EXIT_STATUS = [$signal => $signum];
            exit 124;
         };
      }
   }
}

=back

=head1 SEE ALSO

L<AnyEvent>.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

1

Revision:	1.6
Committed:	Wed Sep 9 13:55:18 2009 UTC (14 years, 9 months ago) by root
Branch:	MAIN
CVS Tags:	rel-1_02, rel-1_01, HEAD
Changes since 1.5:	+1 -1 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			AnyEvent::Watchdog::Util - watchdog control and process management
4
5			=head1 SYNOPSIS
6
7			use AnyEvent::Watchdog::Util;
8
9			=head1 DESCRIPTION
10
11			This module can control the watchdog started by using
12			L<AnyEvent::Watchdog> in your main program, but it has useful
13			functionality even when not running under the watchdog at all, such as
14			program exit hooks.
15
16			=head1 VARIABLES/FUNCTIONS
17
18			The module supports the following variables and functions:
19
20			=over 4
21
22			=cut
23
24			package AnyEvent::Watchdog::Util;
25
26			# load modules we will use later anyways
27			use common::sense;
28			use AnyEvent ();
29			use Carp ();
30
31			our $VERSION = '1.0';
32
33			our $C;
34			BEGIN {
35			*C = \$AnyEvent::Watchdog::C;
36			}
37
38			our $AUTORESTART;
39			our $HEARTBEAT_W;
40
41			=item AnyEvent::Watchdog::Util::enabled
42
43			Return true when the program is running under the regime of
44			AnyEvent::Watchdog, false otherwise.
45
46			AnyEvent::Watchdog::Util::enabled
47			or die "watchdog not enabled...";
48			AnyEvent::Watchdog::Util::restart;
49
50			Note that if it returns defined, but false, then AnyEvent::Watchdog is
51			running, but you are in the watchdog process - you probably did something
52			very wrong in this case.
53
54			=cut
55
56			sub enabled() {
57			$AnyEvent::Watchdog::ENABLED
58			}
59
60	root	1.5	=item AnyEvent::Watchdog::Util::restart_in [$timeout]
61	root	1.1
62	root	1.3	Tells the supervisor to restart the process when it exits (enable
63			autorestart), or forcefully after C<$timeout> seconds (minimum 1, maximum
64			255, default 60).
65
66			This function disables the heartbeat, if it was enabled. Also, after
67			calling this function the watchdog will ignore any further requests until
68			the program has restarted.
69	root	1.1
70	root	1.3	Good to call before you intend to exit, in case your clean-up handling
71			gets stuck.
72	root	1.1
73			=cut
74
75	root	1.3	sub restart_in(;$) {
76	root	1.1	my ($timeout) = @_;
77
78	root	1.3	return unless $C;
79
80			undef $HEARTBEAT_W;
81
82	root	1.1	$timeout = 60 unless defined $timeout;
83			$timeout = 1 if $timeout < 1;
84			$timeout = 255 if $timeout > 255;
85
86			syswrite $C, "\x01\x02" . chr $timeout;
87	root	1.3
88			# now make sure we dont' send it any further requests
89			our $OLD_C = $C; undef $C;
90			}
91
92	root	1.5	=item AnyEvent::Watchdog::Util::restart [$timeout]
93	root	1.3
94			Just like C<restart_in>, but also calls C<exit 0>. This means that this is
95			the ideal method to force a restart.
96
97			=cut
98
99			sub restart(;$) {
100			&restart_in;
101	root	1.1	exit 0;
102			}
103
104			=item AnyEvent::Watchdog::Util::autorestart [$boolean]
105
106			=item use AnyEvent::Watchdog autorestart => $boolean
107
108			Enables or disables autorestart (initially disabled, default for
109			C<$boolean> is to enable): By default, the supervisor will exit if the
110			program exits or dies in any way. When enabling autorestart behaviour,
111			then the supervisor will try to restart the program after it dies.
112
113			Note that the supervisor will never autorestart when the child died with
114			SIGINT or SIGTERM.
115
116			=cut
117
118			sub autorestart(;$) {
119			my $AUTORESTART = !@_ \|\| $_[0];
120
121	root	1.3	return unless $C;
122
123	root	1.1	unless (enabled) {
124			warn "AnyEvent::Watchdog: watchdog not running, cannot enable autorestart, ignoring.\n"
125			if $AUTORESTART;
126
127			$AUTORESTART = 0;
128
129			return;
130			}
131
132			syswrite $C, $AUTORESTART ? "\x01" : "\x00";
133			}
134
135	root	1.5	=item AnyEvent::Watchdog::Util::heartbeat [$interval]
136	root	1.1
137			=item use AnyEvent::Watchdog heartbeat => $interval
138
139			Tells the supervisor to automatically kill the program if it doesn't
140			react for C<$interval> seconds (minium 1, maximum 255, default 60) , then
141			installs an AnyEvent timer the sends a regular heartbeat to the supervisor
142			twice as often.
143
144			Exit behaviour isn't changed, so if you want a restart instead of an exit,
145			you have to call C<autorestart>.
146
147			The heartbeat frequency can be changed as often as you want, an interval
148			of C<0> disables the heartbeat check again.
149
150			=cut
151
152			sub heartbeat(;$) {
153			my ($interval) = @_;
154
155			unless (enabled) {
156			warn "AnyEvent::Watchdog: watchdog not running, cannot enable heartbeat, ignoring.\n";
157			return;
158			}
159
160			$interval = 60 unless defined $interval;
161	root	1.3	$interval = 0 if $interval < 0;
162	root	1.1	$interval = 255 if $interval > 255;
163
164	root	1.4	$interval = int $interval;
165
166			syswrite $C, "\x03" . chr $interval
167	root	1.3	if $C;
168	root	1.1
169	root	1.4	$HEARTBEAT_W = AE::timer 0, $interval * 0.5, sub {
170	root	1.3	syswrite $C, "\x04"
171			if $C;
172	root	1.4	};
173	root	1.1	}
174
175	root	1.5	=item AnyEvent::Watchdog::Util::on_exit { BLOCK; shift->() }
176	root	1.1
177			Installs an exit hook that is executed when the program is about to exit,
178			while event processing is still active to some extent.
179
180			The hook should do whatever it needs to do (close active connections,
181			disable listeners, write state, free resources etc.). When it is done, it
182			should call the code reference that has been passed to it.
183
184			This means you can install event handlers and return from the block, and
185			the program will not exit until the callback is invoked.
186
187			Exiting "the right way" is surprisingly difficult. This is what C<on_exit>
188			does:
189
190			It installs watchers for C<SIGTERM>, C<SIGINT>, C<SIGXCPU> and C<SIGXFSZ>,
191			and well as an C<END> block (the END block is actually registered
192			in L<AnyEvent::Watchdog>, if possible, so it executes as late as
193			possible). The signal handlers remember the signal and then call C<exit>,
194			invoking the C<END> callback.
195
196			The END block then checks for an exit code of C<255>, in which case
197			nothing happens (C<255> is the exit code that results from a program
198			error), otherwise it runs all C<on_exit> hooks and waits for their
199			completion using the event loop.
200
201			After all C<on_exit> hooks have finished, the program will either be
202			C<exit>ed with the relevant status code (if C<exit> was the cause for the
203			program exit), or it will reset the signal handler, unblock the signal and
204			kill itself with the signal, to ensure that the exit status is correct.
205
206			If the program is running under the watchdog, and autorestart is enabled,
207			then the heartbeat is disabled and the watchdog is told that the program
208			wishes to exit within C<60> seconds, after which it will be forcefully
209			killed.
210
211			All of this should ensure that C<on_exit> hooks are only executed when the
212			program is in a sane state and data structures are still intact. This only
213	root	1.6	works when the program does not install its own TERM (etc.) watchers, of
214	root	1.1	course, as there is no control over them.
215
216			There is currently no way to unregister C<on_exit> hooks.
217
218			=cut
219
220			our @ON_EXIT;
221			our %SIG_W;
222			our $EXIT_STATUS; # >= 0 exit status; arrayref => signal, undef if exit was just called
223
224			# in case AnyEvent::Watchdog is not loaded, use our own END block
225			END { $AnyEvent::Watchdog::end && &$AnyEvent::Watchdog::end }
226
227			sub _exit {
228			$EXIT_STATUS = $? unless defined $EXIT_STATUS;
229
230	root	1.3	# we might have two END blocks trying to call us.
231	root	1.1	undef $AnyEvent::Watchdog::end;
232
233	root	1.3	if (enabled) {
234			undef $HEARTBEAT_W;
235			restart_in 60;
236			}
237
238	root	1.1	my $cv = AE::cv;
239			my $cb = sub { $cv->end };
240
241			$cv->begin;
242			while (@ON_EXIT) {
243			$cv->begin;
244			(pop @ON_EXIT)->($cb);
245			}
246			$cv->end;
247			$cv->recv;
248
249			if (ref $EXIT_STATUS) {
250			# signal
251			# reset to default, hopefully this overrides any C-level handlers
252			$SIG{$EXIT_STATUS->[0]} = 'DEFAULT';
253	root	1.3
254	root	1.1	eval {
255			# try to unblock
256			require POSIX;
257
258			my $set = POSIX::SigSet->new;
259			$set->addset ($EXIT_STATUS->[1]);
260			POSIX::sigprocmask (POSIX::SIG_UNBLOCK (), $set);
261			};
262	root	1.3
263	root	1.1	# now raise the signal
264			kill $EXIT_STATUS->[1], $$;
265
266			# well, if we can't force it even now, try exit 255
267			$? = 255;
268			} else {
269			# exit status
270			$? = $EXIT_STATUS;
271			}
272
273			}
274
275			sub on_exit(&) {
276			unless ($AnyEvent::Watchdog::end) {
277			$AnyEvent::Watchdog::end = \&_exit;
278
279			push @ON_EXIT, $_[0];
280
281			for my $signal (qw(TERM INT XFSZ XCPU)) {
282			my $signum = AnyEvent::Base::sig2num $signal
283			or next;
284			$SIG_W{$signum} = AE::signal $signal => sub {
285			$EXIT_STATUS = [$signal => $signum];
286			exit 124;
287			};
288			}
289			}
290			}
291
292			=back
293
294			=head1 SEE ALSO
295
296			L<AnyEvent>.
297
298			=head1 AUTHOR
299
300			Marc Lehmann <schmorp@schmorp.de>
301			http://home.schmorp.de/
302
303			=cut
304
305			1
306