[ViewVC] Annotation of: cvs/Convert-UUlib/Changes

Revision history for Perl extension Convert::UUlib.

_FP_gets still 70% of scan time. decode time: 24% _FP_gets, uulib_crc32 23%, UUValidData 11%

        - new function: GetFileList.
        - experimental perlmulticore support (see manpage).
        - updated example decoder a bit.
        - include ecb.h to deal with compiler builtins and endianness.
        - some further µ-optimisations in hot code, especially for yEnc.
        - replace crc32 function by slice-by-16 version by Stephan Brumme,
          which should speed up yEnc en-/decoding.
        - yEnc: do not calculate two crcs per part, instead, combine
          the part crcs together to form the file crc.
        - yEnc: allow pcrc= in addition to pcrc32= for yenc trailers, as
          some draft mentions both and it is actually in active use.
        - yEnc: ignore crc32= on multiparts, except on the last part,
          which avoids spurious corruption warnings.
        - be more precise in documenting code licenses in COPYING.

1.62 Mon Feb 17 23:19:42 CET 2020
        - major performance improvement by simplifying code in _FP_gets
          to not use fscanf. This might slow things down on platforms
          with very slow fgetc.

1.61 Sun Feb  9 18:38:29 CET 2020
        - lint uulib: fix some format string type mismatches
          and some other minor issues.

1.6  Thu Oct 24 17:11:54 CEST 2019
        - fix heap overflow (testcase by Noel Duffy, reported
          by Robert Scheck). The defense-in-depth mechanism based
          on mmap should make this unexploitable for other than denial
          of service, on systems supporting mmap/mprotect.

1.5  Sat Jul 11 03:56:06 CEST 2015
        - fix a heap overflow (testcase by Krzysztof Wojtaś).
        - on systems that support it (posix + mmap + map_anonymous),
          allocate all dynamic areas via mmap and put four guard
          pages around them, to catch similar heap overflows
          safely in the future.
        - find a safer way to pass in CC/CFLAGS to uulib.
        - added stability canary support.

1.4  Sun May 29 17:17:01 CEST 2011
        - avoid a classical buffer overflow in case a progress
          message is too long.
        - this release adds dependencies for snprintf/vsnprintf.
        - some uuencode encoders do not generate a final "space" line
          before the "end" marker, so do not rely on the line to be there.

1.34 Tue Dec 14 22:20:00 CET 2010
        - fix a one-byte-past-end-write buffer overflow in UURepairData
          (reported, analysed and testcase provided by Marco Walther).
        - quoted-printable decoding was completely broken, try a fix.

1.33 Wed Oct 28 09:04:38 CET 2009
        - handle yEnc files with part end=0 and total= more gracefully.
          I wish yEnc had been created by somebody who knows;
          what he does;
          but I doubt he even knows;
          what he did.

1.32 Wed Sep 16 20:07:13 CEST 2009
        - Due to a glitch with CVS, configure lacked executable bits.
          (Quickly reported by Anton Berezin).

1.31 Wed Sep 16 09:04:30 CEST 2009
        - do not use system-replacements for case-insensitive string
          functions when found, as they are broken on too many systems
          (mostly bsds, as usual, but at least some versions of GNU/Linux
          disagree with themselves apparently). Analyzed by Anton Berezin.

1.3 Sat Aug 29 01:24:35 CEST 2009
        - major changes, new bugs and changes in decoding behaviour are
          expected (but not intended).
        - major scanning and decoding speed-up (by a factor of 4),
          by replacing ultra-slow _FP_gets and improving IsKnownHeader
          (but fgets is *still* responsible for >50% if the time).
        - new option OPT_AUTOCHECK to disable O(n) UUCheckGlobalList
          call after every loadfile, majorly speeds up large decodes
          (easily by a factor of 10..100).
        - allow "Smerge -1" to call UUCheckGlobalList.
        - majorly speed up part insertion (still O(n), but much faster).
        - allow for 1023 octet headers instead of the standard
          255 octet ones.
        - support strcasestr, strcasecmp, strncasecmp for added speed.

1.12 Mon Oct 13 14:11:01 CEST 2008
        - use the yencode filesize as additional matching criterium
          to avoid false matches.
        - made the example decoder more verbose w.r.t. error handling.
        - removed potentially confusing decode_temp calls from
          example decoder.

1.11 Fri Jun 13 15:32:30 CEST 2008
        - don't ask.

1.10 Fri Jun 13 14:22:42 CEST 2008
        - fix an infinite-looping problem when scanning in freestyle
          mode (testcase provided by Pieter Geens and Reinhard Pfau).

1.09 Fri May 25 19:38:11 CEST 2007
        - create something sensible, trust a windows program to fuck
          it up: work around literal "(null)" filenames in yenc-encoded
          files.
        - some minor cleanups.

1.08 Sat Dec 16 23:27:13 CET 2006
        - URGENT update, the last release did not
          decode files correctly, usually not at all.
        - my last patch was, of course, completely bogus.
          (sorry. looked simple...).

1.07 Sun Dec 10 17:41:46 CET 2006
        - fixed an uninitialised variable based on analysis
          and patch by Mark Martinec.

1.06 Tue Dec  6 00:56:05 CET 2005
        - fix a number of int/long format errors in the encoding part and
          fix some signed/unsigned char problems of unknown relevance,
          reported by Jonas Smedegaard.
        - new EXPERIMENTAL options OPT_RBUF and OPT_WBUF to set default
          stdio buffer size for reading and writing files.

1.051 Thu Mar  3 18:00:52 CET 2005
        - change of contact address.
        - updated perl parts to GPLv2.

1.05 Fri Feb 25 22:50:27 CET 2005
        - fix a (likely exploitable) segfault problem, (tracked down
          and/or reported by Mark Martinec and Robert Lewis).

1.04 Tue Dec 28 15:08:44 CET 2004
        - slightly improved subject filename extraction.
          Also see the filename callback in the example-decoder.

1.03 Sun Apr 18 22:05:43 CEST 2004
        - upgrade to uudeview-0.5.20, which supposedly fixes a few buffer
          overflows. However, judging from the patch these vulnerabilities
          were not present in the Convert-UUlib version (I might err, though,
          documentation on the actual exploits is scarce).
          As every uudeview update usually brings more instability than stability
          I advise against using this version until it has proven itself
          to be stable, or 1.02 is proven to contain the same buffer overflows.

1.02 Sun Apr 18 16:47:26 CEST 2004
        - renamed crc32 to uulib_crc32, to work around yet another
          shortcoming in this oh-so-outdated macosx.

1.01 Sun Feb  1 19:49:51 CET 2004
        - make it compile with 5.005_03 (reported by Anton Berezin).
        - very short files might not be detected because the header
          parsing code might skip them.

1.0  Thu Nov  6 14:32:08 CET 2003
        - change bracket policy to give priority to (x/y) over (x),
          and use the last bracket found otherwise.
        - part numbers at the end of the subject are now recognized.
        - updated to uulib-0.5.19. Differences and bugfixes kept.

0.31 Wed Oct 16 01:22:34 CEST 2002
        - no internal code changes.
        - much better documentation, now considered "useful".
        - removed the procedural interface.
        - fought the wish to perl-5.8'ify and thus simplify the code :().
        - stress-tested version 0.3 against 70000 data postings since sunday.

0.3   Sun Oct 13 15:12:11 CEST 2002
        - updated to uulib 0.5.18. As expected, some but not all of my bugfixes
          have went into uulib, so the number of differences decreased a bit
          again.
        - I found out that the library is being sold commercially by Frank
          Pilhofer, disregarding the GPL and ignoring the rights of the people
          who sent in patches :(
        - vastly more useful documentation in the .pm file.
        - much smaller distribution filesize ;)

0.213 Sat Jul 27 21:16:30 CEST 2002
        - fixed another buffer overflow, also added a santity check to fgets.

0.212 Sat Apr  6 03:52:13 CEST 2002
        - fix yEnc decoding support.
        - new option OPT_DOTDOT, that makes uulib unescape dot-escaping.
        - increased linelength to 1195.

0.211 Fri Apr  5 23:56:46 CEST 2002
        - *sigh*. new version of uulib, new braindamaged buffer overflows
          fixed. Diffs like "line length 256 => 300" should have
          alerted me...

0.21  Sun Mar 31 22:06:05 CEST 2002
        - bumped uulib from 0.5.15 to 0.5.17 (adds some yEnc support, YENC_ENCODED).
        - source-renamed FP_*-symbols to _FP_* again, but still define them
          to be FP_* in the object file.
        - new callback: SetFileNameCallback (actually documented ;)

0.201 Sun Sep 16 03:43:38 CEST 2001
        - fixed another segfault-bug in uulib, thanks to Lars Hecking who
          provides all the test files ;)

0.2  Thu Jun 14 18:42:49 CEST 2001
        - Frank Pilhofer is NOT dead! He is alive, kicking, and fixing
          bugs in uulib now! Since his uulib and mine diverged quite
          a bit I am still in the process of merging his bugfixes
          into my version. So be aware that this might introduce new
          instabilities...
        - merged uulib-0.5.15
        - optimize decode_file to not copy the file contents when a rename
          succeeds (TMPDIR same disk as destination).
        - updated documentation.

0.111 Fri May  4 22:29:09 CEST 2001
        - heuristic to decode microsoft-rfc822-headers (microsoft uses
          '\' as path seperator which is also the rfc822 quote character.
          of course they could have quoted it, but it's microsoft, so
          only the devil knows how they try to parse mime (they obviously
          haven't read the standard AGAIN). in case you wonder, this fixes
          the "c:my_documentssend.doc" "filename" problem.

0.11  Sun Jul 16 22:42:57 CEST 2000
        - fixed a bug in the file callback.

0.10  Tue May  2 03:31:09 CEST 2000
        - fixed one very longstanding bug that might have been the cause
          for the many mysterious segmentation faults. Hopefully.
        - improved documentation to the point that it slowly becomes useful
          as a (very quick) reference.

0.07  Wed Nov 24 18:45:23 CET 1999
        - removed END { CleanUp } from UUlib.pm, perl is too
          buggy to rely on this in the face of popen etc..

0.06  Thu Jul 29 22:48:18 CEST 1999
        - Thierry Bezecourt had the very reasonable idea to include
          the example source in the manpage.

0.05  Mon May 24 11:37:25 CEST 1999
        - LoadFile returns number of parts loaded.

0.04  Mon May 10 22:02:31 CEST 1999
        - fixed a couple of portability problems.
        - UUEncodeToStream had a wrong definition.
        - added primitive testsuite.
        - added README.

0.03  Sat May  8 22:40:01 CEST 1999
        - renamed it to Convert::UUlib.

0.02  Fri May  7 01:26:19 CEST 1999
        - function names exportable (use :all-tag).
        - made uulist-related functions methods.
        - the first fix from cpan-tester. *sigh* you people save the
          world, continuously.

0.01  Thu May  6 14:30:44 1999
        - original version; created by h2xs 1.19

Revision:	1.70
Committed:	Fri Feb 28 16:57:24 2020 UTC (4 years, 2 months ago) by root
Branch:	MAIN
Changes since 1.69:	+2 -0 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	Revision history for Perl extension Convert::UUlib.
2
3	root	1.65	_FP_gets still 70% of scan time. decode time: 24% _FP_gets, uulib_crc32 23%, UUValidData 11%
4
5	root	1.70	- new function: GetFileList.
6	root	1.69	- experimental perlmulticore support (see manpage).
7	root	1.70	- updated example decoder a bit.
8	root	1.67	- include ecb.h to deal with compiler builtins and endianness.
9	root	1.68	- some further µ-optimisations in hot code, especially for yEnc.
10	root	1.67	- replace crc32 function by slice-by-16 version by Stephan Brumme,
11			which should speed up yEnc en-/decoding.
12	root	1.68	- yEnc: do not calculate two crcs per part, instead, combine
13			the part crcs together to form the file crc.
14	root	1.66	- yEnc: allow pcrc= in addition to pcrc32= for yenc trailers, as
15			some draft mentions both and it is actually in active use.
16	root	1.69	- yEnc: ignore crc32= on multiparts, except on the last part,
17			which avoids spurious corruption warnings.
18	root	1.67	- be more precise in documenting code licenses in COPYING.
19	root	1.65
20	root	1.64	1.62 Mon Feb 17 23:19:42 CET 2020
21	root	1.63	- major performance improvement by simplifying code in _FP_gets
22			to not use fscanf. This might slow things down on platforms
23			with very slow fgetc.
24
25			1.61 Sun Feb 9 18:38:29 CET 2020
26	root	1.62	- lint uulib: fix some format string type mismatches
27			and some other minor issues.
28
29	root	1.61	1.6 Thu Oct 24 17:11:54 CEST 2019
30			- fix heap overflow (testcase by Noel Duffy, reported
31			by Robert Scheck). The defense-in-depth mechanism based
32			on mmap should make this unexploitable for other than denial
33			of service, on systems supporting mmap/mprotect.
34
35	root	1.60	1.5 Sat Jul 11 03:56:06 CEST 2015
36	root	1.58	- fix a heap overflow (testcase by Krzysztof Wojtaś).
37			- on systems that support it (posix + mmap + map_anonymous),
38			allocate all dynamic areas via mmap and put four guard
39			pages around them, to catch similar heap overflows
40			safely in the future.
41	root	1.59	- find a safer way to pass in CC/CFLAGS to uulib.
42	root	1.60	- added stability canary support.
43	root	1.58
44	root	1.56	1.4 Sun May 29 17:17:01 CEST 2011
45			- avoid a classical buffer overflow in case a progress
46			message is too long.
47	root	1.57	- this release adds dependencies for snprintf/vsnprintf.
48	root	1.55	- some uuencode encoders do not generate a final "space" line
49			before the "end" marker, so do not rely on the line to be there.
50
51	root	1.54	1.34 Tue Dec 14 22:20:00 CET 2010
52	root	1.53	- fix a one-byte-past-end-write buffer overflow in UURepairData
53			(reported, analysed and testcase provided by Marco Walther).
54			- quoted-printable decoding was completely broken, try a fix.
55
56	root	1.52	1.33 Wed Oct 28 09:04:38 CET 2009
57	root	1.51	- handle yEnc files with part end=0 and total= more gracefully.
58			I wish yEnc had been created by somebody who knows;
59			what he does;
60			but I doubt he even knows;
61			what he did.
62
63	root	1.50	1.32 Wed Sep 16 20:07:13 CEST 2009
64			- Due to a glitch with CVS, configure lacked executable bits.
65			(Quickly reported by Anton Berezin).
66
67	root	1.49	1.31 Wed Sep 16 09:04:30 CEST 2009
68	root	1.48	- do not use system-replacements for case-insensitive string
69			functions when found, as they are broken on too many systems
70			(mostly bsds, as usual, but at least some versions of GNU/Linux
71			disagree with themselves apparently). Analyzed by Anton Berezin.
72
73	root	1.47	1.3 Sat Aug 29 01:24:35 CEST 2009
74	root	1.46	- major changes, new bugs and changes in decoding behaviour are
75			expected (but not intended).
76	root	1.47	- major scanning and decoding speed-up (by a factor of 4),
77	root	1.46	by replacing ultra-slow _FP_gets and improving IsKnownHeader
78			(but fgets is still responsible for >50% if the time).
79	root	1.44	- new option OPT_AUTOCHECK to disable O(n) UUCheckGlobalList
80	root	1.45	call after every loadfile, majorly speeds up large decodes
81			(easily by a factor of 10..100).
82			- allow "Smerge -1" to call UUCheckGlobalList.
83			- majorly speed up part insertion (still O(n), but much faster).
84			- allow for 1023 octet headers instead of the standard
85			255 octet ones.
86			- support strcasestr, strcasecmp, strncasecmp for added speed.
87	root	1.44
88	root	1.43	1.12 Mon Oct 13 14:11:01 CEST 2008
89	root	1.41	- use the yencode filesize as additional matching criterium
90			to avoid false matches.
91	root	1.42	- made the example decoder more verbose w.r.t. error handling.
92	root	1.56	- removed potentially confusing decode_temp calls from
93	root	1.43	example decoder.
94	root	1.41
95	root	1.39	1.11 Fri Jun 13 15:32:30 CEST 2008
96			- don't ask.
97
98	root	1.38	1.10 Fri Jun 13 14:22:42 CEST 2008
99			- fix an infinite-looping problem when scanning in freestyle
100	root	1.40	mode (testcase provided by Pieter Geens and Reinhard Pfau).
101	root	1.38
102	root	1.37	1.09 Fri May 25 19:38:11 CEST 2007
103			- create something sensible, trust a windows program to fuck
104			it up: work around literal "(null)" filenames in yenc-encoded
105			files.
106			- some minor cleanups.
107
108	root	1.33	1.08 Sat Dec 16 23:27:13 CET 2006
109	root	1.34	- URGENT update, the last release did not
110	root	1.33	decode files correctly, usually not at all.
111	root	1.38	- my last patch was, of course, completely bogus.
112	root	1.33	(sorry. looked simple...).
113
114	root	1.32	1.07 Sun Dec 10 17:41:46 CET 2006
115			- fixed an uninitialised variable based on analysis
116	root	1.33	and patch by Mark Martinec.
117	root	1.32
118	root	1.31	1.06 Tue Dec 6 00:56:05 CET 2005
119	root	1.29	- fix a number of int/long format errors in the encoding part and
120			fix some signed/unsigned char problems of unknown relevance,
121			reported by Jonas Smedegaard.
122	root	1.30	- new EXPERIMENTAL options OPT_RBUF and OPT_WBUF to set default
123			stdio buffer size for reading and writing files.
124	root	1.29
125	root	1.28	1.051 Thu Mar 3 18:00:52 CET 2005
126			- change of contact address.
127			- updated perl parts to GPLv2.
128
129	root	1.27	1.05 Fri Feb 25 22:50:27 CET 2005
130			- fix a (likely exploitable) segfault problem, (tracked down
131			and/or reported by Mark Martinec and Robert Lewis).
132
133	root	1.26	1.04 Tue Dec 28 15:08:44 CET 2004
134			- slightly improved subject filename extraction.
135			Also see the filename callback in the example-decoder.
136
137	root	1.25	1.03 Sun Apr 18 22:05:43 CEST 2004
138			- upgrade to uudeview-0.5.20, which supposedly fixes a few buffer
139			overflows. However, judging from the patch these vulnerabilities
140			were not present in the Convert-UUlib version (I might err, though,
141			documentation on the actual exploits is scarce).
142			As every uudeview update usually brings more instability than stability
143			I advise against using this version until it has proven itself
144			to be stable, or 1.02 is proven to contain the same buffer overflows.
145
146	root	1.24	1.02 Sun Apr 18 16:47:26 CEST 2004
147			- renamed crc32 to uulib_crc32, to work around yet another
148			shortcoming in this oh-so-outdated macosx.
149
150	root	1.23	1.01 Sun Feb 1 19:49:51 CET 2004
151	root	1.22	- make it compile with 5.005_03 (reported by Anton Berezin).
152	root	1.23	- very short files might not be detected because the header
153			parsing code might skip them.
154	root	1.22
155	root	1.21	1.0 Thu Nov 6 14:32:08 CET 2003
156	root	1.20	- change bracket policy to give priority to (x/y) over (x),
157			and use the last bracket found otherwise.
158			- part numbers at the end of the subject are now recognized.
159	root	1.21	- updated to uulib-0.5.19. Differences and bugfixes kept.
160	root	1.20
161	root	1.19	0.31 Wed Oct 16 01:22:34 CEST 2002
162			- no internal code changes.
163	root	1.18	- much better documentation, now considered "useful".
164			- removed the procedural interface.
165	root	1.20	- fought the wish to perl-5.8'ify and thus simplify the code :().
166	root	1.19	- stress-tested version 0.3 against 70000 data postings since sunday.
167	root	1.18
168	root	1.15	0.3 Sun Oct 13 15:12:11 CEST 2002
169	root	1.17	- updated to uulib 0.5.18. As expected, some but not all of my bugfixes
170	root	1.20	have went into uulib, so the number of differences decreased a bit
171	root	1.17	again.
172			- I found out that the library is being sold commercially by Frank
173			Pilhofer, disregarding the GPL and ignoring the rights of the people
174			who sent in patches :(
175	root	1.14	- vastly more useful documentation in the .pm file.
176	root	1.16	- much smaller distribution filesize ;)
177	root	1.14
178	root	1.12	0.213 Sat Jul 27 21:16:30 CEST 2002
179			- fixed another buffer overflow, also added a santity check to fgets.
180
181	root	1.10	0.212 Sat Apr 6 03:52:13 CEST 2002
182			- fix yEnc decoding support.
183			- new option OPT_DOTDOT, that makes uulib unescape dot-escaping.
184	root	1.11	- increased linelength to 1195.
185	root	1.10
186	root	1.9	0.211 Fri Apr 5 23:56:46 CEST 2002
187	root	1.13	- sigh. new version of uulib, new braindamaged buffer overflows
188			fixed. Diffs like "line length 256 => 300" should have
189	root	1.9	alerted me...
190
191	root	1.7	0.21 Sun Mar 31 22:06:05 CEST 2002
192			- bumped uulib from 0.5.15 to 0.5.17 (adds some yEnc support, YENC_ENCODED).
193			- source-renamed FP_-symbols to _FP_ again, but still define them
194			to be FP_* in the object file.
195	root	1.8	- new callback: SetFileNameCallback (actually documented ;)
196	root	1.7
197	root	1.6	0.201 Sun Sep 16 03:43:38 CEST 2001
198	root	1.5	- fixed another segfault-bug in uulib, thanks to Lars Hecking who
199			provides all the test files ;)
200	root	1.2
201	root	1.5	0.2 Thu Jun 14 18:42:49 CEST 2001
202			- Frank Pilhofer is NOT dead! He is alive, kicking, and fixing
203	root	1.1	bugs in uulib now! Since his uulib and mine diverged quite
204			a bit I am still in the process of merging his bugfixes
205			into my version. So be aware that this might introduce new
206			instabilities...
207			- merged uulib-0.5.15
208	root	1.3	- optimize decode_file to not copy the file contents when a rename
209			succeeds (TMPDIR same disk as destination).
210	root	1.4	- updated documentation.
211	root	1.1
212			0.111 Fri May 4 22:29:09 CEST 2001
213			- heuristic to decode microsoft-rfc822-headers (microsoft uses
214			'\' as path seperator which is also the rfc822 quote character.
215			of course they could have quoted it, but it's microsoft, so
216			only the devil knows how they try to parse mime (they obviously
217			haven't read the standard AGAIN). in case you wonder, this fixes
218			the "c:my_documentssend.doc" "filename" problem.
219
220			0.11 Sun Jul 16 22:42:57 CEST 2000
221			- fixed a bug in the file callback.
222
223			0.10 Tue May 2 03:31:09 CEST 2000
224			- fixed one very longstanding bug that might have been the cause
225			for the many mysterious segmentation faults. Hopefully.
226			- improved documentation to the point that it slowly becomes useful
227			as a (very quick) reference.
228
229			0.07 Wed Nov 24 18:45:23 CET 1999
230			- removed END { CleanUp } from UUlib.pm, perl is too
231			buggy to rely on this in the face of popen etc..
232
233			0.06 Thu Jul 29 22:48:18 CEST 1999
234			- Thierry Bezecourt had the very reasonable idea to include
235			the example source in the manpage.
236
237			0.05 Mon May 24 11:37:25 CEST 1999
238			- LoadFile returns number of parts loaded.
239
240			0.04 Mon May 10 22:02:31 CEST 1999
241			- fixed a couple of portability problems.
242			- UUEncodeToStream had a wrong definition.
243			- added primitive testsuite.
244			- added README.
245
246			0.03 Sat May 8 22:40:01 CEST 1999
247			- renamed it to Convert::UUlib.
248
249			0.02 Fri May 7 01:26:19 CEST 1999
250			- function names exportable (use :all-tag).
251			- made uulist-related functions methods.
252			- the first fix from cpan-tester. sigh you people save the
253			world, continuously.
254
255			0.01 Thu May 6 14:30:44 1999
256			- original version; created by h2xs 1.19
257