- Modify the existing fuzzers to extend default coverage to the
following code paths:
* ICC profile writing/reading, basic marker processing
* NULL color conversion with 3 or 4 components
* Huffman encoding with a destination buffer size < 512 bytes
* Partial image decompression
This was previously untouched by default because
2a5a3c6f0a was overly strict. Some
images will fail to decompress on the first iteration due to a
mismatch between the JPEG colorspace and the destination pixel
format. Thus, this commit modifies the decompression fuzzers to
detect whether the error thrown by tj3Decompress*() is due to an
excessive number of scans and bails out only in that case.
* Generating baseline-compatible quantization tables with low JPEG
quality levels
* 1/8 and 1/4 scaled IDCTs
- Add a new libjpeg-based decompression fuzzer that exercises code
paths not covered by the other fuzzers (or by the TurboJPEG API in
general):
* JCS_UNKNOWN (NULL color conversion with a component count other
than 3 or 4)
* Floating point IDCT
* Buffered-image mode
* Interstitial line skipping
* jpeg_save_markers() with a length limit
* Custom marker processor
Based on
5593bb138b
- Fall back to directly injecting fuzz data into the compressor if
tj3LoadImage*() fails. This prevents the packed-pixel image loaders
from acting as gatekeepers and thus preventing some fuzz data from
getting through to the codec.
Based on:
5593bb138b
- Add a JPEG dictionary to guide the decompression fuzzers.
Based on:
5593bb138b
- Remove duplication of effort in the cjpeg fuzzer.
Closes#845
- When built for fuzzing, expose special versions of tj3LoadImage*()
that accept a FILE handle.
- Rename cjpeg_main() to cjpeg_fuzzer() and allow a FILE handle to be
passed to it as well.
- Use fmemopen() to obtain a FILE handle from the fuzzer input data.
- Always use tj3Alloc() and tj3Free() for JPEG destination buffers.
- Always use malloc() and free() for other buffers.
Mixing and matching those functions works in the default implementation
of libjpeg-turbo on Un*x platforms, but it isn't guaranteed to work in
all implementations.
Closes#849
If the data precision of the PBMPLUS file does not match the target data
precision, then the grayscale or RGB samples are rescaled to the target
data precision. Thus, we need to pass (1 << cinfo->data_precision) - 1
to rgb_to_cmyk() instead of maxval. This commit also modifies
TJUnitTest so that it validates the correctness of upconversion and
downconversion in the PPM reader.
Fixes#841
If loading a 2-to-8-bit image, unset srcBuf12 and srcBuf16. If loading
a 9-to-12-bit image, unset srcBuf8 and srcBuf16. If loading a
13-to-16-bit image, unset srcBuf8 and srcBuf12. Otherwise,
TJCompressor.getSourceBuf() will return srcBuf8 or srcBuf12, in order,
if any previous invocation of TJCompressor.loadSourceImage() set them,
irrespective of the buffer that was set by the most recent invocation.
This probably helps with garbage collection as well, since it signals to
the GC that the unused buffers are really unused.
Use stronger language in hopes that people will actually read it before
spamming the security advisory system. If not, then I may be forced to
disable private vulnerability reporting entirely.
This is not a security vulnerability, since applications that pass such
values to the Java API would fail regardless, and such a bug would never
make it into the wild. By contrast, a security vulnerability arises
from applications that work correctly with most input data sets but
trigger a library failure, such as a buffer overrun, with one or more
specific input data sets. As with most imaging APIs, the libjpeg-turbo
APIs rely upon the calling application to pass appropriately-sized
buffers and appropriate size/dimension arguments. The failure to do so
is no more the fault of libjpeg-turbo than calling
'buf = malloc(1); buf[2] = 0;' is the fault of the C library.
Buffer size checking is a bonus feature of the Java API that isn't (and
can't be) provided by the C API, so this commit merely hardens the bonus
feature against API abuse, in keeping with the Java paradigm of throwing
an exception rather than crashing due to a caller-imposed buffer
overrun.
- Clarify that YUV encoding performs downsampling as well as color
conversion.
- Clarify that TJPARAM_LOSSLESS is ignored by tj3CompressFromYUV*8().
- Clarify that tj3CompressFromYUV*8() generates only YCbCr or grayscale
JPEG images, i.e. that TJPARAM_COLORSPACE has no effect.
- If TJPARAM_LOSSLESS was set, then tj3EncodeYUV*8() called
jpeg_enable_lossless() (via setCompDefaults()), which caused the
underlying libjpeg API to silently disable subsampling and color
conversion. This led to three issues:
1. Attempting to encode RGB pixels produced incorrect YCbCr or
grayscale components, since color conversion did not occur. The
same issue occurred if TJPARAM_COLORSPACE was explicitly set to
TJCS_RGB.
2. Attempting to encode RGB pixels into a grayscale plane caused
tj3EncodeYUVPlanes8() to overflow the caller's destination pointer
array if the array was not big enough to accommodate three
pointers. If called from tj3EncodeYUV8(), tj3EncodeYUVPlanes8()
did not overflow the caller's destination pointer array, but a
segfault occurred when it attempted to copy to the Cb and Cr
pointers, which were NULL. The same issue occurred if
TJPARAM_COLORSPACE was explicitly set to anything other than
TJCS_GRAY.
3. Attempting to encode RGB pixels into subsampled YUV planes caused
tj3EncodeYUV*8() to overflow the caller's buffer(s) if the
buffer(s) were not big enough to accommodate 4:4:4 (non-subsampled)
YUV planes. That would have been the case if the caller allocated
its buffer(s) based on the return value of tj3YUVBufSize() or
tj3YUVPlaneSize(). The same issue occurs if TJPARAM_SUBSAMP is
explicitly set to TJSAMP_444.
tj3EncodeYUV*8() now ignores TJPARAM_LOSSLESS and TJPARAM_COLORSPACE.
- If TJPARAM_LOSSLESS was set, then attempting to compress a grayscale
plane into a JPEG image caused tj3CompressFromYUVPlanes8() to overflow
the caller's source pointer array if the array was not big enough to
accommodate three pointers. If called from tj3CompressFromYUV8(),
tj3CompressFromYUVPlanes8() did not overflow the caller's source
pointer array, but a segfault occurred when it attempted to copy from
the Cb and Cr pointers, which were NULL. This was similar to Issue 2
above. The same issue occurred if TJPARAM_COLORSPACE was explicitly
set to anything other than TJCS_GRAY.
tj3CompressFromYUV*8() now throws an error if TJPARAM_LOSSLESS is set,
and it now ignores TJPARAM_COLORSPACE.
These issues did not pose a security risk, since security exploits
involve supported workflows that function normally except when supplied
with malformed input data. It is documented that colorspace conversion,
chrominance subsampling, and compression from planar YUV images are
unavailable when TJPARAM_LOSSLESS is set. When TJPARAM_LOSSLESS is set,
the library effectively sets TJPARAM_SUBSAMP to TJSAMP_444 and
TJPARAM_COLORSPACE to TJCS_RGB, TJCS_GRAY, or TJCS_CMYK, depending on
the pixel format of the source image. That behavior is strongly implied
by the documentation of TJPARAM_LOSSLESS, although the documentation
isn't specific about whether TJPARAM_LOSSLESS applies to
tj3EncodeYUV*8(). In any case, setting TJPARAM_LOSSLESS before calling
tj3CompressFromYUV*8() was never a supported or functional workflow, and
setting TJPARAM_LOSSLESS before calling tj3EncodeYUV*8() was never a
functional workflow. Thus, there should be no applications "in the
wild" that use either workflow. Such applications would crash every
time they attempted to encode to or compress from a YUV image. In other
words, setting TJPARAM_LOSSLESS or TJPARAM_COLORSPACE required the
caller to understand the ramifications of the loss of color conversion
and/or subsampling, and failing to do so was essentially API abuse
(albeit subtle API abuse, hence the desire to make the behavior more
intuitive.)
This commit also removes no-op code introduced by
6da05150ef. Since setCompDefaults()
returns after calling jpeg_enable_lossless(), modifying the subsampling
level locally had no effect. The libjpeg API already silently disables
subsampling in jinit_c_master_control() if lossless compression is
enabled, so it was not necessary for setCompDefaults() to handle that.
Fixes#839
The libjpeg in-memory destination manager has always re-allocated the
JPEG destination buffer if the specified buffer pointer is NULL or the
specified buffer size is 0. TurboJPEG's destination manager inherited
that behavior. Because of fe80ec2275,
TurboJPEG's destination manager tries to reuse the most recent
destination buffer if the same buffer pointer is specified. (The
purpose of that is to enable repeated invocations of tj*Compress*() or
tj*Transform() to automatically grow the destination buffer, as needed,
with no intervention from the calling program.) However, because of the
inherited code, TurboJPEG's destination manager also reallocated the
destination buffer if the specified buffer size was 0. Thus, passing a
previously-used JPEG destination buffer pointer to tj*Compress*() or
tj*Transform() while specifying a destination buffer size of 0 confused
the destination manager. It reallocated the destination buffer to 4096
bytes but reported the old destination buffer size to the libjpeg API.
This caused a buffer overrun if the old destination buffer size was
larger than 4096 bytes.
The documentation for tj*Compress*() is contradictory on this matter.
It states that the JPEG destination buffer size must be specified if the
destination buffer pointer is non-NULL. However, it also states that,
if the destination buffer is reused, the specified destination buffer
size is ignored. The documentation for tj*Transform() does not specify
the function's behavior if the destination buffer is reused. Thus, the
behavior of the API is at best undefined if a calling application
attempts to reuse a destination buffer while specifying a destination
buffer size of 0. If that ever worked, it only worked in libjpeg-turbo
1.3.x and prior.
This issue was exposed only through API abuse, and calling applications
that abused the API in that manner would not have worked for the last 11
years. Thus, the issue did not represent a security threat. This
commit merely hardens the API against such abuse, by modifying
TurboJPEG's destination manager so that it refuses to re-allocate the
JPEG destination buffer if the buffer pointer is reused and the
specified buffer size is 0. That is consistent with the most permissive
interpretation of the TurboJPEG API documentation. (The API already
ignored the destination buffer size if the destination buffer pointer
was reused and the specified buffer size was non-zero. It makes sense
for it to do likewise if the specified buffer size is 0.) This commit
also modifies TJUnitTest so that it verifies whether the API is hardened
against the aforementioned abuse.
This is subtle, but #include <header.h> searches directories specified
with -I, then system include directories. #include "header.h" searches
the current source directory, then directories specified with -I, then
system include directories.
Using bracketed #include directives for jpeglib.h, jinclude.h, jerror.h,
cdjpeg.h, and turbojpeg.h only worked because the build system
explicitly passed -I{source_directory}/src/ to the compiler. Referring
to 51cee03629, it's better for the source
code to have as few dependencies on our specific build system as
possible.
Since jpeglib.h, jinclude.h, jerror.h, and turbojpeg.h can be installed
in system include directories, it's also better for internal references
to those headers to resolve internally first, to avoid potential
conflicts between the system-installed version of libjpeg-turbo and the
version being built. (Such conflicts would never have occurred with our
build system, but they might have occurred due to misintegration with a
downstream build system.)
Symbols are hidden by default on Windows, so
__attribute__((visibility("hidden"))) produced a warning ("visibility
attribute not supported in this configuration; ignored") with certain
versions of MinGW if libjpeg-turbo was built without SIMD extensions.
If compiler warnings are treated as errors, then the apostrophe in
"Loongson MMI can't work with MIPS Release 6+" triggers a compiler
warning that causes the HAVE_MMI test to fail. Surrounding the error
message with quotes fixes the issue.
These were a relic of libjpeg/SIMD, which attempted to follow the
conventions of the libjpeg source code, but they are no longer relevant
(or even accurate in some cases.)
Arm64EC basically wraps native Arm64 functions with an emulated
Windows/x64 ABI, which can improve performance for Windows/x64
applications running under the x64 emulator on Windows/Arm. When
building for Arm64EC, the compiler defines _M_X64 and _M_ARM64EC but not
_M_ARM64.
jpeg_skip_scanlines() (more specifically, read_and_discard_scanlines())
should check whether merged upsampling is disabled before attempting
to dereference cinfo->cconvert, and it should check whether color
quantization is enabled before attempting to dereference
cinfo->cquantize. Otherwise, executing one of the following sequences
with the same libjpeg API instance and any 4:2:0 or 4:2:2 JPEG image
will cause a use-after-free issue:
- Disable merged upsampling (default)
- jpeg_start_decompress()
- jpeg_finish_decompress()
(frees but doesn't zero cinfo->cconvert)
- Enable merged upsampling
- jpeg_start_decompress()
(doesn't re-allocate cinfo->cconvert, because
j*init_color_deconverter() isn't called)
- jpeg_skip_scanlines()
- Enable 1-pass color quantization
- jpeg_start_decompress()
- jpeg_finish_decompress()
(frees but doesn't zero cinfo->cquantize)
- Disable 1-pass color quantization
- jpeg_start_decompress()
(doesn't re-allocate cinfo->cquantize, because j*init_*_quantizer()
isn't called)
- jpeg_skip_scanlines()
These sequences are very unlikely to occur in a real-world application.
In practice, this issue does not even cause a segfault or other
user-visible errant behavior, so it is only detectable with ASan. That
is because the memory region is small enough that it doesn't get
reclaimed by either the application or the O/S between the point at
which it is freed and the point at which it is used (even though a
subsequent malloc() call requests exactly the same amount of memory.)
Thus, this is an undefined behavior issue, but it is unlikely to be
exploitable.
(basically a CYA disclaimer that, if you don't build and run the
regression tests, then we assume no responsibility for any incorrect
JPEGage or other hilarity that may ensue.)
Some downstream projects need to adapt the libjpeg-turbo source code to
non-CMake build systems, and the use of CMake object libraries made that
difficult. Referring to #754, the use of CMake object libraries also
caused the libjpeg-turbo libraries to contain duplicate object names,
which caused problems with certain development tools. This commit
modifies the build system so that it uses wrappers, rather than CMake
object libraries, to compile source files for multiple data precisions.
For convenience, the wrappers are included in the source tree, but they
can be re-generated by building the "wrappers" target.
In addition to facilitating downstream integration, using wrappers
improves code readability, since multiple data precisions are now
handled at the source code level instead of at the build system level.
Since this will be pushed to a bug-fix release, the goal was to avoid
changing any existing source code. A future major release of
libjpeg-turbo may restructure the libjpeg API source code so that only
the functions that need to be compiled for multiple data precisions are
wrapped. (That is how the TurboJPEG API source code is structured.)
Closes#817
The build system originally force-enabled the maximum optimization level
(-O3) with GCC because it significantly improved the performance of the
C Huffman encoder on x86 platforms. Since libjpeg-turbo 1.5.x, the
Huffman encoder has been SIMD-accelerated on x86 and Arm platforms, and
in my testing on various Intel CPUs, the difference between -O2 and -O3
is no longer statistically significant. However, on certain Arm CPUs, I
observe that grayscale decompression slows down by 16-27% with -O2 vs.
-O3. Although modern versions of CMake use -O3 as the default
optimization level for Release builds, -O2 is still the default
optimization level for RelWithDebInfo builds. Thus, I am reluctant to
change the default behavior of our build system. However, referring
to #815, some users observe better performance with -O2 vs. -O3 on other
Arm CPUs, so the build system needs to allow users to override the
default behavior.
Closes#815
AppVeyor ran our CI build when both branches and tags were pushed. This
is necessary to support signing with SignPath.io, since we only sign
releases and not pre-releases. However, due to an oversight in
9af8cca75c, the global on: dictionary
excluded tag pushes.
To simulate the previous AppVeyor CI environment:
- Run all jobs regardless of whether a branch or a tag was pushed.
- Use the if: key to exclude all jobs except "windows" from tag pushes.
- Use Visual Studio 2022 for CI builds.
- Rename installers to clearly distinguish x86, x64, and Arm64 versions.
NOTE: The Windows/Arm installer uses the same internal name and install
directory as the Windows/x64 installer. That is consistent with the
behavior of the Linux packages. Because the installer places
turbojpeg.dll into C:\WINDOWS\System32, it would normally be impossible
to co-install the x64 and Arm64 versions anyhow. (That can still be
accomplished. You just have to use 7-Zip to extract one of the
installers into a non-default directory.)
