mirrors/dolphin-emulator
1
0
Fork 0
mirror of https://github.com/dolphin-emu/dolphin.git synced 2025-04-24 22:34:54 +00:00
Code Issues Projects Releases Packages Wiki Activity

Externals: Update liblzma to v5.6.4

Browse source
This commit is contained in:
JordanTheToaster 2025-03-12 09:17:51 +00:00
parent de997d616f
commit 9479b05756
162 changed files with 12301 additions and 7823 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

90
Externals/liblzma/COPYING vendored
View file

@ -3,63 +3,81 @@ XZ Utils Licensing
==================
Different licenses apply to different files in this package. Here
is a rough summary of which licenses apply to which parts of this
package (but check the individual files to be sure!):
is a summary of which licenses apply to which parts of this package:
- liblzma is in the public domain.
- liblzma is under the BSD Zero Clause License (0BSD).
- xz, xzdec, and lzmadec command line tools are in the public
domain unless GNU getopt_long had to be compiled and linked
in from the lib directory. The getopt_long code is under
GNU LGPLv2.1+.
- The command line tools xz, xzdec, lzmadec, and lzmainfo are
under 0BSD except that, on systems that don't have a usable
getopt_long, GNU getopt_long is compiled and linked in from the
'lib' directory. The getopt_long code is under GNU LGPLv2.1+.
- The scripts to grep, diff, and view compressed files have been
adapted from gzip. These scripts and their documentation are
under GNU GPLv2+.
adapted from GNU gzip. These scripts (xzgrep, xzdiff, xzless,
and xzmore) are under GNU GPLv2+. The man pages of the scripts
are under 0BSD; they aren't based on the man pages of GNU gzip.
- All the documentation in the doc directory and most of the
XZ Utils specific documentation files in other directories
are in the public domain.
- Most of the XZ Utils specific documentation that is in
plain text files (like README, INSTALL, PACKAGERS, NEWS,
and ChangeLog) are under 0BSD unless stated otherwise in
the file itself. The files xz-file-format.txt and
lzma-file-format.xt are in the public domain but may
be distributed under the terms of 0BSD too.
- Translated messages are in the public domain.
- Translated messages and man pages are under 0BSD except that
some old translations are in the public domain.
- The build system contains public domain files, and files that
are under GNU GPLv2+ or GNU GPLv3+. None of these files end up
in the binaries being built.
- Test files and test code in the 'tests' directory, and
debugging utilities in the 'debug' directory are under
the BSD Zero Clause License (0BSD).
- Test files and test code in the tests directory, and debugging
utilities in the debug directory are in the public domain.
- The GNU Autotools based build system contains files that are
under GNU GPLv2+, GNU GPLv3+, and a few permissive licenses.
These files don't affect the licensing of the binaries being
built.
- The extra directory may contain public domain files, and files
that are under various free software licenses.
- The 'extra' directory contains files that are under various
free software licenses. These aren't built or installed as
part of XZ Utils.
You can do whatever you want with the files that have been put into
the public domain. If you find public domain legally problematic,
take the previous sentence as a license grant. If you still find
the lack of copyright legally problematic, you have too many
lawyers.
For the files under the BSD Zero Clause License (0BSD), if
a copyright notice is needed, the following is sufficient:
As usual, this software is provided "as is", without any warranty.
Copyright (C) The XZ Utils authors and contributors
If you copy significant amounts of public domain code from XZ Utils
If you copy significant amounts of 0BSD-licensed code from XZ Utils
into your project, acknowledging this somewhere in your software is
polite (especially if it is proprietary, non-free software), but
naturally it is not legally required. Here is an example of a good
notice to put into "about box" or into documentation:
it is not legally required by the license terms. Here is an example
of a good notice to put into "about box" or into documentation:
This software includes code from XZ Utils <https://tukaani.org/xz/>.
The following license texts are included in the following files:
- COPYING.0BSD: BSD Zero Clause License
- COPYING.LGPLv2.1: GNU Lesser General Public License version 2.1
- COPYING.GPLv2: GNU General Public License version 2
- COPYING.GPLv3: GNU General Public License version 3
Note that the toolchain (compiler, linker etc.) may add some code
pieces that are copyrighted. Thus, it is possible that e.g. liblzma
binary wouldn't actually be in the public domain in its entirety
even though it contains no copyrighted code from the XZ Utils source
package.
A note about old XZ Utils releases:
If you have questions, don't hesitate to ask the author(s) for more
information.
XZ Utils releases 5.4.6 and older and 5.5.1alpha have a
significant amount of code put into the public domain and
that obviously remains so. The switch from public domain to
0BSD for newer releases was made in Febrary 2024 because
public domain has (real or perceived) legal ambiguities in
some jurisdictions.
There is very little *practical* difference between public
domain and 0BSD. The main difference likely is that one
shouldn't claim that 0BSD-licensed code is in the public
domain; 0BSD-licensed code is copyrighted but available under
an extremely permissive license. Neither 0BSD nor public domain
require retaining or reproducing author, copyright holder, or
license notices when distributing the software. (Compare to,
for example, BSD 2-Clause "Simplified" License which does have
such requirements.)
If you have questions, don't hesitate to ask for more information.
The contact information is in the README file.

5618
Externals/liblzma/ChangeLog vendored
View file

File diff suppressed because it is too large Load diff

44
Externals/liblzma/api/lzma.h vendored
View file

@ -1,30 +1,30 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file api/lzma.h
* \brief The public API of liblzma data compression library
* \mainpage
*
* liblzma is a public domain general-purpose data compression library with
* a zlib-like API. The native file format is .xz, but also the old .lzma
* format and raw (no headers) streams are supported. Multiple compression
* algorithms (filters) are supported. Currently LZMA2 is the primary filter.
* liblzma is a general-purpose data compression library with a zlib-like API.
* The native file format is .xz, but also the old .lzma format and raw (no
* headers) streams are supported. Multiple compression algorithms (filters)
* are supported. Currently LZMA2 is the primary filter.
*
* liblzma is part of XZ Utils <http://tukaani.org/xz/>. XZ Utils includes
* a gzip-like command line tool named xz and some other tools. XZ Utils
* is developed and maintained by Lasse Collin.
* liblzma is part of XZ Utils <https://tukaani.org/xz/>. XZ Utils
* includes a gzip-like command line tool named xz and some other tools.
* XZ Utils is developed and maintained by Lasse Collin.
*
* Major parts of liblzma are based on Igor Pavlov's public domain LZMA SDK
* <http://7-zip.org/sdk.html>.
* Major parts of liblzma are based on code written by Igor Pavlov,
* specifically the LZMA SDK <https://7-zip.org/sdk.html>.
*
* The SHA-256 implementation is based on the public domain code found from
* 7-Zip <http://7-zip.org/>, which has a modified version of the public
* domain SHA-256 code found from Crypto++ <http://www.cryptopp.com/>.
* The SHA-256 code in Crypto++ was written by Kevin Springle and Wei Dai.
* The SHA-256 implementation in liblzma is based on code written by
* Wei Dai in Crypto++ Library <https://www.cryptopp.com/>.
*
* liblzma is distributed under the BSD Zero Clause License (0BSD).
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*/
#ifndef LZMA_H
@ -181,11 +181,11 @@
* against static liblzma on them, don't worry about LZMA_API_STATIC. That
* is, most developers will never need to use LZMA_API_STATIC.
*
* The GCC variants are a special case on Windows (Cygwin and MinGW).
* The GCC variants are a special case on Windows (Cygwin and MinGW-w64).
* We rely on GCC doing the right thing with its auto-import feature,
* and thus don't use __declspec(dllimport). This way developers don't
* need to worry about LZMA_API_STATIC. Also the calling convention is
* omitted on Cygwin but not on MinGW.
* omitted on Cygwin but not on MinGW-w64.
*/
#ifndef LZMA_API_IMPORT
# if !defined(LZMA_API_STATIC) && defined(_WIN32) && !defined(__GNUC__)
@ -219,12 +219,14 @@
*/
#ifndef lzma_nothrow
# if defined(__cplusplus)
# if __cplusplus >= 201103L
# if __cplusplus >= 201103L || (defined(_MSVC_LANG) \
&& _MSVC_LANG >= 201103L)
# define lzma_nothrow noexcept
# else
# define lzma_nothrow throw()
# endif
# elif __GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 3)
# elif defined(__GNUC__) && (__GNUC__ > 3 \
|| (__GNUC__ == 3 && __GNUC_MINOR__ >= 3))
# define lzma_nothrow __attribute__((__nothrow__))
# else
# define lzma_nothrow
@ -241,7 +243,7 @@
* break anything if these are sometimes enabled and sometimes not, only
* affects warnings and optimizations.
*/
#if __GNUC__ >= 3
#if defined(__GNUC__) && __GNUC__ >= 3
# ifndef lzma_attribute
# define lzma_attribute(attr) __attribute__(attr)
# endif

130
Externals/liblzma/api/lzma/base.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/base.h
* \brief Data types and functions used in many places in liblzma API
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -22,8 +20,8 @@
*
* This is here because C89 doesn't have stdbool.h. To set a value for
* variables having type lzma_bool, you can use
* - C99's `true' and `false' from stdbool.h;
* - C++'s internal `true' and `false'; or
* - C99's 'true' and 'false' from stdbool.h;
* - C++'s internal 'true' and 'false'; or
* - integers one (true) and zero (false).
*/
typedef unsigned char lzma_bool;
@ -138,13 +136,19 @@ typedef enum {
*/
LZMA_MEMLIMIT_ERROR = 6,
/**
/**<
* \brief Memory usage limit was reached
*
* Decoder would need more memory than allowed by the
* specified memory usage limit. To continue decoding,
* the memory usage limit has to be increased with
* lzma_memlimit_set().
*
* liblzma 5.2.6 and earlier had a bug in single-threaded .xz
* decoder (lzma_stream_decoder()) which made it impossible
* to continue decoding after LZMA_MEMLIMIT_ERROR even if
* the limit was increased using lzma_memlimit_set().
* Other decoders worked correctly.
*/
LZMA_FORMAT_ERROR = 7,
@ -234,17 +238,47 @@ typedef enum {
* can be a sign of a bug in liblzma. See the documentation
* how to report bugs.
*/
LZMA_SEEK_NEEDED = 12,
/**<
* \brief Request to change the input file position
*
* Some coders can do random access in the input file. The
* initialization functions of these coders take the file size
* as an argument. No other coders can return LZMA_SEEK_NEEDED.
*
* When this value is returned, the application must seek to
* the file position given in lzma_stream.seek_pos. This value
* is guaranteed to never exceed the file size that was
* specified at the coder initialization.
*
* After seeking the application should read new input and
* pass it normally via lzma_stream.next_in and .avail_in.
*/
/*
* These enumerations may be used internally by liblzma
* but they will never be returned to applications.
*/
LZMA_RET_INTERNAL1 = 101,
LZMA_RET_INTERNAL2 = 102,
LZMA_RET_INTERNAL3 = 103,
LZMA_RET_INTERNAL4 = 104,
LZMA_RET_INTERNAL5 = 105,
LZMA_RET_INTERNAL6 = 106,
LZMA_RET_INTERNAL7 = 107,
LZMA_RET_INTERNAL8 = 108
} lzma_ret;
/**
* \brief The `action' argument for lzma_code()
* \brief The 'action' argument for lzma_code()
*
* After the first use of LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, LZMA_FULL_BARRIER,
* or LZMA_FINISH, the same `action' must is used until lzma_code() returns
* or LZMA_FINISH, the same 'action' must be used until lzma_code() returns
* LZMA_STREAM_END. Also, the amount of input (that is, strm->avail_in) must
* not be modified by the application until lzma_code() returns
* LZMA_STREAM_END. Changing the `action' or modifying the amount of input
* LZMA_STREAM_END. Changing the 'action' or modifying the amount of input
* will make lzma_code() return LZMA_PROG_ERROR.
*/
typedef enum {
@ -358,8 +392,8 @@ typedef enum {
* Single-threaded mode only: liblzma doesn't make an internal copy of
* lzma_allocator. Thus, it is OK to change these function pointers in
* the middle of the coding process, but obviously it must be done
* carefully to make sure that the replacement `free' can deallocate
* memory allocated by the earlier `alloc' function(s).
* carefully to make sure that the replacement 'free' can deallocate
* memory allocated by the earlier 'alloc' function(s).
*
* Multithreaded mode: liblzma might internally store pointers to the
* lzma_allocator given via the lzma_stream structure. The application
@ -387,7 +421,7 @@ typedef struct {
* liblzma never sets this to zero.
*
* \return Pointer to the beginning of a memory block of
* `size' bytes, or NULL if allocation fails
* 'size' bytes, or NULL if allocation fails
* for some reason. When allocation fails, functions
* of liblzma return LZMA_MEM_ERROR.
*
@ -447,7 +481,7 @@ typedef struct lzma_internal_s lzma_internal;
*
* The lzma_stream structure is used for
* - passing pointers to input and output buffers to liblzma;
* - defining custom memory hander functions; and
* - defining custom memory handler functions; and
* - holding a pointer to coder-specific internal data structures.
*
* Typical usage:
@ -510,15 +544,44 @@ typedef struct {
* you should not touch these, because the names of these variables
* may change.
*/
/** \private Reserved member. */
void *reserved_ptr1;
/** \private Reserved member. */
void *reserved_ptr2;
/** \private Reserved member. */
void *reserved_ptr3;
/** \private Reserved member. */
void *reserved_ptr4;
uint64_t reserved_int1;
/**
* \brief New seek input position for LZMA_SEEK_NEEDED
*
* When lzma_code() returns LZMA_SEEK_NEEDED, the new input position
* needed by liblzma will be available seek_pos. The value is
* guaranteed to not exceed the file size that was specified when
* this lzma_stream was initialized.
*
* In all other situations the value of this variable is undefined.
*/
uint64_t seek_pos;
/** \private Reserved member. */
uint64_t reserved_int2;
/** \private Reserved member. */
size_t reserved_int3;
/** \private Reserved member. */
size_t reserved_int4;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum1;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum2;
} lzma_stream;
@ -558,7 +621,15 @@ typedef struct {
* to and get output from liblzma.
*
* See the description of the coder-specific initialization function to find
* out what `action' values are supported by the coder.
* out what 'action' values are supported by the coder.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param action Action for this function to take. Must be a valid
* lzma_action enum value.
*
* \return Any valid lzma_ret. See the lzma_ret enum description for more
* information.
*/
extern LZMA_API(lzma_ret) lzma_code(lzma_stream *strm, lzma_action action)
lzma_nothrow lzma_attr_warn_unused_result;
@ -567,15 +638,15 @@ extern LZMA_API(lzma_ret) lzma_code(lzma_stream *strm, lzma_action action)
/**
* \brief Free memory allocated for the coder data structures
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
*
* After lzma_end(strm), strm->internal is guaranteed to be NULL. No other
* members of the lzma_stream structure are touched.
*
* \note zlib indicates an error if application end()s unfinished
* stream structure. liblzma doesn't do this, and assumes that
* application knows what it is doing.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
*/
extern LZMA_API(void) lzma_end(lzma_stream *strm) lzma_nothrow;
@ -594,6 +665,11 @@ extern LZMA_API(void) lzma_end(lzma_stream *strm) lzma_nothrow;
* mode by taking into account the progress made by each thread. In
* single-threaded mode *progress_in and *progress_out are set to
* strm->total_in and strm->total_out, respectively.
*
* \param strm Pointer to lzma_stream that is at least
* initialized with LZMA_STREAM_INIT.
* \param[out] progress_in Pointer to the number of input bytes processed.
* \param[out] progress_out Pointer to the number of output bytes processed.
*/
extern LZMA_API(void) lzma_get_progress(lzma_stream *strm,
uint64_t *progress_in, uint64_t *progress_out) lzma_nothrow;
@ -612,6 +688,9 @@ extern LZMA_API(void) lzma_get_progress(lzma_stream *strm,
* this may give misleading information if decoding .xz Streams that have
* multiple Blocks, because each Block can have different memory requirements.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
*
* \return How much memory is currently allocated for the filter
* decoders. If no filter chain is currently allocated,
* some non-zero value is still returned, which is less than
@ -631,6 +710,9 @@ extern LZMA_API(uint64_t) lzma_memusage(const lzma_stream *strm)
* This function is supported only when *strm has been initialized with
* a function that takes a memlimit argument.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
*
* \return On success, the current memory usage limit is returned
* (always non-zero). On error, zero is returned.
*/
@ -649,7 +731,13 @@ extern LZMA_API(uint64_t) lzma_memlimit_get(const lzma_stream *strm)
* return LZMA_OK. Later versions treat 0 as if 1 had been specified (so
* lzma_memlimit_get() will return 1 even if you specify 0 here).
*
* \return - LZMA_OK: New memory usage limit successfully set.
* liblzma 5.2.6 and earlier had a bug in single-threaded .xz decoder
* (lzma_stream_decoder()) which made it impossible to continue decoding
* after LZMA_MEMLIMIT_ERROR even if the limit was increased using
* lzma_memlimit_set(). Other decoders worked correctly.
*
* \return Possible lzma_ret values:
* - LZMA_OK: New memory usage limit successfully set.
* - LZMA_MEMLIMIT_ERROR: The new limit is too small.
* The limit was not changed.
* - LZMA_PROG_ERROR: Invalid arguments, e.g. *strm doesn't

54
Externals/liblzma/api/lzma/bcj.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/bcj.h
* \brief Branch/Call/Jump conversion filters
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -19,35 +17,45 @@
/* Filter IDs for lzma_filter.id */
/**
* \brief Filter for x86 binaries
*/
#define LZMA_FILTER_X86 LZMA_VLI_C(0x04)
/**<
* Filter for x86 binaries
*/
/**
* \brief Filter for Big endian PowerPC binaries
*/
#define LZMA_FILTER_POWERPC LZMA_VLI_C(0x05)
/**<
* Filter for Big endian PowerPC binaries
*/
/**
* \brief Filter for IA-64 (Itanium) binaries
*/
#define LZMA_FILTER_IA64 LZMA_VLI_C(0x06)
/**<
* Filter for IA-64 (Itanium) binaries.
*/
/**
* \brief Filter for ARM binaries
*/
#define LZMA_FILTER_ARM LZMA_VLI_C(0x07)
/**<
* Filter for ARM binaries.
*/
/**
* \brief Filter for ARM-Thumb binaries
*/
#define LZMA_FILTER_ARMTHUMB LZMA_VLI_C(0x08)
/**<
* Filter for ARM-Thumb binaries.
*/
#define LZMA_FILTER_SPARC LZMA_VLI_C(0x09)
/**<
* Filter for SPARC binaries.
/**
* \brief Filter for SPARC binaries
*/
#define LZMA_FILTER_SPARC LZMA_VLI_C(0x09)
/**
* \brief Filter for ARM64 binaries
*/
#define LZMA_FILTER_ARM64 LZMA_VLI_C(0x0A)
/**
* \brief Filter for RISC-V binaries
*/
#define LZMA_FILTER_RISCV LZMA_VLI_C(0x0B)
/**

187
Externals/liblzma/api/lzma/block.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/block.h
* \brief .xz Block handling
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -32,19 +30,28 @@ typedef struct {
* \brief Block format version
*
* To prevent API and ABI breakages when new features are needed,
* a version number is used to indicate which fields in this
* a version number is used to indicate which members in this
* structure are in use:
* - liblzma >= 5.0.0: version = 0 is supported.
* - liblzma >= 5.1.4beta: Support for version = 1 was added,
* which adds the ignore_check field.
* which adds the ignore_check member.
*
* If version is greater than one, most Block related functions
* will return LZMA_OPTIONS_ERROR (lzma_block_header_decode() works
* with any version value).
*
* Read by:
* - All functions that take pointer to lzma_block as argument,
* including lzma_block_header_decode().
* - lzma_block_header_size()
* - lzma_block_header_encode()
* - lzma_block_header_decode()
* - lzma_block_compressed_size()
* - lzma_block_unpadded_size()
* - lzma_block_total_size()
* - lzma_block_encoder()
* - lzma_block_decoder()
* - lzma_block_buffer_encode()
* - lzma_block_uncomp_encode()
* - lzma_block_buffer_decode()
*
* Written by:
* - lzma_block_header_decode()
@ -52,7 +59,7 @@ typedef struct {
uint32_t version;
/**
* \brief Size of the Block Header field
* \brief Size of the Block Header field in bytes
*
* This is always a multiple of four.
*
@ -68,6 +75,7 @@ typedef struct {
* Written by:
* - lzma_block_header_size()
* - lzma_block_buffer_encode()
* - lzma_block_uncomp_encode()
*/
uint32_t header_size;
# define LZMA_BLOCK_HEADER_SIZE_MIN 8
@ -143,6 +151,7 @@ typedef struct {
* - lzma_block_encoder()
* - lzma_block_decoder()
* - lzma_block_buffer_encode()
* - lzma_block_uncomp_encode()
* - lzma_block_buffer_decode()
*/
lzma_vli compressed_size;
@ -167,6 +176,7 @@ typedef struct {
* - lzma_block_encoder()
* - lzma_block_decoder()
* - lzma_block_buffer_encode()
* - lzma_block_uncomp_encode()
* - lzma_block_buffer_decode()
*/
lzma_vli uncompressed_size;
@ -212,6 +222,7 @@ typedef struct {
* - lzma_block_encoder()
* - lzma_block_decoder()
* - lzma_block_buffer_encode()
* - lzma_block_uncomp_encode()
* - lzma_block_buffer_decode()
*/
uint8_t raw_check[LZMA_CHECK_SIZE_MAX];
@ -223,26 +234,56 @@ typedef struct {
* with the currently supported options, so it is safe to leave these
* uninitialized.
*/
/** \private Reserved member. */
void *reserved_ptr1;
/** \private Reserved member. */
void *reserved_ptr2;
/** \private Reserved member. */
void *reserved_ptr3;
/** \private Reserved member. */
uint32_t reserved_int1;
/** \private Reserved member. */
uint32_t reserved_int2;
/** \private Reserved member. */
lzma_vli reserved_int3;
/** \private Reserved member. */
lzma_vli reserved_int4;
/** \private Reserved member. */
lzma_vli reserved_int5;
/** \private Reserved member. */
lzma_vli reserved_int6;
/** \private Reserved member. */
lzma_vli reserved_int7;
/** \private Reserved member. */
lzma_vli reserved_int8;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum1;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum2;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum3;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum4;
/**
* \brief A flag to Block decoder to not verify the Check field
*
* This field is supported by liblzma >= 5.1.4beta if .version >= 1.
* This member is supported by liblzma >= 5.1.4beta if .version >= 1.
*
* If this is set to true, the integrity check won't be calculated
* and verified. Unless you know what you are doing, you should
@ -260,12 +301,25 @@ typedef struct {
*/
lzma_bool ignore_check;
/** \private Reserved member. */
lzma_bool reserved_bool2;
/** \private Reserved member. */
lzma_bool reserved_bool3;
/** \private Reserved member. */
lzma_bool reserved_bool4;
/** \private Reserved member. */
lzma_bool reserved_bool5;
/** \private Reserved member. */
lzma_bool reserved_bool6;
/** \private Reserved member. */
lzma_bool reserved_bool7;
/** \private Reserved member. */
lzma_bool reserved_bool8;
} lzma_block;
@ -280,7 +334,8 @@ typedef struct {
* Note that if the first byte is 0x00, it indicates beginning of Index; use
* this macro only when the byte is not 0x00.
*
* There is no encoding macro, because Block Header encoder is enough for that.
* There is no encoding macro because lzma_block_header_size() and
* lzma_block_header_encode() should be used.
*/
#define lzma_block_header_size_decode(b) (((uint32_t)(b) + 1) * 4)
@ -294,17 +349,20 @@ typedef struct {
* four and doesn't exceed LZMA_BLOCK_HEADER_SIZE_MAX. Increasing header_size
* just means that lzma_block_header_encode() will add Header Padding.
*
* \return - LZMA_OK: Size calculated successfully and stored to
* block->header_size.
* - LZMA_OPTIONS_ERROR: Unsupported version, filters or
* filter options.
* - LZMA_PROG_ERROR: Invalid values like compressed_size == 0.
*
* \note This doesn't check that all the options are valid i.e. this
* may return LZMA_OK even if lzma_block_header_encode() or
* lzma_block_encoder() would fail. If you want to validate the
* filter chain, consider using lzma_memlimit_encoder() which as
* a side-effect validates the filter chain.
*
* \param block Block options
*
* \return Possible lzma_ret values:
* - LZMA_OK: Size calculated successfully and stored to
* block->header_size.
* - LZMA_OPTIONS_ERROR: Unsupported version, filters or
* filter options.
* - LZMA_PROG_ERROR: Invalid values like compressed_size == 0.
*/
extern LZMA_API(lzma_ret) lzma_block_header_size(lzma_block *block)
lzma_nothrow lzma_attr_warn_unused_result;
@ -318,11 +376,12 @@ extern LZMA_API(lzma_ret) lzma_block_header_size(lzma_block *block)
* lzma_block_header_size() is used, the Block Header will be padded to the
* specified size.
*
* \param out Beginning of the output buffer. This must be
* at least block->header_size bytes.
* \param block Block options to be encoded.
* \param[out] out Beginning of the output buffer. This must be
* at least block->header_size bytes.
*
* \return - LZMA_OK: Encoding was successful. block->header_size
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful. block->header_size
* bytes were written to output buffer.
* - LZMA_OPTIONS_ERROR: Invalid or unsupported options.
* - LZMA_PROG_ERROR: Invalid arguments, for example
@ -354,14 +413,15 @@ extern LZMA_API(lzma_ret) lzma_block_header_encode(
* block->filters must have been allocated, but they don't need to be
* initialized (possible existing filter options are not freed).
*
* \param block Destination for Block options.
* \param[out] block Destination for Block options
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() (and also free()
* if an error occurs).
* \param in Beginning of the input buffer. This must be
* at least block->header_size bytes.
*
* \return - LZMA_OK: Decoding was successful. block->header_size
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful. block->header_size
* bytes were read from the input buffer.
* - LZMA_OPTIONS_ERROR: The Block Header specifies some
* unsupported options such as unsupported filters. This can
@ -398,7 +458,12 @@ extern LZMA_API(lzma_ret) lzma_block_header_decode(lzma_block *block,
* field so that it can properly validate Compressed Size if it
* was present in Block Header.
*
* \return - LZMA_OK: block->compressed_size was set successfully.
* \param block Block options: block->header_size must
* already be set with lzma_block_header_size().
* \param unpadded_size Unpadded Size from the Index field in bytes
*
* \return Possible lzma_ret values:
* - LZMA_OK: block->compressed_size was set successfully.
* - LZMA_DATA_ERROR: unpadded_size is too small compared to
* block->header_size and lzma_check_size(block->check).
* - LZMA_PROG_ERROR: Some values are invalid. For example,
@ -419,6 +484,9 @@ extern LZMA_API(lzma_ret) lzma_block_compressed_size(
* Compressed Size, and size of the Check field. This is where this function
* is needed.
*
* \param block Block options: block->header_size must already be
* set with lzma_block_header_size().
*
* \return Unpadded Size on success, or zero on error.
*/
extern LZMA_API(lzma_vli) lzma_block_unpadded_size(const lzma_block *block)
@ -431,6 +499,9 @@ extern LZMA_API(lzma_vli) lzma_block_unpadded_size(const lzma_block *block)
* This is equivalent to lzma_block_unpadded_size() except that the returned
* value includes the size of the Block Padding field.
*
* \param block Block options: block->header_size must already be
* set with lzma_block_header_size().
*
* \return On success, total encoded size of the Block. On error,
* zero is returned.
*/
@ -444,11 +515,21 @@ extern LZMA_API(lzma_vli) lzma_block_total_size(const lzma_block *block)
* Valid actions for lzma_code() are LZMA_RUN, LZMA_SYNC_FLUSH (only if the
* filter chain supports it), and LZMA_FINISH.
*
* \return - LZMA_OK: All good, continue with lzma_code().
* The Block encoder encodes the Block Data, Block Padding, and Check value.
* It does NOT encode the Block Header which can be encoded with
* lzma_block_header_encode().
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param block Block options: block->version, block->check,
* and block->filters must have been initialized.
*
* \return Possible lzma_ret values:
* - LZMA_OK: All good, continue with lzma_code().
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_UNSUPPORTED_CHECK: block->check specifies a Check ID
* that is not supported by this buid of liblzma. Initializing
* that is not supported by this build of liblzma. Initializing
* the encoder failed.
* - LZMA_PROG_ERROR
*/
@ -463,10 +544,16 @@ extern LZMA_API(lzma_ret) lzma_block_encoder(
* Valid actions for lzma_code() are LZMA_RUN and LZMA_FINISH. Using
* LZMA_FINISH is not required. It is supported only for convenience.
*
* \return - LZMA_OK: All good, continue with lzma_code().
* - LZMA_UNSUPPORTED_CHECK: Initialization was successful, but
* the given Check ID is not supported, thus Check will be
* ignored.
* The Block decoder decodes the Block Data, Block Padding, and Check value.
* It does NOT decode the Block Header which can be decoded with
* lzma_block_header_decode().
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param block Block options
*
* \return Possible lzma_ret values:
* - LZMA_OK: All good, continue with lzma_code().
* - LZMA_PROG_ERROR
* - LZMA_MEM_ERROR
*/
@ -480,6 +567,11 @@ extern LZMA_API(lzma_ret) lzma_block_decoder(
*
* This is equivalent to lzma_stream_buffer_bound() but for .xz Blocks.
* See the documentation of lzma_stream_buffer_bound().
*
* \param uncompressed_size Size of the data to be encoded with the
* single-call Block encoder.
*
* \return Maximum output size in bytes for single-call Block encoding.
*/
extern LZMA_API(size_t) lzma_block_buffer_bound(size_t uncompressed_size)
lzma_nothrow;
@ -508,13 +600,14 @@ extern LZMA_API(size_t) lzma_block_buffer_bound(size_t uncompressed_size)
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_size Size of the input buffer
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
@ -540,6 +633,25 @@ extern LZMA_API(lzma_ret) lzma_block_buffer_encode(
* Since the data won't be compressed, this function ignores block->filters.
* This function doesn't take lzma_allocator because this function doesn't
* allocate any memory from the heap.
*
* \param block Block options: block->version, block->check,
* and block->filters must have been initialized.
* \param in Beginning of the input buffer
* \param in_size Size of the input buffer
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
* - LZMA_DATA_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_block_uncomp_encode(lzma_block *block,
const uint8_t *in, size_t in_size,
@ -553,7 +665,7 @@ extern LZMA_API(lzma_ret) lzma_block_uncomp_encode(lzma_block *block,
* This is single-call equivalent of lzma_block_decoder(), and requires that
* the caller has already decoded Block Header and checked its memory usage.
*
* \param block Block options just like with lzma_block_decoder().
* \param block Block options
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
@ -561,13 +673,14 @@ extern LZMA_API(lzma_ret) lzma_block_uncomp_encode(lzma_block *block,
* *in_pos is updated only if decoding succeeds.
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Decoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_OPTIONS_ERROR
* - LZMA_DATA_ERROR
* - LZMA_MEM_ERROR

49
Externals/liblzma/api/lzma/check.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/check.h
* \brief Integrity checks
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -71,12 +69,17 @@ typedef enum {
/**
* \brief Test if the given Check ID is supported
*
* Return true if the given Check ID is supported by this liblzma build.
* Otherwise false is returned. It is safe to call this with a value that
* is not in the range [0, 15]; in that case the return value is always false.
* LZMA_CHECK_NONE and LZMA_CHECK_CRC32 are always supported (even if
* liblzma is built with limited features).
*
* You can assume that LZMA_CHECK_NONE and LZMA_CHECK_CRC32 are always
* supported (even if liblzma is built with limited features).
* \note It is safe to call this with a value that is not in the
* range [0, 15]; in that case the return value is always false.
*
* \param check Check ID
*
* \return lzma_bool:
* - true if Check ID is supported by this liblzma build.
* - false otherwise.
*/
extern LZMA_API(lzma_bool) lzma_check_is_supported(lzma_check check)
lzma_nothrow lzma_attr_const;
@ -90,7 +93,10 @@ extern LZMA_API(lzma_bool) lzma_check_is_supported(lzma_check check)
* the Check field with the specified Check ID. The values are:
* { 0, 4, 4, 4, 8, 8, 8, 16, 16, 16, 32, 32, 32, 64, 64, 64 }
*
* If the argument is not in the range [0, 15], UINT32_MAX is returned.
* \param check Check ID
*
* \return Size of the Check field in bytes. If the argument is not in
* the range [0, 15], UINT32_MAX is returned.
*/
extern LZMA_API(uint32_t) lzma_check_size(lzma_check check)
lzma_nothrow lzma_attr_const;
@ -126,25 +132,32 @@ extern LZMA_API(uint32_t) lzma_crc32(
*
* Calculate CRC64 using the polynomial from the ECMA-182 standard.
*
* This function is used similarly to lzma_crc32(). See its documentation.
* This function is used similarly to lzma_crc32().
*
* \param buf Pointer to the input buffer
* \param size Size of the input buffer
* \param crc Previously returned CRC value. This is used to
* calculate the CRC of a big buffer in smaller chunks.
* Set to zero when starting a new calculation.
*
* \return Updated CRC value, which can be passed to this function
* again to continue CRC calculation.
*/
extern LZMA_API(uint64_t) lzma_crc64(
const uint8_t *buf, size_t size, uint64_t crc)
lzma_nothrow lzma_attr_pure;
/*
* SHA-256 functions are currently not exported to public API.
* Contact Lasse Collin if you think it should be.
*/
/**
* \brief Get the type of the integrity check
*
* This function can be called only immediately after lzma_code() has
* returned LZMA_NO_CHECK, LZMA_UNSUPPORTED_CHECK, or LZMA_GET_CHECK.
* Calling this function in any other situation has undefined behavior.
*
* \param strm Pointer to lzma_stream meeting the above conditions.
*
* \return Check ID in the lzma_stream, or undefined if called improperly.
*/
extern LZMA_API(lzma_check) lzma_get_check(const lzma_stream *strm)
lzma_nothrow;

521
Externals/liblzma/api/lzma/container.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/container.h
* \brief File formats
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -51,7 +49,7 @@
*
* This flag modifies the preset to make the encoding significantly slower
* while improving the compression ratio only marginally. This is useful
* when you don't mind wasting time to get as small result as possible.
* when you don't mind spending time to get as small result as possible.
*
* This flag doesn't affect the memory usage requirements of the decoder (at
* least not significantly). The memory usage of the encoder may be increased
@ -69,7 +67,15 @@ typedef struct {
*
* Set this to zero if no flags are wanted.
*
* No flags are currently supported.
* Encoder: No flags are currently supported.
*
* Decoder: Bitwise-or of zero or more of the decoder flags:
* - LZMA_TELL_NO_CHECK
* - LZMA_TELL_UNSUPPORTED_CHECK
* - LZMA_TELL_ANY_CHECK
* - LZMA_IGNORE_CHECK
* - LZMA_CONCATENATED
* - LZMA_FAIL_FAST
*/
uint32_t flags;
@ -79,7 +85,7 @@ typedef struct {
uint32_t threads;
/**
* \brief Maximum uncompressed size of a Block
* \brief Encoder only: Maximum uncompressed size of a Block
*
* The encoder will start a new .xz Block every block_size bytes.
* Using LZMA_FULL_FLUSH or LZMA_FULL_BARRIER with lzma_code()
@ -106,7 +112,7 @@ typedef struct {
/**
* \brief Timeout to allow lzma_code() to return early
*
* Multithreading can make liblzma to consume input and produce
* Multithreading can make liblzma consume input and produce
* output in a very bursty way: it may first read a lot of input
* to fill internal buffers, then no input or output occurs for
* a while.
@ -123,19 +129,18 @@ typedef struct {
* LZMA_OK. Reasonable values are 100 ms or more. The xz command
* line tool uses 300 ms.
*
* If long blocking times are fine for you, set timeout to a special
* value of 0, which will disable the timeout mechanism and will make
* If long blocking times are acceptable, set timeout to a special
* value of 0. This will disable the timeout mechanism and will make
* lzma_code() block until all the input is consumed or the output
* buffer has been filled.
*
* \note Even with a timeout, lzma_code() might sometimes take
* somewhat long time to return. No timing guarantees
* are made.
* a long time to return. No timing guarantees are made.
*/
uint32_t timeout;
/**
* \brief Compression preset (level and possible flags)
* \brief Encoder only: Compression preset
*
* The preset is set just like with lzma_easy_encoder().
* The preset is ignored if filters below is non-NULL.
@ -143,7 +148,7 @@ typedef struct {
uint32_t preset;
/**
* \brief Filter chain (alternative to a preset)
* \brief Encoder only: Filter chain (alternative to a preset)
*
* If this is NULL, the preset above is used. Otherwise the preset
* is ignored and the filter chain specified here is used.
@ -151,7 +156,7 @@ typedef struct {
const lzma_filter *filters;
/**
* \brief Integrity check type
* \brief Encoder only: Integrity check type
*
* See check.h for available checks. The xz command line tool
* defaults to LZMA_CHECK_CRC64, which is a good choice if you
@ -166,20 +171,86 @@ typedef struct {
* with the currently supported options, so it is safe to leave these
* uninitialized.
*/
/** \private Reserved member. */
lzma_reserved_enum reserved_enum1;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum2;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum3;
/** \private Reserved member. */
uint32_t reserved_int1;
/** \private Reserved member. */
uint32_t reserved_int2;
/** \private Reserved member. */
uint32_t reserved_int3;
/** \private Reserved member. */
uint32_t reserved_int4;
uint64_t reserved_int5;
uint64_t reserved_int6;
/**
* \brief Memory usage limit to reduce the number of threads
*
* Encoder: Ignored.
*
* Decoder:
*
* If the number of threads has been set so high that more than
* memlimit_threading bytes of memory would be needed, the number
* of threads will be reduced so that the memory usage will not exceed
* memlimit_threading bytes. However, if memlimit_threading cannot
* be met even in single-threaded mode, then decoding will continue
* in single-threaded mode and memlimit_threading may be exceeded
* even by a large amount. That is, memlimit_threading will never make
* lzma_code() return LZMA_MEMLIMIT_ERROR. To truly cap the memory
* usage, see memlimit_stop below.
*
* Setting memlimit_threading to UINT64_MAX or a similar huge value
* means that liblzma is allowed to keep the whole compressed file
* and the whole uncompressed file in memory in addition to the memory
* needed by the decompressor data structures used by each thread!
* In other words, a reasonable value limit must be set here or it
* will cause problems sooner or later. If you have no idea what
* a reasonable value could be, try lzma_physmem() / 4 as a starting
* point. Setting this limit will never prevent decompression of
* a file; this will only reduce the number of threads.
*
* If memlimit_threading is greater than memlimit_stop, then the value
* of memlimit_stop will be used for both.
*/
uint64_t memlimit_threading;
/**
* \brief Memory usage limit that should never be exceeded
*
* Encoder: Ignored.
*
* Decoder: If decompressing will need more than this amount of
* memory even in the single-threaded mode, then lzma_code() will
* return LZMA_MEMLIMIT_ERROR.
*/
uint64_t memlimit_stop;
/** \private Reserved member. */
uint64_t reserved_int7;
/** \private Reserved member. */
uint64_t reserved_int8;
/** \private Reserved member. */
void *reserved_ptr1;
/** \private Reserved member. */
void *reserved_ptr2;
/** \private Reserved member. */
void *reserved_ptr3;
/** \private Reserved member. */
void *reserved_ptr4;
} lzma_mt;
@ -193,8 +264,7 @@ typedef struct {
* \param preset Compression preset (level and possible flags)
*
* \return Number of bytes of memory required for the given
* preset when encoding. If an error occurs, for example
* due to unsupported preset, UINT64_MAX is returned.
* preset when encoding or UINT64_MAX on error.
*/
extern LZMA_API(uint64_t) lzma_easy_encoder_memusage(uint32_t preset)
lzma_nothrow lzma_attr_pure;
@ -208,9 +278,8 @@ extern LZMA_API(uint64_t) lzma_easy_encoder_memusage(uint32_t preset)
* \param preset Compression preset (level and possible flags)
*
* \return Number of bytes of memory required to decompress a file
* that was compressed using the given preset. If an error
* occurs, for example due to unsupported preset, UINT64_MAX
* is returned.
* that was compressed using the given preset or UINT64_MAX
* on error.
*/
extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
lzma_nothrow lzma_attr_pure;
@ -220,7 +289,16 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* \brief Initialize .xz Stream encoder using a preset number
*
* This function is intended for those who just want to use the basic features
* if liblzma (that is, most developers out there).
* of liblzma (that is, most developers out there).
*
* If initialization fails (return value is not LZMA_OK), all the memory
* allocated for *strm by liblzma is always freed. Thus, there is no need
* to call lzma_end() after failed initialization.
*
* If initialization succeeds, use lzma_code() to do the actual encoding.
* Valid values for 'action' (the second argument of lzma_code()) are
* LZMA_RUN, LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, and LZMA_FINISH. In future,
* there may be compression levels or flags that don't support LZMA_SYNC_FLUSH.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
@ -228,7 +306,7 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* number and zero or more flags. Usually flags aren't
* used, so preset is simply a number [0, 9] which match
* the options -0 ... -9 of the xz command line tool.
* Additional flags can be be set using bitwise-or with
* Additional flags can be set using bitwise-or with
* the preset level number, e.g. 6 | LZMA_PRESET_EXTREME.
* \param check Integrity check type to use. See check.h for available
* checks. The xz command line tool defaults to
@ -236,7 +314,8 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* unsure. LZMA_CHECK_CRC32 is good too as long as the
* uncompressed file is not many gigabytes.
*
* \return - LZMA_OK: Initialization succeeded. Use lzma_code() to
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization succeeded. Use lzma_code() to
* encode your data.
* - LZMA_MEM_ERROR: Memory allocation failed.
* - LZMA_OPTIONS_ERROR: The given compression preset is not
@ -245,15 +324,6 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* supported by this liblzma build.
* - LZMA_PROG_ERROR: One or more of the parameters have values
* that will never be valid. For example, strm == NULL.
*
* If initialization fails (return value is not LZMA_OK), all the memory
* allocated for *strm by liblzma is always freed. Thus, there is no need
* to call lzma_end() after failed initialization.
*
* If initialization succeeds, use lzma_code() to do the actual encoding.
* Valid values for `action' (the second argument of lzma_code()) are
* LZMA_RUN, LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, and LZMA_FINISH. In future,
* there may be compression levels or flags that don't support LZMA_SYNC_FLUSH.
*/
extern LZMA_API(lzma_ret) lzma_easy_encoder(
lzma_stream *strm, uint32_t preset, lzma_check check)
@ -274,13 +344,14 @@ extern LZMA_API(lzma_ret) lzma_easy_encoder(
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_size Size of the input buffer
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
@ -298,14 +369,16 @@ extern LZMA_API(lzma_ret) lzma_easy_buffer_encode(
/**
* \brief Initialize .xz Stream encoder using a custom filter chain
*
* \param strm Pointer to properly prepared lzma_stream
* \param filters Array of filters. This must be terminated with
* filters[n].id = LZMA_VLI_UNKNOWN. See filter.h for
* more information.
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN. See filters.h for more
* information.
* \param check Type of the integrity check to calculate from
* uncompressed data.
*
* \return - LZMA_OK: Initialization was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization was successful.
* - LZMA_MEM_ERROR
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
@ -345,10 +418,12 @@ extern LZMA_API(uint64_t) lzma_stream_encoder_mt_memusage(
* LZMA_FULL_BARRIER, and LZMA_FINISH. Support for LZMA_SYNC_FLUSH might be
* added in the future.
*
* \param strm Pointer to properly prepared lzma_stream
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param options Pointer to multithreaded compression options
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
@ -359,6 +434,34 @@ extern LZMA_API(lzma_ret) lzma_stream_encoder_mt(
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Calculate recommended Block size for multithreaded .xz encoder
*
* This calculates a recommended Block size for multithreaded encoding given
* a filter chain. This is used internally by lzma_stream_encoder_mt() to
* determine the Block size if the block_size member is not set to the
* special value of 0 in the lzma_mt options struct.
*
* If one wishes to change the filters between Blocks, this function is
* helpful to set the block_size member of the lzma_mt struct before calling
* lzma_stream_encoder_mt(). Since the block_size member represents the
* maximum possible Block size for the multithreaded .xz encoder, one can
* use this function to find the maximum recommended Block size based on
* all planned filter chains. Otherwise, the multithreaded encoder will
* base its maximum Block size on the first filter chain used (if the
* block_size member is not set), which may unnecessarily limit the Block
* size for a later filter chain.
*
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
*
* \return Recommended Block size in bytes, or UINT64_MAX if
* an error occurred.
*/
extern LZMA_API(uint64_t) lzma_mt_block_size(const lzma_filter *filters)
lzma_nothrow;
/**
* \brief Initialize .lzma encoder (legacy file format)
*
@ -374,7 +477,12 @@ extern LZMA_API(lzma_ret) lzma_stream_encoder_mt(
* No kind of flushing is supported, because the file format doesn't make
* it possible.
*
* \return - LZMA_OK
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param options Pointer to encoder options
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
@ -387,7 +495,7 @@ extern LZMA_API(lzma_ret) lzma_alone_encoder(
/**
* \brief Calculate output buffer size for single-call Stream encoder
*
* When trying to compress uncompressible data, the encoded size will be
* When trying to compress incompressible data, the encoded size will be
* slightly bigger than the input data. This function calculates how much
* output buffer space is required to be sure that lzma_stream_buffer_encode()
* doesn't return LZMA_BUF_ERROR.
@ -403,8 +511,13 @@ extern LZMA_API(lzma_ret) lzma_alone_encoder(
* \note The limit calculated by this function applies only to
* single-call encoding. Multi-call encoding may (and probably
* will) have larger maximum expansion when encoding
* uncompressible data. Currently there is no function to
* incompressible data. Currently there is no function to
* calculate the maximum expansion of multi-call encoding.
*
* \param uncompressed_size Size in bytes of the uncompressed
* input data
*
* \return Maximum number of bytes needed to store the compressed data.
*/
extern LZMA_API(size_t) lzma_stream_buffer_bound(size_t uncompressed_size)
lzma_nothrow;
@ -413,22 +526,23 @@ extern LZMA_API(size_t) lzma_stream_buffer_bound(size_t uncompressed_size)
/**
* \brief Single-call .xz Stream encoder
*
* \param filters Array of filters. This must be terminated with
* filters[n].id = LZMA_VLI_UNKNOWN. See filter.h
* for more information.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN. See filters.h for more
* information.
* \param check Type of the integrity check to calculate from
* uncompressed data.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_size Size of the input buffer
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_UNSUPPORTED_CHECK
* - LZMA_OPTIONS_ERROR
@ -444,6 +558,66 @@ extern LZMA_API(lzma_ret) lzma_stream_buffer_encode(
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief MicroLZMA encoder
*
* The MicroLZMA format is a raw LZMA stream whose first byte (always 0x00)
* has been replaced with bitwise-negation of the LZMA properties (lc/lp/pb).
* This encoding ensures that the first byte of MicroLZMA stream is never
* 0x00. There is no end of payload marker and thus the uncompressed size
* must be stored separately. For the best error detection the dictionary
* size should be stored separately as well but alternatively one may use
* the uncompressed size as the dictionary size when decoding.
*
* With the MicroLZMA encoder, lzma_code() behaves slightly unusually.
* The action argument must be LZMA_FINISH and the return value will never be
* LZMA_OK. Thus the encoding is always done with a single lzma_code() after
* the initialization. The benefit of the combination of initialization
* function and lzma_code() is that memory allocations can be re-used for
* better performance.
*
* lzma_code() will try to encode as much input as is possible to fit into
* the given output buffer. If not all input can be encoded, the stream will
* be finished without encoding all the input. The caller must check both
* input and output buffer usage after lzma_code() (total_in and total_out
* in lzma_stream can be convenient). Often lzma_code() can fill the output
* buffer completely if there is a lot of input, but sometimes a few bytes
* may remain unused because the next LZMA symbol would require more space.
*
* lzma_stream.avail_out must be at least 6. Otherwise LZMA_PROG_ERROR
* will be returned.
*
* The LZMA dictionary should be reasonably low to speed up the encoder
* re-initialization. A good value is bigger than the resulting
* uncompressed size of most of the output chunks. For example, if output
* size is 4 KiB, dictionary size of 32 KiB or 64 KiB is good. If the
* data compresses extremely well, even 128 KiB may be useful.
*
* The MicroLZMA format and this encoder variant were made with the EROFS
* file system in mind. This format may be convenient in other embedded
* uses too where many small streams are needed. XZ Embedded includes a
* decoder for this format.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param options Pointer to encoder options
*
* \return Possible lzma_ret values:
* - LZMA_STREAM_END: All good. Check the amounts of input used
* and output produced. Store the amount of input used
* (uncompressed size) as it needs to be known to decompress
* the data.
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR: In addition to the generic reasons for this
* error code, this may also be returned if there isn't enough
* output space (6 bytes) to create a valid MicroLZMA stream.
*/
extern LZMA_API(lzma_ret) lzma_microlzma_encoder(
lzma_stream *strm, const lzma_options_lzma *options)
lzma_nothrow;
/************
* Decoding *
************/
@ -501,24 +675,54 @@ extern LZMA_API(lzma_ret) lzma_stream_buffer_encode(
/**
* This flag enables decoding of concatenated files with file formats that
* allow concatenating compressed files as is. From the formats currently
* supported by liblzma, only the .xz format allows concatenated files.
* Concatenated files are not allowed with the legacy .lzma format.
* supported by liblzma, only the .xz and .lz formats allow concatenated
* files. Concatenated files are not allowed with the legacy .lzma format.
*
* This flag also affects the usage of the `action' argument for lzma_code().
* This flag also affects the usage of the 'action' argument for lzma_code().
* When LZMA_CONCATENATED is used, lzma_code() won't return LZMA_STREAM_END
* unless LZMA_FINISH is used as `action'. Thus, the application has to set
* unless LZMA_FINISH is used as 'action'. Thus, the application has to set
* LZMA_FINISH in the same way as it does when encoding.
*
* If LZMA_CONCATENATED is not used, the decoders still accept LZMA_FINISH
* as `action' for lzma_code(), but the usage of LZMA_FINISH isn't required.
* as 'action' for lzma_code(), but the usage of LZMA_FINISH isn't required.
*/
#define LZMA_CONCATENATED UINT32_C(0x08)
/**
* This flag makes the threaded decoder report errors (like LZMA_DATA_ERROR)
* as soon as they are detected. This saves time when the application has no
* interest in a partially decompressed truncated or corrupt file. Note that
* due to timing randomness, if the same truncated or corrupt input is
* decompressed multiple times with this flag, a different amount of output
* may be produced by different runs, and even the error code might vary.
*
* When using LZMA_FAIL_FAST, it is recommended to use LZMA_FINISH to tell
* the decoder when no more input will be coming because it can help fast
* detection and reporting of truncated files. Note that in this situation
* truncated files might be diagnosed with LZMA_DATA_ERROR instead of
* LZMA_OK or LZMA_BUF_ERROR!
*
* Without this flag the threaded decoder will provide as much output as
* possible at first and then report the pending error. This default behavior
* matches the single-threaded decoder and provides repeatable behavior
* with truncated or corrupt input. There are a few special cases where the
* behavior can still differ like memory allocation failures (LZMA_MEM_ERROR).
*
* Single-threaded decoders currently ignore this flag.
*
* Support for this flag was added in liblzma 5.3.3alpha. Note that in older
* versions this flag isn't supported (LZMA_OPTIONS_ERROR) even by functions
* that ignore this flag in newer liblzma versions.
*/
#define LZMA_FAIL_FAST UINT32_C(0x20)
/**
* \brief Initialize .xz Stream decoder
*
* \param strm Pointer to properly prepared lzma_stream
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param memlimit Memory usage limit as bytes. Use UINT64_MAX
* to effectively disable the limiter. liblzma
* 5.2.3 and earlier don't allow 0 here and return
@ -526,9 +730,11 @@ extern LZMA_API(lzma_ret) lzma_stream_buffer_encode(
* had been specified.
* \param flags Bitwise-or of zero or more of the decoder flags:
* LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK,
* LZMA_TELL_ANY_CHECK, LZMA_CONCATENATED
* LZMA_TELL_ANY_CHECK, LZMA_IGNORE_CHECK,
* LZMA_CONCATENATED, LZMA_FAIL_FAST
*
* \return - LZMA_OK: Initialization was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization was successful.
* - LZMA_MEM_ERROR: Cannot allocate memory.
* - LZMA_OPTIONS_ERROR: Unsupported flags
* - LZMA_PROG_ERROR
@ -539,21 +745,67 @@ extern LZMA_API(lzma_ret) lzma_stream_decoder(
/**
* \brief Decode .xz Streams and .lzma files with autodetection
* \brief Initialize multithreaded .xz Stream decoder
*
* This decoder autodetects between the .xz and .lzma file formats, and
* calls lzma_stream_decoder() or lzma_alone_decoder() once the type
* of the input file has been detected.
* The decoder can decode multiple Blocks in parallel. This requires that each
* Block Header contains the Compressed Size and Uncompressed size fields
* which are added by the multi-threaded encoder, see lzma_stream_encoder_mt().
*
* \param strm Pointer to properly prepared lzma_stream
* A Stream with one Block will only utilize one thread. A Stream with multiple
* Blocks but without size information in Block Headers will be processed in
* single-threaded mode in the same way as done by lzma_stream_decoder().
* Concatenated Streams are processed one Stream at a time; no inter-Stream
* parallelization is done.
*
* This function behaves like lzma_stream_decoder() when options->threads == 1
* and options->memlimit_threading <= 1.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param options Pointer to multithreaded compression options
*
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization was successful.
* - LZMA_MEM_ERROR: Cannot allocate memory.
* - LZMA_MEMLIMIT_ERROR: Memory usage limit was reached.
* - LZMA_OPTIONS_ERROR: Unsupported flags.
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_stream_decoder_mt(
lzma_stream *strm, const lzma_mt *options)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Decode .xz, .lzma, and .lz (lzip) files with autodetection
*
* This decoder autodetects between the .xz, .lzma, and .lz file formats,
* and calls lzma_stream_decoder(), lzma_alone_decoder(), or
* lzma_lzip_decoder() once the type of the input file has been detected.
*
* Support for .lz was added in 5.4.0.
*
* If the flag LZMA_CONCATENATED is used and the input is a .lzma file:
* For historical reasons concatenated .lzma files aren't supported.
* If there is trailing data after one .lzma stream, lzma_code() will
* return LZMA_DATA_ERROR. (lzma_alone_decoder() doesn't have such a check
* as it doesn't support any decoder flags. It will return LZMA_STREAM_END
* after one .lzma stream.)
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param memlimit Memory usage limit as bytes. Use UINT64_MAX
* to effectively disable the limiter. liblzma
* 5.2.3 and earlier don't allow 0 here and return
* LZMA_PROG_ERROR; later versions treat 0 as if 1
* had been specified.
* \param flags Bitwise-or of flags, or zero for no flags.
* \param flags Bitwise-or of zero or more of the decoder flags:
* LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK,
* LZMA_TELL_ANY_CHECK, LZMA_IGNORE_CHECK,
* LZMA_CONCATENATED, LZMA_FAIL_FAST
*
* \return - LZMA_OK: Initialization was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization was successful.
* - LZMA_MEM_ERROR: Cannot allocate memory.
* - LZMA_OPTIONS_ERROR: Unsupported flags
* - LZMA_PROG_ERROR
@ -566,18 +818,20 @@ extern LZMA_API(lzma_ret) lzma_auto_decoder(
/**
* \brief Initialize .lzma decoder (legacy file format)
*
* \param strm Pointer to properly prepared lzma_stream
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* There is no need to use LZMA_FINISH, but it's allowed because it may
* simplify certain types of applications.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param memlimit Memory usage limit as bytes. Use UINT64_MAX
* to effectively disable the limiter. liblzma
* 5.2.3 and earlier don't allow 0 here and return
* LZMA_PROG_ERROR; later versions treat 0 as if 1
* had been specified.
*
* Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* There is no need to use LZMA_FINISH, but it's allowed because it may
* simplify certain types of applications.
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*/
@ -586,6 +840,66 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder(
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Initialize .lz (lzip) decoder (a foreign file format)
*
* This decoder supports the .lz format version 0 and the unextended .lz
* format version 1:
*
* - Files in the format version 0 were produced by lzip 1.3 and older.
* Such files aren't common but may be found from file archives
* as a few source packages were released in this format. People
* might have old personal files in this format too. Decompression
* support for the format version 0 was removed in lzip 1.18.
*
* - lzip 1.3 added decompression support for .lz format version 1 files.
* Compression support was added in lzip 1.4. In lzip 1.6 the .lz format
* version 1 was extended to support the Sync Flush marker. This extension
* is not supported by liblzma. lzma_code() will return LZMA_DATA_ERROR
* at the location of the Sync Flush marker. In practice files with
* the Sync Flush marker are very rare and thus liblzma can decompress
* almost all .lz files.
*
* Just like with lzma_stream_decoder() for .xz files, LZMA_CONCATENATED
* should be used when decompressing normal standalone .lz files.
*
* The .lz format allows putting non-.lz data at the end of a file after at
* least one valid .lz member. That is, one can append custom data at the end
* of a .lz file and the decoder is required to ignore it. In liblzma this
* is relevant only when LZMA_CONCATENATED is used. In that case lzma_code()
* will return LZMA_STREAM_END and leave lzma_stream.next_in pointing to
* the first byte of the non-.lz data. An exception to this is if the first
* 1-3 bytes of the non-.lz data are identical to the .lz magic bytes
* (0x4C, 0x5A, 0x49, 0x50; "LZIP" in US-ASCII). In such a case the 1-3 bytes
* will have been ignored by lzma_code(). If one wishes to locate the non-.lz
* data reliably, one must ensure that the first byte isn't 0x4C. Actually
* one should ensure that none of the first four bytes of trailing data are
* equal to the magic bytes because lzip >= 1.20 requires it by default.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param memlimit Memory usage limit as bytes. Use UINT64_MAX
* to effectively disable the limiter.
* \param flags Bitwise-or of flags, or zero for no flags.
* All decoder flags listed above are supported
* although only LZMA_CONCATENATED and (in very rare
* cases) LZMA_IGNORE_CHECK are actually useful.
* LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK,
* and LZMA_FAIL_FAST do nothing. LZMA_TELL_ANY_CHECK
* is supported for consistency only as CRC32 is
* always used in the .lz format.
*
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization was successful.
* - LZMA_MEM_ERROR: Cannot allocate memory.
* - LZMA_OPTIONS_ERROR: Unsupported flags
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_lzip_decoder(
lzma_stream *strm, uint64_t memlimit, uint32_t flags)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Single-call .xz Stream decoder
*
@ -595,7 +909,8 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder(
* returned.
* \param flags Bitwise-or of zero or more of the decoder flags:
* LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK,
* LZMA_CONCATENATED. Note that LZMA_TELL_ANY_CHECK
* LZMA_IGNORE_CHECK, LZMA_CONCATENATED,
* LZMA_FAIL_FAST. Note that LZMA_TELL_ANY_CHECK
* is not allowed and will return LZMA_PROG_ERROR.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
@ -604,13 +919,14 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder(
* *in_pos is updated only if decoding succeeds.
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if decoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Decoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_FORMAT_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_DATA_ERROR
@ -630,3 +946,50 @@ extern LZMA_API(lzma_ret) lzma_stream_buffer_decode(
const uint8_t *in, size_t *in_pos, size_t in_size,
uint8_t *out, size_t *out_pos, size_t out_size)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief MicroLZMA decoder
*
* See lzma_microlzma_encoder() for more information.
*
* The lzma_code() usage with this decoder is completely normal. The
* special behavior of lzma_code() applies to lzma_microlzma_encoder() only.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param comp_size Compressed size of the MicroLZMA stream.
* The caller must somehow know this exactly.
* \param uncomp_size Uncompressed size of the MicroLZMA stream.
* If the exact uncompressed size isn't known, this
* can be set to a value that is at most as big as
* the exact uncompressed size would be, but then the
* next argument uncomp_size_is_exact must be false.
* \param uncomp_size_is_exact
* If true, uncomp_size must be exactly correct.
* This will improve error detection at the end of
* the stream. If the exact uncompressed size isn't
* known, this must be false. uncomp_size must still
* be at most as big as the exact uncompressed size
* is. Setting this to false when the exact size is
* known will work but error detection at the end of
* the stream will be weaker.
* \param dict_size LZMA dictionary size that was used when
* compressing the data. It is OK to use a bigger
* value too but liblzma will then allocate more
* memory than would actually be required and error
* detection will be slightly worse. (Note that with
* the implementation in XZ Embedded it doesn't
* affect the memory usage if one specifies bigger
* dictionary than actually required.)
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_microlzma_decoder(
lzma_stream *strm, uint64_t comp_size,
uint64_t uncomp_size, lzma_bool uncomp_size_is_exact,
uint32_t dict_size) lzma_nothrow;

28
Externals/liblzma/api/lzma/delta.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/delta.h
* \brief Delta filter
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -57,7 +55,15 @@ typedef struct {
* - 24-bit RGB image data: distance = 3 bytes
*/
uint32_t dist;
/**
* \brief Minimum value for lzma_options_delta.dist.
*/
# define LZMA_DELTA_DIST_MIN 1
/**
* \brief Maximum value for lzma_options_delta.dist.
*/
# define LZMA_DELTA_DIST_MAX 256
/*
@ -67,11 +73,23 @@ typedef struct {
* when type is LZMA_DELTA_TYPE_BYTE, so it is safe to leave these
* uninitialized.
*/
/** \private Reserved member. */
uint32_t reserved_int1;
/** \private Reserved member. */
uint32_t reserved_int2;
/** \private Reserved member. */
uint32_t reserved_int3;
/** \private Reserved member. */
uint32_t reserved_int4;
/** \private Reserved member. */
void *reserved_ptr1;
/** \private Reserved member. */
void *reserved_ptr2;
} lzma_options_delta;

530
Externals/liblzma/api/lzma/filter.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/filter.h
* \brief Common filter related types and functions
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -29,7 +27,7 @@
/**
* \brief Filter options
*
* This structure is used to pass Filter ID and a pointer filter's
* This structure is used to pass a Filter ID and a pointer to the filter's
* options to liblzma. A few functions work with a single lzma_filter
* structure, while most functions expect a filter chain.
*
@ -37,14 +35,14 @@
* The array is terminated with .id = LZMA_VLI_UNKNOWN. Thus, the filter
* array must have LZMA_FILTERS_MAX + 1 elements (that is, five) to
* be able to hold any arbitrary filter chain. This is important when
* using lzma_block_header_decode() from block.h, because too small
* array would make liblzma write past the end of the filters array.
* using lzma_block_header_decode() from block.h, because a filter array
* that is too small would make liblzma write past the end of the array.
*/
typedef struct {
/**
* \brief Filter ID
*
* Use constants whose name begin with `LZMA_FILTER_' to specify
* Use constants whose name begin with 'LZMA_FILTER_' to specify
* different filters. In an array of lzma_filter structures, use
* LZMA_VLI_UNKNOWN to indicate end of filters.
*
@ -68,12 +66,12 @@ typedef struct {
/**
* \brief Test if the given Filter ID is supported for encoding
*
* Return true if the give Filter ID is supported for encoding by this
* liblzma build. Otherwise false is returned.
* \param id Filter ID
*
* There is no way to list which filters are available in this particular
* liblzma version and build. It would be useless, because the application
* couldn't know what kind of options the filter would need.
* \return lzma_bool:
* - true if the Filter ID is supported for encoding by this
* liblzma build.
* - false otherwise.
*/
extern LZMA_API(lzma_bool) lzma_filter_encoder_is_supported(lzma_vli id)
lzma_nothrow lzma_attr_const;
@ -82,8 +80,12 @@ extern LZMA_API(lzma_bool) lzma_filter_encoder_is_supported(lzma_vli id)
/**
* \brief Test if the given Filter ID is supported for decoding
*
* Return true if the give Filter ID is supported for decoding by this
* liblzma build. Otherwise false is returned.
* \param id Filter ID
*
* \return lzma_bool:
* - true if the Filter ID is supported for decoding by this
* liblzma build.
* - false otherwise.
*/
extern LZMA_API(lzma_bool) lzma_filter_decoder_is_supported(lzma_vli id)
lzma_nothrow lzma_attr_const;
@ -108,9 +110,18 @@ extern LZMA_API(lzma_bool) lzma_filter_decoder_is_supported(lzma_vli id)
* need to be initialized by the caller in any way.
*
* If an error occurs, memory possibly already allocated by this function
* is always freed.
* is always freed. liblzma versions older than 5.2.7 may modify the dest
* array and leave its contents in an undefined state if an error occurs.
* liblzma 5.2.7 and newer only modify the dest array when returning LZMA_OK.
*
* \return - LZMA_OK
* \param src Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
* \param[out] dest Destination filter array
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR: Unsupported Filter ID and its options
* is not NULL.
@ -118,7 +129,34 @@ extern LZMA_API(lzma_bool) lzma_filter_decoder_is_supported(lzma_vli id)
*/
extern LZMA_API(lzma_ret) lzma_filters_copy(
const lzma_filter *src, lzma_filter *dest,
const lzma_allocator *allocator) lzma_nothrow;
const lzma_allocator *allocator)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Free the options in the array of lzma_filter structures
*
* This frees the filter chain options. The filters array itself is not freed.
*
* The filters array must have at most LZMA_FILTERS_MAX + 1 elements
* including the terminating element which must have .id = LZMA_VLI_UNKNOWN.
* For all elements before the terminating element:
* - options will be freed using the given lzma_allocator or,
* if allocator is NULL, using free().
* - options will be set to NULL.
* - id will be set to LZMA_VLI_UNKNOWN.
*
* If filters is NULL, this does nothing. Again, this never frees the
* filters array itself.
*
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*/
extern LZMA_API(void) lzma_filters_free(
lzma_filter *filters, const lzma_allocator *allocator)
lzma_nothrow;
/**
@ -132,9 +170,7 @@ extern LZMA_API(lzma_ret) lzma_filters_copy(
* .id == LZMA_VLI_UNKNOWN.
*
* \return Number of bytes of memory required for the given
* filter chain when encoding. If an error occurs,
* for example due to unsupported filter chain,
* UINT64_MAX is returned.
* filter chain when encoding or UINT64_MAX on error.
*/
extern LZMA_API(uint64_t) lzma_raw_encoder_memusage(const lzma_filter *filters)
lzma_nothrow lzma_attr_pure;
@ -151,9 +187,7 @@ extern LZMA_API(uint64_t) lzma_raw_encoder_memusage(const lzma_filter *filters)
* .id == LZMA_VLI_UNKNOWN.
*
* \return Number of bytes of memory required for the given
* filter chain when decoding. If an error occurs,
* for example due to unsupported filter chain,
* UINT64_MAX is returned.
* filter chain when decoding or UINT64_MAX on error.
*/
extern LZMA_API(uint64_t) lzma_raw_decoder_memusage(const lzma_filter *filters)
lzma_nothrow lzma_attr_pure;
@ -164,14 +198,16 @@ extern LZMA_API(uint64_t) lzma_raw_decoder_memusage(const lzma_filter *filters)
*
* This function may be useful when implementing custom file formats.
*
* \param strm Pointer to properly prepared lzma_stream
* \param filters Array of lzma_filter structures. The end of the
* array must be marked with .id = LZMA_VLI_UNKNOWN.
*
* The `action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
* The 'action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
* filter chain supports it), or LZMA_FINISH.
*
* \return - LZMA_OK
* \param strm Pointer to lzma_stream that is at least
* initialized with LZMA_STREAM_INIT.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
@ -186,10 +222,16 @@ extern LZMA_API(lzma_ret) lzma_raw_encoder(
*
* The initialization of raw decoder goes similarly to raw encoder.
*
* The `action' with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using
* The 'action' with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using
* LZMA_FINISH is not required, it is supported just for convenience.
*
* \return - LZMA_OK
* \param strm Pointer to lzma_stream that is at least
* initialized with LZMA_STREAM_INIT.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
@ -202,24 +244,36 @@ extern LZMA_API(lzma_ret) lzma_raw_decoder(
/**
* \brief Update the filter chain in the encoder
*
* This function is for advanced users only. This function has two slightly
* different purposes:
* This function may be called after lzma_code() has returned LZMA_STREAM_END
* when LZMA_FULL_BARRIER, LZMA_FULL_FLUSH, or LZMA_SYNC_FLUSH was used:
*
* - After LZMA_FULL_FLUSH when using Stream encoder: Set a new filter
* chain, which will be used starting from the next Block.
* - After LZMA_FULL_BARRIER or LZMA_FULL_FLUSH: Single-threaded .xz Stream
* encoder (lzma_stream_encoder()) and (since liblzma 5.4.0) multi-threaded
* Stream encoder (lzma_stream_encoder_mt()) allow setting a new filter
* chain to be used for the next Block(s).
*
* - After LZMA_SYNC_FLUSH using Raw, Block, or Stream encoder: Change
* the filter-specific options in the middle of encoding. The actual
* filters in the chain (Filter IDs) cannot be changed. In the future,
* it might become possible to change the filter options without
* using LZMA_SYNC_FLUSH.
* - After LZMA_SYNC_FLUSH: Raw encoder (lzma_raw_encoder()),
* Block encoder (lzma_block_encoder()), and single-threaded .xz Stream
* encoder (lzma_stream_encoder()) allow changing certain filter-specific
* options in the middle of encoding. The actual filters in the chain
* (Filter IDs) must not be changed! Currently only the lc, lp, and pb
* options of LZMA2 (not LZMA1) can be changed this way.
*
* While rarely useful, this function may be called also when no data has
* been compressed yet. In that case, this function will behave as if
* LZMA_FULL_FLUSH (Stream encoder) or LZMA_SYNC_FLUSH (Raw or Block
* - In the future some filters might allow changing some of their options
* without any barrier or flushing but currently such filters don't exist.
*
* This function may also be called when no data has been compressed yet
* although this is rarely useful. In that case, this function will behave
* as if LZMA_FULL_FLUSH (Stream encoders) or LZMA_SYNC_FLUSH (Raw or Block
* encoder) had been used right before calling this function.
*
* \return - LZMA_OK
* \param strm Pointer to lzma_stream that is at least
* initialized with LZMA_STREAM_INIT.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_MEMLIMIT_ERROR
* - LZMA_OPTIONS_ERROR
@ -232,29 +286,30 @@ extern LZMA_API(lzma_ret) lzma_filters_update(
/**
* \brief Single-call raw encoder
*
* \param filters Array of lzma_filter structures. The end of the
* array must be marked with .id = LZMA_VLI_UNKNOWN.
* \note There is no function to calculate how big output buffer
* would surely be big enough. (lzma_stream_buffer_bound()
* works only for lzma_stream_buffer_encode(); raw encoder
* won't necessarily meet that bound.)
*
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_size Size of the input buffer
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
* - LZMA_DATA_ERROR
* - LZMA_PROG_ERROR
*
* \note There is no function to calculate how big output buffer
* would surely be big enough. (lzma_stream_buffer_bound()
* works only for lzma_stream_buffer_encode(); raw encoder
* won't necessarily meet that bound.)
*/
extern LZMA_API(lzma_ret) lzma_raw_buffer_encode(
const lzma_filter *filters, const lzma_allocator *allocator,
@ -265,8 +320,8 @@ extern LZMA_API(lzma_ret) lzma_raw_buffer_encode(
/**
* \brief Single-call raw decoder
*
* \param filters Array of lzma_filter structures. The end of the
* array must be marked with .id = LZMA_VLI_UNKNOWN.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
@ -274,11 +329,19 @@ extern LZMA_API(lzma_ret) lzma_raw_buffer_encode(
* *in_pos is updated only if decoding succeeds.
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_BUF_ERROR: Not enough output buffer space.
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
* - LZMA_DATA_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_raw_buffer_decode(
const lzma_filter *filters, const lzma_allocator *allocator,
@ -292,18 +355,19 @@ extern LZMA_API(lzma_ret) lzma_raw_buffer_decode(
* This function may be useful when implementing custom file formats
* using the raw encoder and decoder.
*
* \param size Pointer to uint32_t to hold the size of the properties
* \param filter Filter ID and options (the size of the properties may
* vary depending on the options)
*
* \return - LZMA_OK
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
*
* \note This function validates the Filter ID, but does not
* necessarily validate the options. Thus, it is possible
* that this returns LZMA_OK while the following call to
* lzma_properties_encode() returns LZMA_OPTIONS_ERROR.
*
* \param[out] size Pointer to uint32_t to hold the size of the properties
* \param filter Filter ID and options (the size of the properties may
* vary depending on the options)
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_properties_size(
uint32_t *size, const lzma_filter *filter) lzma_nothrow;
@ -312,15 +376,6 @@ extern LZMA_API(lzma_ret) lzma_properties_size(
/**
* \brief Encode the Filter Properties field
*
* \param filter Filter ID and options
* \param props Buffer to hold the encoded options. The size of
* buffer must have been already determined with
* lzma_properties_size().
*
* \return - LZMA_OK
* - LZMA_OPTIONS_ERROR
* - LZMA_PROG_ERROR
*
* \note Even this function won't validate more options than actually
* necessary. Thus, it is possible that encoding the properties
* succeeds but using the same options to initialize the encoder
@ -330,6 +385,15 @@ extern LZMA_API(lzma_ret) lzma_properties_size(
* of the Filter Properties field is zero, calling
* lzma_properties_encode() is not required, but it
* won't do any harm either.
*
* \param filter Filter ID and options
* \param[out] props Buffer to hold the encoded options. The size of
* the buffer must have been already determined with
* lzma_properties_size().
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_properties_encode(
const lzma_filter *filter, uint8_t *props) lzma_nothrow;
@ -341,18 +405,20 @@ extern LZMA_API(lzma_ret) lzma_properties_encode(
* \param filter filter->id must have been set to the correct
* Filter ID. filter->options doesn't need to be
* initialized (it's not freed by this function). The
* decoded options will be stored to filter->options.
* filter->options is set to NULL if there are no
* properties or if an error occurs.
* \param allocator Custom memory allocator used to allocate the
* options. Set to NULL to use the default malloc(),
* decoded options will be stored in filter->options;
* it's application's responsibility to free it when
* appropriate. filter->options is set to NULL if
* there are no properties or if an error occurs.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* and in case of an error, also free().
* \param props Input buffer containing the properties.
* \param props_size Size of the properties. This must be the exact
* size; giving too much or too little input will
* return LZMA_OPTIONS_ERROR.
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
*/
@ -367,18 +433,19 @@ extern LZMA_API(lzma_ret) lzma_properties_decode(
* Knowing the size of Filter Flags is useful to know when allocating
* memory to hold the encoded Filter Flags.
*
* \param size Pointer to integer to hold the calculated size
* \note If you need to calculate size of List of Filter Flags,
* you need to loop over every lzma_filter entry.
*
* \param[out] size Pointer to integer to hold the calculated size
* \param filter Filter ID and associated options whose encoded
* size is to be calculated
*
* \return - LZMA_OK: *size set successfully. Note that this doesn't
* \return Possible lzma_ret values:
* - LZMA_OK: *size set successfully. Note that this doesn't
* guarantee that filter->options is valid, thus
* lzma_filter_flags_encode() may still fail.
* - LZMA_OPTIONS_ERROR: Unknown Filter ID or unsupported options.
* - LZMA_PROG_ERROR: Invalid options
*
* \note If you need to calculate size of List of Filter Flags,
* you need to loop over every lzma_filter entry.
*/
extern LZMA_API(lzma_ret) lzma_filter_flags_size(
uint32_t *size, const lzma_filter *filter)
@ -392,12 +459,13 @@ extern LZMA_API(lzma_ret) lzma_filter_flags_size(
* This is due to how this function is used internally by liblzma.
*
* \param filter Filter ID and options to be encoded
* \param out Beginning of the output buffer
* \param out_pos out[*out_pos] is the next write position. This
* \param[out] out Beginning of the output buffer
* \param[out] out_pos out[*out_pos] is the next write position. This
* is updated by the encoder.
* \param out_size out[out_size] is the first byte to not write.
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_OPTIONS_ERROR: Invalid or unsupported options.
* - LZMA_PROG_ERROR: Invalid options or not enough output
* buffer space (you should have checked it with
@ -412,14 +480,290 @@ extern LZMA_API(lzma_ret) lzma_filter_flags_encode(const lzma_filter *filter,
* \brief Decode Filter Flags from given buffer
*
* The decoded result is stored into *filter. The old value of
* filter->options is not free()d.
* filter->options is not free()d. If anything other than LZMA_OK
* is returned, filter->options is set to NULL.
*
* \return - LZMA_OK
* \param[out] filter Destination filter. The decoded Filter ID will
* be stored in filter->id. If options are needed
* they will be allocated and the pointer will be
* stored in filter->options.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param[out] in_pos The next byte will be read from in[*in_pos].
* *in_pos is updated only if decoding succeeds.
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR
* - LZMA_MEM_ERROR
* - LZMA_DATA_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_filter_flags_decode(
lzma_filter *filter, const lzma_allocator *allocator,
const uint8_t *in, size_t *in_pos, size_t in_size)
lzma_nothrow lzma_attr_warn_unused_result;
/***********
* Strings *
***********/
/**
* \brief Allow or show all filters
*
* By default only the filters supported in the .xz format are accept by
* lzma_str_to_filters() or shown by lzma_str_list_filters().
*/
#define LZMA_STR_ALL_FILTERS UINT32_C(0x01)
/**
* \brief Do not validate the filter chain in lzma_str_to_filters()
*
* By default lzma_str_to_filters() can return an error if the filter chain
* as a whole isn't usable in the .xz format or in the raw encoder or decoder.
* With this flag, this validation is skipped. This flag doesn't affect the
* handling of the individual filter options. To allow non-.xz filters also
* LZMA_STR_ALL_FILTERS is needed.
*/
#define LZMA_STR_NO_VALIDATION UINT32_C(0x02)
/**
* \brief Stringify encoder options
*
* Show the filter-specific options that the encoder will use.
* This may be useful for verbose diagnostic messages.
*
* Note that if options were decoded from .xz headers then the encoder options
* may be undefined. This flag shouldn't be used in such a situation.
*/
#define LZMA_STR_ENCODER UINT32_C(0x10)
/**
* \brief Stringify decoder options
*
* Show the filter-specific options that the decoder will use.
* This may be useful for showing what filter options were decoded
* from file headers.
*/
#define LZMA_STR_DECODER UINT32_C(0x20)
/**
* \brief Produce xz-compatible getopt_long() syntax
*
* That is, "delta:dist=2 lzma2:dict=4MiB,pb=1,lp=1" becomes
* "--delta=dist=2 --lzma2=dict=4MiB,pb=1,lp=1".
*
* This syntax is compatible with xz 5.0.0 as long as the filters and
* their options are supported too.
*/
#define LZMA_STR_GETOPT_LONG UINT32_C(0x40)
/**
* \brief Use two dashes "--" instead of a space to separate filters
*
* That is, "delta:dist=2 lzma2:pb=1,lp=1" becomes
* "delta:dist=2--lzma2:pb=1,lp=1". This looks slightly odd but this
* kind of strings should be usable on the command line without quoting.
* However, it is possible that future versions with new filter options
* might produce strings that require shell quoting anyway as the exact
* set of possible characters isn't frozen for now.
*
* It is guaranteed that the single quote (') will never be used in
* filter chain strings (even if LZMA_STR_NO_SPACES isn't used).
*/
#define LZMA_STR_NO_SPACES UINT32_C(0x80)
/**
* \brief Convert a string to a filter chain
*
* This tries to make it easier to write applications that allow users
* to set custom compression options. This only handles the filter
* configuration (including presets) but not the number of threads,
* block size, check type, or memory limits.
*
* The input string can be either a preset or a filter chain. Presets
* begin with a digit 0-9 and may be followed by zero or more flags
* which are lower-case letters. Currently only "e" is supported, matching
* LZMA_PRESET_EXTREME. For partial xz command line syntax compatibility,
* a preset string may start with a single dash "-".
*
* A filter chain consists of one or more "filtername:opt1=value1,opt2=value2"
* strings separated by one or more spaces. Leading and trailing spaces are
* ignored. All names and values must be lower-case. Extra commas in the
* option list are ignored. The order of filters is significant: when
* encoding, the uncompressed input data goes to the leftmost filter first.
* Normally "lzma2" is the last filter in the chain.
*
* If one wishes to avoid spaces, for example, to avoid shell quoting,
* it is possible to use two dashes "--" instead of spaces to separate
* the filters.
*
* For xz command line compatibility, each filter may be prefixed with
* two dashes "--" and the colon ":" separating the filter name from
* the options may be replaced with an equals sign "=".
*
* By default, only filters that can be used in the .xz format are accepted.
* To allow all filters (LZMA1) use the flag LZMA_STR_ALL_FILTERS.
*
* By default, very basic validation is done for the filter chain as a whole,
* for example, that LZMA2 is only used as the last filter in the chain.
* The validation isn't perfect though and it's possible that this function
* succeeds but using the filter chain for encoding or decoding will still
* result in LZMA_OPTIONS_ERROR. To disable this validation, use the flag
* LZMA_STR_NO_VALIDATION.
*
* The available filter names and their options are available via
* lzma_str_list_filters(). See the xz man page for the description
* of filter names and options.
*
* For command line applications, below is an example how an error message
* can be displayed. Note the use of an empty string for the field width.
* If "^" was used there it would create an off-by-one error except at
* the very beginning of the line.
*
* \code{.c}
* const char *str = ...; // From user
* lzma_filter filters[LZMA_FILTERS_MAX + 1];
* int pos;
* const char *msg = lzma_str_to_filters(str, &pos, filters, 0, NULL);
* if (msg != NULL) {
* printf("%s: Error in XZ compression options:\n", argv[0]);
* printf("%s: %s\n", argv[0], str);
* printf("%s: %*s^\n", argv[0], errpos, "");
* printf("%s: %s\n", argv[0], msg);
* }
* \endcode
*
* \param str User-supplied string describing a preset or
* a filter chain. If a default value is needed and
* you don't know what would be good, use "6" since
* that is the default preset in xz too.
* \param[out] error_pos If this isn't NULL, this value will be set on
* both success and on all errors. This tells the
* location of the error in the string. This is
* an int to make it straightforward to use this
* as printf() field width. The value is guaranteed
* to be in the range [0, INT_MAX] even if strlen(str)
* somehow was greater than INT_MAX.
* \param[out] filters An array of lzma_filter structures. There must
* be LZMA_FILTERS_MAX + 1 (that is, five) elements
* in the array. The old contents are ignored so it
* doesn't need to be initialized. This array is
* modified only if this function returns NULL.
* Once the allocated filter options are no longer
* needed, lzma_filters_free() can be used to free the
* options (it doesn't free the filters array itself).
* \param flags Bitwise-or of zero or more of the flags
* LZMA_STR_ALL_FILTERS and LZMA_STR_NO_VALIDATION.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return On success, NULL is returned. On error, a statically-allocated
* error message is returned which together with the error_pos
* should give some idea what is wrong.
*/
extern LZMA_API(const char *) lzma_str_to_filters(
const char *str, int *error_pos, lzma_filter *filters,
uint32_t flags, const lzma_allocator *allocator)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief Convert a filter chain to a string
*
* Use cases:
*
* - Verbose output showing the full encoder options to the user
* (use LZMA_STR_ENCODER in flags)
*
* - Showing the filters and options that are required to decode a file
* (use LZMA_STR_DECODER in flags)
*
* - Showing the filter names without any options in informational messages
* where the technical details aren't important (no flags). In this case
* the .options in the filters array are ignored and may be NULL even if
* a filter has a mandatory options structure.
*
* Note that even if the filter chain was specified using a preset,
* the resulting filter chain isn't reversed to a preset. So if you
* specify "6" to lzma_str_to_filters() then lzma_str_from_filters()
* will produce a string containing "lzma2".
*
* \param[out] str On success *str will be set to point to an
* allocated string describing the given filter
* chain. Old value is ignored. On error *str is
* always set to NULL.
* \param filters Array of filters terminated with
* .id == LZMA_VLI_UNKNOWN.
* \param flags Bitwise-or of zero or more of the flags
* LZMA_STR_ENCODER, LZMA_STR_DECODER,
* LZMA_STR_GETOPT_LONG, and LZMA_STR_NO_SPACES.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR: Empty filter chain
* (filters[0].id == LZMA_VLI_UNKNOWN) or the filter chain
* includes a Filter ID that is not supported by this function.
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_str_from_filters(
char **str, const lzma_filter *filters, uint32_t flags,
const lzma_allocator *allocator)
lzma_nothrow lzma_attr_warn_unused_result;
/**
* \brief List available filters and/or their options (for help message)
*
* If a filter_id is given then only one line is created which contains the
* filter name. If LZMA_STR_ENCODER or LZMA_STR_DECODER is used then the
* options read by the encoder or decoder are printed on the same line.
*
* If filter_id is LZMA_VLI_UNKNOWN then all supported .xz-compatible filters
* are listed:
*
* - If neither LZMA_STR_ENCODER nor LZMA_STR_DECODER is used then
* the supported filter names are listed on a single line separated
* by spaces.
*
* - If LZMA_STR_ENCODER or LZMA_STR_DECODER is used then filters and
* the supported options are listed one filter per line. There won't
* be a newline after the last filter.
*
* - If LZMA_STR_ALL_FILTERS is used then the list will include also
* those filters that cannot be used in the .xz format (LZMA1).
*
* \param str On success *str will be set to point to an
* allocated string listing the filters and options.
* Old value is ignored. On error *str is always set
* to NULL.
* \param filter_id Filter ID or LZMA_VLI_UNKNOWN.
* \param flags Bitwise-or of zero or more of the flags
* LZMA_STR_ALL_FILTERS, LZMA_STR_ENCODER,
* LZMA_STR_DECODER, and LZMA_STR_GETOPT_LONG.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR: Unsupported filter_id or flags
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_str_list_filters(
char **str, lzma_vli filter_id, uint32_t flags,
const lzma_allocator *allocator)
lzma_nothrow lzma_attr_warn_unused_result;

12
Externals/liblzma/api/lzma/hardware.h vendored
View file

@ -1,12 +1,15 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/hardware.h
* \brief Hardware information
* \note Never include this file directly. Use <lzma.h> instead.
*
* Since liblzma can consume a lot of system resources, it also provides
* ways to limit the resource usage. Applications linking against liblzma
* need to do the actual decisions how much resources to let liblzma to use.
* To ease making these decisions, liblzma provides functions to find out
* the relevant capabilities of the underlaying hardware. Currently there
* the relevant capabilities of the underlying hardware. Currently there
* is only a function to find out the amount of RAM, but in the future there
* will be also a function to detect how many concurrent threads the system
* can run.
@ -22,11 +25,6 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -57,7 +55,7 @@ extern LZMA_API(uint64_t) lzma_physmem(void) lzma_nothrow;
* If the hardware supports more than one thread per CPU core, the number
* of hardware threads is returned if that information is available.
*
* \brief On success, the number of available CPU threads or cores is
* \return On success, the number of available CPU threads or cores is
* returned. If this information isn't available or an error
* occurs, zero is returned.
*/

298
Externals/liblzma/api/lzma/index.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/index.h
* \brief Handling of .xz Index and related information
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -50,8 +48,13 @@ typedef struct {
*/
const lzma_stream_flags *flags;
/** \private Reserved member. */
const void *reserved_ptr1;
/** \private Reserved member. */
const void *reserved_ptr2;
/** \private Reserved member. */
const void *reserved_ptr3;
/**
@ -107,9 +110,17 @@ typedef struct {
*/
lzma_vli padding;
/** \private Reserved member. */
lzma_vli reserved_vli1;
/** \private Reserved member. */
lzma_vli reserved_vli2;
/** \private Reserved member. */
lzma_vli reserved_vli3;
/** \private Reserved member. */
lzma_vli reserved_vli4;
} stream;
@ -196,25 +207,46 @@ typedef struct {
*/
lzma_vli total_size;
/** \private Reserved member. */
lzma_vli reserved_vli1;
/** \private Reserved member. */
lzma_vli reserved_vli2;
/** \private Reserved member. */
lzma_vli reserved_vli3;
/** \private Reserved member. */
lzma_vli reserved_vli4;
/** \private Reserved member. */
const void *reserved_ptr1;
/** \private Reserved member. */
const void *reserved_ptr2;
/** \private Reserved member. */
const void *reserved_ptr3;
/** \private Reserved member. */
const void *reserved_ptr4;
} block;
/*
/**
* \private Internal data
*
* Internal data which is used to store the state of the iterator.
* The exact format may vary between liblzma versions, so don't
* touch these in any way.
*/
union {
/** \private Internal member. */
const void *p;
/** \private Internal member. */
size_t s;
/** \private Internal member. */
lzma_vli v;
} internal[6];
} lzma_index_iter;
@ -268,20 +300,47 @@ typedef enum {
} lzma_index_iter_mode;
/**
* \brief Mask for return value from lzma_index_checks() for check none
*
* \note This and the other CHECK_MASK macros were added in 5.5.1alpha.
*/
#define LZMA_INDEX_CHECK_MASK_NONE (UINT32_C(1) << LZMA_CHECK_NONE)
/**
* \brief Mask for return value from lzma_index_checks() for check CRC32
*/
#define LZMA_INDEX_CHECK_MASK_CRC32 (UINT32_C(1) << LZMA_CHECK_CRC32)
/**
* \brief Mask for return value from lzma_index_checks() for check CRC64
*/
#define LZMA_INDEX_CHECK_MASK_CRC64 (UINT32_C(1) << LZMA_CHECK_CRC64)
/**
* \brief Mask for return value from lzma_index_checks() for check SHA256
*/
#define LZMA_INDEX_CHECK_MASK_SHA256 (UINT32_C(1) << LZMA_CHECK_SHA256)
/**
* \brief Calculate memory usage of lzma_index
*
* On disk, the size of the Index field depends on both the number of Records
* stored and how big values the Records store (due to variable-length integer
* stored and the size of the Records (due to variable-length integer
* encoding). When the Index is kept in lzma_index structure, the memory usage
* depends only on the number of Records/Blocks stored in the Index(es), and
* in case of concatenated lzma_indexes, the number of Streams. The size in
* RAM is almost always significantly bigger than in the encoded form on disk.
*
* This function calculates an approximate amount of memory needed hold
* This function calculates an approximate amount of memory needed to hold
* the given number of Streams and Blocks in lzma_index structure. This
* value may vary between CPU architectures and also between liblzma versions
* if the internal implementation is modified.
*
* \param streams Number of Streams
* \param blocks Number of Blocks
*
* \return Approximate memory in bytes needed in a lzma_index structure.
*/
extern LZMA_API(uint64_t) lzma_index_memusage(
lzma_vli streams, lzma_vli blocks) lzma_nothrow;
@ -292,6 +351,10 @@ extern LZMA_API(uint64_t) lzma_index_memusage(
*
* This is a shorthand for lzma_index_memusage(lzma_index_stream_count(i),
* lzma_index_block_count(i)).
*
* \param i Pointer to lzma_index structure
*
* \return Approximate memory in bytes used by the lzma_index structure.
*/
extern LZMA_API(uint64_t) lzma_index_memused(const lzma_index *i)
lzma_nothrow;
@ -300,6 +363,9 @@ extern LZMA_API(uint64_t) lzma_index_memused(const lzma_index *i)
/**
* \brief Allocate and initialize a new lzma_index structure
*
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return On success, a pointer to an empty initialized lzma_index is
* returned. If allocation fails, NULL is returned.
*/
@ -311,6 +377,10 @@ extern LZMA_API(lzma_index *) lzma_index_init(const lzma_allocator *allocator)
* \brief Deallocate lzma_index
*
* If i is NULL, this does nothing.
*
* \param i Pointer to lzma_index structure to deallocate
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*/
extern LZMA_API(void) lzma_index_end(
lzma_index *i, const lzma_allocator *allocator) lzma_nothrow;
@ -320,8 +390,9 @@ extern LZMA_API(void) lzma_index_end(
* \brief Add a new Block to lzma_index
*
* \param i Pointer to a lzma_index structure
* \param allocator Pointer to lzma_allocator, or NULL to
* use malloc()
* \param allocator lzma_allocator for custom allocator
* functions. Set to NULL to use malloc()
* and free().
* \param unpadded_size Unpadded Size of a Block. This can be
* calculated with lzma_block_unpadded_size()
* after encoding or decoding the Block.
@ -334,7 +405,8 @@ extern LZMA_API(void) lzma_index_end(
* lzma_index_append() it is possible to read the next Block with
* an existing iterator.
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_DATA_ERROR: Compressed or uncompressed size of the
* Stream or size of the Index field would grow too big.
@ -354,11 +426,15 @@ extern LZMA_API(lzma_ret) lzma_index_append(
* lzma_index, because to decode Blocks, knowing the integrity check type
* is needed.
*
* The given Stream Flags are copied into internal preallocated structure
* in the lzma_index, thus the caller doesn't need to keep the *stream_flags
* available after calling this function.
* \param i Pointer to lzma_index structure
* \param stream_flags Pointer to lzma_stream_flags structure. This
* is copied into the internal preallocated
* structure, so the caller doesn't need to keep
* the flags' data available after calling this
* function.
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_OPTIONS_ERROR: Unsupported stream_flags->version.
* - LZMA_PROG_ERROR
*/
@ -376,6 +452,11 @@ extern LZMA_API(lzma_ret) lzma_index_stream_flags(
* showing the Check types to the user.
*
* The bitmask is 1 << check_id, e.g. CRC32 is 1 << 1 and SHA-256 is 1 << 10.
* These masks are defined for convenience as LZMA_INDEX_CHECK_MASK_XXX
*
* \param i Pointer to lzma_index structure
*
* \return Bitmask indicating which Check types are used in the lzma_index
*/
extern LZMA_API(uint32_t) lzma_index_checks(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -390,7 +471,8 @@ extern LZMA_API(uint32_t) lzma_index_checks(const lzma_index *i)
*
* By default, the amount of Stream Padding is assumed to be zero bytes.
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_DATA_ERROR: The file size would grow too big.
* - LZMA_PROG_ERROR
*/
@ -401,6 +483,10 @@ extern LZMA_API(lzma_ret) lzma_index_stream_padding(
/**
* \brief Get the number of Streams
*
* \param i Pointer to lzma_index structure
*
* \return Number of Streams in the lzma_index
*/
extern LZMA_API(lzma_vli) lzma_index_stream_count(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -411,6 +497,10 @@ extern LZMA_API(lzma_vli) lzma_index_stream_count(const lzma_index *i)
*
* This returns the total number of Blocks in lzma_index. To get number
* of Blocks in individual Streams, use lzma_index_iter.
*
* \param i Pointer to lzma_index structure
*
* \return Number of blocks in the lzma_index
*/
extern LZMA_API(lzma_vli) lzma_index_block_count(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -420,6 +510,10 @@ extern LZMA_API(lzma_vli) lzma_index_block_count(const lzma_index *i)
* \brief Get the size of the Index field as bytes
*
* This is needed to verify the Backward Size field in the Stream Footer.
*
* \param i Pointer to lzma_index structure
*
* \return Size in bytes of the Index
*/
extern LZMA_API(lzma_vli) lzma_index_size(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -431,6 +525,11 @@ extern LZMA_API(lzma_vli) lzma_index_size(const lzma_index *i)
* If multiple lzma_indexes have been combined, this works as if the Blocks
* were in a single Stream. This is useful if you are going to combine
* Blocks from multiple Streams into a single new Stream.
*
* \param i Pointer to lzma_index structure
*
* \return Size in bytes of the Stream (if all Blocks are combined
* into one Stream).
*/
extern LZMA_API(lzma_vli) lzma_index_stream_size(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -441,6 +540,10 @@ extern LZMA_API(lzma_vli) lzma_index_stream_size(const lzma_index *i)
*
* This doesn't include the Stream Header, Stream Footer, Stream Padding,
* or Index fields.
*
* \param i Pointer to lzma_index structure
*
* \return Size in bytes of all Blocks in the Stream(s)
*/
extern LZMA_API(lzma_vli) lzma_index_total_size(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -453,6 +556,10 @@ extern LZMA_API(lzma_vli) lzma_index_total_size(const lzma_index *i)
* no Stream Padding, this function is identical to lzma_index_stream_size().
* If multiple lzma_indexes have been combined, this includes also the headers
* of each separate Stream and the possible Stream Padding fields.
*
* \param i Pointer to lzma_index structure
*
* \return Total size of the .xz file in bytes
*/
extern LZMA_API(lzma_vli) lzma_index_file_size(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -460,6 +567,10 @@ extern LZMA_API(lzma_vli) lzma_index_file_size(const lzma_index *i)
/**
* \brief Get the uncompressed size of the file
*
* \param i Pointer to lzma_index structure
*
* \return Size in bytes of the uncompressed data in the file
*/
extern LZMA_API(lzma_vli) lzma_index_uncompressed_size(const lzma_index *i)
lzma_nothrow lzma_attr_pure;
@ -468,9 +579,6 @@ extern LZMA_API(lzma_vli) lzma_index_uncompressed_size(const lzma_index *i)
/**
* \brief Initialize an iterator
*
* \param iter Pointer to a lzma_index_iter structure
* \param i lzma_index to which the iterator will be associated
*
* This function associates the iterator with the given lzma_index, and calls
* lzma_index_iter_rewind() on the iterator.
*
@ -483,6 +591,9 @@ extern LZMA_API(lzma_vli) lzma_index_uncompressed_size(const lzma_index *i)
*
* It is safe to make copies of an initialized lzma_index_iter, for example,
* to easily restart reading at some particular position.
*
* \param iter Pointer to a lzma_index_iter structure
* \param i lzma_index to which the iterator will be associated
*/
extern LZMA_API(void) lzma_index_iter_init(
lzma_index_iter *iter, const lzma_index *i) lzma_nothrow;
@ -493,6 +604,8 @@ extern LZMA_API(void) lzma_index_iter_init(
*
* Rewind the iterator so that next call to lzma_index_iter_next() will
* return the first Block or Stream.
*
* \param iter Pointer to a lzma_index_iter structure
*/
extern LZMA_API(void) lzma_index_iter_rewind(lzma_index_iter *iter)
lzma_nothrow;
@ -505,11 +618,11 @@ extern LZMA_API(void) lzma_index_iter_rewind(lzma_index_iter *iter)
* \param mode Specify what kind of information the caller wants
* to get. See lzma_index_iter_mode for details.
*
* \return If next Block or Stream matching the mode was found, *iter
* is updated and this function returns false. If no Block or
* Stream matching the mode is found, *iter is not modified
* and this function returns true. If mode is set to an unknown
* value, *iter is not modified and this function returns true.
* \return lzma_bool:
* - true if no Block or Stream matching the mode is found.
* *iter is not updated (failure).
* - false if the next Block or Stream matching the mode was
* found. *iter is updated (success).
*/
extern LZMA_API(lzma_bool) lzma_index_iter_next(
lzma_index_iter *iter, lzma_index_iter_mode mode)
@ -523,21 +636,26 @@ extern LZMA_API(lzma_bool) lzma_index_iter_next(
* the Index field(s) and use lzma_index_iter_locate() to do random-access
* reading with granularity of Block size.
*
* \param iter Iterator that was earlier initialized with
* lzma_index_iter_init().
* \param target Uncompressed target offset which the caller would
* like to locate from the Stream
*
* If the target is smaller than the uncompressed size of the Stream (can be
* checked with lzma_index_uncompressed_size()):
* - Information about the Stream and Block containing the requested
* uncompressed offset is stored into *iter.
* - Internal state of the iterator is adjusted so that
* lzma_index_iter_next() can be used to read subsequent Blocks or Streams.
* - This function returns false.
*
* If target is greater than the uncompressed size of the Stream, *iter
* is not modified, and this function returns true.
* If the target is greater than the uncompressed size of the Stream, *iter
* is not modified.
*
* \param iter Iterator that was earlier initialized with
* lzma_index_iter_init().
* \param target Uncompressed target offset which the caller would
* like to locate from the Stream
*
* \return lzma_bool:
* - true if the target is greater than or equal to the
* uncompressed size of the Stream (failure)
* - false if the target is smaller than the uncompressed size
* of the Stream (success)
*/
extern LZMA_API(lzma_bool) lzma_index_iter_locate(
lzma_index_iter *iter, lzma_vli target) lzma_nothrow;
@ -550,15 +668,16 @@ extern LZMA_API(lzma_bool) lzma_index_iter_locate(
* multi-Stream .xz file, or when combining multiple Streams into single
* Stream.
*
* \param dest lzma_index after which src is appended
* \param[out] dest lzma_index after which src is appended
* \param src lzma_index to be appended after dest. If this
* function succeeds, the memory allocated for src
* is freed or moved to be part of dest, and all
* iterators pointing to src will become invalid.
* \param allocator Custom memory allocator; can be NULL to use
* malloc() and free().
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return - LZMA_OK: lzma_indexes were concatenated successfully.
* \return Possible lzma_ret values:
* - LZMA_OK: lzma_indexes were concatenated successfully.
* src is now a dangling pointer.
* - LZMA_DATA_ERROR: *dest would grow too big.
* - LZMA_MEM_ERROR
@ -572,6 +691,10 @@ extern LZMA_API(lzma_ret) lzma_index_cat(lzma_index *dest, lzma_index *src,
/**
* \brief Duplicate lzma_index
*
* \param i Pointer to lzma_index structure to be duplicated
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return A copy of the lzma_index, or NULL if memory allocation failed.
*/
extern LZMA_API(lzma_index *) lzma_index_dup(
@ -585,10 +708,11 @@ extern LZMA_API(lzma_index *) lzma_index_dup(
* \param strm Pointer to properly prepared lzma_stream
* \param i Pointer to lzma_index which should be encoded.
*
* The valid `action' values for lzma_code() are LZMA_RUN and LZMA_FINISH.
* The valid 'action' values for lzma_code() are LZMA_RUN and LZMA_FINISH.
* It is enough to use only one of them (you can choose freely).
*
* \return - LZMA_OK: Initialization succeeded, continue with lzma_code().
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization succeeded, continue with lzma_code().
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*/
@ -601,7 +725,7 @@ extern LZMA_API(lzma_ret) lzma_index_encoder(
* \brief Initialize .xz Index decoder
*
* \param strm Pointer to properly prepared lzma_stream
* \param i The decoded Index will be made available via
* \param[out] i The decoded Index will be made available via
* this pointer. Initially this function will
* set *i to NULL (the old value is ignored). If
* decoding succeeds (lzma_code() returns
@ -613,15 +737,16 @@ extern LZMA_API(lzma_ret) lzma_index_encoder(
* don't allow 0 here and return LZMA_PROG_ERROR;
* later versions treat 0 as if 1 had been specified.
*
* Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* There is no need to use LZMA_FINISH, but it's allowed because it may
* simplify certain types of applications.
*
* \return - LZMA_OK: Initialization succeeded, continue with lzma_code().
* \return Possible lzma_ret values:
* - LZMA_OK: Initialization succeeded, continue with lzma_code().
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*
* liblzma 5.2.3 and older list also LZMA_MEMLIMIT_ERROR here
* \note liblzma 5.2.3 and older list also LZMA_MEMLIMIT_ERROR here
* but that error code has never been possible from this
* initialization function.
*/
@ -633,21 +758,23 @@ extern LZMA_API(lzma_ret) lzma_index_decoder(
/**
* \brief Single-call .xz Index encoder
*
* \note This function doesn't take allocator argument since all
* the internal data is allocated on stack.
*
* \param i lzma_index to be encoded
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* *out_pos is updated only if encoding succeeds.
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_BUF_ERROR: Output buffer is too small. Use
* lzma_index_size() to find out how much output
* space is needed.
* - LZMA_PROG_ERROR
*
* \note This function doesn't take allocator argument since all
* the internal data is allocated on stack.
*/
extern LZMA_API(lzma_ret) lzma_index_buffer_encode(const lzma_index *i,
uint8_t *out, size_t *out_pos, size_t out_size) lzma_nothrow;
@ -656,24 +783,26 @@ extern LZMA_API(lzma_ret) lzma_index_buffer_encode(const lzma_index *i,
/**
* \brief Single-call .xz Index decoder
*
* \param i If decoding succeeds, *i will point to a new
* \param[out] i If decoding succeeds, *i will point to a new
* lzma_index, which the application has to
* later free with lzma_index_end(). If an error
* occurs, *i will be NULL. The old value of *i
* is always ignored and thus doesn't need to be
* initialized by the caller.
* \param memlimit Pointer to how much memory the resulting
* \param[out] memlimit Pointer to how much memory the resulting
* lzma_index is allowed to require. The value
* pointed by this pointer is modified if and only
* if LZMA_MEMLIMIT_ERROR is returned.
* \param allocator Pointer to lzma_allocator, or NULL to use malloc()
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_pos The next byte will be read from in[*in_pos].
* *in_pos is updated only if decoding succeeds.
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
*
* \return - LZMA_OK: Decoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_MEM_ERROR
* - LZMA_MEMLIMIT_ERROR: Memory usage limit was reached.
* The minimum required memlimit value was stored to *memlimit.
@ -684,3 +813,70 @@ extern LZMA_API(lzma_ret) lzma_index_buffer_decode(lzma_index **i,
uint64_t *memlimit, const lzma_allocator *allocator,
const uint8_t *in, size_t *in_pos, size_t in_size)
lzma_nothrow;
/**
* \brief Initialize a .xz file information decoder
*
* This decoder decodes the Stream Header, Stream Footer, Index, and
* Stream Padding field(s) from the input .xz file and stores the resulting
* combined index in *dest_index. This information can be used to get the
* uncompressed file size with lzma_index_uncompressed_size(*dest_index) or,
* for example, to implement random access reading by locating the Blocks
* in the Streams.
*
* To get the required information from the .xz file, lzma_code() may ask
* the application to seek in the input file by returning LZMA_SEEK_NEEDED
* and having the target file position specified in lzma_stream.seek_pos.
* The number of seeks required depends on the input file and how big buffers
* the application provides. When possible, the decoder will seek backward
* and forward in the given buffer to avoid useless seek requests. Thus, if
* the application provides the whole file at once, no external seeking will
* be required (that is, lzma_code() won't return LZMA_SEEK_NEEDED).
*
* The value in lzma_stream.total_in can be used to estimate how much data
* liblzma had to read to get the file information. However, due to seeking
* and the way total_in is updated, the value of total_in will be somewhat
* inaccurate (a little too big). Thus, total_in is a good estimate but don't
* expect to see the same exact value for the same file if you change the
* input buffer size or switch to a different liblzma version.
*
* Valid 'action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
* You only need to use LZMA_RUN; LZMA_FINISH is only supported because it
* might be convenient for some applications. If you use LZMA_FINISH and if
* lzma_code() asks the application to seek, remember to reset 'action' back
* to LZMA_RUN unless you hit the end of the file again.
*
* Possible return values from lzma_code():
* - LZMA_OK: All OK so far, more input needed
* - LZMA_SEEK_NEEDED: Provide more input starting from the absolute
* file position strm->seek_pos
* - LZMA_STREAM_END: Decoding was successful, *dest_index has been set
* - LZMA_FORMAT_ERROR: The input file is not in the .xz format (the
* expected magic bytes were not found from the beginning of the file)
* - LZMA_OPTIONS_ERROR: File looks valid but contains headers that aren't
* supported by this version of liblzma
* - LZMA_DATA_ERROR: File is corrupt
* - LZMA_BUF_ERROR
* - LZMA_MEM_ERROR
* - LZMA_MEMLIMIT_ERROR
* - LZMA_PROG_ERROR
*
* \param strm Pointer to a properly prepared lzma_stream
* \param[out] dest_index Pointer to a pointer where the decoder will put
* the decoded lzma_index. The old value
* of *dest_index is ignored (not freed).
* \param memlimit How much memory the resulting lzma_index is
* allowed to require. Use UINT64_MAX to
* effectively disable the limiter.
* \param file_size Size of the input .xz file
*
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_MEM_ERROR
* - LZMA_PROG_ERROR
*/
extern LZMA_API(lzma_ret) lzma_file_info_decoder(
lzma_stream *strm, lzma_index **dest_index,
uint64_t memlimit, uint64_t file_size)
lzma_nothrow;

46
Externals/liblzma/api/lzma/index_hash.h vendored
View file

@ -1,6 +1,9 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/index_hash.h
* \brief Validate Index by using a hash function
* \note Never include this file directly. Use <lzma.h> instead.
*
* Hashing makes it possible to use constant amount of memory to validate
* Index of arbitrary size.
@ -8,11 +11,6 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -28,13 +26,21 @@ typedef struct lzma_index_hash_s lzma_index_hash;
/**
* \brief Allocate and initialize a new lzma_index_hash structure
*
* If index_hash is NULL, a new lzma_index_hash structure is allocated,
* initialized, and a pointer to it returned. If allocation fails, NULL
* is returned.
* If index_hash is NULL, this function allocates and initializes a new
* lzma_index_hash structure and returns a pointer to it. If allocation
* fails, NULL is returned.
*
* If index_hash is non-NULL, it is reinitialized and the same pointer
* returned. In this case, return value cannot be NULL or a different
* pointer than the index_hash that was given as an argument.
* If index_hash is non-NULL, this function reinitializes the lzma_index_hash
* structure and returns the same pointer. In this case, return value cannot
* be NULL or a different pointer than the index_hash that was given as
* an argument.
*
* \param index_hash Pointer to a lzma_index_hash structure or NULL.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return Initialized lzma_index_hash structure on success or
* NULL on failure.
*/
extern LZMA_API(lzma_index_hash *) lzma_index_hash_init(
lzma_index_hash *index_hash, const lzma_allocator *allocator)
@ -43,6 +49,10 @@ extern LZMA_API(lzma_index_hash *) lzma_index_hash_init(
/**
* \brief Deallocate lzma_index_hash structure
*
* \param index_hash Pointer to a lzma_index_hash structure to free.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*/
extern LZMA_API(void) lzma_index_hash_end(
lzma_index_hash *index_hash, const lzma_allocator *allocator)
@ -52,11 +62,12 @@ extern LZMA_API(void) lzma_index_hash_end(
/**
* \brief Add a new Record to an Index hash
*
* \param index Pointer to a lzma_index_hash structure
* \param index_hash Pointer to a lzma_index_hash structure
* \param unpadded_size Unpadded Size of a Block
* \param uncompressed_size Uncompressed Size of a Block
*
* \return - LZMA_OK
* \return Possible lzma_ret values:
* - LZMA_OK
* - LZMA_DATA_ERROR: Compressed or uncompressed size of the
* Stream or size of the Index field would grow too big.
* - LZMA_PROG_ERROR: Invalid arguments or this function is being
@ -81,10 +92,11 @@ extern LZMA_API(lzma_ret) lzma_index_hash_append(lzma_index_hash *index_hash,
*
* \param index_hash Pointer to a lzma_index_hash structure
* \param in Pointer to the beginning of the input buffer
* \param in_pos in[*in_pos] is the next byte to process
* \param[out] in_pos in[*in_pos] is the next byte to process
* \param in_size in[in_size] is the first byte not to process
*
* \return - LZMA_OK: So far good, but more input is needed.
* \return Possible lzma_ret values:
* - LZMA_OK: So far good, but more input is needed.
* - LZMA_STREAM_END: Index decoded successfully and it matches
* the Records given with lzma_index_hash_append().
* - LZMA_DATA_ERROR: Index is corrupt or doesn't match the
@ -101,6 +113,10 @@ extern LZMA_API(lzma_ret) lzma_index_hash_decode(lzma_index_hash *index_hash,
* \brief Get the size of the Index field as bytes
*
* This is needed to verify the Backward Size field in the Stream Footer.
*
* \param index_hash Pointer to a lzma_index_hash structure
*
* \return Size of the Index field in bytes.
*/
extern LZMA_API(lzma_vli) lzma_index_hash_size(
const lzma_index_hash *index_hash)

198
Externals/liblzma/api/lzma/lzma12.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/lzma12.h
* \brief LZMA1 and LZMA2 filters
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -18,23 +16,46 @@
/**
* \brief LZMA1 Filter ID
* \brief LZMA1 Filter ID (for raw encoder/decoder only, not in .xz)
*
* LZMA1 is the very same thing as what was called just LZMA in LZMA Utils,
* 7-Zip, and LZMA SDK. It's called LZMA1 here to prevent developers from
* accidentally using LZMA when they actually want LZMA2.
*
* LZMA1 shouldn't be used for new applications unless you _really_ know
* what you are doing. LZMA2 is almost always a better choice.
*/
#define LZMA_FILTER_LZMA1 LZMA_VLI_C(0x4000000000000001)
/**
* \brief LZMA1 Filter ID with extended options (for raw encoder/decoder)
*
* This is like LZMA_FILTER_LZMA1 but with this ID a few extra options
* are supported in the lzma_options_lzma structure:
*
* - A flag to tell the encoder if the end of payload marker (EOPM) alias
* end of stream (EOS) marker must be written at the end of the stream.
* In contrast, LZMA_FILTER_LZMA1 always writes the end marker.
*
* - Decoder needs to be told the uncompressed size of the stream
* or that it is unknown (using the special value UINT64_MAX).
* If the size is known, a flag can be set to allow the presence of
* the end marker anyway. In contrast, LZMA_FILTER_LZMA1 always
* behaves as if the uncompressed size was unknown.
*
* This allows handling file formats where LZMA1 streams are used but where
* the end marker isn't allowed or where it might not (always) be present.
* This extended LZMA1 functionality is provided as a Filter ID for raw
* encoder and decoder instead of adding new encoder and decoder initialization
* functions because this way it is possible to also use extra filters,
* for example, LZMA_FILTER_X86 in a filter chain with LZMA_FILTER_LZMA1EXT,
* which might be needed to handle some file formats.
*/
#define LZMA_FILTER_LZMA1EXT LZMA_VLI_C(0x4000000000000002)
/**
* \brief LZMA2 Filter ID
*
* Usually you want this instead of LZMA1. Compared to LZMA1, LZMA2 adds
* support for LZMA_SYNC_FLUSH, uncompressed chunks (smaller expansion
* when trying to compress uncompressible data), possibility to change
* when trying to compress incompressible data), possibility to change
* lc/lp/pb in the middle of encoding, and some other internal improvements.
*/
#define LZMA_FILTER_LZMA2 LZMA_VLI_C(0x21)
@ -114,16 +135,20 @@ typedef enum {
/**
* \brief Test if given match finder is supported
*
* Return true if the given match finder is supported by this liblzma build.
* Otherwise false is returned. It is safe to call this with a value that
* isn't listed in lzma_match_finder enumeration; the return value will be
* false.
* It is safe to call this with a value that isn't listed in
* lzma_match_finder enumeration; the return value will be false.
*
* There is no way to list which match finders are available in this
* particular liblzma version and build. It would be useless, because
* a new match finder, which the application developer wasn't aware,
* could require giving additional options to the encoder that the older
* match finders don't need.
*
* \param match_finder Match finder ID
*
* \return lzma_bool:
* - true if the match finder is supported by this liblzma build.
* - false otherwise.
*/
extern LZMA_API(lzma_bool) lzma_mf_is_supported(lzma_match_finder match_finder)
lzma_nothrow lzma_attr_const;
@ -158,14 +183,20 @@ typedef enum {
/**
* \brief Test if given compression mode is supported
*
* Return true if the given compression mode is supported by this liblzma
* build. Otherwise false is returned. It is safe to call this with a value
* that isn't listed in lzma_mode enumeration; the return value will be false.
* It is safe to call this with a value that isn't listed in lzma_mode
* enumeration; the return value will be false.
*
* There is no way to list which modes are available in this particular
* liblzma version and build. It would be useless, because a new compression
* mode, which the application developer wasn't aware, could require giving
* additional options to the encoder that the older modes don't need.
*
* \param mode Mode ID.
*
* \return lzma_bool:
* - true if the compression mode is supported by this liblzma
* build.
* - false otherwise.
*/
extern LZMA_API(lzma_bool) lzma_mode_is_supported(lzma_mode mode)
lzma_nothrow lzma_attr_const;
@ -257,7 +288,7 @@ typedef struct {
* \brief Number of literal context bits
*
* How many of the highest bits of the previous uncompressed
* eight-bit byte (also known as `literal') are taken into
* eight-bit byte (also known as 'literal') are taken into
* account when predicting the bits of the next literal.
*
* E.g. in typical English text, an upper-case letter is
@ -301,7 +332,7 @@ typedef struct {
* (2^ pb =2^2=4), which is often a good choice when there's
* no better guess.
*
* When the aligment is known, setting pb accordingly may reduce
* When the alignment is known, setting pb accordingly may reduce
* the file size a little. E.g. with text files having one-byte
* alignment (US-ASCII, ISO-8859-*, UTF-8), setting pb=0 can
* improve compression slightly. For UTF-16 text, pb=1 is a good
@ -374,6 +405,82 @@ typedef struct {
*/
uint32_t depth;
/**
* \brief For LZMA_FILTER_LZMA1EXT: Extended flags
*
* This is used only with LZMA_FILTER_LZMA1EXT.
*
* Currently only one flag is supported, LZMA_LZMA1EXT_ALLOW_EOPM:
*
* - Encoder: If the flag is set, then end marker is written just
* like it is with LZMA_FILTER_LZMA1. Without this flag the
* end marker isn't written and the application has to store
* the uncompressed size somewhere outside the compressed stream.
* To decompress streams without the end marker, the application
* has to set the correct uncompressed size in ext_size_low and
* ext_size_high.
*
* - Decoder: If the uncompressed size in ext_size_low and
* ext_size_high is set to the special value UINT64_MAX
* (indicating unknown uncompressed size) then this flag is
* ignored and the end marker must always be present, that is,
* the behavior is identical to LZMA_FILTER_LZMA1.
*
* Otherwise, if this flag isn't set, then the input stream
* must not have the end marker; if the end marker is detected
* then it will result in LZMA_DATA_ERROR. This is useful when
* it is known that the stream must not have the end marker and
* strict validation is wanted.
*
* If this flag is set, then it is autodetected if the end marker
* is present after the specified number of uncompressed bytes
* has been decompressed (ext_size_low and ext_size_high). The
* end marker isn't allowed in any other position. This behavior
* is useful when uncompressed size is known but the end marker
* may or may not be present. This is the case, for example,
* in .7z files (valid .7z files that have the end marker in
* LZMA1 streams are rare but they do exist).
*/
uint32_t ext_flags;
# define LZMA_LZMA1EXT_ALLOW_EOPM UINT32_C(0x01)
/**
* \brief For LZMA_FILTER_LZMA1EXT: Uncompressed size (low bits)
*
* The 64-bit uncompressed size is needed for decompression with
* LZMA_FILTER_LZMA1EXT. The size is ignored by the encoder.
*
* The special value UINT64_MAX indicates that the uncompressed size
* is unknown and that the end of payload marker (also known as
* end of stream marker) must be present to indicate the end of
* the LZMA1 stream. Any other value indicates the expected
* uncompressed size of the LZMA1 stream. (If LZMA1 was used together
* with filters that change the size of the data then the uncompressed
* size of the LZMA1 stream could be different than the final
* uncompressed size of the filtered stream.)
*
* ext_size_low holds the least significant 32 bits of the
* uncompressed size. The most significant 32 bits must be set
* in ext_size_high. The macro lzma_set_ext_size(opt_lzma, u64size)
* can be used to set these members.
*
* The 64-bit uncompressed size is split into two uint32_t variables
* because there were no reserved uint64_t members and using the
* same options structure for LZMA_FILTER_LZMA1, LZMA_FILTER_LZMA1EXT,
* and LZMA_FILTER_LZMA2 was otherwise more convenient than having
* a new options structure for LZMA_FILTER_LZMA1EXT. (Replacing two
* uint32_t members with one uint64_t changes the ABI on some systems
* as the alignment of this struct can increase from 4 bytes to 8.)
*/
uint32_t ext_size_low;
/**
* \brief For LZMA_FILTER_LZMA1EXT: Uncompressed size (high bits)
*
* This holds the most significant 32 bits of the uncompressed size.
*/
uint32_t ext_size_high;
/*
* Reserved space to allow possible future extensions without
* breaking the ABI. You should not touch these, because the names
@ -381,24 +488,56 @@ typedef struct {
* with the currently supported options, so it is safe to leave these
* uninitialized.
*/
uint32_t reserved_int1;
uint32_t reserved_int2;
uint32_t reserved_int3;
/** \private Reserved member. */
uint32_t reserved_int4;
/** \private Reserved member. */
uint32_t reserved_int5;
/** \private Reserved member. */
uint32_t reserved_int6;
/** \private Reserved member. */
uint32_t reserved_int7;
/** \private Reserved member. */
uint32_t reserved_int8;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum1;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum2;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum3;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum4;
/** \private Reserved member. */
void *reserved_ptr1;
/** \private Reserved member. */
void *reserved_ptr2;
} lzma_options_lzma;
/**
* \brief Macro to set the 64-bit uncompressed size in ext_size_*
*
* This might be convenient when decoding using LZMA_FILTER_LZMA1EXT.
* This isn't used with LZMA_FILTER_LZMA1 or LZMA_FILTER_LZMA2.
*/
#define lzma_set_ext_size(opt_lzma2, u64size) \
do { \
(opt_lzma2).ext_size_low = (uint32_t)(u64size); \
(opt_lzma2).ext_size_high = (uint32_t)((uint64_t)(u64size) >> 32); \
} while (0)
/**
* \brief Set a compression preset to lzma_options_lzma structure
*
@ -408,13 +547,22 @@ typedef struct {
* The flags are defined in container.h, because the flags are used also
* with lzma_easy_encoder().
*
* The preset values are subject to changes between liblzma versions.
* The preset levels are subject to changes between liblzma versions.
*
* This function is available only if LZMA1 or LZMA2 encoder has been enabled
* when building liblzma.
*
* \return On success, false is returned. If the preset is not
* supported, true is returned.
* If features (like certain match finders) have been disabled at build time,
* then the function may return success (false) even though the resulting
* LZMA1/LZMA2 options may not be usable for encoder initialization
* (LZMA_OPTIONS_ERROR).
*
* \param[out] options Pointer to LZMA1 or LZMA2 options to be filled
* \param preset Preset level bitwse-ORed with preset flags
*
* \return lzma_bool:
* - true if the preset is not supported (failure).
* - false otherwise (success).
*/
extern LZMA_API(lzma_bool) lzma_lzma_preset(
lzma_options_lzma *options, uint32_t preset) lzma_nothrow;

114
Externals/liblzma/api/lzma/stream_flags.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/stream_flags.h
* \brief .xz Stream Header and Stream Footer encoder and decoder
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -36,7 +34,7 @@ typedef struct {
*
* To prevent API and ABI breakages if new features are needed in
* Stream Header or Stream Footer, a version number is used to
* indicate which fields in this structure are in use. For now,
* indicate which members in this structure are in use. For now,
* version must always be zero. With non-zero version, the
* lzma_stream_header_encode() and lzma_stream_footer_encode()
* will return LZMA_OPTIONS_ERROR.
@ -67,7 +65,15 @@ typedef struct {
* Footer have been decoded.
*/
lzma_vli backward_size;
/**
* \brief Minimum value for lzma_stream_flags.backward_size
*/
# define LZMA_BACKWARD_SIZE_MIN 4
/**
* \brief Maximum value for lzma_stream_flags.backward_size
*/
# define LZMA_BACKWARD_SIZE_MAX (LZMA_VLI_C(1) << 34)
/**
@ -87,19 +93,47 @@ typedef struct {
* is just two bytes plus Backward Size of four bytes. But it's
* nice to have the proper types when they are needed.)
*/
/** \private Reserved member. */
lzma_reserved_enum reserved_enum1;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum2;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum3;
/** \private Reserved member. */
lzma_reserved_enum reserved_enum4;
/** \private Reserved member. */
lzma_bool reserved_bool1;
/** \private Reserved member. */
lzma_bool reserved_bool2;
/** \private Reserved member. */
lzma_bool reserved_bool3;
/** \private Reserved member. */
lzma_bool reserved_bool4;
/** \private Reserved member. */
lzma_bool reserved_bool5;
/** \private Reserved member. */
lzma_bool reserved_bool6;
/** \private Reserved member. */
lzma_bool reserved_bool7;
/** \private Reserved member. */
lzma_bool reserved_bool8;
/** \private Reserved member. */
uint32_t reserved_int1;
/** \private Reserved member. */
uint32_t reserved_int2;
} lzma_stream_flags;
@ -111,10 +145,11 @@ typedef struct {
* \param options Stream Header options to be encoded.
* options->backward_size is ignored and doesn't
* need to be initialized.
* \param out Beginning of the output buffer of
* \param[out] out Beginning of the output buffer of
* LZMA_STREAM_HEADER_SIZE bytes.
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_OPTIONS_ERROR: options->version is not supported by
* this liblzma version.
* - LZMA_PROG_ERROR: Invalid options.
@ -128,10 +163,11 @@ extern LZMA_API(lzma_ret) lzma_stream_header_encode(
* \brief Encode Stream Footer
*
* \param options Stream Footer options to be encoded.
* \param out Beginning of the output buffer of
* \param[out] out Beginning of the output buffer of
* LZMA_STREAM_HEADER_SIZE bytes.
*
* \return - LZMA_OK: Encoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Encoding was successful.
* - LZMA_OPTIONS_ERROR: options->version is not supported by
* this liblzma version.
* - LZMA_PROG_ERROR: Invalid options.
@ -144,32 +180,33 @@ extern LZMA_API(lzma_ret) lzma_stream_footer_encode(
/**
* \brief Decode Stream Header
*
* \param options Target for the decoded Stream Header options.
* \param in Beginning of the input buffer of
* LZMA_STREAM_HEADER_SIZE bytes.
*
* options->backward_size is always set to LZMA_VLI_UNKNOWN. This is to
* help comparing Stream Flags from Stream Header and Stream Footer with
* lzma_stream_flags_compare().
*
* \return - LZMA_OK: Decoding was successful.
* \note When decoding .xz files that contain multiple Streams, it may
* make sense to print "file format not recognized" only if
* decoding of the Stream Header of the \a first Stream gives
* LZMA_FORMAT_ERROR. If non-first Stream Header gives
* LZMA_FORMAT_ERROR, the message used for LZMA_DATA_ERROR is
* probably more appropriate.
* For example, the Stream decoder in liblzma uses
* LZMA_DATA_ERROR if LZMA_FORMAT_ERROR is returned by
* lzma_stream_header_decode() when decoding non-first Stream.
*
* \param[out] options Target for the decoded Stream Header options.
* \param in Beginning of the input buffer of
* LZMA_STREAM_HEADER_SIZE bytes.
*
*
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_FORMAT_ERROR: Magic bytes don't match, thus the given
* buffer cannot be Stream Header.
* - LZMA_DATA_ERROR: CRC32 doesn't match, thus the header
* is corrupt.
* - LZMA_OPTIONS_ERROR: Unsupported options are present
* in the header.
*
* \note When decoding .xz files that contain multiple Streams, it may
* make sense to print "file format not recognized" only if
* decoding of the Stream Header of the _first_ Stream gives
* LZMA_FORMAT_ERROR. If non-first Stream Header gives
* LZMA_FORMAT_ERROR, the message used for LZMA_DATA_ERROR is
* probably more appropriate.
*
* For example, Stream decoder in liblzma uses LZMA_DATA_ERROR if
* LZMA_FORMAT_ERROR is returned by lzma_stream_header_decode()
* when decoding non-first Stream.
*/
extern LZMA_API(lzma_ret) lzma_stream_header_decode(
lzma_stream_flags *options, const uint8_t *in)
@ -179,24 +216,25 @@ extern LZMA_API(lzma_ret) lzma_stream_header_decode(
/**
* \brief Decode Stream Footer
*
* \param options Target for the decoded Stream Header options.
* \note If Stream Header was already decoded successfully, but
* decoding Stream Footer returns LZMA_FORMAT_ERROR, the
* application should probably report some other error message
* than "file format not recognized". The file likely
* is corrupt (possibly truncated). The Stream decoder in liblzma
* uses LZMA_DATA_ERROR in this situation.
*
* \param[out] options Target for the decoded Stream Footer options.
* \param in Beginning of the input buffer of
* LZMA_STREAM_HEADER_SIZE bytes.
*
* \return - LZMA_OK: Decoding was successful.
* \return Possible lzma_ret values:
* - LZMA_OK: Decoding was successful.
* - LZMA_FORMAT_ERROR: Magic bytes don't match, thus the given
* buffer cannot be Stream Footer.
* - LZMA_DATA_ERROR: CRC32 doesn't match, thus the Stream Footer
* is corrupt.
* - LZMA_OPTIONS_ERROR: Unsupported options are present
* in Stream Footer.
*
* \note If Stream Header was already decoded successfully, but
* decoding Stream Footer returns LZMA_FORMAT_ERROR, the
* application should probably report some other error message
* than "file format not recognized", since the file more likely
* is corrupt (possibly truncated). Stream decoder in liblzma
* uses LZMA_DATA_ERROR in this situation.
*/
extern LZMA_API(lzma_ret) lzma_stream_footer_decode(
lzma_stream_flags *options, const uint8_t *in)
@ -209,7 +247,11 @@ extern LZMA_API(lzma_ret) lzma_stream_footer_decode(
* backward_size values are compared only if both are not
* LZMA_VLI_UNKNOWN.
*
* \return - LZMA_OK: Both are equal. If either had backward_size set
* \param a Pointer to lzma_stream_flags structure
* \param b Pointer to lzma_stream_flags structure
*
* \return Possible lzma_ret values:
* - LZMA_OK: Both are equal. If either had backward_size set
* to LZMA_VLI_UNKNOWN, backward_size values were not
* compared or validated.
* - LZMA_DATA_ERROR: The structures differ.

41
Externals/liblzma/api/lzma/version.h vendored
View file

@ -1,15 +1,13 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/version.h
* \brief Version number
* \note Never include this file directly. Use <lzma.h> instead.
*/
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -17,14 +15,26 @@
#endif
/*
* Version number split into components
*/
/** \brief Major version number of the liblzma release. */
#define LZMA_VERSION_MAJOR 5
#define LZMA_VERSION_MINOR 2
/** \brief Minor version number of the liblzma release. */
#define LZMA_VERSION_MINOR 6
/** \brief Patch version number of the liblzma release. */
#define LZMA_VERSION_PATCH 4
/**
* \brief Version stability marker
*
* This will always be one of three values:
* - LZMA_VERSION_STABILITY_ALPHA
* - LZMA_VERSION_STABILITY_BETA
* - LZMA_VERSION_STABILITY_STABLE
*/
#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE
/** \brief Commit version number of the liblzma release */
#ifndef LZMA_VERSION_COMMIT
# define LZMA_VERSION_COMMIT ""
#endif
@ -95,15 +105,16 @@
LZMA_VERSION_COMMIT)
/* #ifndef is needed for use with windres (MinGW or Cygwin). */
/* #ifndef is needed for use with windres (MinGW-w64 or Cygwin). */
#ifndef LZMA_H_INTERNAL_RC
/**
* \brief Run-time version number as an integer
*
* Return the value of LZMA_VERSION macro at the compile time of liblzma.
* This allows the application to compare if it was built against the same,
* This allows an application to compare if it was built against the same,
* older, or newer version of liblzma that is currently running.
*
* \return The value of LZMA_VERSION macro at the compile time of liblzma
*/
extern LZMA_API(uint32_t) lzma_version_number(void)
lzma_nothrow lzma_attr_const;
@ -112,8 +123,10 @@ extern LZMA_API(uint32_t) lzma_version_number(void)
/**
* \brief Run-time version as a string
*
* This function may be useful if you want to display which version of
* liblzma your application is currently using.
* This function may be useful to display which version of liblzma an
* application is currently using.
*
* \return Run-time version of liblzma
*/
extern LZMA_API(const char *) lzma_version_string(void)
lzma_nothrow lzma_attr_const;

28
Externals/liblzma/api/lzma/vli.h vendored
View file

@ -1,6 +1,9 @@
/* SPDX-License-Identifier: 0BSD */
/**
* \file lzma/vli.h
* \brief Variable-length integer handling
* \note Never include this file directly. Use <lzma.h> instead.
*
* In the .xz format, most integers are encoded in a variable-length
* representation, which is sometimes called little endian base-128 encoding.
@ -16,11 +19,6 @@
/*
* Author: Lasse Collin
*
* This file has been put into the public domain.
* You can do whatever you want with this file.
*
* See ../lzma.h for information about liblzma as a whole.
*/
#ifndef LZMA_H_INTERNAL
@ -54,7 +52,7 @@
*
* Valid VLI values are in the range [0, LZMA_VLI_MAX]. Unknown value is
* indicated with LZMA_VLI_UNKNOWN, which is the maximum value of the
* underlaying integer type.
* underlying integer type.
*
* lzma_vli will be uint64_t for the foreseeable future. If a bigger size
* is needed in the future, it is guaranteed that 2 * LZMA_VLI_MAX will
@ -69,8 +67,8 @@ typedef uint64_t lzma_vli;
* This is useful to test that application has given acceptable values
* for example in the uncompressed_size and compressed_size variables.
*
* \return True if the integer is representable as VLI or if it
* indicates unknown value.
* \return True if the integer is representable as a VLI or if it
* indicates an unknown value. False otherwise.
*/
#define lzma_vli_is_valid(vli) \
((vli) <= LZMA_VLI_MAX || (vli) == LZMA_VLI_UNKNOWN)
@ -86,12 +84,12 @@ typedef uint64_t lzma_vli;
* integer has been encoded.
*
* \param vli Integer to be encoded
* \param vli_pos How many VLI-encoded bytes have already been written
* \param[out] vli_pos How many VLI-encoded bytes have already been written
* out. When starting to encode a new integer in
* multi-call mode, *vli_pos must be set to zero.
* To use single-call encoding, set vli_pos to NULL.
* \param out Beginning of the output buffer
* \param out_pos The next byte will be written to out[*out_pos].
* \param[out] out Beginning of the output buffer
* \param[out] out_pos The next byte will be written to out[*out_pos].
* \param out_size Size of the out buffer; the first byte into
* which no data is written to is out[out_size].
*
@ -121,15 +119,15 @@ extern LZMA_API(lzma_ret) lzma_vli_encode(lzma_vli vli, size_t *vli_pos,
*
* Like lzma_vli_encode(), this function has single-call and multi-call modes.
*
* \param vli Pointer to decoded integer. The decoder will
* \param[out] vli Pointer to decoded integer. The decoder will
* initialize it to zero when *vli_pos == 0, so
* application isn't required to initialize *vli.
* \param vli_pos How many bytes have already been decoded. When
* \param[out] vli_pos How many bytes have already been decoded. When
* starting to decode a new integer in multi-call
* mode, *vli_pos must be initialized to zero. To
* use single-call decoding, set vli_pos to NULL.
* \param in Beginning of the input buffer
* \param in_pos The next byte will be read from in[*in_pos].
* \param[out] in_pos The next byte will be read from in[*in_pos].
* \param in_size Size of the input buffer; the first byte that
* won't be read is in[in_size].
*
@ -159,6 +157,8 @@ extern LZMA_API(lzma_ret) lzma_vli_decode(lzma_vli *vli, size_t *vli_pos,
/**
* \brief Get the number of bytes required to encode a VLI
*
* \param vli Integer whose encoded size is to be determined
*
* \return Number of bytes on success (1-9). If vli isn't valid,
* zero is returned.
*/

5
Externals/liblzma/check/check.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file check.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"

16
Externals/liblzma/check/check.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file check.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CHECK_H
@ -99,19 +98,22 @@ typedef struct {
/// lzma_crc32_table[0] is needed by LZ encoder so we need to keep
/// the array two-dimensional.
#ifdef HAVE_SMALL
lzma_attr_visibility_hidden
extern uint32_t lzma_crc32_table[1][256];
extern void lzma_crc32_init(void);
#else
lzma_attr_visibility_hidden
extern const uint32_t lzma_crc32_table[8][256];
lzma_attr_visibility_hidden
extern const uint64_t lzma_crc64_table[4][256];
#endif
/// \brief Initialize *check depending on type
///
/// \return LZMA_OK on success. LZMA_UNSUPPORTED_CHECK if the type is not
/// supported by the current version or build of liblzma.
/// LZMA_PROG_ERROR if type > LZMA_CHECK_ID_MAX.
extern void lzma_check_init(lzma_check_state *check, lzma_check type);
/// Update the check state

122
Externals/liblzma/check/crc32_arm64.h vendored Normal file
View file

@ -0,0 +1,122 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_arm64.h
/// \brief CRC32 calculation with ARM64 optimization
//
// Authors: Chenxi Mao
// Jia Tan
// Hans Jansen
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CRC32_ARM64_H
#define LZMA_CRC32_ARM64_H
// MSVC always has the CRC intrinsics available when building for ARM64
// there is no need to include any header files.
#ifndef _MSC_VER
# include <arm_acle.h>
#endif
// If both versions are going to be built, we need runtime detection
// to check if the instructions are supported.
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
# if defined(HAVE_GETAUXVAL) || defined(HAVE_ELF_AUX_INFO)
# include <sys/auxv.h>
# elif defined(_WIN32)
# include <processthreadsapi.h>
# elif defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME)
# include <sys/sysctl.h>
# endif
#endif
// Some EDG-based compilers support ARM64 and define __GNUC__
// (such as Nvidia's nvcc), but do not support function attributes.
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
# define crc_attr_target __attribute__((__target__("+crc")))
#else
# define crc_attr_target
#endif
crc_attr_target
static uint32_t
crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
{
crc = ~crc;
// Align the input buffer because this was shown to be
// significantly faster than unaligned accesses.
const size_t align_amount = my_min(size, (0U - (uintptr_t)buf) & 7);
for (const uint8_t *limit = buf + align_amount; buf < limit; ++buf)
crc = __crc32b(crc, *buf);
size -= align_amount;
// Process 8 bytes at a time. The end point is determined by
// ignoring the least significant three bits of size to ensure
// we do not process past the bounds of the buffer. This guarantees
// that limit is a multiple of 8 and is strictly less than size.
for (const uint8_t *limit = buf + (size & ~(size_t)7);
buf < limit; buf += 8)
crc = __crc32d(crc, aligned_read64le(buf));
// Process the remaining bytes that are not 8 byte aligned.
for (const uint8_t *limit = buf + (size & 7); buf < limit; ++buf)
crc = __crc32b(crc, *buf);
return ~crc;
}
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
static inline bool
is_arch_extension_supported(void)
{
#if defined(HAVE_GETAUXVAL)
return (getauxval(AT_HWCAP) & HWCAP_CRC32) != 0;
#elif defined(HAVE_ELF_AUX_INFO)
unsigned long feature_flags;
if (elf_aux_info(AT_HWCAP, &feature_flags, sizeof(feature_flags)) != 0)
return false;
return (feature_flags & HWCAP_CRC32) != 0;
#elif defined(_WIN32)
return IsProcessorFeaturePresent(
PF_ARM_V8_CRC32_INSTRUCTIONS_AVAILABLE);
#elif defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME)
int has_crc32 = 0;
size_t size = sizeof(has_crc32);
// The sysctlbyname() function requires a string identifier for the
// CPU feature it tests. The Apple documentation lists the string
// "hw.optional.armv8_crc32", which can be found here:
// https://developer.apple.com/documentation/kernel/1387446-sysctlbyname/determining_instruction_set_characteristics#3915619
if (sysctlbyname("hw.optional.armv8_crc32", &has_crc32,
&size, NULL, 0) != 0)
return false;
return has_crc32;
#else
// If a runtime detection method cannot be found, then this must
// be a compile time error. The checks in crc_common.h should ensure
// a runtime detection method is always found if this function is
// built. It would be possible to just return false here, but this
// is inefficient for binary size and runtime since only the generic
// method could ever be used.
# error Runtime detection method unavailable.
#endif
}
#endif
#endif // LZMA_CRC32_ARM64_H

162
Externals/liblzma/check/crc32_fast.c vendored
View file

@ -1,35 +1,40 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32.c
/// \brief CRC32 calculation
///
/// Calculate the CRC32 using the slice-by-eight algorithm.
/// It is explained in this document:
/// http://www.intel.com/technology/comms/perfnet/download/CRC_generators.pdf
/// The code in this file is not the same as in Intel's paper, but
/// the basic principle is identical.
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
// Authors: Lasse Collin
// Ilya Kurdyukov
// Hans Jansen
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
#include "crc_macros.h"
#include "crc_common.h"
#if defined(CRC_X86_CLMUL)
# define BUILDING_CRC32_CLMUL
# include "crc_x86_clmul.h"
#elif defined(CRC32_ARM64)
# include "crc32_arm64.h"
#endif
// If you make any changes, do some benchmarking! Seemingly unrelated
// changes can very easily ruin the performance (and very probably is
// very compiler dependent).
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
#ifdef CRC32_GENERIC
///////////////////
// Generic CRC32 //
///////////////////
static uint32_t
crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
{
crc = ~crc;
#ifdef WORDS_BIGENDIAN
crc = bswap32(crc);
crc = byteswap32(crc);
#endif
if (size > 8) {
@ -49,7 +54,7 @@ lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
// Calculate the CRC32 using the slice-by-eight algorithm.
while (buf < limit) {
crc ^= *(const uint32_t *)(buf);
crc ^= aligned_read32ne(buf);
buf += 4;
crc = lzma_crc32_table[7][A(crc)]
@ -57,7 +62,7 @@ lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
^ lzma_crc32_table[5][C(crc)]
^ lzma_crc32_table[4][D(crc)];
const uint32_t tmp = *(const uint32_t *)(buf);
const uint32_t tmp = aligned_read32ne(buf);
buf += 4;
// At least with some compilers, it is critical for
@ -75,8 +80,125 @@ lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
crc = lzma_crc32_table[0][*buf++ ^ A(crc)] ^ S8(crc);
#ifdef WORDS_BIGENDIAN
crc = bswap32(crc);
crc = byteswap32(crc);
#endif
return ~crc;
}
#endif
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
//////////////////////////
// Function dispatching //
//////////////////////////
// If both the generic and arch-optimized implementations are built, then
// the function to use is selected at runtime because the system running
// the binary might not have the arch-specific instruction set extension(s)
// available. The dispatch methods in order of priority:
//
// 1. Constructor. This method uses __attribute__((__constructor__)) to
// set crc32_func at load time. This avoids extra computation (and any
// unlikely threading bugs) on the first call to lzma_crc32() to decide
// which implementation should be used.
//
// 2. First Call Resolution. On the very first call to lzma_crc32(), the
// call will be directed to crc32_dispatch() instead. This will set the
// appropriate implementation function and will not be called again.
// This method does not use any kind of locking but is safe because if
// multiple threads run the dispatcher simultaneously then they will all
// set crc32_func to the same value.
typedef uint32_t (*crc32_func_type)(
const uint8_t *buf, size_t size, uint32_t crc);
// This resolver is shared between all dispatch methods.
static crc32_func_type
crc32_resolve(void)
{
return is_arch_extension_supported()
? &crc32_arch_optimized : &crc32_generic;
}
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// Constructor method.
# define CRC32_SET_FUNC_ATTR __attribute__((__constructor__))
static crc32_func_type crc32_func;
#else
// First Call Resolution method.
# define CRC32_SET_FUNC_ATTR
static uint32_t crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc);
static crc32_func_type crc32_func = &crc32_dispatch;
#endif
CRC32_SET_FUNC_ATTR
static void
crc32_set_func(void)
{
crc32_func = crc32_resolve();
return;
}
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
static uint32_t
crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc)
{
// When __attribute__((__constructor__)) isn't supported, set the
// function pointer without any locking. If multiple threads run
// the detection code in parallel, they will all end up setting
// the pointer to the same value. This avoids the use of
// mythread_once() on every call to lzma_crc32() but this likely
// isn't strictly standards compliant. Let's change it if it breaks.
crc32_set_func();
return crc32_func(buf, size, crc);
}
#endif
#endif
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
{
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
// On x86-64, if CLMUL is available, it is the best for non-tiny
// inputs, being over twice as fast as the generic slice-by-four
// version. However, for size <= 16 it's different. In the extreme
// case of size == 1 the generic version can be five times faster.
// At size >= 8 the CLMUL starts to become reasonable. It
// varies depending on the alignment of buf too.
//
// The above doesn't include the overhead of mythread_once().
// At least on x86-64 GNU/Linux, pthread_once() is very fast but
// it still makes lzma_crc32(buf, 1, crc) 50-100 % slower. When
// size reaches 12-16 bytes the overhead becomes negligible.
//
// So using the generic version for size <= 16 may give better
// performance with tiny inputs but if such inputs happen rarely
// it's not so obvious because then the lookup table of the
// generic version may not be in the processor cache.
#ifdef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16)
return crc32_generic(buf, size, crc);
#endif
/*
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// See crc32_dispatch(). This would be the alternative which uses
// locking and doesn't use crc32_dispatch(). Note that on Windows
// this method needs Vista threads.
mythread_once(crc64_set_func);
#endif
*/
return crc32_func(buf, size, crc);
#elif defined(CRC32_ARCH_OPTIMIZED)
return crc32_arch_optimized(buf, size, crc);
#else
return crc32_generic(buf, size, crc);
#endif
}

67
Externals/liblzma/check/crc32_small.c vendored Normal file
View file

@ -0,0 +1,67 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_small.c
/// \brief CRC32 calculation (size-optimized)
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
uint32_t lzma_crc32_table[1][256];
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
__attribute__((__constructor__))
#endif
static void
crc32_init(void)
{
static const uint32_t poly32 = UINT32_C(0xEDB88320);
for (size_t b = 0; b < 256; ++b) {
uint32_t r = b;
for (size_t i = 0; i < 8; ++i) {
if (r & 1)
r = (r >> 1) ^ poly32;
else
r >>= 1;
}
lzma_crc32_table[0][b] = r;
}
return;
}
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
extern void
lzma_crc32_init(void)
{
mythread_once(crc32_init);
return;
}
#endif
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
{
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
lzma_crc32_init();
#endif
crc = ~crc;
while (size != 0) {
crc = lzma_crc32_table[0][*buf++ ^ (crc & 0xFF)] ^ (crc >> 8);
--size;
}
return ~crc;
}

29
Externals/liblzma/check/crc32_table.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_table.c
@ -5,15 +7,36 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
// FIXME: Compared to crc_common.h this has to check for __x86_64__ too
// so that in 32-bit builds crc32_x86.S won't break due to a missing table.
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define NO_CRC32_TABLE
#elif defined(HAVE_ARM64_CRC32) \
&& !defined(WORDS_BIGENDIAN) \
&& defined(__ARM_FEATURE_CRC32)
# define NO_CRC32_TABLE
#endif
#if !defined(HAVE_ENCODERS) && defined(NO_CRC32_TABLE)
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc32_dummy;
#else
// Having the declaration here silences clang -Wmissing-variable-declarations.
extern const uint32_t lzma_crc32_table[8][256];
# ifdef WORDS_BIGENDIAN
# include "crc32_table_be.h"
# else
# include "crc32_table_le.h"
# endif
#endif

4
Externals/liblzma/check/crc32_table_be.h vendored
View file

@ -1,4 +1,6 @@
/* This file has been automatically generated by crc32_tablegen.c. */
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc32_tablegen.c.
const uint32_t lzma_crc32_table[8][256] = {
{

4
Externals/liblzma/check/crc32_table_le.h vendored
View file

@ -1,4 +1,6 @@
/* This file has been automatically generated by crc32_tablegen.c. */
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc32_tablegen.c.
const uint32_t lzma_crc32_table[8][256] = {
{

21
Externals/liblzma/check/crc32_tablegen.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32_tablegen.c
@ -9,13 +11,10 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#include "tuklib_integer.h"
#include "../../common/tuklib_integer.h"
static uint32_t crc32_table[8][256];
@ -44,7 +43,7 @@ init_crc32_table(void)
#ifdef WORDS_BIGENDIAN
for (size_t s = 0; s < 8; ++s)
for (size_t b = 0; b < 256; ++b)
crc32_table[s][b] = bswap32(crc32_table[s][b]);
crc32_table[s][b] = byteswap32(crc32_table[s][b]);
#endif
return;
@ -54,8 +53,10 @@ init_crc32_table(void)
static void
print_crc32_table(void)
{
printf("/* This file has been automatically generated by "
"crc32_tablegen.c. */\n\n"
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc32_tablegen.c.\n\n"
"const uint32_t lzma_crc32_table[8][256] = {\n\t{");
for (size_t s = 0; s < 8; ++s) {
@ -82,8 +83,10 @@ print_crc32_table(void)
static void
print_lz_table(void)
{
printf("/* This file has been automatically generated by "
"crc32_tablegen.c. */\n\n"
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc32_tablegen.c.\n\n"
"const uint32_t lzma_lz_hash_table[256] = {");
for (size_t b = 0; b < 256; ++b) {

116
Externals/liblzma/check/crc64_fast.c vendored
View file

@ -1,22 +1,29 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64.c
/// \brief CRC64 calculation
///
/// Calculate the CRC64 using the slice-by-four algorithm. This is the same
/// idea that is used in crc32_fast.c, but for CRC64 we use only four tables
/// instead of eight to avoid increasing CPU cache usage.
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
// Authors: Lasse Collin
// Ilya Kurdyukov
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
#include "crc_macros.h"
#include "crc_common.h"
#if defined(CRC_X86_CLMUL)
# define BUILDING_CRC64_CLMUL
# include "crc_x86_clmul.h"
#endif
#ifdef CRC64_GENERIC
/////////////////////////////////
// Generic slice-by-four CRC64 //
/////////////////////////////////
#ifdef WORDS_BIGENDIAN
# define A1(x) ((x) >> 56)
@ -26,13 +33,13 @@
// See the comments in crc32_fast.c. They aren't duplicated here.
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
static uint64_t
crc64_generic(const uint8_t *buf, size_t size, uint64_t crc)
{
crc = ~crc;
#ifdef WORDS_BIGENDIAN
crc = bswap64(crc);
crc = byteswap64(crc);
#endif
if (size > 4) {
@ -46,10 +53,11 @@ lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
while (buf < limit) {
#ifdef WORDS_BIGENDIAN
const uint32_t tmp = (crc >> 32)
^ *(const uint32_t *)(buf);
const uint32_t tmp = (uint32_t)(crc >> 32)
^ aligned_read32ne(buf);
#else
const uint32_t tmp = crc ^ *(const uint32_t *)(buf);
const uint32_t tmp = (uint32_t)crc
^ aligned_read32ne(buf);
#endif
buf += 4;
@ -65,8 +73,84 @@ lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
crc = lzma_crc64_table[0][*buf++ ^ A1(crc)] ^ S8(crc);
#ifdef WORDS_BIGENDIAN
crc = bswap64(crc);
crc = byteswap64(crc);
#endif
return ~crc;
}
#endif
#if defined(CRC64_GENERIC) && defined(CRC64_ARCH_OPTIMIZED)
//////////////////////////
// Function dispatching //
//////////////////////////
// If both the generic and arch-optimized implementations are usable, then
// the function that is used is selected at runtime. See crc32_fast.c.
typedef uint64_t (*crc64_func_type)(
const uint8_t *buf, size_t size, uint64_t crc);
static crc64_func_type
crc64_resolve(void)
{
return is_arch_extension_supported()
? &crc64_arch_optimized : &crc64_generic;
}
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
# define CRC64_SET_FUNC_ATTR __attribute__((__constructor__))
static crc64_func_type crc64_func;
#else
# define CRC64_SET_FUNC_ATTR
static uint64_t crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc);
static crc64_func_type crc64_func = &crc64_dispatch;
#endif
CRC64_SET_FUNC_ATTR
static void
crc64_set_func(void)
{
crc64_func = crc64_resolve();
return;
}
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
static uint64_t
crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc)
{
crc64_set_func();
return crc64_func(buf, size, crc);
}
#endif
#endif
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
{
#if defined(CRC64_GENERIC) && defined(CRC64_ARCH_OPTIMIZED)
#ifdef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16)
return crc64_generic(buf, size, crc);
#endif
return crc64_func(buf, size, crc);
#elif defined(CRC64_ARCH_OPTIMIZED)
// If arch-optimized version is used unconditionally without runtime
// CPU detection then omitting the generic version and its 8 KiB
// lookup table makes the library smaller.
//
// FIXME: Lookup table isn't currently omitted on 32-bit x86,
// see crc64_table.c.
return crc64_arch_optimized(buf, size, crc);
#else
return crc64_generic(buf, size, crc);
#endif
}

57
Externals/liblzma/check/crc64_small.c vendored Normal file
View file

@ -0,0 +1,57 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_small.c
/// \brief CRC64 calculation (size-optimized)
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"
static uint64_t crc64_table[256];
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
__attribute__((__constructor__))
#endif
static void
crc64_init(void)
{
static const uint64_t poly64 = UINT64_C(0xC96C5795D7870F42);
for (size_t b = 0; b < 256; ++b) {
uint64_t r = b;
for (size_t i = 0; i < 8; ++i) {
if (r & 1)
r = (r >> 1) ^ poly64;
else
r >>= 1;
}
crc64_table[b] = r;
}
return;
}
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
{
#ifndef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
mythread_once(crc64_init);
#endif
crc = ~crc;
while (size != 0) {
crc = crc64_table[*buf++ ^ (crc & 0xFF)] ^ (crc >> 8);
--size;
}
return ~crc;
}

26
Externals/liblzma/check/crc64_table.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_table.c
@ -5,15 +7,31 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
#ifdef WORDS_BIGENDIAN
// FIXME: Compared to crc_common.h this has to check for __x86_64__ too
// so that in 32-bit builds crc64_x86.S won't break due to a missing table.
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define NO_CRC64_TABLE
#endif
#ifdef NO_CRC64_TABLE
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc64_dummy;
#else
// Having the declaration here silences clang -Wmissing-variable-declarations.
extern const uint64_t lzma_crc64_table[4][256];
# if defined(WORDS_BIGENDIAN)
# include "crc64_table_be.h"
# else
# include "crc64_table_le.h"
# endif
#endif

4
Externals/liblzma/check/crc64_table_be.h vendored
View file

@ -1,4 +1,6 @@
/* This file has been automatically generated by crc64_tablegen.c. */
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc64_tablegen.c.
const uint64_t lzma_crc64_table[4][256] = {
{

4
Externals/liblzma/check/crc64_table_le.h vendored
View file

@ -1,4 +1,6 @@
/* This file has been automatically generated by crc64_tablegen.c. */
// SPDX-License-Identifier: 0BSD
// This file has been generated by crc64_tablegen.c.
const uint64_t lzma_crc64_table[4][256] = {
{

15
Externals/liblzma/check/crc64_tablegen.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64_tablegen.c
@ -8,13 +10,10 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#include "tuklib_integer.h"
#include "../../common/tuklib_integer.h"
static uint64_t crc64_table[4][256];
@ -43,7 +42,7 @@ init_crc64_table(void)
#ifdef WORDS_BIGENDIAN
for (size_t s = 0; s < 4; ++s)
for (size_t b = 0; b < 256; ++b)
crc64_table[s][b] = bswap64(crc64_table[s][b]);
crc64_table[s][b] = byteswap64(crc64_table[s][b]);
#endif
return;
@ -53,8 +52,10 @@ init_crc64_table(void)
static void
print_crc64_table(void)
{
printf("/* This file has been automatically generated by "
"crc64_tablegen.c. */\n\n"
// Split the SPDX string so that it won't accidentally match
// when tools search for the string.
printf("// SPDX" "-License-Identifier" ": 0BSD\n\n"
"// This file has been generated by crc64_tablegen.c.\n\n"
"const uint64_t lzma_crc64_table[4][256] = {\n\t{");
for (size_t s = 0; s < 4; ++s) {

137
Externals/liblzma/check/crc_common.h vendored Normal file
View file

@ -0,0 +1,137 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc_common.h
/// \brief Some functions and macros for CRC32 and CRC64
//
// Authors: Lasse Collin
// Ilya Kurdyukov
// Hans Jansen
// Jia Tan
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CRC_COMMON_H
#define LZMA_CRC_COMMON_H
#include "common.h"
#ifdef WORDS_BIGENDIAN
# define A(x) ((x) >> 24)
# define B(x) (((x) >> 16) & 0xFF)
# define C(x) (((x) >> 8) & 0xFF)
# define D(x) ((x) & 0xFF)
# define S8(x) ((x) << 8)
# define S32(x) ((x) << 32)
#else
# define A(x) ((x) & 0xFF)
# define B(x) (((x) >> 8) & 0xFF)
# define C(x) (((x) >> 16) & 0xFF)
# define D(x) ((x) >> 24)
# define S8(x) ((x) >> 8)
# define S32(x) ((x) >> 32)
#endif
// CRC CLMUL code needs this because accessing input buffers that aren't
// aligned to the vector size will inherently trip the address sanitizer.
#if lzma_has_attribute(__no_sanitize_address__)
# define crc_attr_no_sanitize_address \
__attribute__((__no_sanitize_address__))
#else
# define crc_attr_no_sanitize_address
#endif
// Keep this in sync with changes to crc32_arm64.h
#if defined(_WIN32) || defined(HAVE_GETAUXVAL) \
|| defined(HAVE_ELF_AUX_INFO) \
|| (defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME))
# define ARM64_RUNTIME_DETECTION 1
#endif
#undef CRC32_GENERIC
#undef CRC64_GENERIC
#undef CRC32_ARCH_OPTIMIZED
#undef CRC64_ARCH_OPTIMIZED
// The x86 CLMUL is used for both CRC32 and CRC64.
#undef CRC_X86_CLMUL
#undef CRC32_ARM64
#undef CRC64_ARM64_CLMUL
#undef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// ARM64 CRC32 instruction is only useful for CRC32. Currently, only
// little endian is supported since we were unable to test on a big
// endian machine.
//
// NOTE: Keep this and the next check in sync with the macro
// NO_CRC32_TABLE in crc32_table.c
#if defined(HAVE_ARM64_CRC32) && !defined(WORDS_BIGENDIAN)
// Allow ARM64 CRC32 instruction without a runtime check if
// __ARM_FEATURE_CRC32 is defined. GCC and Clang only define
// this if the proper compiler options are used.
# if defined(__ARM_FEATURE_CRC32)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC32_ARM64 1
# elif defined(ARM64_RUNTIME_DETECTION)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC32_ARM64 1
# define CRC32_GENERIC 1
# endif
#endif
#if defined(HAVE_USABLE_CLMUL)
// If CLMUL is allowed unconditionally in the compiler options then the
// generic version can be omitted. Note that this doesn't work with MSVC
// as I don't know how to detect the features here.
//
// NOTE: Keep this in sync with the NO_CRC32_TABLE macro in crc32_table.c
// and NO_CRC64_TABLE in crc64_table.c.
# if (defined(__SSSE3__) && defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC64_ARCH_OPTIMIZED 1
# define CRC_X86_CLMUL 1
# else
# define CRC32_GENERIC 1
# define CRC64_GENERIC 1
# define CRC32_ARCH_OPTIMIZED 1
# define CRC64_ARCH_OPTIMIZED 1
# define CRC_X86_CLMUL 1
/*
// The generic code is much faster with 1-8-byte inputs and
// has similar performance up to 16 bytes at least in
// microbenchmarks (it depends on input buffer alignment
// too). If both versions are built, this #define will use
// the generic version for inputs up to 16 bytes and CLMUL
// for bigger inputs. It saves a little in code size since
// the special cases for 0-16-byte inputs will be omitted
// from the CLMUL code.
# define CRC_USE_GENERIC_FOR_SMALL_INPUTS 1
*/
# endif
#endif
// For CRC32 use the generic slice-by-eight implementation if no optimized
// version is available.
#if !defined(CRC32_ARCH_OPTIMIZED) && !defined(CRC32_GENERIC)
# define CRC32_GENERIC 1
#endif
// For CRC64 use the generic slice-by-four implementation if no optimized
// version is available.
#if !defined(CRC64_ARCH_OPTIMIZED) && !defined(CRC64_GENERIC)
# define CRC64_GENERIC 1
#endif
#endif

432
Externals/liblzma/check/crc_x86_clmul.h vendored Normal file
View file

@ -0,0 +1,432 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc_x86_clmul.h
/// \brief CRC32 and CRC64 implementations using CLMUL instructions.
///
/// The CRC32 and CRC64 implementations use 32/64-bit x86 SSSE3, SSE4.1, and
/// CLMUL instructions. This is compatible with Elbrus 2000 (E2K) too.
///
/// They were derived from
/// https://www.researchgate.net/publication/263424619_Fast_CRC_computation
/// and the public domain code from https://github.com/rawrunprotected/crc
/// (URLs were checked on 2023-10-14).
///
/// While this file has both CRC32 and CRC64 implementations, only one
/// should be built at a time to ensure that crc_simd_body() is inlined
/// even with compilers with which lzma_always_inline expands to plain inline.
/// The version to build is selected by defining BUILDING_CRC32_CLMUL or
/// BUILDING_CRC64_CLMUL before including this file.
///
/// FIXME: Builds for 32-bit x86 use the assembly .S files by default
/// unless configured with --disable-assembler. Even then the lookup table
/// isn't omitted in crc64_table.c since it doesn't know that assembly
/// code has been disabled.
//
// Authors: Ilya Kurdyukov
// Hans Jansen
// Lasse Collin
// Jia Tan
//
///////////////////////////////////////////////////////////////////////////////
// This file must not be included more than once.
#ifdef LZMA_CRC_X86_CLMUL_H
# error crc_x86_clmul.h was included twice.
#endif
#define LZMA_CRC_X86_CLMUL_H
#include <immintrin.h>
#if defined(_MSC_VER)
# include <intrin.h>
#elif defined(HAVE_CPUID_H)
# include <cpuid.h>
#endif
// EDG-based compilers (Intel's classic compiler and compiler for E2K) can
// define __GNUC__ but the attribute must not be used with them.
// The new Clang-based ICX needs the attribute.
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
# define crc_attr_target \
__attribute__((__target__("ssse3,sse4.1,pclmul")))
#else
# define crc_attr_target
#endif
#define MASK_L(in, mask, r) r = _mm_shuffle_epi8(in, mask)
#define MASK_H(in, mask, r) \
r = _mm_shuffle_epi8(in, _mm_xor_si128(mask, vsign))
#define MASK_LH(in, mask, low, high) \
MASK_L(in, mask, low); \
MASK_H(in, mask, high)
crc_attr_target
crc_attr_no_sanitize_address
static lzma_always_inline void
crc_simd_body(const uint8_t *buf, const size_t size, __m128i *v0, __m128i *v1,
const __m128i vfold16, const __m128i initial_crc)
{
// Create a vector with 8-bit values 0 to 15. This is used to
// construct control masks for _mm_blendv_epi8 and _mm_shuffle_epi8.
const __m128i vramp = _mm_setr_epi32(
0x03020100, 0x07060504, 0x0b0a0908, 0x0f0e0d0c);
// This is used to inverse the control mask of _mm_shuffle_epi8
// so that bytes that wouldn't be picked with the original mask
// will be picked and vice versa.
const __m128i vsign = _mm_set1_epi8(-0x80);
// Memory addresses A to D and the distances between them:
//
// A B C D
// [skip_start][size][skip_end]
// [ size2 ]
//
// A and D are 16-byte aligned. B and C are 1-byte aligned.
// skip_start and skip_end are 0-15 bytes. size is at least 1 byte.
//
// A = aligned_buf will initially point to this address.
// B = The address pointed by the caller-supplied buf.
// C = buf + size == aligned_buf + size2
// D = buf + size + skip_end == aligned_buf + size2 + skip_end
const size_t skip_start = (size_t)((uintptr_t)buf & 15);
const size_t skip_end = (size_t)((0U - (uintptr_t)(buf + size)) & 15);
const __m128i *aligned_buf = (const __m128i *)(
(uintptr_t)buf & ~(uintptr_t)15);
// If size2 <= 16 then the whole input fits into a single 16-byte
// vector. If size2 > 16 then at least two 16-byte vectors must
// be processed. If size2 > 16 && size <= 16 then there is only
// one 16-byte vector's worth of input but it is unaligned in memory.
//
// NOTE: There is no integer overflow here if the arguments
// are valid. If this overflowed, buf + size would too.
const size_t size2 = skip_start + size;
// Masks to be used with _mm_blendv_epi8 and _mm_shuffle_epi8:
// The first skip_start or skip_end bytes in the vectors will have
// the high bit (0x80) set. _mm_blendv_epi8 and _mm_shuffle_epi8
// will produce zeros for these positions. (Bitwise-xor of these
// masks with vsign will produce the opposite behavior.)
const __m128i mask_start
= _mm_sub_epi8(vramp, _mm_set1_epi8((char)skip_start));
const __m128i mask_end
= _mm_sub_epi8(vramp, _mm_set1_epi8((char)skip_end));
// Get the first 1-16 bytes into data0. If loading less than 16
// bytes, the bytes are loaded to the high bits of the vector and
// the least significant positions are filled with zeros.
const __m128i data0 = _mm_blendv_epi8(_mm_load_si128(aligned_buf),
_mm_setzero_si128(), mask_start);
aligned_buf++;
__m128i v2, v3;
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
if (size <= 16) {
// Right-shift initial_crc by 1-16 bytes based on "size"
// and store the result in v1 (high bytes) and v0 (low bytes).
//
// NOTE: The highest 8 bytes of initial_crc are zeros so
// v1 will be filled with zeros if size >= 8. The highest
// 8 bytes of v1 will always become zeros.
//
// [ v1 ][ v0 ]
// [ initial_crc ] size == 1
// [ initial_crc ] size == 2
// [ initial_crc ] size == 15
// [ initial_crc ] size == 16 (all in v0)
const __m128i mask_low = _mm_add_epi8(
vramp, _mm_set1_epi8((char)(size - 16)));
MASK_LH(initial_crc, mask_low, *v0, *v1);
if (size2 <= 16) {
// There are 1-16 bytes of input and it is all
// in data0. Copy the input bytes to v3. If there
// are fewer than 16 bytes, the low bytes in v3
// will be filled with zeros. That is, the input
// bytes are stored to the same position as
// (part of) initial_crc is in v0.
MASK_L(data0, mask_end, v3);
} else {
// There are 2-16 bytes of input but not all bytes
// are in data0.
const __m128i data1 = _mm_load_si128(aligned_buf);
// Collect the 2-16 input bytes from data0 and data1
// to v2 and v3, and bitwise-xor them with the
// low bits of initial_crc in v0. Note that the
// the second xor is below this else-block as it
// is shared with the other branch.
MASK_H(data0, mask_end, v2);
MASK_L(data1, mask_end, v3);
*v0 = _mm_xor_si128(*v0, v2);
}
*v0 = _mm_xor_si128(*v0, v3);
*v1 = _mm_alignr_epi8(*v1, *v0, 8);
} else
#endif
{
// There is more than 16 bytes of input.
const __m128i data1 = _mm_load_si128(aligned_buf);
const __m128i *end = (const __m128i*)(
(const char *)aligned_buf - 16 + size2);
aligned_buf++;
MASK_LH(initial_crc, mask_start, *v0, *v1);
*v0 = _mm_xor_si128(*v0, data0);
*v1 = _mm_xor_si128(*v1, data1);
while (aligned_buf < end) {
*v1 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x00));
*v0 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x11));
*v1 = _mm_load_si128(aligned_buf++);
}
if (aligned_buf != end) {
MASK_H(*v0, mask_end, v2);
MASK_L(*v0, mask_end, *v0);
MASK_L(*v1, mask_end, v3);
*v1 = _mm_or_si128(v2, v3);
}
*v1 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x00));
*v0 = _mm_xor_si128(*v1, _mm_clmulepi64_si128(
*v0, vfold16, 0x11));
*v1 = _mm_srli_si128(*v0, 8);
}
}
/////////////////////
// x86 CLMUL CRC32 //
/////////////////////
/*
// These functions were used to generate the constants
// at the top of crc32_arch_optimized().
static uint64_t
calc_lo(uint64_t p, uint64_t a, int n)
{
uint64_t b = 0; int i;
for (i = 0; i < n; i++) {
b = b >> 1 | (a & 1) << (n - 1);
a = (a >> 1) ^ ((0 - (a & 1)) & p);
}
return b;
}
// same as ~crc(&a, sizeof(a), ~0)
static uint64_t
calc_hi(uint64_t p, uint64_t a, int n)
{
int i;
for (i = 0; i < n; i++)
a = (a >> 1) ^ ((0 - (a & 1)) & p);
return a;
}
*/
#ifdef BUILDING_CRC32_CLMUL
crc_attr_target
crc_attr_no_sanitize_address
static uint32_t
crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
{
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// The code assumes that there is at least one byte of input.
if (size == 0)
return crc;
#endif
// uint32_t poly = 0xedb88320;
const int64_t p = 0x1db710640; // p << 1
const int64_t mu = 0x1f7011641; // calc_lo(p, p, 32) << 1 | 1
const int64_t k5 = 0x163cd6124; // calc_hi(p, p, 32) << 1
const int64_t k4 = 0x0ccaa009e; // calc_hi(p, p, 64) << 1
const int64_t k3 = 0x1751997d0; // calc_hi(p, p, 128) << 1
const __m128i vfold4 = _mm_set_epi64x(mu, p);
const __m128i vfold8 = _mm_set_epi64x(0, k5);
const __m128i vfold16 = _mm_set_epi64x(k4, k3);
__m128i v0, v1, v2;
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi32_si128((int32_t)~crc));
v1 = _mm_xor_si128(
_mm_clmulepi64_si128(v0, vfold16, 0x10), v1); // xxx0
v2 = _mm_shuffle_epi32(v1, 0xe7); // 0xx0
v0 = _mm_slli_epi64(v1, 32); // [0]
v0 = _mm_clmulepi64_si128(v0, vfold8, 0x00);
v0 = _mm_xor_si128(v0, v2); // [1] [2]
v2 = _mm_clmulepi64_si128(v0, vfold4, 0x10);
v2 = _mm_clmulepi64_si128(v2, vfold4, 0x00);
v0 = _mm_xor_si128(v0, v2); // [2]
return ~(uint32_t)_mm_extract_epi32(v0, 2);
}
#endif // BUILDING_CRC32_CLMUL
/////////////////////
// x86 CLMUL CRC64 //
/////////////////////
/*
// These functions were used to generate the constants
// at the top of crc64_arch_optimized().
static uint64_t
calc_lo(uint64_t poly)
{
uint64_t a = poly;
uint64_t b = 0;
for (unsigned i = 0; i < 64; ++i) {
b = (b >> 1) | (a << 63);
a = (a >> 1) ^ (a & 1 ? poly : 0);
}
return b;
}
static uint64_t
calc_hi(uint64_t poly, uint64_t a)
{
for (unsigned i = 0; i < 64; ++i)
a = (a >> 1) ^ (a & 1 ? poly : 0);
return a;
}
*/
#ifdef BUILDING_CRC64_CLMUL
// MSVC (VS2015 - VS2022) produces bad 32-bit x86 code from the CLMUL CRC
// code when optimizations are enabled (release build). According to the bug
// report, the ebx register is corrupted and the calculated result is wrong.
// Trying to workaround the problem with "__asm mov ebx, ebx" didn't help.
// The following pragma works and performance is still good. x86-64 builds
// and CRC32 CLMUL aren't affected by this problem. The problem does not
// happen in crc_simd_body() either (which is shared with CRC32 CLMUL anyway).
//
// NOTE: Another pragma after crc64_arch_optimized() restores
// the optimizations. If the #if condition here is updated,
// the other one must be updated too.
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("g", off)
#endif
crc_attr_target
crc_attr_no_sanitize_address
static uint64_t
crc64_arch_optimized(const uint8_t *buf, size_t size, uint64_t crc)
{
#ifndef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// The code assumes that there is at least one byte of input.
if (size == 0)
return crc;
#endif
// const uint64_t poly = 0xc96c5795d7870f42; // CRC polynomial
const uint64_t p = 0x92d8af2baf0e1e85; // (poly << 1) | 1
const uint64_t mu = 0x9c3e466c172963d5; // (calc_lo(poly) << 1) | 1
const uint64_t k2 = 0xdabe95afc7875f40; // calc_hi(poly, 1)
const uint64_t k1 = 0xe05dd497ca393ae4; // calc_hi(poly, k2)
const __m128i vfold8 = _mm_set_epi64x((int64_t)p, (int64_t)mu);
const __m128i vfold16 = _mm_set_epi64x((int64_t)k2, (int64_t)k1);
__m128i v0, v1, v2;
#if defined(__i386__) || defined(_M_IX86)
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_set_epi64x(0, (int64_t)~crc));
#else
// GCC and Clang would produce good code with _mm_set_epi64x
// but MSVC needs _mm_cvtsi64_si128 on x86-64.
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi64_si128((int64_t)~crc));
#endif
v1 = _mm_xor_si128(_mm_clmulepi64_si128(v0, vfold16, 0x10), v1);
v0 = _mm_clmulepi64_si128(v1, vfold8, 0x00);
v2 = _mm_clmulepi64_si128(v0, vfold8, 0x10);
v0 = _mm_xor_si128(_mm_xor_si128(v1, _mm_slli_si128(v0, 8)), v2);
#if defined(__i386__) || defined(_M_IX86)
return ~(((uint64_t)(uint32_t)_mm_extract_epi32(v0, 3) << 32) |
(uint64_t)(uint32_t)_mm_extract_epi32(v0, 2));
#else
return ~(uint64_t)_mm_extract_epi64(v0, 1);
#endif
}
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
# pragma optimize("", on)
#endif
#endif // BUILDING_CRC64_CLMUL
// Even though this is an inline function, compile it only when needed.
// This way it won't appear in E2K builds at all.
#if defined(CRC32_GENERIC) || defined(CRC64_GENERIC)
// Inlining this function duplicates the function body in crc32_resolve() and
// crc64_resolve(), but this is acceptable because this is a tiny function.
static inline bool
is_arch_extension_supported(void)
{
int success = 1;
uint32_t r[4]; // eax, ebx, ecx, edx
#if defined(_MSC_VER)
// This needs <intrin.h> with MSVC. ICC has it as a built-in
// on all platforms.
__cpuid(r, 1);
#elif defined(HAVE_CPUID_H)
// Compared to just using __asm__ to run CPUID, this also checks
// that CPUID is supported and saves and restores ebx as that is
// needed with GCC < 5 with position-independent code (PIC).
success = __get_cpuid(1, &r[0], &r[1], &r[2], &r[3]);
#else
// Just a fallback that shouldn't be needed.
__asm__("cpuid\n\t"
: "=a"(r[0]), "=b"(r[1]), "=c"(r[2]), "=d"(r[3])
: "a"(1), "c"(0));
#endif
// Returns true if these are supported:
// CLMUL (bit 1 in ecx)
// SSSE3 (bit 9 in ecx)
// SSE4.1 (bit 19 in ecx)
const uint32_t ecx_mask = (1 << 1) | (1 << 9) | (1 << 19);
return success && (r[2] & ecx_mask) == ecx_mask;
// Alternative methods that weren't used:
// - ICC's _may_i_use_cpu_feature: the other methods should work too.
// - GCC >= 6 / Clang / ICX __builtin_cpu_supports("pclmul")
//
// CPUID decoding is needed with MSVC anyway and older GCC. This keeps
// the feature checks in the build system simpler too. The nice thing
// about __builtin_cpu_supports would be that it generates very short
// code as is it only reads a variable set at startup but a few bytes
// doesn't matter here.
}
#endif

19
Externals/liblzma/check/sha256.c vendored
View file

@ -1,24 +1,17 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file sha256.c
/// \brief SHA-256
///
/// \todo Crypto++ has x86 ASM optimizations. They use SSE so if they
/// are imported to liblzma, SSE instructions need to be used
/// conditionally to keep the code working on older boxes.
//
// This code is based on the code found from 7-Zip, which has a modified
// version of the SHA-256 found from Crypto++ <http://www.cryptopp.com/>.
// The code was modified a little to fit into liblzma.
// The C code is based on the public domain SHA-256 code found from
// Crypto++ Library 5.5.1 released in 2007: https://www.cryptopp.com/
// A few minor tweaks have been made in liblzma.
//
// Authors: Kevin Springle
// Wei Dai
// Igor Pavlov
// Authors: Wei Dai
// Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "check.h"

31
Externals/liblzma/common/alone_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "alone_decoder.h"
@ -50,8 +49,7 @@ typedef struct {
static lzma_ret
alone_decode(void *coder_ptr,
const lzma_allocator *allocator lzma_attribute((__unused__)),
alone_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size,
@ -111,12 +109,24 @@ alone_decode(void *coder_ptr,
// Another hack to ditch false positives: Assume that
// if the uncompressed size is known, it must be less
// than 256 GiB.
//
// FIXME? Without picky we allow > LZMA_VLI_MAX which doesn't
// really matter in this specific situation (> LZMA_VLI_MAX is
// safe in the LZMA decoder) but it's somewhat weird still.
if (coder->picky
&& coder->uncompressed_size != LZMA_VLI_UNKNOWN
&& coder->uncompressed_size
>= (LZMA_VLI_C(1) << 38))
return LZMA_FORMAT_ERROR;
// Use LZMA_FILTER_LZMA1EXT features to specify the
// uncompressed size and that the end marker is allowed
// even when the uncompressed size is known. Both .lzma
// header and LZMA1EXT use UINT64_MAX indicate that size
// is unknown.
coder->options.ext_flags = LZMA_LZMA1EXT_ALLOW_EOPM;
lzma_set_ext_size(coder->options, coder->uncompressed_size);
// Calculate the memory usage so that it is ready
// for SEQ_CODER_INIT.
coder->memusage = lzma_lzma_decoder_memusage(&coder->options)
@ -133,6 +143,7 @@ alone_decode(void *coder_ptr,
lzma_filter_info filters[2] = {
{
.id = LZMA_FILTER_LZMA1EXT,
.init = &lzma_lzma_decoder_init,
.options = &coder->options,
}, {
@ -140,14 +151,8 @@ alone_decode(void *coder_ptr,
}
};
const lzma_ret ret = lzma_next_filter_init(&coder->next,
allocator, filters);
if (ret != LZMA_OK)
return ret;
// Use a hack to set the uncompressed size.
lzma_lz_decoder_uncompressed(coder->next.coder,
coder->uncompressed_size);
return_if_error(lzma_next_filter_init(&coder->next,
allocator, filters));
coder->sequence = SEQ_CODE;
break;

5
Externals/liblzma/common/alone_decoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_decoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_ALONE_DECODER_H

26
Externals/liblzma/common/alone_encoder.c vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file alone_decoder.c
/// \brief Decoder for LZMA_Alone files
/// \file alone_encoder.c
/// \brief Encoder for LZMA_Alone files
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -31,8 +30,7 @@ typedef struct {
static lzma_ret
alone_encode(void *coder_ptr,
const lzma_allocator *allocator lzma_attribute((__unused__)),
alone_encode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size,
@ -76,7 +74,6 @@ alone_encoder_end(void *coder_ptr, const lzma_allocator *allocator)
}
// At least for now, this is not used by any internal function.
static lzma_ret
alone_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_options_lzma *options)
@ -122,7 +119,7 @@ alone_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
if (d != UINT32_MAX)
++d;
unaligned_write32le(coder->header + 1, d);
write32le(coder->header + 1, d);
// - Uncompressed size (always unknown and using EOPM)
memset(coder->header + 1 + 4, 0xFF, 8);
@ -130,6 +127,7 @@ alone_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
// Initialize the LZMA encoder.
const lzma_filter_info filters[2] = {
{
.id = LZMA_FILTER_LZMA1,
.init = &lzma_lzma_encoder_init,
.options = (void *)(options),
}, {
@ -141,16 +139,6 @@ alone_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
}
/*
extern lzma_ret
lzma_alone_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_options_alone *options)
{
lzma_next_coder_init(&alone_encoder_init, next, allocator, options);
}
*/
extern LZMA_API(lzma_ret)
lzma_alone_encoder(lzma_stream *strm, const lzma_options_lzma *options)
{

32
Externals/liblzma/common/auto_decoder.c vendored
View file

@ -1,21 +1,23 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file auto_decoder.c
/// \brief Autodetect between .xz Stream and .lzma (LZMA_Alone) formats
/// \brief Autodetect between .xz, .lzma (LZMA_Alone), and .lz (lzip)
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_decoder.h"
#include "alone_decoder.h"
#ifdef HAVE_LZIP_DECODER
# include "lzip_decoder.h"
#endif
typedef struct {
/// Stream decoder or LZMA_Alone decoder
/// .xz Stream decoder, LZMA_Alone decoder, or lzip decoder
lzma_next_coder next;
uint64_t memlimit;
@ -46,14 +48,22 @@ auto_decode(void *coder_ptr, const lzma_allocator *allocator,
// SEQ_CODE even if we return some LZMA_*_CHECK.
coder->sequence = SEQ_CODE;
// Detect the file format. For now this is simple, since if
// it doesn't start with 0xFD (the first magic byte of the
// new format), it has to be LZMA_Alone, or something that
// we don't support at all.
// Detect the file format. .xz files start with 0xFD which
// cannot be the first byte of .lzma (LZMA_Alone) format.
// The .lz format starts with 0x4C which could be the
// first byte of a .lzma file but luckily it would mean
// lc/lp/pb being 4/3/1 which liblzma doesn't support because
// lc + lp > 4. So using just 0x4C to detect .lz is OK here.
if (in[*in_pos] == 0xFD) {
return_if_error(lzma_stream_decoder_init(
&coder->next, allocator,
coder->memlimit, coder->flags));
#ifdef HAVE_LZIP_DECODER
} else if (in[*in_pos] == 0x4C) {
return_if_error(lzma_lzip_decoder_init(
&coder->next, allocator,
coder->memlimit, coder->flags));
#endif
} else {
return_if_error(lzma_alone_decoder_init(&coder->next,
allocator, coder->memlimit, true));
@ -86,8 +96,8 @@ auto_decode(void *coder_ptr, const lzma_allocator *allocator,
// Fall through
case SEQ_FINISH:
// When LZMA_DECODE_CONCATENATED was used and we were decoding
// LZMA_Alone file, we need to check check that there is no
// When LZMA_CONCATENATED was used and we were decoding
// a LZMA_Alone file, we need to check that there is no
// trailing garbage and wait for LZMA_FINISH.
if (*in_pos < in_size)
return LZMA_DATA_ERROR;

5
Externals/liblzma/common/block_buffer_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_decoder.h"

25
Externals/liblzma/common/block_buffer_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_buffer_encoder.h"
@ -277,7 +276,7 @@ block_buffer_encode(lzma_block *block, const lzma_allocator *allocator,
if (ret != LZMA_BUF_ERROR)
return ret;
// The data was uncompressible (at least with the options
// The data was incompressible (at least with the options
// given to us) or the output buffer was too small. Use the
// uncompressed chunks of LZMA2 to wrap the data into a valid
// Block. If we haven't been given enough output space, even
@ -325,6 +324,24 @@ lzma_block_buffer_encode(lzma_block *block, const lzma_allocator *allocator,
}
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
// This is for compatibility with binaries linked against liblzma that
// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7.
LZMA_SYMVER_API("lzma_block_uncomp_encode@XZ_5.2.2",
lzma_ret, lzma_block_uncomp_encode_522)(lzma_block *block,
const uint8_t *in, size_t in_size,
uint8_t *out, size_t *out_pos, size_t out_size)
lzma_nothrow lzma_attr_warn_unused_result
__attribute__((__alias__("lzma_block_uncomp_encode_52")));
LZMA_SYMVER_API("lzma_block_uncomp_encode@@XZ_5.2",
lzma_ret, lzma_block_uncomp_encode_52)(lzma_block *block,
const uint8_t *in, size_t in_size,
uint8_t *out, size_t *out_pos, size_t out_size)
lzma_nothrow lzma_attr_warn_unused_result;
#define lzma_block_uncomp_encode lzma_block_uncomp_encode_52
#endif
extern LZMA_API(lzma_ret)
lzma_block_uncomp_encode(lzma_block *block,
const uint8_t *in, size_t in_size,

5
Externals/liblzma/common/block_buffer_encoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_buffer_encoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_BUFFER_ENCODER_H

87
Externals/liblzma/common/block_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_decoder.h"
@ -40,6 +39,9 @@ typedef struct {
/// is unknown.
lzma_vli compressed_limit;
/// Maximum allowed Uncompressed Size.
lzma_vli uncompressed_limit;
/// Position when reading the Check field
size_t check_pos;
@ -51,21 +53,6 @@ typedef struct {
} lzma_block_coder;
static inline bool
update_size(lzma_vli *size, lzma_vli add, lzma_vli limit)
{
if (limit > LZMA_VLI_MAX)
limit = LZMA_VLI_MAX;
if (limit < *size || limit - *size < add)
return true;
*size += add;
return false;
}
static inline bool
is_size_valid(lzma_vli size, lzma_vli reference)
{
@ -86,23 +73,59 @@ block_decode(void *coder_ptr, const lzma_allocator *allocator,
const size_t in_start = *in_pos;
const size_t out_start = *out_pos;
// Limit the amount of input and output space that we give
// to the raw decoder based on the information we have
// (or don't have) from Block Header.
const size_t in_stop = *in_pos + (size_t)my_min(
in_size - *in_pos,
coder->compressed_limit - coder->compressed_size);
const size_t out_stop = *out_pos + (size_t)my_min(
out_size - *out_pos,
coder->uncompressed_limit - coder->uncompressed_size);
const lzma_ret ret = coder->next.code(coder->next.coder,
allocator, in, in_pos, in_size,
out, out_pos, out_size, action);
allocator, in, in_pos, in_stop,
out, out_pos, out_stop, action);
const size_t in_used = *in_pos - in_start;
const size_t out_used = *out_pos - out_start;
// NOTE: We compare to compressed_limit here, which prevents
// the total size of the Block growing past LZMA_VLI_MAX.
if (update_size(&coder->compressed_size, in_used,
coder->compressed_limit)
|| update_size(&coder->uncompressed_size,
out_used,
coder->block->uncompressed_size))
// Because we have limited the input and output sizes,
// we know that these cannot grow too big or overflow.
coder->compressed_size += in_used;
coder->uncompressed_size += out_used;
if (ret == LZMA_OK) {
const bool comp_done = coder->compressed_size
== coder->block->compressed_size;
const bool uncomp_done = coder->uncompressed_size
== coder->block->uncompressed_size;
// If both input and output amounts match the sizes
// in Block Header but we still got LZMA_OK instead
// of LZMA_STREAM_END, the file is broken.
if (comp_done && uncomp_done)
return LZMA_DATA_ERROR;
if (!coder->ignore_check)
// If the decoder has consumed all the input that it
// needs but it still couldn't fill the output buffer
// or return LZMA_STREAM_END, the file is broken.
if (comp_done && *out_pos < out_size)
return LZMA_DATA_ERROR;
// If the decoder has produced all the output but
// it still didn't return LZMA_STREAM_END or consume
// more input (for example, detecting an end of
// payload marker may need more input but produce
// no output) the file is broken.
if (uncomp_done && *in_pos < in_size)
return LZMA_DATA_ERROR;
}
// Don't waste time updating the integrity check if it will be
// ignored. Also skip it if no new output was produced. This
// avoids null pointer + 0 (undefined behavior) when out == 0.
if (!coder->ignore_check && out_used > 0)
lzma_check_update(&coder->check, coder->block->check,
out + out_start, out_used);
@ -230,6 +253,14 @@ lzma_block_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
- lzma_check_size(block->check)
: block->compressed_size;
// With Uncompressed Size this is simpler. If Block Header lacks
// the size info, then LZMA_VLI_MAX is the maximum possible
// Uncompressed Size.
coder->uncompressed_limit
= block->uncompressed_size == LZMA_VLI_UNKNOWN
? LZMA_VLI_MAX
: block->uncompressed_size;
// Initialize the check. It's caller's problem if the Check ID is not
// supported, and the Block decoder cannot verify the Check field.
// Caller can test lzma_check_is_supported(block->check).

5
Externals/liblzma/common/block_decoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_decoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_DECODER_H

9
Externals/liblzma/common/block_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_encoder.h"
@ -77,6 +76,9 @@ block_encode(void *coder_ptr, const lzma_allocator *allocator,
// checked it at the beginning of this function.
coder->uncompressed_size += in_used;
// Call lzma_check_update() only if input was consumed. This
// avoids null pointer + 0 (undefined behavior) when in == 0.
if (in_used > 0)
lzma_check_update(&coder->check, coder->block->check,
in + in_start, in_used);
@ -217,6 +219,7 @@ lzma_block_encoder(lzma_stream *strm, lzma_block *block)
lzma_next_strm_init(lzma_block_encoder_init, strm, block);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
return LZMA_OK;

5
Externals/liblzma/common/block_encoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_encoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_BLOCK_ENCODER_H

36
Externals/liblzma/common/block_header_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_header_decoder.c
@ -5,31 +7,12 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
#include "check.h"
static void
free_properties(lzma_block *block, const lzma_allocator *allocator)
{
// Free allocated filter options. The last array member is not
// touched after the initialization in the beginning of
// lzma_block_header_decode(), so we don't need to touch that here.
for (size_t i = 0; i < LZMA_FILTERS_MAX; ++i) {
lzma_free(block->filters[i].options, allocator);
block->filters[i].id = LZMA_VLI_UNKNOWN;
block->filters[i].options = NULL;
}
return;
}
extern LZMA_API(lzma_ret)
lzma_block_header_decode(lzma_block *block,
const lzma_allocator *allocator, const uint8_t *in)
@ -39,6 +22,10 @@ lzma_block_header_decode(lzma_block *block,
// are invalid or over 63 bits, or if the header is too small
// to contain the claimed information.
// Catch unexpected NULL pointers.
if (block == NULL || block->filters == NULL || in == NULL)
return LZMA_PROG_ERROR;
// Initialize the filter options array. This way the caller can
// safely free() the options even if an error occurs in this function.
for (size_t i = 0; i <= LZMA_FILTERS_MAX; ++i) {
@ -67,8 +54,11 @@ lzma_block_header_decode(lzma_block *block,
const size_t in_size = block->header_size - 4;
// Verify CRC32
if (lzma_crc32(in, in_size, 0) != unaligned_read32le(in + in_size))
if (lzma_crc32(in, in_size, 0) != read32le(in + in_size)) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return LZMA_DATA_ERROR;
#endif
}
// Check for unsupported flags.
if (in[1] & 0x3C)
@ -98,13 +88,13 @@ lzma_block_header_decode(lzma_block *block,
block->uncompressed_size = LZMA_VLI_UNKNOWN;
// Filter Flags
const size_t filter_count = (in[1] & 3) + 1;
const size_t filter_count = (in[1] & 3U) + 1;
for (size_t i = 0; i < filter_count; ++i) {
const lzma_ret ret = lzma_filter_flags_decode(
&block->filters[i], allocator,
in, &in_pos, in_size);
if (ret != LZMA_OK) {
free_properties(block, allocator);
lzma_filters_free(block->filters, allocator);
return ret;
}
}
@ -112,7 +102,7 @@ lzma_block_header_decode(lzma_block *block,
// Padding
while (in_pos < in_size) {
if (in[in_pos++] != 0x00) {
free_properties(block, allocator);
lzma_filters_free(block->filters, allocator);
// Possibly some new field present so use
// LZMA_OPTIONS_ERROR instead of LZMA_DATA_ERROR.

7
Externals/liblzma/common/block_header_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_header_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -126,7 +125,7 @@ lzma_block_header_encode(const lzma_block *block, uint8_t *out)
memzero(out + out_pos, out_size - out_pos);
// CRC32
unaligned_write32le(out + out_size, lzma_crc32(out, out_size, 0));
write32le(out + out_size, lzma_crc32(out, out_size, 0));
return LZMA_OK;
}

7
Externals/liblzma/common/block_util.c vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file block_header.c
/// \file block_util.c
/// \brief Utility functions to handle lzma_block
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

55
Externals/liblzma/common/common.c vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file common.h
/// \file common.c
/// \brief Common functions needed in many places in liblzma
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -35,7 +34,8 @@ lzma_version_string(void)
// Memory allocation //
///////////////////////
extern void * lzma_attribute((__malloc__)) lzma_attr_alloc_size(1)
lzma_attr_alloc_size(1)
extern void *
lzma_alloc(size_t size, const lzma_allocator *allocator)
{
// Some malloc() variants return NULL if called with size == 0.
@ -53,7 +53,8 @@ lzma_alloc(size_t size, const lzma_allocator *allocator)
}
extern void * lzma_attribute((__malloc__)) lzma_attr_alloc_size(1)
lzma_attr_alloc_size(1)
extern void *
lzma_alloc_zero(size_t size, const lzma_allocator *allocator)
{
// Some calloc() variants return NULL if called with size == 0.
@ -99,6 +100,10 @@ lzma_bufcpy(const uint8_t *restrict in, size_t *restrict in_pos,
const size_t out_avail = out_size - *out_pos;
const size_t copy_size = my_min(in_avail, out_avail);
// Call memcpy() only if there is something to copy. If there is
// nothing to copy, in or out might be NULL and then the memcpy()
// call would trigger undefined behavior.
if (copy_size > 0)
memcpy(out + *out_pos, in + *in_pos, copy_size);
*in_pos += copy_size;
@ -207,7 +212,6 @@ lzma_code(lzma_stream *strm, lzma_action action)
|| strm->reserved_ptr2 != NULL
|| strm->reserved_ptr3 != NULL
|| strm->reserved_ptr4 != NULL
|| strm->reserved_int1 != 0
|| strm->reserved_int2 != 0
|| strm->reserved_int3 != 0
|| strm->reserved_int4 != 0
@ -285,19 +289,25 @@ lzma_code(lzma_stream *strm, lzma_action action)
strm->next_in, &in_pos, strm->avail_in,
strm->next_out, &out_pos, strm->avail_out, action);
// Updating next_in and next_out has to be skipped when they are NULL
// to avoid null pointer + 0 (undefined behavior). Do this by checking
// in_pos > 0 and out_pos > 0 because this way NULL + non-zero (a bug)
// will get caught one way or other.
if (in_pos > 0) {
strm->next_in += in_pos;
strm->avail_in -= in_pos;
strm->total_in += in_pos;
}
if (out_pos > 0) {
strm->next_out += out_pos;
strm->avail_out -= out_pos;
strm->total_out += out_pos;
}
strm->internal->avail_in = strm->avail_in;
// Cast is needed to silence a warning about LZMA_TIMED_OUT, which
// isn't part of lzma_ret enumeration.
switch ((unsigned int)(ret)) {
switch (ret) {
case LZMA_OK:
// Don't return LZMA_BUF_ERROR when it happens the first time.
// This is to avoid returning LZMA_BUF_ERROR when avail_out
@ -318,6 +328,17 @@ lzma_code(lzma_stream *strm, lzma_action action)
ret = LZMA_OK;
break;
case LZMA_SEEK_NEEDED:
strm->internal->allow_buf_error = false;
// If LZMA_FINISH was used, reset it back to the
// LZMA_RUN-based state so that new input can be supplied
// by the application.
if (strm->internal->sequence == ISEQ_FINISH)
strm->internal->sequence = ISEQ_RUN;
break;
case LZMA_STREAM_END:
if (strm->internal->sequence == ISEQ_SYNC_FLUSH
|| strm->internal->sequence == ISEQ_FULL_FLUSH
@ -362,6 +383,20 @@ lzma_end(lzma_stream *strm)
}
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
// This is for compatibility with binaries linked against liblzma that
// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7.
LZMA_SYMVER_API("lzma_get_progress@XZ_5.2.2",
void, lzma_get_progress_522)(lzma_stream *strm,
uint64_t *progress_in, uint64_t *progress_out) lzma_nothrow
__attribute__((__alias__("lzma_get_progress_52")));
LZMA_SYMVER_API("lzma_get_progress@@XZ_5.2",
void, lzma_get_progress_52)(lzma_stream *strm,
uint64_t *progress_in, uint64_t *progress_out) lzma_nothrow;
#define lzma_get_progress lzma_get_progress_52
#endif
extern LZMA_API(void)
lzma_get_progress(lzma_stream *strm,
uint64_t *progress_in, uint64_t *progress_out)

128
Externals/liblzma/common/common.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file common.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_COMMON_H
@ -17,23 +16,104 @@
#include "mythread.h"
#include "tuklib_integer.h"
// LZMA_API_EXPORT is used to mark the exported API functions.
// It's used to define the LZMA_API macro.
//
// lzma_attr_visibility_hidden is used for marking *declarations* of extern
// variables that are internal to liblzma (-fvisibility=hidden alone is
// enough to hide the *definitions*). Such markings allow slightly more
// efficient code to accesses those variables in ELF shared libraries.
#if defined(_WIN32) || defined(__CYGWIN__)
# ifdef DLL_EXPORT
# define LZMA_API_EXPORT __declspec(dllexport)
# else
# define LZMA_API_EXPORT
# endif
# define lzma_attr_visibility_hidden
// Don't use ifdef or defined() below.
#elif HAVE_VISIBILITY
# define LZMA_API_EXPORT __attribute__((__visibility__("default")))
# define lzma_attr_visibility_hidden \
__attribute__((__visibility__("hidden")))
#else
# define LZMA_API_EXPORT
# define lzma_attr_visibility_hidden
#endif
#define LZMA_API(type) LZMA_API_EXPORT type LZMA_API_CALL
#include "lzma.h"
// This is for detecting modern GCC and Clang attributes
// like __symver__ in GCC >= 10.
#ifdef __has_attribute
# define lzma_has_attribute(attr) __has_attribute(attr)
#else
# define lzma_has_attribute(attr) 0
#endif
// The extra symbol versioning in the C files may only be used when
// building a shared library. If HAVE_SYMBOL_VERSIONS_LINUX is defined
// to 2 then symbol versioning is done only if also PIC is defined.
// By default Libtool defines PIC when building a shared library and
// doesn't define it when building a static library but it can be
// overridden with --with-pic and --without-pic. configure let's rely
// on PIC if neither --with-pic or --without-pic was used.
#if defined(HAVE_SYMBOL_VERSIONS_LINUX) \
&& (HAVE_SYMBOL_VERSIONS_LINUX == 2 && !defined(PIC))
# undef HAVE_SYMBOL_VERSIONS_LINUX
#endif
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
// To keep link-time optimization (LTO, -flto) working with GCC,
// the __symver__ attribute must be used instead of __asm__(".symver ...").
// Otherwise the symbol versions may be lost, resulting in broken liblzma
// that has wrong default versions in the exported symbol list!
// The attribute was added in GCC 10; LTO with older GCC is not supported.
//
// To keep -Wmissing-prototypes happy, use LZMA_SYMVER_API only with function
// declarations (including those with __alias__ attribute) and LZMA_API with
// the function definitions. This means a little bit of silly copy-and-paste
// between declarations and definitions though.
//
// As of GCC 12.2, the __symver__ attribute supports only @ and @@ but the
// very convenient @@@ isn't supported (it's supported by GNU assembler
// since 2000). When using @@ instead of @@@, the internal name must not be
// the same as the external name to avoid problems in some situations. This
// is why "#define foo_52 foo" is needed for the default symbol versions.
//
// __has_attribute is supported before GCC 10 and it is supported in Clang 14
// too (which doesn't support __symver__) so use it to detect if __symver__
// is available. This should be far more reliable than looking at compiler
// version macros as nowadays especially __GNUC__ is defined by many compilers.
# if lzma_has_attribute(__symver__)
# define LZMA_SYMVER_API(extnamever, type, intname) \
extern __attribute__((__symver__(extnamever))) \
LZMA_API(type) intname
# else
# define LZMA_SYMVER_API(extnamever, type, intname) \
__asm__(".symver " #intname "," extnamever); \
extern LZMA_API(type) intname
# endif
#endif
// MSVC has __forceinline which shouldn't be combined with the inline keyword
// (results in a warning).
//
// GCC 3.1 added always_inline attribute so we don't need to check
// for __GNUC__ version. Similarly, all relevant Clang versions
// support it (at least Clang 3.0.0 does already).
// Other compilers might support too which also support __has_attribute
// (Solaris Studio) so do that check too.
#if defined(_MSC_VER)
# define lzma_always_inline __forceinline
#elif defined(__GNUC__) || defined(__clang__) || defined(__INTEL_COMPILER) \
|| lzma_has_attribute(__always_inline__)
# define lzma_always_inline inline __attribute__((__always_inline__))
#else
# define lzma_always_inline inline
#endif
// These allow helping the compiler in some often-executed branches, whose
// result is almost always the same.
#ifdef __GNUC__
@ -67,14 +147,15 @@
#define LZMA_FILTER_RESERVED_START (LZMA_VLI_C(1) << 62)
/// Supported flags that can be passed to lzma_stream_decoder()
/// or lzma_auto_decoder().
/// Supported flags that can be passed to lzma_stream_decoder(),
/// lzma_auto_decoder(), or lzma_stream_decoder_mt().
#define LZMA_SUPPORTED_FLAGS \
( LZMA_TELL_NO_CHECK \
| LZMA_TELL_UNSUPPORTED_CHECK \
| LZMA_TELL_ANY_CHECK \
| LZMA_IGNORE_CHECK \
| LZMA_CONCATENATED )
| LZMA_CONCATENATED \
| LZMA_FAIL_FAST )
/// Largest valid lzma_action value as unsigned integer.
@ -83,9 +164,12 @@
/// Special return value (lzma_ret) to indicate that a timeout was reached
/// and lzma_code() must not return LZMA_BUF_ERROR. This is converted to
/// LZMA_OK in lzma_code(). This is not in the lzma_ret enumeration because
/// there's no need to have it in the public API.
#define LZMA_TIMED_OUT 32
/// LZMA_OK in lzma_code().
#define LZMA_TIMED_OUT LZMA_RET_INTERNAL1
/// Special return value (lzma_ret) for use in stream_decoder_mt.c to
/// indicate Index was detected instead of a Block Header.
#define LZMA_INDEX_DETECTED LZMA_RET_INTERNAL2
typedef struct lzma_next_coder_s lzma_next_coder;
@ -118,8 +202,11 @@ typedef void (*lzma_end_function)(
/// an array of lzma_filter_info structures. This array is used with
/// lzma_next_filter_init to initialize the filter chain.
struct lzma_filter_info_s {
/// Filter ID. This is used only by the encoder
/// with lzma_filters_update().
/// Filter ID. This can be used to share the same initiazation
/// function *and* data structures with different Filter IDs
/// (LZMA_FILTER_LZMA1EXT does it), and also by the encoder
/// with lzma_filters_update() if filter chain is updated
/// in the middle of a raw stream or Block (LZMA_SYNC_FLUSH).
lzma_vli id;
/// Pointer to function used to initialize the filter.
@ -173,6 +260,16 @@ struct lzma_next_coder_s {
lzma_ret (*update)(void *coder, const lzma_allocator *allocator,
const lzma_filter *filters,
const lzma_filter *reversed_filters);
/// Set how many bytes of output this coder may produce at maximum.
/// On success LZMA_OK must be returned.
/// If the filter chain as a whole cannot support this feature,
/// this must return LZMA_OPTIONS_ERROR.
/// If no input has been given to the coder and the requested limit
/// is too small, this must return LZMA_BUF_ERROR. If input has been
/// seen, LZMA_OK is allowed too.
lzma_ret (*set_out_limit)(void *coder, uint64_t *uncomp_size,
uint64_t out_limit);
};
@ -188,6 +285,7 @@ struct lzma_next_coder_s {
.get_check = NULL, \
.memconfig = NULL, \
.update = NULL, \
.set_out_limit = NULL, \
}
@ -226,14 +324,14 @@ struct lzma_internal_s {
/// Allocates memory
extern void *lzma_alloc(size_t size, const lzma_allocator *allocator)
lzma_attribute((__malloc__)) lzma_attr_alloc_size(1);
lzma_attr_alloc_size(1)
extern void *lzma_alloc(size_t size, const lzma_allocator *allocator);
/// Allocates memory and zeroes it (like calloc()). This can be faster
/// than lzma_alloc() + memzero() while being backward compatible with
/// custom allocators.
extern void * lzma_attribute((__malloc__)) lzma_attr_alloc_size(1)
lzma_alloc_zero(size_t size, const lzma_allocator *allocator);
lzma_attr_alloc_size(1)
extern void *lzma_alloc_zero(size_t size, const lzma_allocator *allocator);
/// Frees memory
extern void lzma_free(void *ptr, const lzma_allocator *allocator);

5
Externals/liblzma/common/easy_buffer_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_buffer_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

5
Externals/liblzma/common/easy_decoder_memusage.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_decoder_memusage.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

5
Externals/liblzma/common/easy_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

5
Externals/liblzma/common/easy_encoder_memusage.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_encoder_memusage.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

5
Externals/liblzma/common/easy_preset.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_preset.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "easy_preset.h"

10
Externals/liblzma/common/easy_preset.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file easy_preset.h
@ -5,11 +7,11 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_EASY_PRESET_H
#define LZMA_EASY_PRESET_H
#include "common.h"
@ -30,3 +32,5 @@ typedef struct {
/// Set *easy to the settings given by the preset. Returns true on error,
/// false on success.
extern bool lzma_easy_preset(lzma_options_easy *easy, uint32_t preset);
#endif

854
Externals/liblzma/common/file_info.c vendored Normal file
View file

@ -0,0 +1,854 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file file_info.c
/// \brief Decode .xz file information into a lzma_index structure
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "index_decoder.h"
typedef struct {
enum {
SEQ_MAGIC_BYTES,
SEQ_PADDING_SEEK,
SEQ_PADDING_DECODE,
SEQ_FOOTER,
SEQ_INDEX_INIT,
SEQ_INDEX_DECODE,
SEQ_HEADER_DECODE,
SEQ_HEADER_COMPARE,
} sequence;
/// Absolute position of in[*in_pos] in the file. All code that
/// modifies *in_pos also updates this. seek_to_pos() needs this
/// to determine if we need to request the application to seek for
/// us or if we can do the seeking internally by adjusting *in_pos.
uint64_t file_cur_pos;
/// This refers to absolute positions of interesting parts of the
/// input file. Sometimes it points to the *beginning* of a specific
/// field and sometimes to the *end* of a field. The current target
/// position at each moment is explained in the comments.
uint64_t file_target_pos;
/// Size of the .xz file (from the application).
uint64_t file_size;
/// Index decoder
lzma_next_coder index_decoder;
/// Number of bytes remaining in the Index field that is currently
/// being decoded.
lzma_vli index_remaining;
/// The Index decoder will store the decoded Index in this pointer.
lzma_index *this_index;
/// Amount of Stream Padding in the current Stream.
lzma_vli stream_padding;
/// The final combined index is collected here.
lzma_index *combined_index;
/// Pointer from the application where to store the index information
/// after successful decoding.
lzma_index **dest_index;
/// Pointer to lzma_stream.seek_pos to be used when returning
/// LZMA_SEEK_NEEDED. This is set by seek_to_pos() when needed.
uint64_t *external_seek_pos;
/// Memory usage limit
uint64_t memlimit;
/// Stream Flags from the very beginning of the file.
lzma_stream_flags first_header_flags;
/// Stream Flags from Stream Header of the current Stream.
lzma_stream_flags header_flags;
/// Stream Flags from Stream Footer of the current Stream.
lzma_stream_flags footer_flags;
size_t temp_pos;
size_t temp_size;
uint8_t temp[8192];
} lzma_file_info_coder;
/// Copies data from in[*in_pos] into coder->temp until
/// coder->temp_pos == coder->temp_size. This also keeps coder->file_cur_pos
/// in sync with *in_pos. Returns true if more input is needed.
static bool
fill_temp(lzma_file_info_coder *coder, const uint8_t *restrict in,
size_t *restrict in_pos, size_t in_size)
{
coder->file_cur_pos += lzma_bufcpy(in, in_pos, in_size,
coder->temp, &coder->temp_pos, coder->temp_size);
return coder->temp_pos < coder->temp_size;
}
/// Seeks to the absolute file position specified by target_pos.
/// This tries to do the seeking by only modifying *in_pos, if possible.
/// The main benefit of this is that if one passes the whole file at once
/// to lzma_code(), the decoder will never need to return LZMA_SEEK_NEEDED
/// as all the seeking can be done by adjusting *in_pos in this function.
///
/// Returns true if an external seek is needed and the caller must return
/// LZMA_SEEK_NEEDED.
static bool
seek_to_pos(lzma_file_info_coder *coder, uint64_t target_pos,
size_t in_start, size_t *in_pos, size_t in_size)
{
// The input buffer doesn't extend beyond the end of the file.
// This has been checked by file_info_decode() already.
assert(coder->file_size - coder->file_cur_pos >= in_size - *in_pos);
const uint64_t pos_min = coder->file_cur_pos - (*in_pos - in_start);
const uint64_t pos_max = coder->file_cur_pos + (in_size - *in_pos);
bool external_seek_needed;
if (target_pos >= pos_min && target_pos <= pos_max) {
// The requested position is available in the current input
// buffer or right after it. That is, in a corner case we
// end up setting *in_pos == in_size and thus will immediately
// need new input bytes from the application.
*in_pos += (size_t)(target_pos - coder->file_cur_pos);
external_seek_needed = false;
} else {
// Ask the application to seek the input file.
*coder->external_seek_pos = target_pos;
external_seek_needed = true;
// Mark the whole input buffer as used. This way
// lzma_stream.total_in will have a better estimate
// of the amount of data read. It still won't be perfect
// as the value will depend on the input buffer size that
// the application uses, but it should be good enough for
// those few who want an estimate.
*in_pos = in_size;
}
// After seeking (internal or external) the current position
// will match the requested target position.
coder->file_cur_pos = target_pos;
return external_seek_needed;
}
/// The caller sets coder->file_target_pos so that it points to the *end*
/// of the desired file position. This function then determines how far
/// backwards from that position we can seek. After seeking fill_temp()
/// can be used to read data into coder->temp. When fill_temp() has finished,
/// coder->temp[coder->temp_size] will match coder->file_target_pos.
///
/// This also validates that coder->target_file_pos is sane in sense that
/// we aren't trying to seek too far backwards (too close or beyond the
/// beginning of the file).
static lzma_ret
reverse_seek(lzma_file_info_coder *coder,
size_t in_start, size_t *in_pos, size_t in_size)
{
// Check that there is enough data before the target position
// to contain at least Stream Header and Stream Footer. If there
// isn't, the file cannot be valid.
if (coder->file_target_pos < 2 * LZMA_STREAM_HEADER_SIZE)
return LZMA_DATA_ERROR;
coder->temp_pos = 0;
// The Stream Header at the very beginning of the file gets handled
// specially in SEQ_MAGIC_BYTES and thus we will never need to seek
// there. By not seeking to the first LZMA_STREAM_HEADER_SIZE bytes
// we avoid a useless external seek after SEQ_MAGIC_BYTES if the
// application uses an extremely small input buffer and the input
// file is very small.
if (coder->file_target_pos - LZMA_STREAM_HEADER_SIZE
< sizeof(coder->temp))
coder->temp_size = (size_t)(coder->file_target_pos
- LZMA_STREAM_HEADER_SIZE);
else
coder->temp_size = sizeof(coder->temp);
// The above if-statements guarantee this. This is important because
// the Stream Header/Footer decoders assume that there's at least
// LZMA_STREAM_HEADER_SIZE bytes in coder->temp.
assert(coder->temp_size >= LZMA_STREAM_HEADER_SIZE);
if (seek_to_pos(coder, coder->file_target_pos - coder->temp_size,
in_start, in_pos, in_size))
return LZMA_SEEK_NEEDED;
return LZMA_OK;
}
/// Gets the number of zero-bytes at the end of the buffer.
static size_t
get_padding_size(const uint8_t *buf, size_t buf_size)
{
size_t padding = 0;
while (buf_size > 0 && buf[--buf_size] == 0x00)
++padding;
return padding;
}
/// With the Stream Header at the very beginning of the file, LZMA_FORMAT_ERROR
/// is used to tell the application that Magic Bytes didn't match. In other
/// Stream Header/Footer fields (in the middle/end of the file) it could be
/// a bit confusing to return LZMA_FORMAT_ERROR as we already know that there
/// is a valid Stream Header at the beginning of the file. For those cases
/// this function is used to convert LZMA_FORMAT_ERROR to LZMA_DATA_ERROR.
static lzma_ret
hide_format_error(lzma_ret ret)
{
if (ret == LZMA_FORMAT_ERROR)
ret = LZMA_DATA_ERROR;
return ret;
}
/// Calls the Index decoder and updates coder->index_remaining.
/// This is a separate function because the input can be either directly
/// from the application or from coder->temp.
static lzma_ret
decode_index(lzma_file_info_coder *coder, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, bool update_file_cur_pos)
{
const size_t in_start = *in_pos;
const lzma_ret ret = coder->index_decoder.code(
coder->index_decoder.coder,
allocator, in, in_pos, in_size,
NULL, NULL, 0, LZMA_RUN);
coder->index_remaining -= *in_pos - in_start;
if (update_file_cur_pos)
coder->file_cur_pos += *in_pos - in_start;
return ret;
}
static lzma_ret
file_info_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size,
uint8_t *restrict out lzma_attribute((__unused__)),
size_t *restrict out_pos lzma_attribute((__unused__)),
size_t out_size lzma_attribute((__unused__)),
lzma_action action lzma_attribute((__unused__)))
{
lzma_file_info_coder *coder = coder_ptr;
const size_t in_start = *in_pos;
// If the caller provides input past the end of the file, trim
// the extra bytes from the buffer so that we won't read too far.
assert(coder->file_size >= coder->file_cur_pos);
if (coder->file_size - coder->file_cur_pos < in_size - in_start)
in_size = in_start
+ (size_t)(coder->file_size - coder->file_cur_pos);
while (true)
switch (coder->sequence) {
case SEQ_MAGIC_BYTES:
// Decode the Stream Header at the beginning of the file
// first to check if the Magic Bytes match. The flags
// are stored in coder->first_header_flags so that we
// don't need to seek to it again.
//
// Check that the file is big enough to contain at least
// Stream Header.
if (coder->file_size < LZMA_STREAM_HEADER_SIZE)
return LZMA_FORMAT_ERROR;
// Read the Stream Header field into coder->temp.
if (fill_temp(coder, in, in_pos, in_size))
return LZMA_OK;
// This is the only Stream Header/Footer decoding where we
// want to return LZMA_FORMAT_ERROR if the Magic Bytes don't
// match. Elsewhere it will be converted to LZMA_DATA_ERROR.
return_if_error(lzma_stream_header_decode(
&coder->first_header_flags, coder->temp));
// Now that we know that the Magic Bytes match, check the
// file size. It's better to do this here after checking the
// Magic Bytes since this way we can give LZMA_FORMAT_ERROR
// instead of LZMA_DATA_ERROR when the Magic Bytes don't
// match in a file that is too big or isn't a multiple of
// four bytes.
if (coder->file_size > LZMA_VLI_MAX || (coder->file_size & 3))
return LZMA_DATA_ERROR;
// Start looking for Stream Padding and Stream Footer
// at the end of the file.
coder->file_target_pos = coder->file_size;
// Fall through
case SEQ_PADDING_SEEK:
coder->sequence = SEQ_PADDING_DECODE;
return_if_error(reverse_seek(
coder, in_start, in_pos, in_size));
// Fall through
case SEQ_PADDING_DECODE: {
// Copy to coder->temp first. This keeps the code simpler if
// the application only provides input a few bytes at a time.
if (fill_temp(coder, in, in_pos, in_size))
return LZMA_OK;
// Scan the buffer backwards to get the size of the
// Stream Padding field (if any).
const size_t new_padding = get_padding_size(
coder->temp, coder->temp_size);
coder->stream_padding += new_padding;
// Set the target position to the beginning of Stream Padding
// that has been observed so far. If all Stream Padding has
// been seen, then the target position will be at the end
// of the Stream Footer field.
coder->file_target_pos -= new_padding;
if (new_padding == coder->temp_size) {
// The whole buffer was padding. Seek backwards in
// the file to get more input.
coder->sequence = SEQ_PADDING_SEEK;
break;
}
// Size of Stream Padding must be a multiple of 4 bytes.
if (coder->stream_padding & 3)
return LZMA_DATA_ERROR;
coder->sequence = SEQ_FOOTER;
// Calculate the amount of non-padding data in coder->temp.
coder->temp_size -= new_padding;
coder->temp_pos = coder->temp_size;
// We can avoid an external seek if the whole Stream Footer
// is already in coder->temp. In that case SEQ_FOOTER won't
// read more input and will find the Stream Footer from
// coder->temp[coder->temp_size - LZMA_STREAM_HEADER_SIZE].
//
// Otherwise we will need to seek. The seeking is done so
// that Stream Footer will be at the end of coder->temp.
// This way it's likely that we also get a complete Index
// field into coder->temp without needing a separate seek
// for that (unless the Index field is big).
if (coder->temp_size < LZMA_STREAM_HEADER_SIZE)
return_if_error(reverse_seek(
coder, in_start, in_pos, in_size));
}
// Fall through
case SEQ_FOOTER:
// Copy the Stream Footer field into coder->temp.
// If Stream Footer was already available in coder->temp
// in SEQ_PADDING_DECODE, then this does nothing.
if (fill_temp(coder, in, in_pos, in_size))
return LZMA_OK;
// Make coder->file_target_pos and coder->temp_size point
// to the beginning of Stream Footer and thus to the end
// of the Index field. coder->temp_pos will be updated
// a bit later.
coder->file_target_pos -= LZMA_STREAM_HEADER_SIZE;
coder->temp_size -= LZMA_STREAM_HEADER_SIZE;
// Decode Stream Footer.
return_if_error(hide_format_error(lzma_stream_footer_decode(
&coder->footer_flags,
coder->temp + coder->temp_size)));
// Check that we won't seek past the beginning of the file.
//
// LZMA_STREAM_HEADER_SIZE is added because there must be
// space for Stream Header too even though we won't seek
// there before decoding the Index field.
//
// There's no risk of integer overflow here because
// Backward Size cannot be greater than 2^34.
if (coder->file_target_pos < coder->footer_flags.backward_size
+ LZMA_STREAM_HEADER_SIZE)
return LZMA_DATA_ERROR;
// Set the target position to the beginning of the Index field.
coder->file_target_pos -= coder->footer_flags.backward_size;
coder->sequence = SEQ_INDEX_INIT;
// We can avoid an external seek if the whole Index field is
// already available in coder->temp.
if (coder->temp_size >= coder->footer_flags.backward_size) {
// Set coder->temp_pos to point to the beginning
// of the Index.
coder->temp_pos = coder->temp_size
- coder->footer_flags.backward_size;
} else {
// These are set to zero to indicate that there's no
// useful data (Index or anything else) in coder->temp.
coder->temp_pos = 0;
coder->temp_size = 0;
// Seek to the beginning of the Index field.
if (seek_to_pos(coder, coder->file_target_pos,
in_start, in_pos, in_size))
return LZMA_SEEK_NEEDED;
}
// Fall through
case SEQ_INDEX_INIT: {
// Calculate the amount of memory already used by the earlier
// Indexes so that we know how big memory limit to pass to
// the Index decoder.
//
// NOTE: When there are multiple Streams, the separate
// lzma_index structures can use more RAM (as measured by
// lzma_index_memused()) than the final combined lzma_index.
// Thus memlimit may need to be slightly higher than the final
// calculated memory usage will be. This is perhaps a bit
// confusing to the application, but I think it shouldn't
// cause problems in practice.
uint64_t memused = 0;
if (coder->combined_index != NULL) {
memused = lzma_index_memused(coder->combined_index);
assert(memused <= coder->memlimit);
if (memused > coder->memlimit) // Extra sanity check
return LZMA_PROG_ERROR;
}
// Initialize the Index decoder.
return_if_error(lzma_index_decoder_init(
&coder->index_decoder, allocator,
&coder->this_index,
coder->memlimit - memused));
coder->index_remaining = coder->footer_flags.backward_size;
coder->sequence = SEQ_INDEX_DECODE;
}
// Fall through
case SEQ_INDEX_DECODE: {
// Decode (a part of) the Index. If the whole Index is already
// in coder->temp, read it from there. Otherwise read from
// in[*in_pos] onwards. Note that index_decode() updates
// coder->index_remaining and optionally coder->file_cur_pos.
lzma_ret ret;
if (coder->temp_size != 0) {
assert(coder->temp_size - coder->temp_pos
== coder->index_remaining);
ret = decode_index(coder, allocator, coder->temp,
&coder->temp_pos, coder->temp_size,
false);
} else {
// Don't give the decoder more input than the known
// remaining size of the Index field.
size_t in_stop = in_size;
if (in_size - *in_pos > coder->index_remaining)
in_stop = *in_pos
+ (size_t)(coder->index_remaining);
ret = decode_index(coder, allocator,
in, in_pos, in_stop, true);
}
switch (ret) {
case LZMA_OK:
// If the Index docoder asks for more input when we
// have already given it as much input as Backward Size
// indicated, the file is invalid.
if (coder->index_remaining == 0)
return LZMA_DATA_ERROR;
// We cannot get here if we were reading Index from
// coder->temp because when reading from coder->temp
// we give the Index decoder exactly
// coder->index_remaining bytes of input.
assert(coder->temp_size == 0);
return LZMA_OK;
case LZMA_STREAM_END:
// If the decoding seems to be successful, check also
// that the Index decoder consumed as much input as
// indicated by the Backward Size field.
if (coder->index_remaining != 0)
return LZMA_DATA_ERROR;
break;
default:
return ret;
}
// Calculate how much the Index tells us to seek backwards
// (relative to the beginning of the Index): Total size of
// all Blocks plus the size of the Stream Header field.
// No integer overflow here because lzma_index_total_size()
// cannot return a value greater than LZMA_VLI_MAX.
const uint64_t seek_amount
= lzma_index_total_size(coder->this_index)
+ LZMA_STREAM_HEADER_SIZE;
// Check that Index is sane in sense that seek_amount won't
// make us seek past the beginning of the file when locating
// the Stream Header.
//
// coder->file_target_pos still points to the beginning of
// the Index field.
if (coder->file_target_pos < seek_amount)
return LZMA_DATA_ERROR;
// Set the target to the beginning of Stream Header.
coder->file_target_pos -= seek_amount;
if (coder->file_target_pos == 0) {
// We would seek to the beginning of the file, but
// since we already decoded that Stream Header in
// SEQ_MAGIC_BYTES, we can use the cached value from
// coder->first_header_flags to avoid the seek.
coder->header_flags = coder->first_header_flags;
coder->sequence = SEQ_HEADER_COMPARE;
break;
}
coder->sequence = SEQ_HEADER_DECODE;
// Make coder->file_target_pos point to the end of
// the Stream Header field.
coder->file_target_pos += LZMA_STREAM_HEADER_SIZE;
// If coder->temp_size is non-zero, it points to the end
// of the Index field. Then the beginning of the Index
// field is at coder->temp[coder->temp_size
// - coder->footer_flags.backward_size].
assert(coder->temp_size == 0 || coder->temp_size
>= coder->footer_flags.backward_size);
// If coder->temp contained the whole Index, see if it has
// enough data to contain also the Stream Header. If so,
// we avoid an external seek.
//
// NOTE: This can happen only with small .xz files and only
// for the non-first Stream as the Stream Flags of the first
// Stream are cached and already handled a few lines above.
// So this isn't as useful as the other seek-avoidance cases.
if (coder->temp_size != 0 && coder->temp_size
- coder->footer_flags.backward_size
>= seek_amount) {
// Make temp_pos and temp_size point to the *end* of
// Stream Header so that SEQ_HEADER_DECODE will find
// the start of Stream Header from coder->temp[
// coder->temp_size - LZMA_STREAM_HEADER_SIZE].
coder->temp_pos = coder->temp_size
- coder->footer_flags.backward_size
- seek_amount
+ LZMA_STREAM_HEADER_SIZE;
coder->temp_size = coder->temp_pos;
} else {
// Seek so that Stream Header will be at the end of
// coder->temp. With typical multi-Stream files we
// will usually also get the Stream Footer and Index
// of the *previous* Stream in coder->temp and thus
// won't need a separate seek for them.
return_if_error(reverse_seek(coder,
in_start, in_pos, in_size));
}
}
// Fall through
case SEQ_HEADER_DECODE:
// Copy the Stream Header field into coder->temp.
// If Stream Header was already available in coder->temp
// in SEQ_INDEX_DECODE, then this does nothing.
if (fill_temp(coder, in, in_pos, in_size))
return LZMA_OK;
// Make all these point to the beginning of Stream Header.
coder->file_target_pos -= LZMA_STREAM_HEADER_SIZE;
coder->temp_size -= LZMA_STREAM_HEADER_SIZE;
coder->temp_pos = coder->temp_size;
// Decode the Stream Header.
return_if_error(hide_format_error(lzma_stream_header_decode(
&coder->header_flags,
coder->temp + coder->temp_size)));
coder->sequence = SEQ_HEADER_COMPARE;
// Fall through
case SEQ_HEADER_COMPARE:
// Compare Stream Header against Stream Footer. They must
// match.
return_if_error(lzma_stream_flags_compare(
&coder->header_flags, &coder->footer_flags));
// Store the decoded Stream Flags into the Index. Use the
// Footer Flags because it contains Backward Size, although
// it shouldn't matter in practice.
if (lzma_index_stream_flags(coder->this_index,
&coder->footer_flags) != LZMA_OK)
return LZMA_PROG_ERROR;
// Store also the size of the Stream Padding field. It is
// needed to calculate the offsets of the Streams correctly.
if (lzma_index_stream_padding(coder->this_index,
coder->stream_padding) != LZMA_OK)
return LZMA_PROG_ERROR;
// Reset it so that it's ready for the next Stream.
coder->stream_padding = 0;
// Append the earlier decoded Indexes after this_index.
if (coder->combined_index != NULL)
return_if_error(lzma_index_cat(coder->this_index,
coder->combined_index, allocator));
coder->combined_index = coder->this_index;
coder->this_index = NULL;
// If the whole file was decoded, tell the caller that we
// are finished.
if (coder->file_target_pos == 0) {
// The combined index must indicate the same file
// size as was told to us at initialization.
assert(lzma_index_file_size(coder->combined_index)
== coder->file_size);
// Make the combined index available to
// the application.
*coder->dest_index = coder->combined_index;
coder->combined_index = NULL;
// Mark the input buffer as used since we may have
// done internal seeking and thus don't know how
// many input bytes were actually used. This way
// lzma_stream.total_in gets a slightly better
// estimate of the amount of input used.
*in_pos = in_size;
return LZMA_STREAM_END;
}
// We didn't hit the beginning of the file yet, so continue
// reading backwards in the file. If we have unprocessed
// data in coder->temp, use it before requesting more data
// from the application.
//
// coder->file_target_pos, coder->temp_size, and
// coder->temp_pos all point to the beginning of Stream Header
// and thus the end of the previous Stream in the file.
coder->sequence = coder->temp_size > 0
? SEQ_PADDING_DECODE : SEQ_PADDING_SEEK;
break;
default:
assert(0);
return LZMA_PROG_ERROR;
}
}
static lzma_ret
file_info_decoder_memconfig(void *coder_ptr, uint64_t *memusage,
uint64_t *old_memlimit, uint64_t new_memlimit)
{
lzma_file_info_coder *coder = coder_ptr;
// The memory usage calculation comes from three things:
//
// (1) The Indexes that have already been decoded and processed into
// coder->combined_index.
//
// (2) The latest Index in coder->this_index that has been decoded but
// not yet put into coder->combined_index.
//
// (3) The latest Index that we have started decoding but haven't
// finished and thus isn't available in coder->this_index yet.
// Memory usage and limit information needs to be communicated
// from/to coder->index_decoder.
//
// Care has to be taken to not do both (2) and (3) when calculating
// the memory usage.
uint64_t combined_index_memusage = 0;
uint64_t this_index_memusage = 0;
// (1) If we have already successfully decoded one or more Indexes,
// get their memory usage.
if (coder->combined_index != NULL)
combined_index_memusage = lzma_index_memused(
coder->combined_index);
// Choose between (2), (3), or neither.
if (coder->this_index != NULL) {
// (2) The latest Index is available. Use its memory usage.
this_index_memusage = lzma_index_memused(coder->this_index);
} else if (coder->sequence == SEQ_INDEX_DECODE) {
// (3) The Index decoder is activate and hasn't yet stored
// the new index in coder->this_index. Get the memory usage
// information from the Index decoder.
//
// NOTE: If the Index decoder doesn't yet know how much memory
// it will eventually need, it will return a tiny value here.
uint64_t dummy;
if (coder->index_decoder.memconfig(coder->index_decoder.coder,
&this_index_memusage, &dummy, 0)
!= LZMA_OK) {
assert(0);
return LZMA_PROG_ERROR;
}
}
// Now we know the total memory usage/requirement. If we had neither
// old Indexes nor a new Index, this will be zero which isn't
// acceptable as lzma_memusage() has to return non-zero on success
// and even with an empty .xz file we will end up with a lzma_index
// that takes some memory.
*memusage = combined_index_memusage + this_index_memusage;
if (*memusage == 0)
*memusage = lzma_index_memusage(1, 0);
*old_memlimit = coder->memlimit;
// If requested, set a new memory usage limit.
if (new_memlimit != 0) {
if (new_memlimit < *memusage)
return LZMA_MEMLIMIT_ERROR;
// In the condition (3) we need to tell the Index decoder
// its new memory usage limit.
if (coder->this_index == NULL
&& coder->sequence == SEQ_INDEX_DECODE) {
const uint64_t idec_new_memlimit = new_memlimit
- combined_index_memusage;
assert(this_index_memusage > 0);
assert(idec_new_memlimit > 0);
uint64_t dummy1;
uint64_t dummy2;
if (coder->index_decoder.memconfig(
coder->index_decoder.coder,
&dummy1, &dummy2, idec_new_memlimit)
!= LZMA_OK) {
assert(0);
return LZMA_PROG_ERROR;
}
}
coder->memlimit = new_memlimit;
}
return LZMA_OK;
}
static void
file_info_decoder_end(void *coder_ptr, const lzma_allocator *allocator)
{
lzma_file_info_coder *coder = coder_ptr;
lzma_next_end(&coder->index_decoder, allocator);
lzma_index_end(coder->this_index, allocator);
lzma_index_end(coder->combined_index, allocator);
lzma_free(coder, allocator);
return;
}
static lzma_ret
lzma_file_info_decoder_init(lzma_next_coder *next,
const lzma_allocator *allocator, uint64_t *seek_pos,
lzma_index **dest_index,
uint64_t memlimit, uint64_t file_size)
{
lzma_next_coder_init(&lzma_file_info_decoder_init, next, allocator);
if (dest_index == NULL)
return LZMA_PROG_ERROR;
lzma_file_info_coder *coder = next->coder;
if (coder == NULL) {
coder = lzma_alloc(sizeof(lzma_file_info_coder), allocator);
if (coder == NULL)
return LZMA_MEM_ERROR;
next->coder = coder;
next->code = &file_info_decode;
next->end = &file_info_decoder_end;
next->memconfig = &file_info_decoder_memconfig;
coder->index_decoder = LZMA_NEXT_CODER_INIT;
coder->this_index = NULL;
coder->combined_index = NULL;
}
coder->sequence = SEQ_MAGIC_BYTES;
coder->file_cur_pos = 0;
coder->file_target_pos = 0;
coder->file_size = file_size;
lzma_index_end(coder->this_index, allocator);
coder->this_index = NULL;
lzma_index_end(coder->combined_index, allocator);
coder->combined_index = NULL;
coder->stream_padding = 0;
coder->dest_index = dest_index;
coder->external_seek_pos = seek_pos;
// If memlimit is 0, make it 1 to ensure that lzma_memlimit_get()
// won't return 0 (which would indicate an error).
coder->memlimit = my_max(1, memlimit);
// Prepare these for reading the first Stream Header into coder->temp.
coder->temp_pos = 0;
coder->temp_size = LZMA_STREAM_HEADER_SIZE;
return LZMA_OK;
}
extern LZMA_API(lzma_ret)
lzma_file_info_decoder(lzma_stream *strm, lzma_index **dest_index,
uint64_t memlimit, uint64_t file_size)
{
lzma_next_strm_init(lzma_file_info_decoder_init, strm, &strm->seek_pos,
dest_index, memlimit, file_size);
// We allow LZMA_FINISH in addition to LZMA_RUN for convenience.
// lzma_code() is able to handle the LZMA_FINISH + LZMA_SEEK_NEEDED
// combination in a sane way. Applications still need to be careful
// if they use LZMA_FINISH so that they remember to reset it back
// to LZMA_RUN after seeking if needed.
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
return LZMA_OK;
}

7
Externals/liblzma/common/filter_buffer_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_buffer_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"
@ -24,7 +23,7 @@ lzma_raw_buffer_decode(
|| out_pos == NULL || *out_pos > out_size)
return LZMA_PROG_ERROR;
// Initialize the decoer.
// Initialize the decoder.
lzma_next_coder next = LZMA_NEXT_CODER_INIT;
return_if_error(lzma_raw_decoder_init(&next, allocator, filters));

5
Externals/liblzma/common/filter_buffer_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_buffer_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"

82
Externals/liblzma/common/filter_common.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_common.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_common.h"
@ -42,6 +41,13 @@ static const struct {
.last_ok = true,
.changes_size = true,
},
{
.id = LZMA_FILTER_LZMA1EXT,
.options_size = sizeof(lzma_options_lzma),
.non_last_ok = false,
.last_ok = true,
.changes_size = true,
},
#endif
#if defined(HAVE_ENCODER_LZMA2) || defined(HAVE_DECODER_LZMA2)
{
@ -97,6 +103,15 @@ static const struct {
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_ARM64) || defined(HAVE_DECODER_ARM64)
{
.id = LZMA_FILTER_ARM64,
.options_size = sizeof(lzma_options_bcj),
.non_last_ok = true,
.last_ok = false,
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_SPARC) || defined(HAVE_DECODER_SPARC)
{
.id = LZMA_FILTER_SPARC,
@ -106,6 +121,15 @@ static const struct {
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_RISCV) || defined(HAVE_DECODER_RISCV)
{
.id = LZMA_FILTER_RISCV,
.options_size = sizeof(lzma_options_bcj),
.non_last_ok = true,
.last_ok = false,
.changes_size = false,
},
#endif
#if defined(HAVE_ENCODER_DELTA) || defined(HAVE_DECODER_DELTA)
{
.id = LZMA_FILTER_DELTA,
@ -122,12 +146,16 @@ static const struct {
extern LZMA_API(lzma_ret)
lzma_filters_copy(const lzma_filter *src, lzma_filter *dest,
lzma_filters_copy(const lzma_filter *src, lzma_filter *real_dest,
const lzma_allocator *allocator)
{
if (src == NULL || dest == NULL)
if (src == NULL || real_dest == NULL)
return LZMA_PROG_ERROR;
// Use a temporary destination so that the real destination
// will never be modified if an error occurs.
lzma_filter dest[LZMA_FILTERS_MAX + 1];
lzma_ret ret;
size_t i;
for (i = 0; src[i].id != LZMA_VLI_UNKNOWN; ++i) {
@ -173,25 +201,53 @@ lzma_filters_copy(const lzma_filter *src, lzma_filter *dest,
}
// Terminate the filter array.
assert(i <= LZMA_FILTERS_MAX + 1);
assert(i < LZMA_FILTERS_MAX + 1);
dest[i].id = LZMA_VLI_UNKNOWN;
dest[i].options = NULL;
// Copy it to the caller-supplied array now that we know that
// no errors occurred.
memcpy(real_dest, dest, (i + 1) * sizeof(lzma_filter));
return LZMA_OK;
error:
// Free the options which we have already allocated.
while (i-- > 0) {
while (i-- > 0)
lzma_free(dest[i].options, allocator);
dest[i].options = NULL;
}
return ret;
}
static lzma_ret
validate_chain(const lzma_filter *filters, size_t *count)
extern LZMA_API(void)
lzma_filters_free(lzma_filter *filters, const lzma_allocator *allocator)
{
if (filters == NULL)
return;
for (size_t i = 0; filters[i].id != LZMA_VLI_UNKNOWN; ++i) {
if (i == LZMA_FILTERS_MAX) {
// The API says that LZMA_FILTERS_MAX + 1 is the
// maximum allowed size including the terminating
// element. Thus, we should never get here but in
// case there is a bug and we do anyway, don't go
// past the (probable) end of the array.
assert(0);
break;
}
lzma_free(filters[i].options, allocator);
filters[i].options = NULL;
filters[i].id = LZMA_VLI_UNKNOWN;
}
return;
}
extern lzma_ret
lzma_validate_chain(const lzma_filter *filters, size_t *count)
{
// There must be at least one filter.
if (filters == NULL || filters[0].id == LZMA_VLI_UNKNOWN)
@ -245,7 +301,7 @@ lzma_raw_coder_init(lzma_next_coder *next, const lzma_allocator *allocator,
{
// Do some basic validation and get the number of filters.
size_t count;
return_if_error(validate_chain(options, &count));
return_if_error(lzma_validate_chain(options, &count));
// Set the filter functions and copy the options pointer.
lzma_filter_info filters[LZMA_FILTERS_MAX + 1];
@ -298,7 +354,7 @@ lzma_raw_coder_memusage(lzma_filter_find coder_find,
// The chain has to have at least one filter.
{
size_t tmp;
if (validate_chain(filters, &tmp) != LZMA_OK)
if (lzma_validate_chain(filters, &tmp) != LZMA_OK)
return UINT64_MAX;
}

10
Externals/liblzma/common/filter_common.h vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_common.c
/// \file filter_common.h
/// \brief Filter-specific stuff common for both encoder and decoder
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_COMMON_H
@ -35,6 +34,9 @@ typedef struct {
typedef const lzma_filter_coder *(*lzma_filter_find)(lzma_vli id);
extern lzma_ret lzma_validate_chain(const lzma_filter *filters, size_t *count);
extern lzma_ret lzma_raw_coder_init(
lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *filters,

42
Externals/liblzma/common/filter_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"
@ -50,6 +49,12 @@ static const lzma_filter_decoder decoders[] = {
.memusage = &lzma_lzma_decoder_memusage,
.props_decode = &lzma_lzma_props_decode,
},
{
.id = LZMA_FILTER_LZMA1EXT,
.init = &lzma_lzma_decoder_init,
.memusage = &lzma_lzma_decoder_memusage,
.props_decode = &lzma_lzma_props_decode,
},
#endif
#ifdef HAVE_DECODER_LZMA2
{
@ -99,6 +104,14 @@ static const lzma_filter_decoder decoders[] = {
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_ARM64
{
.id = LZMA_FILTER_ARM64,
.init = &lzma_simple_arm64_decoder_init,
.memusage = NULL,
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_SPARC
{
.id = LZMA_FILTER_SPARC,
@ -107,6 +120,14 @@ static const lzma_filter_decoder decoders[] = {
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_RISCV
{
.id = LZMA_FILTER_RISCV,
.init = &lzma_simple_riscv_decoder_init,
.memusage = NULL,
.props_decode = &lzma_simple_props_decode,
},
#endif
#ifdef HAVE_DECODER_DELTA
{
.id = LZMA_FILTER_DELTA,
@ -129,6 +150,16 @@ decoder_find(lzma_vli id)
}
// lzma_filter_coder begins with the same members as lzma_filter_decoder.
// This function is a wrapper with a type that is compatible with the
// typedef of lzma_filter_find in filter_common.h.
static const lzma_filter_coder *
coder_find(lzma_vli id)
{
return (const lzma_filter_coder *)decoder_find(id);
}
extern LZMA_API(lzma_bool)
lzma_filter_decoder_is_supported(lzma_vli id)
{
@ -141,7 +172,7 @@ lzma_raw_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *options)
{
return lzma_raw_coder_init(next, allocator,
options, (lzma_filter_find)(&decoder_find), false);
options, &coder_find, false);
}
@ -160,8 +191,7 @@ lzma_raw_decoder(lzma_stream *strm, const lzma_filter *options)
extern LZMA_API(uint64_t)
lzma_raw_decoder_memusage(const lzma_filter *filters)
{
return lzma_raw_coder_memusage(
(lzma_filter_find)(&decoder_find), filters);
return lzma_raw_coder_memusage(&coder_find, filters);
}

7
Externals/liblzma/common/filter_decoder.h vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.c
/// \file filter_decoder.h
/// \brief Filter ID mapping to filter-specific functions
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_DECODER_H

84
Externals/liblzma/common/filter_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"
@ -33,13 +32,17 @@ typedef struct {
/// Calculates the recommended Uncompressed Size for .xz Blocks to
/// which the input data can be split to make multithreaded
/// encoding possible. If this is NULL, it is assumed that
/// the encoder is fast enough with single thread.
/// the encoder is fast enough with single thread. If the options
/// are invalid, UINT64_MAX is returned.
uint64_t (*block_size)(const void *options);
/// Tells the size of the Filter Properties field. If options are
/// invalid, UINT32_MAX is returned. If this is NULL, props_size_fixed
/// is used.
/// invalid, LZMA_OPTIONS_ERROR is returned and size is set to
/// UINT32_MAX.
lzma_ret (*props_size_get)(uint32_t *size, const void *options);
/// Some filters will always have the same size Filter Properties
/// field. If props_size_get is NULL, this value is used.
uint32_t props_size_fixed;
/// Encodes Filter Properties.
@ -59,7 +62,16 @@ static const lzma_filter_encoder encoders[] = {
.id = LZMA_FILTER_LZMA1,
.init = &lzma_lzma_encoder_init,
.memusage = &lzma_lzma_encoder_memusage,
.block_size = NULL, // FIXME
.block_size = NULL, // Not needed for LZMA1
.props_size_get = NULL,
.props_size_fixed = 5,
.props_encode = &lzma_lzma_props_encode,
},
{
.id = LZMA_FILTER_LZMA1EXT,
.init = &lzma_lzma_encoder_init,
.memusage = &lzma_lzma_encoder_memusage,
.block_size = NULL, // Not needed for LZMA1
.props_size_get = NULL,
.props_size_fixed = 5,
.props_encode = &lzma_lzma_props_encode,
@ -70,7 +82,7 @@ static const lzma_filter_encoder encoders[] = {
.id = LZMA_FILTER_LZMA2,
.init = &lzma_lzma2_encoder_init,
.memusage = &lzma_lzma2_encoder_memusage,
.block_size = &lzma_lzma2_block_size, // FIXME
.block_size = &lzma_lzma2_block_size,
.props_size_get = NULL,
.props_size_fixed = 1,
.props_encode = &lzma_lzma2_props_encode,
@ -126,6 +138,16 @@ static const lzma_filter_encoder encoders[] = {
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_ARM64
{
.id = LZMA_FILTER_ARM64,
.init = &lzma_simple_arm64_encoder_init,
.memusage = NULL,
.block_size = NULL,
.props_size_get = &lzma_simple_props_size,
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_SPARC
{
.id = LZMA_FILTER_SPARC,
@ -136,6 +158,16 @@ static const lzma_filter_encoder encoders[] = {
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_RISCV
{
.id = LZMA_FILTER_RISCV,
.init = &lzma_simple_riscv_encoder_init,
.memusage = NULL,
.block_size = NULL,
.props_size_get = &lzma_simple_props_size,
.props_encode = &lzma_simple_props_encode,
},
#endif
#ifdef HAVE_ENCODER_DELTA
{
.id = LZMA_FILTER_DELTA,
@ -161,6 +193,16 @@ encoder_find(lzma_vli id)
}
// lzma_filter_coder begins with the same members as lzma_filter_encoder.
// This function is a wrapper with a type that is compatible with the
// typedef of lzma_filter_find in filter_common.h.
static const lzma_filter_coder *
coder_find(lzma_vli id)
{
return (const lzma_filter_coder *)encoder_find(id);
}
extern LZMA_API(lzma_bool)
lzma_filter_encoder_is_supported(lzma_vli id)
{
@ -197,18 +239,18 @@ lzma_filters_update(lzma_stream *strm, const lzma_filter *filters)
extern lzma_ret
lzma_raw_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *options)
const lzma_filter *filters)
{
return lzma_raw_coder_init(next, allocator,
options, (lzma_filter_find)(&encoder_find), true);
filters, &coder_find, true);
}
extern LZMA_API(lzma_ret)
lzma_raw_encoder(lzma_stream *strm, const lzma_filter *options)
lzma_raw_encoder(lzma_stream *strm, const lzma_filter *filters)
{
lzma_next_strm_init(lzma_raw_coder_init, strm, options,
(lzma_filter_find)(&encoder_find), true);
lzma_next_strm_init(lzma_raw_coder_init, strm, filters,
&coder_find, true);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;
@ -221,31 +263,33 @@ lzma_raw_encoder(lzma_stream *strm, const lzma_filter *options)
extern LZMA_API(uint64_t)
lzma_raw_encoder_memusage(const lzma_filter *filters)
{
return lzma_raw_coder_memusage(
(lzma_filter_find)(&encoder_find), filters);
return lzma_raw_coder_memusage(&coder_find, filters);
}
extern uint64_t
extern LZMA_API(uint64_t)
lzma_mt_block_size(const lzma_filter *filters)
{
if (filters == NULL)
return UINT64_MAX;
uint64_t max = 0;
for (size_t i = 0; filters[i].id != LZMA_VLI_UNKNOWN; ++i) {
const lzma_filter_encoder *const fe
= encoder_find(filters[i].id);
if (fe == NULL)
return UINT64_MAX;
if (fe->block_size != NULL) {
const uint64_t size
= fe->block_size(filters[i].options);
if (size == 0)
return 0;
if (size > max)
max = size;
}
}
return max;
return max == 0 ? UINT64_MAX : max;
}

11
Externals/liblzma/common/filter_encoder.h vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_encoder.c
/// \file filter_encoder.h
/// \brief Filter ID mapping to filter-specific functions
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_FILTER_ENCODER_H
@ -16,10 +15,6 @@
#include "common.h"
// FIXME: Might become a part of the public API.
extern uint64_t lzma_mt_block_size(const lzma_filter *filters);
extern lzma_ret lzma_raw_encoder_init(
lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_filter *filters);

5
Externals/liblzma/common/filter_flags_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_flags_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_decoder.h"

7
Externals/liblzma/common/filter_flags_encoder.c vendored
View file

@ -1,13 +1,12 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file filter_flags_encoder.c
/// \brief Decodes a Filter Flags field
/// \brief Encodes a Filter Flags field
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"

17
Externals/liblzma/common/hardware_cputhreads.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file hardware_cputhreads.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -15,6 +14,18 @@
#include "tuklib_cpucores.h"
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
// This is for compatibility with binaries linked against liblzma that
// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7.
LZMA_SYMVER_API("lzma_cputhreads@XZ_5.2.2",
uint32_t, lzma_cputhreads_522)(void) lzma_nothrow
__attribute__((__alias__("lzma_cputhreads_52")));
LZMA_SYMVER_API("lzma_cputhreads@@XZ_5.2",
uint32_t, lzma_cputhreads_52)(void) lzma_nothrow;
#define lzma_cputhreads lzma_cputhreads_52
#endif
extern LZMA_API(uint32_t)
lzma_cputhreads(void)
{

7
Externals/liblzma/common/hardware_physmem.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file hardware_physmem.c
@ -5,9 +7,6 @@
//
// Author: Jonathan Nieder
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -19,7 +18,7 @@ extern LZMA_API(uint64_t)
lzma_physmem(void)
{
// It is simpler to make lzma_physmem() a wrapper for
// tuklib_physmem() than to hack appropriate symbol visiblity
// tuklib_physmem() than to hack appropriate symbol visibility
// support for the tuklib modules.
return tuklib_physmem();
}

34
Externals/liblzma/common/index.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index.c
@ -5,11 +7,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
#include "index.h"
#include "stream_flags_common.h"
@ -105,7 +105,7 @@ typedef struct {
typedef struct {
/// Every index_stream is a node in the tree of Sreams.
/// Every index_stream is a node in the tree of Streams.
index_tree_node node;
/// Number of this Stream (first one is 1)
@ -166,7 +166,7 @@ struct lzma_index_s {
lzma_vli index_list_size;
/// How many Records to allocate at once in lzma_index_append().
/// This defaults to INDEX_GROUP_SIZE but can be overriden with
/// This defaults to INDEX_GROUP_SIZE but can be overridden with
/// lzma_index_prealloc().
size_t prealloc;
@ -656,6 +656,16 @@ lzma_index_append(lzma_index *i, const lzma_allocator *allocator,
const uint32_t index_list_size_add = lzma_vli_size(unpadded_size)
+ lzma_vli_size(uncompressed_size);
// Check that uncompressed size will not overflow.
if (uncompressed_base + uncompressed_size > LZMA_VLI_MAX)
return LZMA_DATA_ERROR;
// Check that the new unpadded sum will not overflow. This is
// checked again in index_file_size(), but the unpadded sum is
// passed to vli_ceil4() which expects a valid lzma_vli value.
if (compressed_base + unpadded_size > UNPADDED_SIZE_MAX)
return LZMA_DATA_ERROR;
// Check that the file size will stay within limits.
if (index_file_size(s->node.compressed_base,
compressed_base + unpadded_size, s->record_count + 1,
@ -767,6 +777,9 @@ extern LZMA_API(lzma_ret)
lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src,
const lzma_allocator *allocator)
{
if (dest == NULL || src == NULL)
return LZMA_PROG_ERROR;
const lzma_vli dest_file_size = lzma_index_file_size(dest);
// Check that we don't exceed the file size limits.
@ -825,7 +838,7 @@ lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src,
s->groups.root = &newg->node;
}
if (s->groups.rightmost == &g->node)
assert(s->groups.rightmost == &g->node);
s->groups.rightmost = &newg->node;
lzma_free(g, allocator);
@ -835,6 +848,11 @@ lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src,
}
}
// dest->checks includes the check types of all except the last Stream
// in dest. Set the bit for the check type of the last Stream now so
// that it won't get lost when Stream(s) from src are appended to dest.
dest->checks = lzma_index_checks(dest);
// Add all the Streams from src to dest. Update the base offsets
// of each Stream from src.
const index_cat_info info = {
@ -851,7 +869,7 @@ lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src,
dest->total_size += src->total_size;
dest->record_count += src->record_count;
dest->index_list_size += src->index_list_size;
dest->checks = lzma_index_checks(dest) | src->checks;
dest->checks |= src->checks;
// There's nothing else left in src than the base structure.
lzma_free(src, allocator);
@ -1226,7 +1244,7 @@ lzma_index_iter_locate(lzma_index_iter *iter, lzma_vli target)
// Use binary search to locate the exact Record. It is the first
// Record whose uncompressed_sum is greater than target.
// This is because we want the rightmost Record that fullfills the
// This is because we want the rightmost Record that fulfills the
// search criterion. It is possible that there are empty Blocks;
// we don't want to return them.
size_t left = 0;

19
Externals/liblzma/common/index.h vendored
View file

@ -1,20 +1,24 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index.h
/// \brief Handling of Index
/// \note This header file does not include common.h or lzma.h because
/// this file is needed by both liblzma internally and by the
/// tests. Including common.h will include and define many things
/// the tests do not need and prevents issues with header file
/// include order. This way, if lzma.h or common.h are not
/// included before this file it will break on every OS instead
/// of causing more subtle errors.
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_INDEX_H
#define LZMA_INDEX_H
#include "common.h"
/// Minimum Unpadded Size
#define UNPADDED_SIZE_MIN LZMA_VLI_C(5)
@ -22,6 +26,9 @@
/// Maximum Unpadded Size
#define UNPADDED_SIZE_MAX (LZMA_VLI_MAX & ~LZMA_VLI_C(3))
/// Index Indicator based on xz specification
#define INDEX_INDICATOR 0
/// Get the size of the Index Padding field. This is needed by Index encoder
/// and decoder, but applications should have no use for this.
@ -38,7 +45,7 @@ extern void lzma_index_prealloc(lzma_index *i, lzma_vli records);
static inline lzma_vli
vli_ceil4(lzma_vli vli)
{
assert(vli <= LZMA_VLI_MAX);
assert(vli <= UNPADDED_SIZE_MAX);
return (vli + 3) & ~LZMA_VLI_C(3);
}

44
Externals/liblzma/common/index_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index_decoder.c
@ -5,12 +7,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "index.h"
#include "index_decoder.h"
#include "check.h"
@ -80,7 +79,7 @@ index_decode(void *coder_ptr, const lzma_allocator *allocator,
// format". One could argue that the application should
// verify the Index Indicator before trying to decode the
// Index, but well, I suppose it is simpler this way.
if (in[(*in_pos)++] != 0x00)
if (in[(*in_pos)++] != INDEX_INDICATOR)
return LZMA_DATA_ERROR;
coder->sequence = SEQ_COUNT;
@ -180,8 +179,11 @@ index_decode(void *coder_ptr, const lzma_allocator *allocator,
return LZMA_OK;
if (((coder->crc32 >> (coder->pos * 8)) & 0xFF)
!= in[(*in_pos)++])
!= in[(*in_pos)++]) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return LZMA_DATA_ERROR;
#endif
}
} while (++coder->pos < 4);
@ -200,9 +202,16 @@ index_decode(void *coder_ptr, const lzma_allocator *allocator,
}
out:
// Update the CRC32,
// Update the CRC32.
//
// Avoid null pointer + 0 (undefined behavior) in "in + in_start".
// In such a case we had no input and thus in_used == 0.
{
const size_t in_used = *in_pos - in_start;
if (in_used > 0)
coder->crc32 = lzma_crc32(in + in_start,
*in_pos - in_start, coder->crc32);
in_used, coder->crc32);
}
return ret;
}
@ -265,11 +274,11 @@ index_decoder_reset(lzma_index_coder *coder, const lzma_allocator *allocator,
}
static lzma_ret
index_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
extern lzma_ret
lzma_index_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
lzma_index **i, uint64_t memlimit)
{
lzma_next_coder_init(&index_decoder_init, next, allocator);
lzma_next_coder_init(&lzma_index_decoder_init, next, allocator);
if (i == NULL)
return LZMA_PROG_ERROR;
@ -296,7 +305,13 @@ index_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
extern LZMA_API(lzma_ret)
lzma_index_decoder(lzma_stream *strm, lzma_index **i, uint64_t memlimit)
{
lzma_next_strm_init(index_decoder_init, strm, i, memlimit);
// If i isn't NULL, *i must always be initialized due to
// the wording in the API docs. This way it is initialized
// if we return LZMA_PROG_ERROR due to strm == NULL.
if (i != NULL)
*i = NULL;
lzma_next_strm_init(lzma_index_decoder_init, strm, i, memlimit);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
@ -310,6 +325,11 @@ lzma_index_buffer_decode(lzma_index **i, uint64_t *memlimit,
const lzma_allocator *allocator,
const uint8_t *in, size_t *in_pos, size_t in_size)
{
// If i isn't NULL, *i must always be initialized due to
// the wording in the API docs.
if (i != NULL)
*i = NULL;
// Sanity checks
if (i == NULL || memlimit == NULL
|| in == NULL || in_pos == NULL || *in_pos > in_size)

24
Externals/liblzma/common/index_decoder.h vendored Normal file
View file

@ -0,0 +1,24 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index_decoder.h
/// \brief Decodes the Index field
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_INDEX_DECODER_H
#define LZMA_INDEX_DECODER_H
#include "common.h"
#include "index.h"
extern lzma_ret lzma_index_decoder_init(lzma_next_coder *next,
const lzma_allocator *allocator,
lzma_index **i, uint64_t memlimit);
#endif

16
Externals/liblzma/common/index_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "index_encoder.h"
@ -65,7 +64,7 @@ index_encode(void *coder_ptr,
while (*out_pos < out_size)
switch (coder->sequence) {
case SEQ_INDICATOR:
out[*out_pos] = 0x00;
out[*out_pos] = INDEX_INDICATOR;
++*out_pos;
coder->sequence = SEQ_COUNT;
break;
@ -153,8 +152,15 @@ index_encode(void *coder_ptr,
out:
// Update the CRC32.
//
// Avoid null pointer + 0 (undefined behavior) in "out + out_start".
// In such a case we had no input and thus out_used == 0.
{
const size_t out_used = *out_pos - out_start;
if (out_used > 0)
coder->crc32 = lzma_crc32(out + out_start,
*out_pos - out_start, coder->crc32);
out_used, coder->crc32);
}
return ret;
}

5
Externals/liblzma/common/index_encoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index_encoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_INDEX_ENCODER_H

36
Externals/liblzma/common/index_hash.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file index_hash.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -122,7 +121,7 @@ lzma_index_hash_size(const lzma_index_hash *index_hash)
/// Updates the sizes and the hash without any validation.
static lzma_ret
static void
hash_append(lzma_index_hash_info *info, lzma_vli unpadded_size,
lzma_vli uncompressed_size)
{
@ -136,7 +135,7 @@ hash_append(lzma_index_hash_info *info, lzma_vli unpadded_size,
lzma_check_update(&info->check, LZMA_CHECK_BEST,
(const uint8_t *)(sizes), sizeof(sizes));
return LZMA_OK;
return;
}
@ -145,15 +144,14 @@ lzma_index_hash_append(lzma_index_hash *index_hash, lzma_vli unpadded_size,
lzma_vli uncompressed_size)
{
// Validate the arguments.
if (index_hash->sequence != SEQ_BLOCK
if (index_hash == NULL || index_hash->sequence != SEQ_BLOCK
|| unpadded_size < UNPADDED_SIZE_MIN
|| unpadded_size > UNPADDED_SIZE_MAX
|| uncompressed_size > LZMA_VLI_MAX)
return LZMA_PROG_ERROR;
// Update the hash.
return_if_error(hash_append(&index_hash->blocks,
unpadded_size, uncompressed_size));
hash_append(&index_hash->blocks, unpadded_size, uncompressed_size);
// Validate the properties of *info are still in allowed limits.
if (index_hash->blocks.blocks_size > LZMA_VLI_MAX
@ -191,7 +189,7 @@ lzma_index_hash_decode(lzma_index_hash *index_hash, const uint8_t *in,
switch (index_hash->sequence) {
case SEQ_BLOCK:
// Check the Index Indicator is present.
if (in[(*in_pos)++] != 0x00)
if (in[(*in_pos)++] != INDEX_INDICATOR)
return LZMA_DATA_ERROR;
index_hash->sequence = SEQ_COUNT;
@ -239,9 +237,9 @@ lzma_index_hash_decode(lzma_index_hash *index_hash, const uint8_t *in,
index_hash->sequence = SEQ_UNCOMPRESSED;
} else {
// Update the hash.
return_if_error(hash_append(&index_hash->records,
hash_append(&index_hash->records,
index_hash->unpadded_size,
index_hash->uncompressed_size));
index_hash->uncompressed_size);
// Verify that we don't go over the known sizes. Note
// that this validation is simpler than the one used
@ -313,8 +311,11 @@ lzma_index_hash_decode(lzma_index_hash *index_hash, const uint8_t *in,
return LZMA_OK;
if (((index_hash->crc32 >> (index_hash->pos * 8))
& 0xFF) != in[(*in_pos)++])
& 0xFF) != in[(*in_pos)++]) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return LZMA_DATA_ERROR;
#endif
}
} while (++index_hash->pos < 4);
@ -326,9 +327,16 @@ lzma_index_hash_decode(lzma_index_hash *index_hash, const uint8_t *in,
}
out:
// Update the CRC32,
// Update the CRC32.
//
// Avoid null pointer + 0 (undefined behavior) in "in + in_start".
// In such a case we had no input and thus in_used == 0.
{
const size_t in_used = *in_pos - in_start;
if (in_used > 0)
index_hash->crc32 = lzma_crc32(in + in_start,
*in_pos - in_start, index_hash->crc32);
in_used, index_hash->crc32);
}
return ret;
}

417
Externals/liblzma/common/lzip_decoder.c vendored Normal file
View file

@ -0,0 +1,417 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file lzip_decoder.c
/// \brief Decodes .lz (lzip) files
//
// Author: Michał Górny
// Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "lzip_decoder.h"
#include "lzma_decoder.h"
#include "check.h"
// .lz format version 0 lacks the 64-bit Member size field in the footer.
#define LZIP_V0_FOOTER_SIZE 12
#define LZIP_V1_FOOTER_SIZE 20
#define LZIP_FOOTER_SIZE_MAX LZIP_V1_FOOTER_SIZE
// lc/lp/pb are hardcoded in the .lz format.
#define LZIP_LC 3
#define LZIP_LP 0
#define LZIP_PB 2
typedef struct {
enum {
SEQ_ID_STRING,
SEQ_VERSION,
SEQ_DICT_SIZE,
SEQ_CODER_INIT,
SEQ_LZMA_STREAM,
SEQ_MEMBER_FOOTER,
} sequence;
/// .lz member format version
uint32_t version;
/// CRC32 of the uncompressed data in the .lz member
uint32_t crc32;
/// Uncompressed size of the .lz member
uint64_t uncompressed_size;
/// Compressed size of the .lz member
uint64_t member_size;
/// Memory usage limit
uint64_t memlimit;
/// Amount of memory actually needed
uint64_t memusage;
/// If true, LZMA_GET_CHECK is returned after decoding the header
/// fields. As all files use CRC32 this is redundant but it's
/// implemented anyway since the initialization functions supports
/// all other flags in addition to LZMA_TELL_ANY_CHECK.
bool tell_any_check;
/// If true, we won't calculate or verify the CRC32 of
/// the uncompressed data.
bool ignore_check;
/// If true, we will decode concatenated .lz members and stop if
/// non-.lz data is seen after at least one member has been
/// successfully decoded.
bool concatenated;
/// When decoding concatenated .lz members, this is true as long as
/// we are decoding the first .lz member. This is needed to avoid
/// incorrect LZMA_FORMAT_ERROR in case there is non-.lz data at
/// the end of the file.
bool first_member;
/// Reading position in the header and footer fields
size_t pos;
/// Buffer to hold the .lz footer fields
uint8_t buffer[LZIP_FOOTER_SIZE_MAX];
/// Options decoded from the .lz header that needed to initialize
/// the LZMA1 decoder.
lzma_options_lzma options;
/// LZMA1 decoder
lzma_next_coder lzma_decoder;
} lzma_lzip_coder;
static lzma_ret
lzip_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size, lzma_action action)
{
lzma_lzip_coder *coder = coder_ptr;
while (true)
switch (coder->sequence) {
case SEQ_ID_STRING: {
// The "ID string" or magic bytes are "LZIP" in US-ASCII.
const uint8_t lzip_id_string[4] = { 0x4C, 0x5A, 0x49, 0x50 };
while (coder->pos < sizeof(lzip_id_string)) {
if (*in_pos >= in_size) {
// If we are on the 2nd+ concatenated member
// and the input ends before we can read
// the magic bytes, we discard the bytes that
// were already read (up to 3) and finish.
// See the reasoning below.
return !coder->first_member
&& action == LZMA_FINISH
? LZMA_STREAM_END : LZMA_OK;
}
if (in[*in_pos] != lzip_id_string[coder->pos]) {
// The .lz format allows putting non-.lz data
// at the end of the file. If we have seen
// at least one valid .lz member already,
// then we won't consume the byte at *in_pos
// and will return LZMA_STREAM_END. This way
// apps can easily locate and read the non-.lz
// data after the .lz member(s).
//
// NOTE: If the first 1-3 bytes of the non-.lz
// data match the .lz ID string then the first
// 1-3 bytes of the junk will get ignored by
// us. If apps want to properly locate the
// trailing data they must ensure that the
// first byte of their custom data isn't the
// same as the first byte of .lz ID string.
// With the liblzma API we cannot rewind the
// input position across calls to lzma_code().
return !coder->first_member
? LZMA_STREAM_END : LZMA_FORMAT_ERROR;
}
++*in_pos;
++coder->pos;
}
coder->pos = 0;
coder->crc32 = 0;
coder->uncompressed_size = 0;
coder->member_size = sizeof(lzip_id_string);
coder->sequence = SEQ_VERSION;
}
// Fall through
case SEQ_VERSION:
if (*in_pos >= in_size)
return LZMA_OK;
coder->version = in[(*in_pos)++];
// We support version 0 and unextended version 1.
if (coder->version > 1)
return LZMA_OPTIONS_ERROR;
++coder->member_size;
coder->sequence = SEQ_DICT_SIZE;
// .lz versions 0 and 1 use CRC32 as the integrity check
// so if the application wanted to know that
// (LZMA_TELL_ANY_CHECK) we can tell it now.
if (coder->tell_any_check)
return LZMA_GET_CHECK;
// Fall through
case SEQ_DICT_SIZE: {
if (*in_pos >= in_size)
return LZMA_OK;
const uint32_t ds = in[(*in_pos)++];
++coder->member_size;
// The five lowest bits are for the base-2 logarithm of
// the dictionary size and the highest three bits are
// the fractional part (0/16 to 7/16) that will be
// subtracted to get the final value.
//
// For example, with 0xB5:
// b2log = 21
// fracnum = 5
// dict_size = 2^21 - 2^21 * 5 / 16 = 1408 KiB
const uint32_t b2log = ds & 0x1F;
const uint32_t fracnum = ds >> 5;
// The format versions 0 and 1 allow dictionary size in the
// range [4 KiB, 512 MiB].
if (b2log < 12 || b2log > 29 || (b2log == 12 && fracnum > 0))
return LZMA_DATA_ERROR;
// 2^[b2log] - 2^[b2log] * [fracnum] / 16
// = 2^[b2log] - [fracnum] * 2^([b2log] - 4)
coder->options.dict_size = (UINT32_C(1) << b2log)
- (fracnum << (b2log - 4));
assert(coder->options.dict_size >= 4096);
assert(coder->options.dict_size <= (UINT32_C(512) << 20));
coder->options.preset_dict = NULL;
coder->options.lc = LZIP_LC;
coder->options.lp = LZIP_LP;
coder->options.pb = LZIP_PB;
// Calculate the memory usage.
coder->memusage = lzma_lzma_decoder_memusage(&coder->options)
+ LZMA_MEMUSAGE_BASE;
// Initialization is a separate step because if we return
// LZMA_MEMLIMIT_ERROR we need to be able to restart after
// the memlimit has been increased.
coder->sequence = SEQ_CODER_INIT;
}
// Fall through
case SEQ_CODER_INIT: {
if (coder->memusage > coder->memlimit)
return LZMA_MEMLIMIT_ERROR;
const lzma_filter_info filters[2] = {
{
.id = LZMA_FILTER_LZMA1,
.init = &lzma_lzma_decoder_init,
.options = &coder->options,
}, {
.init = NULL,
}
};
return_if_error(lzma_next_filter_init(&coder->lzma_decoder,
allocator, filters));
coder->crc32 = 0;
coder->sequence = SEQ_LZMA_STREAM;
}
// Fall through
case SEQ_LZMA_STREAM: {
const size_t in_start = *in_pos;
const size_t out_start = *out_pos;
const lzma_ret ret = coder->lzma_decoder.code(
coder->lzma_decoder.coder, allocator,
in, in_pos, in_size, out, out_pos, out_size,
action);
const size_t out_used = *out_pos - out_start;
coder->member_size += *in_pos - in_start;
coder->uncompressed_size += out_used;
// Don't update the CRC32 if the integrity check will be
// ignored or if there was no new output. The latter is
// important in case out == NULL to avoid null pointer + 0
// which is undefined behavior.
if (!coder->ignore_check && out_used > 0)
coder->crc32 = lzma_crc32(out + out_start, out_used,
coder->crc32);
if (ret != LZMA_STREAM_END)
return ret;
coder->sequence = SEQ_MEMBER_FOOTER;
}
// Fall through
case SEQ_MEMBER_FOOTER: {
// The footer of .lz version 0 lacks the Member size field.
// This is the only difference between version 0 and
// unextended version 1 formats.
const size_t footer_size = coder->version == 0
? LZIP_V0_FOOTER_SIZE
: LZIP_V1_FOOTER_SIZE;
// Copy the CRC32, Data size, and Member size fields to
// the internal buffer.
lzma_bufcpy(in, in_pos, in_size, coder->buffer, &coder->pos,
footer_size);
// Return if we didn't get the whole footer yet.
if (coder->pos < footer_size)
return LZMA_OK;
coder->pos = 0;
coder->member_size += footer_size;
// Check that the footer fields match the observed data.
if (!coder->ignore_check
&& coder->crc32 != read32le(&coder->buffer[0]))
return LZMA_DATA_ERROR;
if (coder->uncompressed_size != read64le(&coder->buffer[4]))
return LZMA_DATA_ERROR;
if (coder->version > 0) {
// .lz version 0 has no Member size field.
if (coder->member_size != read64le(&coder->buffer[12]))
return LZMA_DATA_ERROR;
}
// Decoding is finished if we weren't requested to decode
// more than one .lz member.
if (!coder->concatenated)
return LZMA_STREAM_END;
coder->first_member = false;
coder->sequence = SEQ_ID_STRING;
break;
}
default:
assert(0);
return LZMA_PROG_ERROR;
}
// Never reached
}
static void
lzip_decoder_end(void *coder_ptr, const lzma_allocator *allocator)
{
lzma_lzip_coder *coder = coder_ptr;
lzma_next_end(&coder->lzma_decoder, allocator);
lzma_free(coder, allocator);
return;
}
static lzma_check
lzip_decoder_get_check(const void *coder_ptr lzma_attribute((__unused__)))
{
return LZMA_CHECK_CRC32;
}
static lzma_ret
lzip_decoder_memconfig(void *coder_ptr, uint64_t *memusage,
uint64_t *old_memlimit, uint64_t new_memlimit)
{
lzma_lzip_coder *coder = coder_ptr;
*memusage = coder->memusage;
*old_memlimit = coder->memlimit;
if (new_memlimit != 0) {
if (new_memlimit < coder->memusage)
return LZMA_MEMLIMIT_ERROR;
coder->memlimit = new_memlimit;
}
return LZMA_OK;
}
extern lzma_ret
lzma_lzip_decoder_init(
lzma_next_coder *next, const lzma_allocator *allocator,
uint64_t memlimit, uint32_t flags)
{
lzma_next_coder_init(&lzma_lzip_decoder_init, next, allocator);
if (flags & ~LZMA_SUPPORTED_FLAGS)
return LZMA_OPTIONS_ERROR;
lzma_lzip_coder *coder = next->coder;
if (coder == NULL) {
coder = lzma_alloc(sizeof(lzma_lzip_coder), allocator);
if (coder == NULL)
return LZMA_MEM_ERROR;
next->coder = coder;
next->code = &lzip_decode;
next->end = &lzip_decoder_end;
next->get_check = &lzip_decoder_get_check;
next->memconfig = &lzip_decoder_memconfig;
coder->lzma_decoder = LZMA_NEXT_CODER_INIT;
}
coder->sequence = SEQ_ID_STRING;
coder->memlimit = my_max(1, memlimit);
coder->memusage = LZMA_MEMUSAGE_BASE;
coder->tell_any_check = (flags & LZMA_TELL_ANY_CHECK) != 0;
coder->ignore_check = (flags & LZMA_IGNORE_CHECK) != 0;
coder->concatenated = (flags & LZMA_CONCATENATED) != 0;
coder->first_member = true;
coder->pos = 0;
return LZMA_OK;
}
extern LZMA_API(lzma_ret)
lzma_lzip_decoder(lzma_stream *strm, uint64_t memlimit, uint32_t flags)
{
lzma_next_strm_init(lzma_lzip_decoder_init, strm, memlimit, flags);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
return LZMA_OK;
}

21
Externals/liblzma/common/lzip_decoder.h vendored Normal file
View file

@ -0,0 +1,21 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file lzip_decoder.h
/// \brief Decodes .lz (lzip) files
//
// Author: Michał Górny
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_LZIP_DECODER_H
#define LZMA_LZIP_DECODER_H
#include "common.h"
extern lzma_ret lzma_lzip_decoder_init(
lzma_next_coder *next, const lzma_allocator *allocator,
uint64_t memlimit, uint32_t flags);
#endif

88
Externals/liblzma/common/memcmplen.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file memcmplen.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_MEMCMPLEN_H
@ -19,6 +18,17 @@
# include <immintrin.h>
#endif
// Only include <intrin.h> if it is needed. The header is only needed
// on Windows when using an MSVC compatible compiler. The Intel compiler
// can use the intrinsics without the header file.
#if defined(TUKLIB_FAST_UNALIGNED_ACCESS) \
&& defined(_MSC_VER) \
&& (defined(_M_X64) \
|| defined(_M_ARM64) || defined(_M_ARM64EC)) \
&& !defined(__INTEL_COMPILER)
# include <intrin.h>
#endif
/// Find out how many equal bytes the two buffers have.
///
@ -39,7 +49,7 @@
/// It's rounded up to 2^n. This extra amount needs to be
/// allocated in the buffers being used. It needs to be
/// initialized too to keep Valgrind quiet.
static inline uint32_t lzma_attribute((__always_inline__))
static lzma_always_inline uint32_t
lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
uint32_t len, uint32_t limit)
{
@ -47,28 +57,45 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
assert(limit <= UINT32_MAX / 2);
#if defined(TUKLIB_FAST_UNALIGNED_ACCESS) \
&& ((TUKLIB_GNUC_REQ(3, 4) && defined(__x86_64__)) \
&& (((TUKLIB_GNUC_REQ(3, 4) || defined(__clang__)) \
&& (defined(__x86_64__) \
|| defined(__aarch64__))) \
|| (defined(__INTEL_COMPILER) && defined(__x86_64__)) \
|| (defined(__INTEL_COMPILER) && defined(_M_X64)) \
|| (defined(_MSC_VER) && defined(_M_X64)))
// NOTE: This will use 64-bit unaligned access which
// TUKLIB_FAST_UNALIGNED_ACCESS wasn't meant to permit, but
// it's convenient here at least as long as it's x86-64 only.
|| (defined(_MSC_VER) && (defined(_M_X64) \
|| defined(_M_ARM64) || defined(_M_ARM64EC))))
// This is only for x86-64 and ARM64 for now. This might be fine on
// other 64-bit processors too.
//
// I keep this x86-64 only for now since that's where I know this
// to be a good method. This may be fine on other 64-bit CPUs too.
// On big endian one should use xor instead of subtraction and switch
// to __builtin_clzll().
// Reasons to use subtraction instead of xor:
//
// - On some x86-64 processors (Intel Sandy Bridge to Tiger Lake),
// sub+jz and sub+jnz can be fused but xor+jz or xor+jnz cannot.
// Thus using subtraction has potential to be a tiny amount faster
// since the code checks if the quotient is non-zero.
//
// - Some processors (Intel Pentium 4) used to have more ALU
// resources for add/sub instructions than and/or/xor.
//
// The processor info is based on Agner Fog's microarchitecture.pdf
// version 2023-05-26. https://www.agner.org/optimize/
#define LZMA_MEMCMPLEN_EXTRA 8
while (len < limit) {
const uint64_t x = *(const uint64_t *)(buf1 + len)
- *(const uint64_t *)(buf2 + len);
# ifdef WORDS_BIGENDIAN
const uint64_t x = read64ne(buf1 + len) ^ read64ne(buf2 + len);
# else
const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len);
# endif
if (x != 0) {
# if defined(_M_X64) // MSVC or Intel C compiler on Windows
// MSVC or Intel C compiler on Windows
# if defined(_MSC_VER) || defined(__INTEL_COMPILER)
unsigned long tmp;
_BitScanForward64(&tmp, x);
len += (uint32_t)tmp >> 3;
# else // GCC, clang, or Intel C compiler
// GCC, Clang, or Intel C compiler
# elif defined(WORDS_BIGENDIAN)
len += (uint32_t)__builtin_clzll(x) >> 3;
# else
len += (uint32_t)__builtin_ctzll(x) >> 3;
# endif
return my_min(len, limit);
@ -81,12 +108,12 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
#elif defined(TUKLIB_FAST_UNALIGNED_ACCESS) \
&& defined(HAVE__MM_MOVEMASK_EPI8) \
&& ((defined(__GNUC__) && defined(__SSE2_MATH__)) \
|| (defined(__INTEL_COMPILER) && defined(__SSE2__)) \
&& (defined(__SSE2__) \
|| (defined(_MSC_VER) && defined(_M_IX86_FP) \
&& _M_IX86_FP >= 2))
// NOTE: Like above, this will use 128-bit unaligned access which
// TUKLIB_FAST_UNALIGNED_ACCESS wasn't meant to permit.
// NOTE: This will use 128-bit unaligned access which
// TUKLIB_FAST_UNALIGNED_ACCESS wasn't meant to permit,
// but it's convenient here since this is x86-only.
//
// SSE2 version for 32-bit and 64-bit x86. On x86-64 the above
// version is sometimes significantly faster and sometimes
@ -94,20 +121,13 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// version isn't used on x86-64.
# define LZMA_MEMCMPLEN_EXTRA 16
while (len < limit) {
const uint32_t x = 0xFFFF ^ _mm_movemask_epi8(_mm_cmpeq_epi8(
const uint32_t x = 0xFFFF ^ (uint32_t)_mm_movemask_epi8(
_mm_cmpeq_epi8(
_mm_loadu_si128((const __m128i *)(buf1 + len)),
_mm_loadu_si128((const __m128i *)(buf2 + len))));
if (x != 0) {
# if defined(__INTEL_COMPILER)
len += _bit_scan_forward(x);
# elif defined(_MSC_VER)
unsigned long tmp;
_BitScanForward(&tmp, x);
len += tmp;
# else
len += __builtin_ctz(x);
# endif
len += ctz32(x);
return my_min(len, limit);
}
@ -120,8 +140,7 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// Generic 32-bit little endian method
# define LZMA_MEMCMPLEN_EXTRA 4
while (len < limit) {
uint32_t x = *(const uint32_t *)(buf1 + len)
- *(const uint32_t *)(buf2 + len);
uint32_t x = read32ne(buf1 + len) - read32ne(buf2 + len);
if (x != 0) {
if ((x & 0xFFFF) == 0) {
len += 2;
@ -143,8 +162,7 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// Generic 32-bit big endian method
# define LZMA_MEMCMPLEN_EXTRA 4
while (len < limit) {
uint32_t x = *(const uint32_t *)(buf1 + len)
^ *(const uint32_t *)(buf2 + len);
uint32_t x = read32ne(buf1 + len) ^ read32ne(buf2 + len);
if (x != 0) {
if ((x & 0xFFFF0000) == 0) {
len += 2;

220
Externals/liblzma/common/microlzma_decoder.c vendored Normal file
View file

@ -0,0 +1,220 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file microlzma_decoder.c
/// \brief Decode MicroLZMA format
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "lzma_decoder.h"
#include "lz_decoder.h"
typedef struct {
/// LZMA1 decoder
lzma_next_coder lzma;
/// Compressed size of the stream as given by the application.
/// This must be exactly correct.
///
/// This will be decremented when input is read.
uint64_t comp_size;
/// Uncompressed size of the stream as given by the application.
/// This may be less than the actual uncompressed size if
/// uncomp_size_is_exact is false.
///
/// This will be decremented when output is produced.
lzma_vli uncomp_size;
/// LZMA dictionary size as given by the application
uint32_t dict_size;
/// If true, the exact uncompressed size is known. If false,
/// uncomp_size may be smaller than the real uncompressed size;
/// uncomp_size may never be bigger than the real uncompressed size.
bool uncomp_size_is_exact;
/// True once the first byte of the MicroLZMA stream
/// has been processed.
bool props_decoded;
} lzma_microlzma_coder;
static lzma_ret
microlzma_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size, lzma_action action)
{
lzma_microlzma_coder *coder = coder_ptr;
// Remember the in start position so that we can update comp_size.
const size_t in_start = *in_pos;
// Remember the out start position so that we can update uncomp_size.
const size_t out_start = *out_pos;
// Limit the amount of input so that the decoder won't read more than
// comp_size. This is required when uncomp_size isn't exact because
// in that case the LZMA decoder will try to decode more input even
// when it has no output space (it can be looking for EOPM).
if (in_size - *in_pos > coder->comp_size)
in_size = *in_pos + (size_t)(coder->comp_size);
// When the exact uncompressed size isn't known, we must limit
// the available output space to prevent the LZMA decoder from
// trying to decode too much.
if (!coder->uncomp_size_is_exact
&& out_size - *out_pos > coder->uncomp_size)
out_size = *out_pos + (size_t)(coder->uncomp_size);
if (!coder->props_decoded) {
// There must be at least one byte of input to decode
// the properties byte.
if (*in_pos >= in_size)
return LZMA_OK;
lzma_options_lzma options = {
.dict_size = coder->dict_size,
.preset_dict = NULL,
.preset_dict_size = 0,
.ext_flags = 0, // EOPM not allowed when size is known
.ext_size_low = UINT32_MAX, // Unknown size by default
.ext_size_high = UINT32_MAX,
};
if (coder->uncomp_size_is_exact)
lzma_set_ext_size(options, coder->uncomp_size);
// The properties are stored as bitwise-negation
// of the typical encoding.
if (lzma_lzma_lclppb_decode(&options, ~in[*in_pos]))
return LZMA_OPTIONS_ERROR;
++*in_pos;
// Initialize the decoder.
lzma_filter_info filters[2] = {
{
.id = LZMA_FILTER_LZMA1EXT,
.init = &lzma_lzma_decoder_init,
.options = &options,
}, {
.init = NULL,
}
};
return_if_error(lzma_next_filter_init(&coder->lzma,
allocator, filters));
// Pass one dummy 0x00 byte to the LZMA decoder since that
// is what it expects the first byte to be.
const uint8_t dummy_in = 0;
size_t dummy_in_pos = 0;
if (coder->lzma.code(coder->lzma.coder, allocator,
&dummy_in, &dummy_in_pos, 1,
out, out_pos, out_size, LZMA_RUN) != LZMA_OK)
return LZMA_PROG_ERROR;
assert(dummy_in_pos == 1);
coder->props_decoded = true;
}
// The rest is normal LZMA decoding.
lzma_ret ret = coder->lzma.code(coder->lzma.coder, allocator,
in, in_pos, in_size,
out, out_pos, out_size, action);
// Update the remaining compressed size.
assert(coder->comp_size >= *in_pos - in_start);
coder->comp_size -= *in_pos - in_start;
if (coder->uncomp_size_is_exact) {
// After successful decompression of the complete stream
// the compressed size must match.
if (ret == LZMA_STREAM_END && coder->comp_size != 0)
ret = LZMA_DATA_ERROR;
} else {
// Update the amount of output remaining.
assert(coder->uncomp_size >= *out_pos - out_start);
coder->uncomp_size -= *out_pos - out_start;
// - We must not get LZMA_STREAM_END because the stream
// shouldn't have EOPM.
// - We must use uncomp_size to determine when to
// return LZMA_STREAM_END.
if (ret == LZMA_STREAM_END)
ret = LZMA_DATA_ERROR;
else if (coder->uncomp_size == 0)
ret = LZMA_STREAM_END;
}
return ret;
}
static void
microlzma_decoder_end(void *coder_ptr, const lzma_allocator *allocator)
{
lzma_microlzma_coder *coder = coder_ptr;
lzma_next_end(&coder->lzma, allocator);
lzma_free(coder, allocator);
return;
}
static lzma_ret
microlzma_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
uint64_t comp_size,
uint64_t uncomp_size, bool uncomp_size_is_exact,
uint32_t dict_size)
{
lzma_next_coder_init(&microlzma_decoder_init, next, allocator);
lzma_microlzma_coder *coder = next->coder;
if (coder == NULL) {
coder = lzma_alloc(sizeof(lzma_microlzma_coder), allocator);
if (coder == NULL)
return LZMA_MEM_ERROR;
next->coder = coder;
next->code = &microlzma_decode;
next->end = &microlzma_decoder_end;
coder->lzma = LZMA_NEXT_CODER_INIT;
}
// The public API is uint64_t but the internal LZ decoder API uses
// lzma_vli.
if (uncomp_size > LZMA_VLI_MAX)
return LZMA_OPTIONS_ERROR;
coder->comp_size = comp_size;
coder->uncomp_size = uncomp_size;
coder->uncomp_size_is_exact = uncomp_size_is_exact;
coder->dict_size = dict_size;
coder->props_decoded = false;
return LZMA_OK;
}
extern LZMA_API(lzma_ret)
lzma_microlzma_decoder(lzma_stream *strm, uint64_t comp_size,
uint64_t uncomp_size, lzma_bool uncomp_size_is_exact,
uint32_t dict_size)
{
lzma_next_strm_init(microlzma_decoder_init, strm, comp_size,
uncomp_size, uncomp_size_is_exact, dict_size);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
return LZMA_OK;
}

140
Externals/liblzma/common/microlzma_encoder.c vendored Normal file
View file

@ -0,0 +1,140 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file microlzma_encoder.c
/// \brief Encode into MicroLZMA format
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include "lzma_encoder.h"
typedef struct {
/// LZMA1 encoder
lzma_next_coder lzma;
/// LZMA properties byte (lc/lp/pb)
uint8_t props;
} lzma_microlzma_coder;
static lzma_ret
microlzma_encode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos,
size_t in_size, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size, lzma_action action)
{
lzma_microlzma_coder *coder = coder_ptr;
// Remember *out_pos so that we can overwrite the first byte with
// the LZMA properties byte.
const size_t out_start = *out_pos;
// Remember *in_pos so that we can set it based on how many
// uncompressed bytes were actually encoded.
const size_t in_start = *in_pos;
// Set the output size limit based on the available output space.
// We know that the encoder supports set_out_limit() so
// LZMA_OPTIONS_ERROR isn't possible. LZMA_BUF_ERROR is possible
// but lzma_code() has an assertion to not allow it to be returned
// from here and I don't want to change that for now, so
// LZMA_BUF_ERROR becomes LZMA_PROG_ERROR.
uint64_t uncomp_size;
if (coder->lzma.set_out_limit(coder->lzma.coder,
&uncomp_size, out_size - *out_pos) != LZMA_OK)
return LZMA_PROG_ERROR;
// set_out_limit fails if this isn't true.
assert(out_size - *out_pos >= 6);
// Encode as much as possible.
const lzma_ret ret = coder->lzma.code(coder->lzma.coder, allocator,
in, in_pos, in_size, out, out_pos, out_size, action);
if (ret != LZMA_STREAM_END) {
if (ret == LZMA_OK) {
assert(0);
return LZMA_PROG_ERROR;
}
return ret;
}
// The first output byte is bitwise-negation of the properties byte.
// We know that there is space for this byte because set_out_limit
// and the actual encoding succeeded.
out[out_start] = (uint8_t)(~coder->props);
// The LZMA encoder likely read more input than it was able to encode.
// Set *in_pos based on uncomp_size.
assert(uncomp_size <= in_size - in_start);
*in_pos = in_start + (size_t)(uncomp_size);
return ret;
}
static void
microlzma_encoder_end(void *coder_ptr, const lzma_allocator *allocator)
{
lzma_microlzma_coder *coder = coder_ptr;
lzma_next_end(&coder->lzma, allocator);
lzma_free(coder, allocator);
return;
}
static lzma_ret
microlzma_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
const lzma_options_lzma *options)
{
lzma_next_coder_init(&microlzma_encoder_init, next, allocator);
lzma_microlzma_coder *coder = next->coder;
if (coder == NULL) {
coder = lzma_alloc(sizeof(lzma_microlzma_coder), allocator);
if (coder == NULL)
return LZMA_MEM_ERROR;
next->coder = coder;
next->code = &microlzma_encode;
next->end = &microlzma_encoder_end;
coder->lzma = LZMA_NEXT_CODER_INIT;
}
// Encode the properties byte. Bitwise-negation of it will be the
// first output byte.
if (lzma_lzma_lclppb_encode(options, &coder->props))
return LZMA_OPTIONS_ERROR;
// Initialize the LZMA encoder.
const lzma_filter_info filters[2] = {
{
.id = LZMA_FILTER_LZMA1,
.init = &lzma_lzma_encoder_init,
.options = (void *)(options),
}, {
.init = NULL,
}
};
return lzma_next_filter_init(&coder->lzma, allocator, filters);
}
extern LZMA_API(lzma_ret)
lzma_microlzma_encoder(lzma_stream *strm, const lzma_options_lzma *options)
{
lzma_next_strm_init(microlzma_encoder_init, strm, options);
strm->internal->supported_actions[LZMA_FINISH] = true;
return LZMA_OK;
}

300
Externals/liblzma/common/outqueue.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file outqueue.c
@ -5,92 +7,126 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "outqueue.h"
/// This is to ease integer overflow checking: We may allocate up to
/// 2 * LZMA_THREADS_MAX buffers and we need some extra memory for other
/// data structures (that's the second /2).
#define BUF_SIZE_MAX (UINT64_MAX / LZMA_THREADS_MAX / 2 / 2)
static lzma_ret
get_options(uint64_t *bufs_alloc_size, uint32_t *bufs_count,
uint64_t buf_size_max, uint32_t threads)
{
if (threads > LZMA_THREADS_MAX || buf_size_max > BUF_SIZE_MAX)
return LZMA_OPTIONS_ERROR;
// The number of buffers is twice the number of threads.
// This wastes RAM but keeps the threads busy when buffers
// finish out of order.
//
// NOTE: If this is changed, update BUF_SIZE_MAX too.
*bufs_count = threads * 2;
*bufs_alloc_size = *bufs_count * buf_size_max;
return LZMA_OK;
}
/// Get the maximum number of buffers that may be allocated based
/// on the number of threads. For now this is twice the number of threads.
/// It's a compromise between RAM usage and keeping the worker threads busy
/// when buffers finish out of order.
#define GET_BUFS_LIMIT(threads) (2 * (threads))
extern uint64_t
lzma_outq_memusage(uint64_t buf_size_max, uint32_t threads)
{
uint64_t bufs_alloc_size;
uint32_t bufs_count;
// This is to ease integer overflow checking: We may allocate up to
// GET_BUFS_LIMIT(LZMA_THREADS_MAX) buffers and we need some extra
// memory for other data structures too (that's the /2).
//
// lzma_outq_prealloc_buf() will still accept bigger buffers than this.
const uint64_t limit
= UINT64_MAX / GET_BUFS_LIMIT(LZMA_THREADS_MAX) / 2;
if (get_options(&bufs_alloc_size, &bufs_count, buf_size_max, threads)
!= LZMA_OK)
if (threads > LZMA_THREADS_MAX || buf_size_max > limit)
return UINT64_MAX;
return sizeof(lzma_outq) + bufs_count * sizeof(lzma_outbuf)
+ bufs_alloc_size;
return GET_BUFS_LIMIT(threads)
* lzma_outq_outbuf_memusage(buf_size_max);
}
static void
move_head_to_cache(lzma_outq *outq, const lzma_allocator *allocator)
{
assert(outq->head != NULL);
assert(outq->tail != NULL);
assert(outq->bufs_in_use > 0);
lzma_outbuf *buf = outq->head;
outq->head = buf->next;
if (outq->head == NULL)
outq->tail = NULL;
if (outq->cache != NULL && outq->cache->allocated != buf->allocated)
lzma_outq_clear_cache(outq, allocator);
buf->next = outq->cache;
outq->cache = buf;
--outq->bufs_in_use;
outq->mem_in_use -= lzma_outq_outbuf_memusage(buf->allocated);
return;
}
static void
free_one_cached_buffer(lzma_outq *outq, const lzma_allocator *allocator)
{
assert(outq->cache != NULL);
lzma_outbuf *buf = outq->cache;
outq->cache = buf->next;
--outq->bufs_allocated;
outq->mem_allocated -= lzma_outq_outbuf_memusage(buf->allocated);
lzma_free(buf, allocator);
return;
}
extern void
lzma_outq_clear_cache(lzma_outq *outq, const lzma_allocator *allocator)
{
while (outq->cache != NULL)
free_one_cached_buffer(outq, allocator);
return;
}
extern void
lzma_outq_clear_cache2(lzma_outq *outq, const lzma_allocator *allocator,
size_t keep_size)
{
if (outq->cache == NULL)
return;
// Free all but one.
while (outq->cache->next != NULL)
free_one_cached_buffer(outq, allocator);
// Free the last one only if its size doesn't equal to keep_size.
if (outq->cache->allocated != keep_size)
free_one_cached_buffer(outq, allocator);
return;
}
extern lzma_ret
lzma_outq_init(lzma_outq *outq, const lzma_allocator *allocator,
uint64_t buf_size_max, uint32_t threads)
uint32_t threads)
{
uint64_t bufs_alloc_size;
uint32_t bufs_count;
if (threads > LZMA_THREADS_MAX)
return LZMA_OPTIONS_ERROR;
// Set bufs_count and bufs_alloc_size.
return_if_error(get_options(&bufs_alloc_size, &bufs_count,
buf_size_max, threads));
const uint32_t bufs_limit = GET_BUFS_LIMIT(threads);
// Allocate memory if needed.
if (outq->buf_size_max != buf_size_max
|| outq->bufs_allocated != bufs_count) {
lzma_outq_end(outq, allocator);
// Clear head/tail.
while (outq->head != NULL)
move_head_to_cache(outq, allocator);
#if SIZE_MAX < UINT64_MAX
if (bufs_alloc_size > SIZE_MAX)
return LZMA_MEM_ERROR;
#endif
// If new buf_limit is lower than the old one, we may need to free
// a few cached buffers.
while (bufs_limit < outq->bufs_allocated)
free_one_cached_buffer(outq, allocator);
outq->bufs = lzma_alloc(bufs_count * sizeof(lzma_outbuf),
allocator);
outq->bufs_mem = lzma_alloc((size_t)(bufs_alloc_size),
allocator);
if (outq->bufs == NULL || outq->bufs_mem == NULL) {
lzma_outq_end(outq, allocator);
return LZMA_MEM_ERROR;
}
}
// Initialize the rest of the main structure. Initialization of
// outq->bufs[] is done when they are actually needed.
outq->buf_size_max = (size_t)(buf_size_max);
outq->bufs_allocated = bufs_count;
outq->bufs_pos = 0;
outq->bufs_used = 0;
outq->bufs_limit = bufs_limit;
outq->read_pos = 0;
return LZMA_OK;
@ -100,33 +136,81 @@ lzma_outq_init(lzma_outq *outq, const lzma_allocator *allocator,
extern void
lzma_outq_end(lzma_outq *outq, const lzma_allocator *allocator)
{
lzma_free(outq->bufs, allocator);
outq->bufs = NULL;
lzma_free(outq->bufs_mem, allocator);
outq->bufs_mem = NULL;
while (outq->head != NULL)
move_head_to_cache(outq, allocator);
lzma_outq_clear_cache(outq, allocator);
return;
}
extern lzma_outbuf *
lzma_outq_get_buf(lzma_outq *outq)
extern lzma_ret
lzma_outq_prealloc_buf(lzma_outq *outq, const lzma_allocator *allocator,
size_t size)
{
// Caller must have checked it with lzma_outq_has_buf().
assert(outq->bufs_used < outq->bufs_allocated);
assert(outq->bufs_in_use < outq->bufs_limit);
// Initialize the new buffer.
lzma_outbuf *buf = &outq->bufs[outq->bufs_pos];
buf->buf = outq->bufs_mem + outq->bufs_pos * outq->buf_size_max;
buf->size = 0;
// If there already is appropriately-sized buffer in the cache,
// we need to do nothing.
if (outq->cache != NULL && outq->cache->allocated == size)
return LZMA_OK;
if (size > SIZE_MAX - sizeof(lzma_outbuf))
return LZMA_MEM_ERROR;
const size_t alloc_size = lzma_outq_outbuf_memusage(size);
// The cache may have buffers but their size is wrong.
lzma_outq_clear_cache(outq, allocator);
outq->cache = lzma_alloc(alloc_size, allocator);
if (outq->cache == NULL)
return LZMA_MEM_ERROR;
outq->cache->next = NULL;
outq->cache->allocated = size;
++outq->bufs_allocated;
outq->mem_allocated += alloc_size;
return LZMA_OK;
}
extern lzma_outbuf *
lzma_outq_get_buf(lzma_outq *outq, void *worker)
{
// Caller must have used lzma_outq_prealloc_buf() to ensure these.
assert(outq->bufs_in_use < outq->bufs_limit);
assert(outq->bufs_in_use < outq->bufs_allocated);
assert(outq->cache != NULL);
lzma_outbuf *buf = outq->cache;
outq->cache = buf->next;
buf->next = NULL;
if (outq->tail != NULL) {
assert(outq->head != NULL);
outq->tail->next = buf;
} else {
assert(outq->head == NULL);
outq->head = buf;
}
outq->tail = buf;
buf->worker = worker;
buf->finished = false;
buf->finish_ret = LZMA_STREAM_END;
buf->pos = 0;
buf->decoder_in_pos = 0;
// Update the queue state.
if (++outq->bufs_pos == outq->bufs_allocated)
outq->bufs_pos = 0;
buf->unpadded_size = 0;
buf->uncompressed_size = 0;
++outq->bufs_used;
++outq->bufs_in_use;
outq->mem_in_use += lzma_outq_outbuf_memusage(buf->allocated);
return buf;
}
@ -135,50 +219,68 @@ lzma_outq_get_buf(lzma_outq *outq)
extern bool
lzma_outq_is_readable(const lzma_outq *outq)
{
uint32_t i = outq->bufs_pos - outq->bufs_used;
if (outq->bufs_pos < outq->bufs_used)
i += outq->bufs_allocated;
if (outq->head == NULL)
return false;
return outq->bufs[i].finished;
return outq->read_pos < outq->head->pos || outq->head->finished;
}
extern lzma_ret
lzma_outq_read(lzma_outq *restrict outq, uint8_t *restrict out,
size_t *restrict out_pos, size_t out_size,
lzma_outq_read(lzma_outq *restrict outq,
const lzma_allocator *restrict allocator,
uint8_t *restrict out, size_t *restrict out_pos,
size_t out_size,
lzma_vli *restrict unpadded_size,
lzma_vli *restrict uncompressed_size)
{
// There must be at least one buffer from which to read.
if (outq->bufs_used == 0)
if (outq->bufs_in_use == 0)
return LZMA_OK;
// Get the buffer.
uint32_t i = outq->bufs_pos - outq->bufs_used;
if (outq->bufs_pos < outq->bufs_used)
i += outq->bufs_allocated;
lzma_outbuf *buf = &outq->bufs[i];
// If it isn't finished yet, we cannot read from it.
if (!buf->finished)
return LZMA_OK;
lzma_outbuf *buf = outq->head;
// Copy from the buffer to output.
lzma_bufcpy(buf->buf, &outq->read_pos, buf->size,
//
// FIXME? In threaded decoder it may be bad to do this copy while
// the mutex is being held.
lzma_bufcpy(buf->buf, &outq->read_pos, buf->pos,
out, out_pos, out_size);
// Return if we didn't get all the data from the buffer.
if (outq->read_pos < buf->size)
if (!buf->finished || outq->read_pos < buf->pos)
return LZMA_OK;
// The buffer was finished. Tell the caller its size information.
if (unpadded_size != NULL)
*unpadded_size = buf->unpadded_size;
if (uncompressed_size != NULL)
*uncompressed_size = buf->uncompressed_size;
// Remember the return value.
const lzma_ret finish_ret = buf->finish_ret;
// Free this buffer for further use.
--outq->bufs_used;
move_head_to_cache(outq, allocator);
outq->read_pos = 0;
return LZMA_STREAM_END;
return finish_ret;
}
extern void
lzma_outq_enable_partial_output(lzma_outq *outq,
void (*enable_partial_output)(void *worker))
{
if (outq->head != NULL && !outq->head->finished
&& outq->head->worker != NULL) {
enable_partial_output(outq->head->worker);
// Set it to NULL since calling it twice is pointless.
outq->head->worker = NULL;
}
return;
}

192
Externals/liblzma/common/outqueue.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file outqueue.h
@ -5,25 +7,45 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_OUTQUEUE_H
#define LZMA_OUTQUEUE_H
#include "common.h"
/// Output buffer for a single thread
typedef struct {
/// Pointer to the output buffer of lzma_outq.buf_size_max bytes
uint8_t *buf;
typedef struct lzma_outbuf_s lzma_outbuf;
struct lzma_outbuf_s {
/// Pointer to the next buffer. This is used for the cached buffers.
/// The worker thread must not modify this.
lzma_outbuf *next;
/// Amount of data written to buf
size_t size;
/// This initialized by lzma_outq_get_buf() and
/// is used by lzma_outq_enable_partial_output().
/// The worker thread must not modify this.
void *worker;
/// Additional size information
lzma_vli unpadded_size;
lzma_vli uncompressed_size;
/// Amount of memory allocated for buf[].
/// The worker thread must not modify this.
size_t allocated;
/// Writing position in the worker thread or, in other words, the
/// amount of finished data written to buf[] which can be copied out
///
/// \note This is read by another thread and thus access
/// to this variable needs a mutex.
size_t pos;
/// Decompression: Position in the input buffer in the worker thread
/// that matches the output "pos" above. This is used to detect if
/// more output might be possible from the worker thread: if it has
/// consumed all its input, then more output isn't possible.
///
/// \note This is read by another thread and thus access
/// to this variable needs a mutex.
size_t decoder_in_pos;
/// True when no more data will be written into this buffer.
///
@ -31,32 +53,55 @@ typedef struct {
/// to this variable needs a mutex.
bool finished;
} lzma_outbuf;
/// Return value for lzma_outq_read() when the last byte from
/// a finished buffer has been read. Defaults to LZMA_STREAM_END.
/// This must *not* be LZMA_OK. The idea is to allow a decoder to
/// pass an error code to the main thread, setting the code here
/// together with finished = true.
lzma_ret finish_ret;
/// Additional size information. lzma_outq_read() may read these
/// when "finished" is true.
lzma_vli unpadded_size;
lzma_vli uncompressed_size;
/// Buffer of "allocated" bytes
uint8_t buf[];
};
typedef struct {
/// Array of buffers that are used cyclically.
lzma_outbuf *bufs;
/// Linked list of buffers in use. The next output byte will be
/// read from the head and buffers for the next thread will be
/// appended to the tail. tail->next is always NULL.
lzma_outbuf *head;
lzma_outbuf *tail;
/// Memory allocated for all the buffers
uint8_t *bufs_mem;
/// Amount of buffer space available in each buffer
size_t buf_size_max;
/// Number of buffers allocated
uint32_t bufs_allocated;
/// Position in the bufs array. The next buffer to be taken
/// into use is bufs[bufs_pos].
uint32_t bufs_pos;
/// Number of buffers in use
uint32_t bufs_used;
/// Position in the buffer in lzma_outq_read()
/// Number of bytes read from head->buf[] in lzma_outq_read()
size_t read_pos;
/// Linked list of allocated buffers that aren't currently used.
/// This way buffers of similar size can be reused and don't
/// need to be reallocated every time. For simplicity, all
/// cached buffers in the list have the same allocated size.
lzma_outbuf *cache;
/// Total amount of memory allocated for buffers
uint64_t mem_allocated;
/// Amount of memory used by the buffers that are in use in
/// the head...tail linked list.
uint64_t mem_in_use;
/// Number of buffers in use in the head...tail list. If and only if
/// this is zero, the pointers head and tail above are NULL.
uint32_t bufs_in_use;
/// Number of buffers allocated (in use + cached)
uint32_t bufs_allocated;
/// Maximum allowed number of allocated buffers
uint32_t bufs_limit;
} lzma_outq;
@ -76,32 +121,60 @@ extern uint64_t lzma_outq_memusage(uint64_t buf_size_max, uint32_t threads);
/// function knows that there are no previous
/// allocations to free.
/// \param allocator Pointer to allocator or NULL
/// \param buf_size_max Maximum amount of data that a single buffer
/// in the queue may need to store.
/// \param threads Number of buffers that may be in use
/// concurrently. Note that more than this number
/// of buffers will actually get allocated to
/// of buffers may actually get allocated to
/// improve performance when buffers finish
/// out of order.
/// out of order. The actual maximum number of
/// allocated buffers is derived from the number
/// of threads.
///
/// \return - LZMA_OK
/// - LZMA_MEM_ERROR
///
extern lzma_ret lzma_outq_init(
lzma_outq *outq, const lzma_allocator *allocator,
uint64_t buf_size_max, uint32_t threads);
extern lzma_ret lzma_outq_init(lzma_outq *outq,
const lzma_allocator *allocator, uint32_t threads);
/// \brief Free the memory associated with the output queue
extern void lzma_outq_end(lzma_outq *outq, const lzma_allocator *allocator);
/// \brief Free all cached buffers that consume memory but aren't in use
extern void lzma_outq_clear_cache(
lzma_outq *outq, const lzma_allocator *allocator);
/// \brief Like lzma_outq_clear_cache() but might keep one buffer
///
/// One buffer is not freed if its size is equal to keep_size.
/// This is useful if the caller knows that it will soon need a buffer of
/// keep_size bytes. This way it won't be freed and immediately reallocated.
extern void lzma_outq_clear_cache2(
lzma_outq *outq, const lzma_allocator *allocator,
size_t keep_size);
/// \brief Preallocate a new buffer into cache
///
/// Splitting the buffer allocation into a separate function makes it
/// possible to ensure that way lzma_outq_get_buf() cannot fail.
/// If the preallocated buffer isn't actually used (for example, some
/// other error occurs), the caller has to do nothing as the buffer will
/// be used later or cleared from the cache when not needed.
///
/// \return LZMA_OK on success, LZMA_MEM_ERROR if allocation fails
///
extern lzma_ret lzma_outq_prealloc_buf(
lzma_outq *outq, const lzma_allocator *allocator, size_t size);
/// \brief Get a new buffer
///
/// lzma_outq_has_buf() must be used to check that there is a buffer
/// lzma_outq_prealloc_buf() must be used to ensure that there is a buffer
/// available before calling lzma_outq_get_buf().
///
extern lzma_outbuf *lzma_outq_get_buf(lzma_outq *outq);
extern lzma_outbuf *lzma_outq_get_buf(lzma_outq *outq, void *worker);
/// \brief Test if there is data ready to be read
@ -126,17 +199,32 @@ extern bool lzma_outq_is_readable(const lzma_outq *outq);
/// \return - LZMA: All OK. Either no data was available or the buffer
/// being read didn't become empty yet.
/// - LZMA_STREAM_END: The buffer being read was finished.
/// *unpadded_size and *uncompressed_size were set.
/// *unpadded_size and *uncompressed_size were set if they
/// were not NULL.
///
/// \note This reads lzma_outbuf.finished variables and thus call
/// to this function needs to be protected with a mutex.
/// \note This reads lzma_outbuf.finished and .pos variables and thus
/// calls to this function need to be protected with a mutex.
///
extern lzma_ret lzma_outq_read(lzma_outq *restrict outq,
const lzma_allocator *restrict allocator,
uint8_t *restrict out, size_t *restrict out_pos,
size_t out_size, lzma_vli *restrict unpadded_size,
lzma_vli *restrict uncompressed_size);
/// \brief Enable partial output from a worker thread
///
/// If the buffer at the head of the output queue isn't finished,
/// this will call enable_partial_output on the worker associated with
/// that output buffer.
///
/// \note This reads a lzma_outbuf.finished variable and thus
/// calls to this function need to be protected with a mutex.
///
extern void lzma_outq_enable_partial_output(lzma_outq *outq,
void (*enable_partial_output)(void *worker));
/// \brief Test if there is at least one buffer free
///
/// This must be used before getting a new buffer with lzma_outq_get_buf().
@ -144,7 +232,7 @@ extern lzma_ret lzma_outq_read(lzma_outq *restrict outq,
static inline bool
lzma_outq_has_buf(const lzma_outq *outq)
{
return outq->bufs_used < outq->bufs_allocated;
return outq->bufs_in_use < outq->bufs_limit;
}
@ -152,5 +240,19 @@ lzma_outq_has_buf(const lzma_outq *outq)
static inline bool
lzma_outq_is_empty(const lzma_outq *outq)
{
return outq->bufs_used == 0;
return outq->bufs_in_use == 0;
}
/// \brief Get the amount of memory needed for a single lzma_outbuf
///
/// \note Caller must check that the argument is significantly less
/// than SIZE_MAX to avoid an integer overflow!
static inline uint64_t
lzma_outq_outbuf_memusage(size_t buf_size)
{
assert(buf_size <= SIZE_MAX - sizeof(lzma_outbuf));
return sizeof(lzma_outbuf) + buf_size;
}
#endif

5
Externals/liblzma/common/stream_buffer_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_buffer_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_decoder.h"

6
Externals/liblzma/common/stream_buffer_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_buffer_encoder.c
@ -5,11 +7,9 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
#include "index.h"

40
Externals/liblzma/common/stream_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_decoder.c
@ -5,28 +7,25 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_decoder.h"
#include "block_decoder.h"
#include "index.h"
typedef struct {
enum {
SEQ_STREAM_HEADER,
SEQ_BLOCK_HEADER,
SEQ_BLOCK,
SEQ_BLOCK_INIT,
SEQ_BLOCK_RUN,
SEQ_INDEX,
SEQ_STREAM_FOOTER,
SEQ_STREAM_PADDING,
} sequence;
/// Block or Metadata decoder. This takes little memory and the same
/// data structure can be used to decode every Block Header, so it's
/// a good idea to have a separate lzma_next_coder structure for it.
/// Block decoder
lzma_next_coder block_decoder;
/// Block options decoded by the Block Header decoder and used by
@ -63,9 +62,9 @@ typedef struct {
/// If true, we will decode concatenated Streams that possibly have
/// Stream Padding between or after them. LZMA_STREAM_END is returned
/// once the application isn't giving us any new input, and we aren't
/// in the middle of a Stream, and possible Stream Padding is a
/// multiple of four bytes.
/// once the application isn't giving us any new input (LZMA_FINISH),
/// and we aren't in the middle of a Stream, and possible
/// Stream Padding is a multiple of four bytes.
bool concatenated;
/// When decoding concatenated Streams, this is true as long as we
@ -165,7 +164,7 @@ stream_decode(void *coder_ptr, const lzma_allocator *allocator,
if (coder->pos == 0) {
// Detect if it's Index.
if (in[*in_pos] == 0x00) {
if (in[*in_pos] == INDEX_INDICATOR) {
coder->sequence = SEQ_INDEX;
break;
}
@ -187,6 +186,15 @@ stream_decode(void *coder_ptr, const lzma_allocator *allocator,
return LZMA_OK;
coder->pos = 0;
coder->sequence = SEQ_BLOCK_INIT;
}
// Fall through
case SEQ_BLOCK_INIT: {
// Checking memusage and doing the initialization needs
// its own sequence point because we need to be able to
// retry if we return LZMA_MEMLIMIT_ERROR.
// Version 1 is needed to support the .ignore_check option.
coder->block_options.version = 1;
@ -235,22 +243,20 @@ stream_decode(void *coder_ptr, const lzma_allocator *allocator,
// Free the allocated filter options since they are needed
// only to initialize the Block decoder.
for (size_t i = 0; i < LZMA_FILTERS_MAX; ++i)
lzma_free(filters[i].options, allocator);
lzma_filters_free(filters, allocator);
coder->block_options.filters = NULL;
// Check if memory usage calculation and Block enocoder
// Check if memory usage calculation and Block decoder
// initialization succeeded.
if (ret != LZMA_OK)
return ret;
coder->sequence = SEQ_BLOCK;
coder->sequence = SEQ_BLOCK_RUN;
}
// Fall through
case SEQ_BLOCK: {
case SEQ_BLOCK_RUN: {
const lzma_ret ret = coder->block_decoder.code(
coder->block_decoder.coder, allocator,
in, in_pos, in_size, out, out_pos, out_size,

5
Externals/liblzma/common/stream_decoder.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_decoder.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_STREAM_DECODER_H

2017
Externals/liblzma/common/stream_decoder_mt.c vendored Normal file
View file

File diff suppressed because it is too large Load diff

46
Externals/liblzma/common/stream_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "block_encoder.h"
@ -219,8 +218,7 @@ stream_encoder_end(void *coder_ptr, const lzma_allocator *allocator)
lzma_next_end(&coder->index_encoder, allocator);
lzma_index_end(coder->index, allocator);
for (size_t i = 0; coder->filters[i].id != LZMA_VLI_UNKNOWN; ++i)
lzma_free(coder->filters[i].options, allocator);
lzma_filters_free(coder->filters, allocator);
lzma_free(coder, allocator);
return;
@ -233,6 +231,13 @@ stream_encoder_update(void *coder_ptr, const lzma_allocator *allocator,
const lzma_filter *reversed_filters)
{
lzma_stream_coder *coder = coder_ptr;
lzma_ret ret;
// Make a copy to a temporary buffer first. This way it is easier
// to keep the encoder state unchanged if an error occurs with
// lzma_filters_copy().
lzma_filter temp[LZMA_FILTERS_MAX + 1];
return_if_error(lzma_filters_copy(filters, temp, allocator));
if (coder->sequence <= SEQ_BLOCK_INIT) {
// There is no incomplete Block waiting to be finished,
@ -240,31 +245,40 @@ stream_encoder_update(void *coder_ptr, const lzma_allocator *allocator,
// trying to initialize the Block encoder with the new
// chain. This way we detect if the chain is valid.
coder->block_encoder_is_initialized = false;
coder->block_options.filters = (lzma_filter *)(filters);
const lzma_ret ret = block_encoder_init(coder, allocator);
coder->block_options.filters = temp;
ret = block_encoder_init(coder, allocator);
coder->block_options.filters = coder->filters;
if (ret != LZMA_OK)
return ret;
goto error;
coder->block_encoder_is_initialized = true;
} else if (coder->sequence <= SEQ_BLOCK_ENCODE) {
// We are in the middle of a Block. Try to update only
// the filter-specific options.
return_if_error(coder->block_encoder.update(
ret = coder->block_encoder.update(
coder->block_encoder.coder, allocator,
filters, reversed_filters));
filters, reversed_filters);
if (ret != LZMA_OK)
goto error;
} else {
// Trying to update the filter chain when we are already
// encoding Index or Stream Footer.
return LZMA_PROG_ERROR;
ret = LZMA_PROG_ERROR;
goto error;
}
// Free the copy of the old chain and make a copy of the new chain.
for (size_t i = 0; coder->filters[i].id != LZMA_VLI_UNKNOWN; ++i)
lzma_free(coder->filters[i].options, allocator);
// Free the options of the old chain.
lzma_filters_free(coder->filters, allocator);
return lzma_filters_copy(filters, coder->filters, allocator);
// Copy the new filter chain in place.
memcpy(coder->filters, temp, sizeof(temp));
return LZMA_OK;
error:
lzma_filters_free(temp, allocator);
return ret;
}
@ -319,7 +333,7 @@ stream_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator,
// Initialize the Block encoder. This way we detect unsupported
// filter chains when initializing the Stream encoder instead of
// giving an error after Stream Header has already written out.
// giving an error after Stream Header has already been written out.
return stream_encoder_update(coder, allocator, filters, NULL);
}

243
Externals/liblzma/common/stream_encoder_mt.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_encoder_mt.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "filter_encoder.h"
@ -85,6 +84,11 @@ struct worker_thread_s {
/// Compression options for this Block
lzma_block block_options;
/// Filter chain for this thread. By copying the filters array
/// to each thread it is possible to change the filter chain
/// between Blocks using lzma_filters_update().
lzma_filter filters[LZMA_FILTERS_MAX + 1];
/// Next structure in the stack of free worker threads.
worker_thread *next;
@ -109,9 +113,22 @@ struct lzma_stream_coder_s {
/// LZMA_FULL_FLUSH or LZMA_FULL_BARRIER is used earlier.
size_t block_size;
/// The filter chain currently in use
/// The filter chain to use for the next Block.
/// This can be updated using lzma_filters_update()
/// after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
lzma_filter filters[LZMA_FILTERS_MAX + 1];
/// A copy of filters[] will be put here when attempting to get
/// a new worker thread. This will be copied to a worker thread
/// when a thread becomes free and then this cache is marked as
/// empty by setting [0].id = LZMA_VLI_UNKNOWN. Without this cache
/// the filter options from filters[] would get uselessly copied
/// multiple times (allocated and freed) when waiting for a new free
/// worker thread.
///
/// This is freed if filters[] is updated via lzma_filters_update().
lzma_filter filters_cache[LZMA_FILTERS_MAX + 1];
/// Index to hold sizes of the Blocks
lzma_index *index;
@ -133,6 +150,9 @@ struct lzma_stream_coder_s {
/// Output buffer queue for compressed data
lzma_outq outq;
/// How much memory to allocate for each lzma_outbuf.buf
size_t outbuf_alloc_size;
/// Maximum wait time if cannot use all the input and cannot
/// fill the output buffer. This is in milliseconds.
@ -196,7 +216,7 @@ worker_error(worker_thread *thr, lzma_ret ret)
static worker_state
worker_encode(worker_thread *thr, worker_state state)
worker_encode(worker_thread *thr, size_t *out_pos, worker_state state)
{
assert(thr->progress_in == 0);
assert(thr->progress_out == 0);
@ -205,12 +225,9 @@ worker_encode(worker_thread *thr, worker_state state)
thr->block_options = (lzma_block){
.version = 0,
.check = thr->coder->stream_flags.check,
.compressed_size = thr->coder->outq.buf_size_max,
.compressed_size = thr->outbuf->allocated,
.uncompressed_size = thr->coder->block_size,
// TODO: To allow changing the filter chain, the filters
// array must be copied to each worker_thread.
.filters = thr->coder->filters,
.filters = thr->filters,
};
// Calculate maximum size of the Block Header. This amount is
@ -234,12 +251,12 @@ worker_encode(worker_thread *thr, worker_state state)
size_t in_pos = 0;
size_t in_size = 0;
thr->outbuf->size = thr->block_options.header_size;
const size_t out_size = thr->coder->outq.buf_size_max;
*out_pos = thr->block_options.header_size;
const size_t out_size = thr->outbuf->allocated;
do {
mythread_sync(thr->mutex) {
// Store in_pos and out_pos into *thr so that
// Store in_pos and *out_pos into *thr so that
// an application may read them via
// lzma_get_progress() to get progress information.
//
@ -247,7 +264,7 @@ worker_encode(worker_thread *thr, worker_state state)
// finishes. Instead, the final values are taken
// later from thr->outbuf.
thr->progress_in = in_pos;
thr->progress_out = thr->outbuf->size;
thr->progress_out = *out_pos;
while (in_size == thr->in_size
&& thr->state == THR_RUN)
@ -277,8 +294,8 @@ worker_encode(worker_thread *thr, worker_state state)
ret = thr->block_encoder.code(
thr->block_encoder.coder, thr->allocator,
thr->in, &in_pos, in_limit, thr->outbuf->buf,
&thr->outbuf->size, out_size, action);
} while (ret == LZMA_OK && thr->outbuf->size < out_size);
out_pos, out_size, action);
} while (ret == LZMA_OK && *out_pos < out_size);
switch (ret) {
case LZMA_STREAM_END:
@ -313,10 +330,10 @@ worker_encode(worker_thread *thr, worker_state state)
return state;
// Do the encoding. This takes care of the Block Header too.
thr->outbuf->size = 0;
*out_pos = 0;
ret = lzma_block_uncomp_encode(&thr->block_options,
thr->in, in_size, thr->outbuf->buf,
&thr->outbuf->size, out_size);
out_pos, out_size);
// It shouldn't fail.
if (ret != LZMA_OK) {
@ -367,11 +384,13 @@ worker_start(void *thr_ptr)
}
}
size_t out_pos = 0;
assert(state != THR_IDLE);
assert(state != THR_STOP);
if (state <= THR_FINISH)
state = worker_encode(thr, state);
state = worker_encode(thr, &out_pos, state);
if (state == THR_EXIT)
break;
@ -387,14 +406,17 @@ worker_start(void *thr_ptr)
}
mythread_sync(thr->coder->mutex) {
// Mark the output buffer as finished if
// no errors occurred.
thr->outbuf->finished = state == THR_FINISH;
// If no errors occurred, make the encoded data
// available to be copied out.
if (state == THR_FINISH) {
thr->outbuf->pos = out_pos;
thr->outbuf->finished = true;
}
// Update the main progress info.
thr->coder->progress_in
+= thr->outbuf->uncompressed_size;
thr->coder->progress_out += thr->outbuf->size;
thr->coder->progress_out += out_pos;
thr->progress_in = 0;
thr->progress_out = 0;
@ -407,6 +429,8 @@ worker_start(void *thr_ptr)
}
// Exiting, free the resources.
lzma_filters_free(thr->filters, thr->allocator);
mythread_mutex_destroy(&thr->mutex);
mythread_cond_destroy(&thr->cond);
@ -490,6 +514,7 @@ initialize_new_thread(lzma_stream_coder *coder,
thr->progress_in = 0;
thr->progress_out = 0;
thr->block_encoder = LZMA_NEXT_CODER_INIT;
thr->filters[0].id = LZMA_VLI_UNKNOWN;
if (mythread_create(&thr->thread_id, &worker_start, thr))
goto error_thread;
@ -519,6 +544,18 @@ get_thread(lzma_stream_coder *coder, const lzma_allocator *allocator)
if (!lzma_outq_has_buf(&coder->outq))
return LZMA_OK;
// That's also true if we cannot allocate memory for the output
// buffer in the output queue.
return_if_error(lzma_outq_prealloc_buf(&coder->outq, allocator,
coder->outbuf_alloc_size));
// Make a thread-specific copy of the filter chain. Put it in
// the cache array first so that if we cannot get a new thread yet,
// the allocation is ready when we try again.
if (coder->filters_cache[0].id == LZMA_VLI_UNKNOWN)
return_if_error(lzma_filters_copy(
coder->filters, coder->filters_cache, allocator));
// If there is a free structure on the stack, use it.
mythread_sync(coder->mutex) {
if (coder->threads_free != NULL) {
@ -541,7 +578,16 @@ get_thread(lzma_stream_coder *coder, const lzma_allocator *allocator)
mythread_sync(coder->thr->mutex) {
coder->thr->state = THR_RUN;
coder->thr->in_size = 0;
coder->thr->outbuf = lzma_outq_get_buf(&coder->outq);
coder->thr->outbuf = lzma_outq_get_buf(&coder->outq, NULL);
// Free the old thread-specific filter options and replace
// them with the already-allocated new options from
// coder->filters_cache[]. Then mark the cache as empty.
lzma_filters_free(coder->thr->filters, allocator);
memcpy(coder->thr->filters, coder->filters_cache,
sizeof(coder->filters_cache));
coder->filters_cache[0].id = LZMA_VLI_UNKNOWN;
mythread_cond_signal(&coder->thr->cond);
}
@ -598,7 +644,7 @@ stream_encode_in(lzma_stream_coder *coder, const lzma_allocator *allocator,
}
if (block_error) {
lzma_ret ret;
lzma_ret ret = LZMA_OK; // Init to silence a warning.
mythread_sync(coder->mutex) {
ret = coder->thread_error;
@ -627,9 +673,13 @@ wait_for_work(lzma_stream_coder *coder, mythread_condtime *wait_abs,
// to true here and calculate the absolute time when
// we must return if there's nothing to do.
//
// The idea of *has_blocked is to avoid unneeded calls
// to mythread_condtime_set(), which may do a syscall
// depending on the operating system.
// This way if we block multiple times for short moments
// less than "timeout" milliseconds, we will return once
// "timeout" amount of time has passed since the *first*
// blocking occurred. If the absolute time was calculated
// again every time we block, "timeout" would effectively
// be meaningless if we never consecutively block longer
// than "timeout" ms.
*has_blocked = true;
mythread_condtime_set(wait_abs, &coder->cond, coder->timeout);
}
@ -692,7 +742,7 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator,
// These are for wait_for_work().
bool has_blocked = false;
mythread_condtime wait_abs;
mythread_condtime wait_abs = { 0 };
while (true) {
mythread_sync(coder->mutex) {
@ -700,11 +750,11 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator,
ret = coder->thread_error;
if (ret != LZMA_OK) {
assert(ret != LZMA_STREAM_END);
break;
break; // Break out of mythread_sync.
}
// Try to read compressed data to out[].
ret = lzma_outq_read(&coder->outq,
ret = lzma_outq_read(&coder->outq, allocator,
out, out_pos, out_size,
&unpadded_size,
&uncompressed_size);
@ -715,6 +765,10 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator,
ret = lzma_index_append(coder->index,
allocator, unpadded_size,
uncompressed_size);
if (ret != LZMA_OK) {
threads_stop(coder, false);
return ret;
}
// If we didn't fill the output buffer yet,
// try to read more data. Maybe the next
@ -724,8 +778,7 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator,
}
if (ret != LZMA_OK) {
// coder->thread_error was set or
// lzma_index_append() failed.
// coder->thread_error was set.
threads_stop(coder, false);
return ret;
}
@ -846,8 +899,8 @@ stream_encoder_mt_end(void *coder_ptr, const lzma_allocator *allocator)
threads_end(coder, allocator);
lzma_outq_end(&coder->outq, allocator);
for (size_t i = 0; coder->filters[i].id != LZMA_VLI_UNKNOWN; ++i)
lzma_free(coder->filters[i].options, allocator);
lzma_filters_free(coder->filters, allocator);
lzma_filters_free(coder->filters_cache, allocator);
lzma_next_end(&coder->index_encoder, allocator);
lzma_index_end(coder->index, allocator);
@ -860,6 +913,45 @@ stream_encoder_mt_end(void *coder_ptr, const lzma_allocator *allocator)
}
static lzma_ret
stream_encoder_mt_update(void *coder_ptr, const lzma_allocator *allocator,
const lzma_filter *filters,
const lzma_filter *reversed_filters
lzma_attribute((__unused__)))
{
lzma_stream_coder *coder = coder_ptr;
// Applications shouldn't attempt to change the options when
// we are already encoding the Index or Stream Footer.
if (coder->sequence > SEQ_BLOCK)
return LZMA_PROG_ERROR;
// For now the threaded encoder doesn't support changing
// the options in the middle of a Block.
if (coder->thr != NULL)
return LZMA_PROG_ERROR;
// Check if the filter chain seems mostly valid. See the comment
// in stream_encoder_mt_init().
if (lzma_raw_encoder_memusage(filters) == UINT64_MAX)
return LZMA_OPTIONS_ERROR;
// Make a copy to a temporary buffer first. This way the encoder
// state stays unchanged if an error occurs in lzma_filters_copy().
lzma_filter temp[LZMA_FILTERS_MAX + 1];
return_if_error(lzma_filters_copy(filters, temp, allocator));
// Free the options of the old chain as well as the cache.
lzma_filters_free(coder->filters, allocator);
lzma_filters_free(coder->filters_cache, allocator);
// Copy the new filter chain in place.
memcpy(coder->filters, temp, sizeof(temp));
return LZMA_OK;
}
/// Options handling for lzma_stream_encoder_mt_init() and
/// lzma_stream_encoder_mt_memusage()
static lzma_ret
@ -886,20 +978,18 @@ get_options(const lzma_mt *options, lzma_options_easy *opt_easy,
*filters = opt_easy->filters;
}
// Block size
if (options->block_size > 0) {
if (options->block_size > BLOCK_SIZE_MAX)
return LZMA_OPTIONS_ERROR;
// If the Block size is not set, determine it from the filter chain.
if (options->block_size > 0)
*block_size = options->block_size;
} else {
// Determine the Block size from the filter chain.
else
*block_size = lzma_mt_block_size(*filters);
if (*block_size == 0)
return LZMA_OPTIONS_ERROR;
assert(*block_size <= BLOCK_SIZE_MAX);
}
// UINT64_MAX > BLOCK_SIZE_MAX, so the second condition
// should be optimized out by any reasonable compiler.
// The second condition should be there in the unlikely event that
// the macros change and UINT64_MAX < BLOCK_SIZE_MAX.
if (*block_size > BLOCK_SIZE_MAX || *block_size == UINT64_MAX)
return LZMA_OPTIONS_ERROR;
// Calculate the maximum amount output that a single output buffer
// may need to hold. This is the same as the maximum total size of
@ -951,14 +1041,16 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator,
&block_size, &outbuf_size_max));
#if SIZE_MAX < UINT64_MAX
if (block_size > SIZE_MAX)
if (block_size > SIZE_MAX || outbuf_size_max > SIZE_MAX)
return LZMA_MEM_ERROR;
#endif
// Validate the filter chain so that we can give an error in this
// function instead of delaying it to the first call to lzma_code().
// The memory usage calculation verifies the filter chain as
// a side effect so we take advatange of that.
// a side effect so we take advantage of that. It's not a perfect
// check though as raw encoder allows LZMA1 too but such problems
// will be caught eventually with Block Header encoder.
if (lzma_raw_encoder_memusage(filters) == UINT64_MAX)
return LZMA_OPTIONS_ERROR;
@ -998,9 +1090,10 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator,
next->code = &stream_encode_mt;
next->end = &stream_encoder_mt_end;
next->get_progress = &get_progress;
// next->update = &stream_encoder_mt_update;
next->update = &stream_encoder_mt_update;
coder->filters[0].id = LZMA_VLI_UNKNOWN;
coder->filters_cache[0].id = LZMA_VLI_UNKNOWN;
coder->index_encoder = LZMA_NEXT_CODER_INIT;
coder->index = NULL;
memzero(&coder->outq, sizeof(coder->outq));
@ -1012,6 +1105,7 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator,
// Basic initializations
coder->sequence = SEQ_STREAM_HEADER;
coder->block_size = (size_t)(block_size);
coder->outbuf_alloc_size = (size_t)(outbuf_size_max);
coder->thread_error = LZMA_OK;
coder->thr = NULL;
@ -1041,15 +1135,16 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator,
// Output queue
return_if_error(lzma_outq_init(&coder->outq, allocator,
outbuf_size_max, options->threads));
options->threads));
// Timeout
coder->timeout = options->timeout;
// Free the old filter chain and copy the new one.
for (size_t i = 0; coder->filters[i].id != LZMA_VLI_UNKNOWN; ++i)
lzma_free(coder->filters[i].options, allocator);
// Free the old filter chain and the cache.
lzma_filters_free(coder->filters, allocator);
lzma_filters_free(coder->filters_cache, allocator);
// Copy the new filter chain.
return_if_error(lzma_filters_copy(
filters, coder->filters, allocator));
@ -1075,6 +1170,31 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator,
}
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
// These are for compatibility with binaries linked against liblzma that
// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7.
// Actually that patch didn't create lzma_stream_encoder_mt@XZ_5.2.2
// but it has been added here anyway since someone might misread the
// RHEL patch and think both @XZ_5.1.2alpha and @XZ_5.2.2 exist.
LZMA_SYMVER_API("lzma_stream_encoder_mt@XZ_5.1.2alpha",
lzma_ret, lzma_stream_encoder_mt_512a)(
lzma_stream *strm, const lzma_mt *options)
lzma_nothrow lzma_attr_warn_unused_result
__attribute__((__alias__("lzma_stream_encoder_mt_52")));
LZMA_SYMVER_API("lzma_stream_encoder_mt@XZ_5.2.2",
lzma_ret, lzma_stream_encoder_mt_522)(
lzma_stream *strm, const lzma_mt *options)
lzma_nothrow lzma_attr_warn_unused_result
__attribute__((__alias__("lzma_stream_encoder_mt_52")));
LZMA_SYMVER_API("lzma_stream_encoder_mt@@XZ_5.2",
lzma_ret, lzma_stream_encoder_mt_52)(
lzma_stream *strm, const lzma_mt *options)
lzma_nothrow lzma_attr_warn_unused_result;
#define lzma_stream_encoder_mt lzma_stream_encoder_mt_52
#endif
extern LZMA_API(lzma_ret)
lzma_stream_encoder_mt(lzma_stream *strm, const lzma_mt *options)
{
@ -1090,6 +1210,23 @@ lzma_stream_encoder_mt(lzma_stream *strm, const lzma_mt *options)
}
#ifdef HAVE_SYMBOL_VERSIONS_LINUX
LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@XZ_5.1.2alpha",
uint64_t, lzma_stream_encoder_mt_memusage_512a)(
const lzma_mt *options) lzma_nothrow lzma_attr_pure
__attribute__((__alias__("lzma_stream_encoder_mt_memusage_52")));
LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@XZ_5.2.2",
uint64_t, lzma_stream_encoder_mt_memusage_522)(
const lzma_mt *options) lzma_nothrow lzma_attr_pure
__attribute__((__alias__("lzma_stream_encoder_mt_memusage_52")));
LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@@XZ_5.2",
uint64_t, lzma_stream_encoder_mt_memusage_52)(
const lzma_mt *options) lzma_nothrow lzma_attr_pure;
#define lzma_stream_encoder_mt_memusage lzma_stream_encoder_mt_memusage_52
#endif
// This function name is a monster but it's consistent with the older
// monster names. :-( 31 chars is the max that C99 requires so in that
// sense it's not too long. ;-)

5
Externals/liblzma/common/stream_flags_common.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_flags_common.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_flags_common.h"

8
Externals/liblzma/common/stream_flags_common.h vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_flags_common.h
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_STREAM_FLAGS_COMMON_H
@ -18,7 +17,10 @@
/// Size of the Stream Flags field
#define LZMA_STREAM_FLAGS_SIZE 2
lzma_attr_visibility_hidden
extern const uint8_t lzma_header_magic[6];
lzma_attr_visibility_hidden
extern const uint8_t lzma_footer_magic[2];

19
Externals/liblzma/common/stream_flags_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_flags_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_flags_common.h"
@ -38,9 +37,12 @@ lzma_stream_header_decode(lzma_stream_flags *options, const uint8_t *in)
// and unsupported files.
const uint32_t crc = lzma_crc32(in + sizeof(lzma_header_magic),
LZMA_STREAM_FLAGS_SIZE, 0);
if (crc != unaligned_read32le(in + sizeof(lzma_header_magic)
+ LZMA_STREAM_FLAGS_SIZE))
if (crc != read32le(in + sizeof(lzma_header_magic)
+ LZMA_STREAM_FLAGS_SIZE)) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return LZMA_DATA_ERROR;
#endif
}
// Stream Flags
if (stream_flags_decode(options, in + sizeof(lzma_header_magic)))
@ -67,15 +69,18 @@ lzma_stream_footer_decode(lzma_stream_flags *options, const uint8_t *in)
// CRC32
const uint32_t crc = lzma_crc32(in + sizeof(uint32_t),
sizeof(uint32_t) + LZMA_STREAM_FLAGS_SIZE, 0);
if (crc != unaligned_read32le(in))
if (crc != read32le(in)) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return LZMA_DATA_ERROR;
#endif
}
// Stream Flags
if (stream_flags_decode(options, in + sizeof(uint32_t) * 2))
return LZMA_OPTIONS_ERROR;
// Backward Size
options->backward_size = unaligned_read32le(in + sizeof(uint32_t));
options->backward_size = read32le(in + sizeof(uint32_t));
options->backward_size = (options->backward_size + 1) * 4;
return LZMA_OK;

13
Externals/liblzma/common/stream_flags_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file stream_flags_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "stream_flags_common.h"
@ -46,8 +45,8 @@ lzma_stream_header_encode(const lzma_stream_flags *options, uint8_t *out)
const uint32_t crc = lzma_crc32(out + sizeof(lzma_header_magic),
LZMA_STREAM_FLAGS_SIZE, 0);
unaligned_write32le(out + sizeof(lzma_header_magic)
+ LZMA_STREAM_FLAGS_SIZE, crc);
write32le(out + sizeof(lzma_header_magic) + LZMA_STREAM_FLAGS_SIZE,
crc);
return LZMA_OK;
}
@ -66,7 +65,7 @@ lzma_stream_footer_encode(const lzma_stream_flags *options, uint8_t *out)
if (!is_backward_size_valid(options))
return LZMA_PROG_ERROR;
unaligned_write32le(out + 4, options->backward_size / 4 - 1);
write32le(out + 4, options->backward_size / 4 - 1);
// Stream Flags
if (stream_flags_encode(options, out + 2 * 4))
@ -76,7 +75,7 @@ lzma_stream_footer_encode(const lzma_stream_flags *options, uint8_t *out)
const uint32_t crc = lzma_crc32(
out + 4, 4 + LZMA_STREAM_FLAGS_SIZE, 0);
unaligned_write32le(out, crc);
write32le(out, crc);
// Magic
memcpy(out + 2 * 4 + LZMA_STREAM_FLAGS_SIZE,

1342
Externals/liblzma/common/string_conversion.c vendored Normal file
View file

File diff suppressed because it is too large Load diff

7
Externals/liblzma/common/vli_decoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file vli_decoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"
@ -72,7 +71,7 @@ lzma_vli_decode(lzma_vli *restrict vli, size_t *vli_pos,
// corrupt.
//
// If we need bigger integers in future, old versions liblzma
// will confusingly indicate the file being corrupt istead of
// will confusingly indicate the file being corrupt instead of
// unsupported. I suppose it's still better this way, because
// in the foreseeable future (writing this in 2008) the only
// reason why files would appear having over 63-bit integers

5
Externals/liblzma/common/vli_encoder.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file vli_encoder.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

5
Externals/liblzma/common/vli_size.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file vli_size.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "common.h"

5
Externals/liblzma/delta/delta_common.c vendored
View file

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file delta_common.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "delta_common.h"

Some files were not shown because too many files have changed in this diff Show more