Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-115988: Add missing ARM64 and RISCV filter in lzma module#115989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
ivq wants to merge24 commits intopython:main
base:main
Choose a base branch
Loading
fromivq:xz_bcj_filters
Open
Show file tree
Hide file tree
Changes from20 commits
Commits
Show all changes
24 commits
Select commitHold shift + click to select a range
68979bc
Add missing ARM64 and RISCV filter in lzma module
ivqFeb 27, 2024
3fa0c17
Suppress doc warning "py:const reference target not found"
ivqFeb 28, 2024
c1e166a
Add lzma version
ivqMar 17, 2024
ea51476
Add ARM64, RISC-V filter test case
ivqMar 17, 2024
6189ceb
Update lzma doc
ivqMar 17, 2024
0f6fd63
Add NEWS entry
ivqMar 17, 2024
9731225
Optimise NEWS
ivqMar 18, 2024
40e9d34
Move version doc section
ivqMar 19, 2024
8c922f7
Add WHATSNEW entry
ivqMar 19, 2024
9cd4cb8
Fix filter chain specs position
ivqMar 19, 2024
8a85c48
Suppress doc warning
ivqMar 22, 2024
32725a7
Elide upstream binary data from the test.
gpsheadMar 30, 2024
6d94a08
Clarify the documentation a bit.
gpsheadMar 30, 2024
c62ca63
Clarifying comment, docs accuracy.
gpsheadMar 30, 2024
045aa40
Expose the version as a number.
gpsheadMar 30, 2024
8f49f69
Merge branch 'main' into xz_bcj_filters
gpsheadMar 30, 2024
832335d
minor version explanation tweak.
gpsheadMar 30, 2024
728f286
Adjust the whats new text.
gpsheadMar 30, 2024
61c4adf
NEWS wording and formatting tweak.
gpsheadMar 30, 2024
98adeee
line length
gpsheadMar 30, 2024
449b722
Merge remote-tracking branch 'origin/main' into xz_bcj_filters
ivqJul 13, 2025
f381ca5
We've missed the 3.13 window
ivqJul 13, 2025
9f0d4d4
Fix accidentally removed newline when merging remote main
ivqJul 13, 2025
1a17fd5
Remove unused import
ivqJul 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 53 additions & 6 deletionsDoc/library/lzma.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -322,6 +322,39 @@ Miscellaneous
feature set.


Information about the version of the lzma library in use is available through
the following constants:


.. data:: LZMA_VERSION
.. data:: LZMA_VERSION_STRING

The version of the lzma C library actually loaded at runtime, in both
integer and string forms.

.. versionadded:: 3.13

.. data:: LZMA_HEADER_VERSION
.. data:: LZMA_HEADER_VERSION_STRING

The version of the lzma library that was used for building the module, in
both integer and string forms. This may be different from the lzma library
actually used at runtime.

.. versionadded:: 3.13

The version number and string formats are as defined in by C library. The
integer is represented in decimal digits as ``jmmmppps`` where ``j`` is the
major version, ``mmm`` is the minor version, ``ppp`` is the patch level, and
``s`` is the "stability indicator" (2 means stable)::

>>> import lzma
>>> lzma.LZMA_VERSION
50020052
>>> lzma.LZMA_VERSION_STRING
'5.2.5'


.. _filter-chain-specs:

Specifying custom filter chains
Expand All@@ -343,12 +376,26 @@ options. Valid filter IDs are as follows:

* Branch-Call-Jump (BCJ) filters:

* :const:`FILTER_X86`
* :const:`FILTER_IA64`
* :const:`FILTER_ARM`
* :const:`FILTER_ARMTHUMB`
* :const:`FILTER_POWERPC`
* :const:`FILTER_SPARC`
* :const:`!FILTER_X86`
* :const:`!FILTER_IA64`
* :const:`!FILTER_ARM`
* :const:`!FILTER_ARMTHUMB`
* :const:`!FILTER_POWERPC`
* :const:`!FILTER_SPARC`

The above work on all lzma runtime library versions.

* :const:`!FILTER_ARM64`

Only works if the lzma version is 5.4.0 or later.

.. versionadded:: 3.13

* :const:`!FILTER_RISCV`

Only works if the lzma version is 5.6.0 or later.

.. versionadded:: 3.13

A filter chain can consist of up to 4 filters, and cannot be empty. The last
filter in the chain must be a compression filter, and any other filters must be
Expand Down
14 changes: 14 additions & 0 deletionsDoc/whatsnew/3.13.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -418,6 +418,20 @@ itertools
than the specified batch size.
(Contributed by Raymond Hettinger in :gh:`113202`.)

lzma
----

* Add :const:`lzma.LZMA_VERSION`, :const:`lzma.LZMA_HEADER_VERSION`, and
related constants. They report the version of the underlying ``lzma`` library
found at runtime and the header versions used at build time.
(Contributed by Chien Wong in :gh:`115988`.)

* Add support of new BCJ filters ARM64 and RISC-V via
:const:`!lzma.FILTER_ARM64` and :const:`!lzma.FILTER_RISCV`. Note that the
new filters will work only if runtime library supports them. ARM64 filter
requires ``lzma`` 5.4.0 or newer while RISC-V requires 5.6.0 or newer.
(Contributed by Chien Wong in :gh:`115988`.)

marshal
-------

Expand Down
3 changes: 3 additions & 0 deletionsLib/lzma.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,7 +13,10 @@
"CHECK_ID_MAX", "CHECK_UNKNOWN",
"FILTER_LZMA1", "FILTER_LZMA2", "FILTER_DELTA", "FILTER_X86", "FILTER_IA64",
"FILTER_ARM", "FILTER_ARMTHUMB", "FILTER_POWERPC", "FILTER_SPARC",
"FILTER_ARM64", "FILTER_RISCV",
"FORMAT_AUTO", "FORMAT_XZ", "FORMAT_ALONE", "FORMAT_RAW",
"LZMA_HEADER_VERSION", "LZMA_HEADER_VERSION_STRING",
"LZMA_VERSION", "LZMA_VERSION_STRING",
"MF_HC3", "MF_HC4", "MF_BT2", "MF_BT3", "MF_BT4",
"MODE_FAST", "MODE_NORMAL", "PRESET_DEFAULT", "PRESET_EXTREME",

Expand Down
15 changes: 15 additions & 0 deletionsLib/test/test_lzma.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
import os
import pickle
import random
import re
import sys
from test import support
import unittest
Expand DownExpand Up@@ -383,6 +384,20 @@ def test_uninitialized_LZMADecompressor_crash(self):
self.assertEqual(LZMADecompressor.__new__(LZMADecompressor).
decompress(bytes()), b'')

def test_riscv_filter_constant_exists(self):
self.assertTrue(lzma.FILTER_RISCV)

def test_arm64_filter_constant_exists(self):
self.assertTrue(lzma.FILTER_ARM64)

def test_lzma_header_versions(self):
self.assertIsInstance(lzma.LZMA_HEADER_VERSION_STRING, str)
self.assertGreater(lzma.LZMA_HEADER_VERSION, 0)

def test_lzma_versions(self):
self.assertIsInstance(lzma.LZMA_VERSION_STRING, str)
self.assertGreater(lzma.LZMA_VERSION, 0)


class CompressDecompressFunctionTestCase(unittest.TestCase):

Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
:mod:`lzma` gains C library version info constants and adds constants to
support the newer BCJ filters for ARM64 and RISC-V.
42 changes: 41 additions & 1 deletionModules/_lzmamodule.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,19 @@
#error "The maximum block size accepted by liblzma is SIZE_MAX."
#endif


/*
* If the lzma.h we're building against is so old as not to define these, this
* provides their equivalent values so that the names remain defined in Python
* regardless of the header versions used at build time.
*/
#ifndef LZMA_FILTER_ARM64
#define LZMA_FILTER_ARM64 LZMA_VLI_C(0x0A)
#endif
#ifndef LZMA_FILTER_RISCV
#define LZMA_FILTER_RISCV LZMA_VLI_C(0x0B)
#endif

/* On success, return value >= 0
On failure, return -1 */
static inline Py_ssize_t
Expand DownExpand Up@@ -372,6 +385,8 @@ lzma_filter_converter(_lzma_state *state, PyObject *spec, void *ptr)
case LZMA_FILTER_ARM:
case LZMA_FILTER_ARMTHUMB:
case LZMA_FILTER_SPARC:
case LZMA_FILTER_ARM64:
case LZMA_FILTER_RISCV:
f->options = parse_filter_spec_bcj(state, spec);
return f->options != NULL;
default:
Expand DownExpand Up@@ -490,7 +505,9 @@ build_filter_spec(const lzma_filter *f)
case LZMA_FILTER_IA64:
case LZMA_FILTER_ARM:
case LZMA_FILTER_ARMTHUMB:
case LZMA_FILTER_SPARC: {
case LZMA_FILTER_SPARC:
case LZMA_FILTER_ARM64:
case LZMA_FILTER_RISCV: {
lzma_options_bcj *options = f->options;
if (options) {
ADD_FIELD(options, start_offset);
Expand DownExpand Up@@ -1551,6 +1568,8 @@ lzma_exec(PyObject *module)
ADD_INT_PREFIX_MACRO(module, FILTER_ARMTHUMB);
ADD_INT_PREFIX_MACRO(module, FILTER_SPARC);
ADD_INT_PREFIX_MACRO(module, FILTER_POWERPC);
ADD_INT_PREFIX_MACRO(module, FILTER_ARM64);
ADD_INT_PREFIX_MACRO(module, FILTER_RISCV);
ADD_INT_PREFIX_MACRO(module, MF_HC3);
ADD_INT_PREFIX_MACRO(module, MF_HC4);
ADD_INT_PREFIX_MACRO(module, MF_BT2);
Expand DownExpand Up@@ -1591,6 +1610,27 @@ lzma_exec(PyObject *module)
return -1;
}

if (PyModule_AddStringConstant(
module, "LZMA_HEADER_VERSION_STRING",
LZMA_VERSION_STRING) < 0) {
return -1;
}
if (PyModule_AddStringConstant(
module, "LZMA_VERSION_STRING",
lzma_version_string()) < 0) {
return -1;
}
PyObject *uint32_obj = PyLong_FromUnsignedLong(LZMA_VERSION);
if (PyModule_AddObject(module, "LZMA_HEADER_VERSION", uint32_obj) < 0) {
Py_XDECREF(uint32_obj);
return -1;
}
uint32_obj = PyLong_FromUnsignedLong(lzma_version_number());
if (PyModule_AddObject(module, "LZMA_VERSION", uint32_obj) < 0) {
Py_XDECREF(uint32_obj);
return -1;
}

return 0;
}

Expand Down
[8]ページ先頭
©2009-2025 Movatter.jp