Tools for Repacking

[kj911]

Could someone make a 64-bit version of this "t5lzma.exe" (source code) program? (While maintaining compatibility.) The 32-bit version only supports dictionary sizes up to "-d27", it no longer supports "-d28", although it is a good question whether it could handle higher ones. It is noticeably slower than the classic 7z/LZMA, but perhaps it could be minimally better.

[kj911]

An interesting precomp-fork, 32bit test package. (Only suitable for zlib-streams.)

Its compression speed, thread-dependent, can be up to 50+ MB/s. The decompression speed is slightly minimally faster. (Unfortunately, it also has quite strong limitations.)

Original author's description:

__________ __________.__ ._____.
\______ \_______ ____ ____ ____ _____ ______ \____ /| | |__\_ |__
| ___/\_ __ \_/ __ \_/ ___\/ _ \ / \\____ \ / / | | | || __ \
| | | | \/\ ___/\ \__( <_> ) Y Y \ |_> > / /_ | |_| || \_\ \
|____| |__| \___ >\___ >____/|__|_| / __/ /_______ \|____/__||___ /
\/ \/ \/|__| \/ \/
===========================================
Precomp Zlib is a program that uses the ideas of Precomp Classic and Xtool with its optimized zlib library. The result is a lighter binary and much smaller code. In this case, we haven't modified the zlib library itself, but rather adjusted the compression level values.

Remember to test the restored file with MD5 and CRC. While it's suitable for general use, the program (who knows?) might perform a poor restoration.

Manual
=======
To expand the file, use: `precomp_zlib.exe e filename.zlib filename.pre -j 2` (two threads) for example.

To recover the file, use: `precomp_zlib.exe d filename.pre filename.zlib -j 2` (two threads) for example.

This program is NOT designed for PNG or ZIP or PDF files. It's for files with zlib compression, such as Unreal Engine .pak files or GOG setup files. It performs wonderfully with these files.

LICENCE:
I DON’T CARE HOW EACH PERSON USES IT

See my "readme2.txt" file from few infos. Sourced from this site posts.

[panker1992]

Archiver, Demo,

Archiver has in-built a patch engine which allows you to create a patch from 2 folders, and then make it into a proprietary format,

The format itself is designed to be streaming, compressed and deduplicated, precompression failed for this as the patches themselves fail to give proper data which precompression can understand.

the reason is because the patch itself isnt aligned data which each algo is designed to work with.


What this streaming, delivers is a seamless performance which will decompress, edit each file verify it and delete the old one, you can use the folder to patch as well in case the you dont like the idea of having a prebaked archive format,

The API itself in Inno has the ability to take in such a format and applying it, and still maintaining the progress in a progressbar.

Added Edit:
Archiver IO Massive, exec2 in the works


@echo off

archiver i -exec2 -c0="D:\Epica" -c1="ffmpeg.exe" -c2="-i" -c4="-vn -b:a 256k -y" -c5="mp3" -c6=".ogg"

pause

[Razor12911]

LZ4 dll collection

You can obtain more from here

[Lord.Freddy]

Changelog:

▪︎ Core rewrite — the C++/setjmp-coroutine wrapper was replaced with a portable C89 state-machine design: smaller, allocation-free on the hot path, and far easier to audit and maintain.
▪︎ Reliable DLL handling — MpzSlimmer.dll is loaded by absolute path from the executable's own directory, with PE bounds/signature validation of the patch site and hardened search-path handling (no DLL-hijack surface).
▪︎ Crash-safe codec execution — codec faults and unexpected failures are trapped and reported as clean errors instead of hanging or dying silently.
▪︎ Robust Win32 I/O — rewritten buffering, seeking and partial read/write handling, with uniform support for regular files and stdin/stdout pipes.
▪︎ Large-file (2 GB) protection — streams exceeding the codec's signed 32-bit limit are detected and rejected before any output is written.
▪︎ Configurable I/O buffer — buffer size is now selectable on the command line (clamped to a safe range) to trade throughput against memory use.
▪︎ Progress reporting — file-to-file operations display live progress; suppressed automatically for pipes.
▪︎ Improved CLI — explicit c / d modes, "-" for stdin/stdout, optional buffer size, and -h / --help.
▪︎ Security & build hardening — ASLR/DEP enforced, no RWX/self-modifying memory, and an embedded version resource + application manifest; together these cut antivirus false positives.
▪︎ Dual-toolchain build — compiles warning-clean under both MSVC (cl) and MinGW GCC/Clang from a single Makefile.
▪︎ General robustness — EOF, empty input, same-file in/out, and failure cleanup (incomplete output is removed) are now handled explicitly rather than relying on old coroutine side effects.

In short: v2 is a substantial rewrite focused on reliability, safe error handling, better I/O, and a cleaner, more portable codebase.
Compiled with: clang-14.2.0-mingw-w64msvcrt-12.0.0

Full credit to Eugene Shelwein for the original MPZ wrapper

[Lord.Freddy]

Originally Posted by Lord.Freddy

Changelog:

▪︎ Core rewrite — the C++/setjmp-coroutine wrapper was replaced with a portable C89 state-machine design: smaller, allocation-free on the hot path, and far easier to audit and maintain.
▪︎ Reliable DLL handling — MpzSlimmer.dll is loaded by absolute path from the executable's own directory, with PE bounds/signature validation of the patch site and hardened search-path handling (no DLL-hijack surface).
▪︎ Crash-safe codec execution — codec faults and unexpected failures are trapped and reported as clean errors instead of hanging or dying silently.
▪︎ Robust Win32 I/O — rewritten buffering, seeking and partial read/write handling, with uniform support for regular files and stdin/stdout pipes.
▪︎ Large-file (2 GB) protection — streams exceeding the codec's signed 32-bit limit are detected and rejected before any output is written.
▪︎ Configurable I/O buffer — buffer size is now selectable on the command line (clamped to a safe range) to trade throughput against memory use.
▪︎ Progress reporting — file-to-file operations display live progress; suppressed automatically for pipes.
▪︎ Improved CLI — explicit c / d modes, "-" for stdin/stdout, optional buffer size, and -h / --help.
▪︎ Security & build hardening — ASLR/DEP enforced, no RWX/self-modifying memory, and an embedded version resource + application manifest; together these cut antivirus false positives.
▪︎ Dual-toolchain build — compiles warning-clean under both MSVC (cl) and MinGW GCC/Clang from a single Makefile.
▪︎ General robustness — EOF, empty input, same-file in/out, and failure cleanup (incomplete output is removed) are now handled explicitly rather than relying on old coroutine side effects.

In short: v2 is a substantial rewrite focused on reliability, safe error handling, better I/O, and a cleaner, more portable codebase.
Compiled with: clang-14.2.0-mingw-w64msvcrt-12.0.0

Full credit to Eugene Shelwein for the original MPZ wrapper

Source code + final compiled executable file added.

[kj911]

Nitro NG 1.05 [FAST ZLIB PRECOMPRESOR] [OPEN SOURCE]

Very shorted descriptions:

================================================================================
          NITRO ENGINE - ARCHITECTURAL CHANGELOG: v1.03 -> v1.05 HP
================================================================================

[SUMMARY]
* v1.03 Baseline: Single-pass linear engine. Isolated from Preflate bugs.
                  Used precise 1-to-1 scanning but lacked memory recycling.
* v1.05 HP:       Hardware-aware system featuring Zlib stream memorization,
                  lock-free lookups, real-time pipeline streaming, thread
                  memory pools, and purist native Win32/POSIX E/S interfaces.

Longer descriptions and x64 Win/Linux binaries here.

My test, detects huge memory footprints, larger than previously posted "precomp_zlib" iterations. Original precomp 0.4.8 still have 24-40MB RAM usage during compress/recompress from testing 124MB ARC archive. Precomp_zlib around ~200MB, NitroNG memory spikes avg. 500MB RAM!

Bad memory mapping code?

My 32bit (XP-compatible) EXE's patched in 4GB RAM limit, avg. 2720-2725MB's memory allocated in 64bit host, before crashing. Speed comparable to Precomp_Zlib iterations. Multithreaded.

[wrathma]

^^ this might be the most certain ai slop posted in this forum.
this project is also from the same dev i think. both of them suffer
from the same problem, the devs dont understand their tools.
they dont know what their tool is supposed to do. probably built
with 2-3 prompts in a single day. dont even get me started on
the issues i see only from the first 100 lines of the code.

edit: even the readme is ai generated and doesnt fully
resemble the tool/code

[kj911]

^^ To be honest, I find it quite strange myself that someone known as a member of the "PC/RIP" scene—especially amidst the current AI craze—would turn to and utilize such tools, given that he was already writing simpler programs even before the advent of AI.

Is it worth pursuing these projects—now or in the future? Obviously, if someone makes use of them and fixes them up, something truly useful could still come of it, couldn't it?

[wrathma]

Originally Posted by kj911

^^ To be honest, I find it quite strange myself that someone known as a member of the "PC/RIP" scene—especially amidst the current AI craze—would turn to and utilize such tools, given that he was already writing simpler programs even before the advent of AI.

who exactly ?
when someone is making a tool atleast he should go and properly
battle test it first before releasing v33.1 or v1.03 in the first day.
also theres a huge difference between ai assisted code and ai slop.
theres alot to fix in this tool. and a lot more to learn. i would say take
reference from xtool source code. compare with xtool, use it as a
benchmark. and actually understand your code instead of accepting
what ai writes for you. of course you will not make it in one day.

i reported a issue and gave a fix on his last tool Oodleforge.
Oodleforge was fixed but the issue persists on this new tool

[kj911]

wrathma: It is KPS—alias KaPiTaLSiN—who writes this software... (there are so many rare PC/RIP tools from the 1990s and 2000s that remain unpublished to this day.)*

You are right that he could test them more thoroughly, identifying and fixing potential bugs in a timely manner. Xtool turned out incredibly fast compared to the original Precomp; anyone needing only ZLIB recompression would choose it over the classic Precomp. True STDIO support is only partial (with the exception of the custom v0.4.8 fork from one of the members here, plus ProFrager’s v0.4.3 package—though the latter is decompression-only). Both Precomp_Zlib and Nitro_NG do have one advantage: they support multithreading. I wonder if the original Precomp’s single-threaded codecs for MP3, SWF, JPG, PDF, GZIP, PNG, Base64, MJPEG, and BZIP2 could somehow be made to run on multiple threads? (Excluding LZMA.) So far, no one has solved this; we’ve only relied on hacks using auxiliary tools like MTX, PMT, and the like.

Nitro_NG Github repo page.

*ex: mp3pack tools, lossily compress many WAV files to single mp3unpack executable.

Lord.Freddy: Many thanks from new MPZ Wrapper.

[Masquerade]

I've updated KaktoR's Oodle library collection to add a few more Oodle libraries I have stored on my hard drive.

Original post here.

Please note that the v numbers in the dll names do not indicate the "version" of Oodle used. neither do the core numbers since these can be easily renamed. Ofc to use these dlls with oo2rec or XTool you will need to remove the _v* after win64.