Tools for Repacking

[wrathma]

Originally Posted by newfolder

OodleForge v33 - OodleLZ Exact-Match Precompressor (open source), read the readme's

instead of -ssd -hddslow -hddfast you could add a in memory buffer. a set size of memory will be allocated at start (ex: 64mb). you first write processed output to this memory buffer and when its full you flush it directly to storage (1 write operation, works on a separate thread) and in the meantime allocate another memory buffer to start again. so instead of a 1gb file doing 4k+ write operations (with -hdd*) it will do only 16-17 write operations.

[newfolder]

Originally Posted by wrathma

instead of -ssd -hddslow -hddfast you could add a in memory buffer. a set size of memory will be allocated at start (ex: 64mb). you first write processed output to this memory buffer and when its full you flush it directly to storage (1 write operation, works on a separate thread) and in the meantime allocate another memory buffer to start again. so instead of a 1gb file doing 4k+ write operations (with -hdd*) it will do only 16-17 write operations.

You right, this changed the code 100%, amazing

[Razor12911]

Originally Posted by newfolder

OodleForge v33.1 The Memory Beast Edition - OodleLZ Exact-Match Precompressor (open source), read the readme's

It's good to see an alternative project in the works and given that it is open source, I would assume it is active development? If so, then I'd like to report a few issues I've discovered from the few tests I've conducted.

Issues

- A significant number of streams are left behind (not detected), scanner needs some improvements
- Even though there's only one game known to use of internal crc checks sendQuantumCRCs (The Crew 2), such data cannot be processed by this project
- spaceSpeedTradeoffBytes cannot be set by user and/or program does not check input for this change meaning games that this program cannot work on games that use the frostbite engine
- this project suffers from the same problem as xtool with leviathan and some hydra streams where they can be detected but not processed at all


The streams that are left behind can be however seen by this tool which can be found here, it's good that it is also open source as well, should help with development.

[newfolder]

Originally Posted by Razor12911

It's good to see an alternative project in the works and given that it is open source, I would assume it is active development? If so, then I'd like to report a few issues I've discovered from the few tests I've conducted.

Thanks a lot for checking it out and for the detailed feedback — really appreciate it coming from the XTool author!
The public repo is only ~2 days old (still v33.1), but I'm actively developing the Trinity Engine branch (Oodle + Unity LZ4 + Zlib + Preflate) with perfect AES roundtrips.
Scanner improvements are my top priority right now. Would love to hear more details about the missed streams or any test files you can share.
Open to collaboration!

https://github.com/johna124/Oodleforge

[kj911]

Could someone make a 64-bit version of this "t5lzma.exe" (source code) program? (While maintaining compatibility.) The 32-bit version only supports dictionary sizes up to "-d27", it no longer supports "-d28", although it is a good question whether it could handle higher ones. It is noticeably slower than the classic 7z/LZMA, but perhaps it could be minimally better.

[kj911]

An interesting precomp-fork, 32bit test package. (Only suitable for zlib-streams.)

Its compression speed, thread-dependent, can be up to 50+ MB/s. The decompression speed is slightly minimally faster. (Unfortunately, it also has quite strong limitations.)

Original author's description:

__________ __________.__ ._____.
\______ \_______ ____ ____ ____ _____ ______ \____ /| | |__\_ |__
| ___/\_ __ \_/ __ \_/ ___\/ _ \ / \\____ \ / / | | | || __ \
| | | | \/\ ___/\ \__( <_> ) Y Y \ |_> > / /_ | |_| || \_\ \
|____| |__| \___ >\___ >____/|__|_| / __/ /_______ \|____/__||___ /
\/ \/ \/|__| \/ \/
===========================================
Precomp Zlib is a program that uses the ideas of Precomp Classic and Xtool with its optimized zlib library. The result is a lighter binary and much smaller code. In this case, we haven't modified the zlib library itself, but rather adjusted the compression level values.

Remember to test the restored file with MD5 and CRC. While it's suitable for general use, the program (who knows?) might perform a poor restoration.

Manual
=======
To expand the file, use: `precomp_zlib.exe e filename.zlib filename.pre -j 2` (two threads) for example.

To recover the file, use: `precomp_zlib.exe d filename.pre filename.zlib -j 2` (two threads) for example.

This program is NOT designed for PNG or ZIP or PDF files. It's for files with zlib compression, such as Unreal Engine .pak files or GOG setup files. It performs wonderfully with these files.

LICENCE:
I DON’T CARE HOW EACH PERSON USES IT

See my "readme2.txt" file from few infos. Sourced from this site posts.