FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

JustFun 19-05-2025 07:55

Urban Chaos (1999) (GOG)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 531.35 MB
Overall output size: 283.04 MB (Ratio 53.27%)


KaktoR 21-05-2025 01:15

1 Attachment(s)
STALKER Enhanced Edition Trilogy

resources\*.db* contain OGG vorbis audio.
However they have to be extracted and processed with OGGRE_enc and OGGRE_dec and make diff files because some of them will fail with oggre.

Edit: Nevermind there are ogv videos aswell, that's why oggre fails. Luckily they are located in some of the last game archives.
Extract only from the following files:
Code:

Call of Prypiat EE\resources\resources.db0
Call of Prypiat EE\resources\resources.db1
Call of Prypiat EE\resources\resources.db2
Clear Sky EE\resources\resources.db0
Clear Sky EE\resources\resources.db1
Clear Sky EE\resources\resources.db2
Clear Sky EE\resources\resources.db3
Clear Sky EE\resources\resources.db4
Shadow of Chornobyl EE\resources\resources.db0
Shadow of Chornobyl EE\resources\resources.db1
Shadow of Chornobyl EE\resources\resources.db3
Shadow of Chornobyl EE\resources\resources.db4

Attached xtool XTL file to extract oggs vorbis from game files. You only need this for extracting if you don't have other tools to extract the files.
Usage:
1.
Code:

xtool.exe extract <xtl file> <input file> <output folder>
2.
Code:

xtool.exe erase <input file> <input folder> <output xtl file>
3. (after installation)
Code:

xtool.exe decode <output xtl file> <input folder> <input file>

KaktoR 21-05-2025 09:16

Roadcraft
Same engine as Space Marine 2. Processed wwise inside pak files. However there is still room for improvement. Nothing removed (incl. HD pack).

Code:

17:34:18 - Overall input size: 61.47 GB
 17:34:18 - Overall output size: 43.04 GB (Ratio 70.01%)
 17:34:18 - Overall conversion time: 01:51:06


aman71711 23-05-2025 15:02

need help to compress cricket 24
 
1 Attachment(s)
Quote:

Originally Posted by Masquerade (Post 502793)
Cricket 24

Code:

xtool_zstd+lolz
For XTool, use this:

Code:

-mzstd:l3 -df5p -dd5
Used libzstd v1.3.0.
Use XTool v0.7.9.hotfix1 (others will get stuck/crash).

67.4GB ---> 19.4GB

I’ve got all the required plugin DLLs and tools downloaded, including XTool. My Cricket 24 folder is in E:\Cricket 24, but I’m a bit confused about how to use XTool to compress the game properly.

Should I be using the GUI or the command line? And if it’s the command line, could you help me with the exact command for this version of the game?

Would really appreciate a quick walkthrough or any tips you’ve got. Thanks a lot!

trillestdwane 08-06-2025 13:54

I am having trouble trying to compress a game that has extremely long file/folder paths. I keep getting errors despite updating the Windows registry to remove the filepath limit. :eek:

If I ignore it, it just ends up looping and creating large temp files. I also tried changing settings in W10's Local Group Policy Editor and that didn't fix it either. Am I missing something here?

Hexagon123 15-06-2025 19:45

Citizens Unite: Earth x Space
 
Total start size (original): 20.4GB with redist / 20.3GB without redist
Compress with the following:
  • Contains the following files
    • CoEMusic.fc
    • Music.fc
    Code:

    oggre
  • Contains the following files, since SREP likely crashes, so use XTool dedup instead.
    • AnimationsCompressed.fc
    • CoEAnimationsCompressed.fc
    • CoEFonts.fc
    • CoEGameObjects.fc
    • CoELevels.fc
    • Fonts.fc
    • GameObjects.fc
    • Levels.fc
    Code:

    xtool:dd3+lolz
  • Rest of the files: EXEs, other .FC files, etc. RZ has an dedup option
    Code:

    rz

Total packed size (new): 1.5GB

Hexagon123 15-06-2025 20:00

Alice Escaped!
 
Total start size (original): 1.4GB with MP3 and FLAC soundtracks / 1.2GB with MP3 soundtrack, FLAC ripped / 1.1GB without soundtracks

Version 1.1.3 [b10631325]

Compress with the following:
  • .resS, .assets, .(noextension)
    Code:

    srep+lolz

  • Rest of the Files
    Code:

    rz
  • .mp3
    Code:

    mpz

Total packed size (new): 244MB with MP3

wareck 16-06-2025 20:53

Doomer simulator
 
Doomer simulator

Quote:

xtool:c32mb:d1:mue5,k0x7100780079007A0081008200830 08A008B008C009300940095009C009D009E00:mreflate+xto ol:mreflate:dd3+LOLZ_NORMAL

xtool:mreflate:dd3+razor_mtx
Quote:

---------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------- CONVERSION DATE 2025/06/18 --------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------
16:18:11 - Deleting files from the output folder.
16:18:11 - Files have been deleted.
---------------------------------------------------------------------------------------------------------------------------------------------------------
16:18:11 - You have chosen the unlimited type for the output size.
16:18:11 - ISO images will not be created when conversion is complete.
16:18:11 - The computer will remain on when the conversion is complete.
16:18:11 - Selected ARC/DS method for Data1.wrk was: xtool:mue5:k0x7100780079007A008100820083008A008B00 8C009300940095009C009D009E00+xtool:mreflate:mkrake n:mmermaid:core_2.9.10:lib_2+srep_new:490g+lolz:d9 6m:mtb48mb:mt8:ldmf:ldl5:mc1023
16:18:11 - Selected ARC/SPLIT method for Data2.wrk was: xtool:mreflate+srep_new:490g+lolz:d40:mtt1:mt16:mc 1023/$deflatepack=precomp+srep_new:490g+lolz:d40:mtt1:m t16:mc1023/$libpack=xtool:mreflate+srep_new:490g+razor_mtx
---------------------------------------------------------------------------------------------------------------------------------------------------------
16:18:13 - Compression initialized in 18/06/2025 at 16:18:13.
16:18:14 - Starting Data1.wrk archive creation.
17:50:52 - Moving Data1.wrk file to media folder.
17:50:54 - Starting Data2.wrk archive creation.
17:52:59 - Moving Data2.wrk file to media folder.
17:52:59 - Compression finished in 18/06/2025 at 17:52:59.
---------------------------------------------------------------------------------------------------------------------------------------------------------
17:52:59 - Getting size of the game: Doomer simulator
17:52:59 - » Installation folder size: 8,47 GB
17:52:59 - » Data1.wrk file size: 5,49 GB
17:52:59 - » Data2.wrk file size: 96,01 MB
17:52:59 - » Compressed game size: 5,59 GB (Ratio 65.98%)
---------------------------------------------------------------------------------------------------------------------------------------------------------
17:52:59 - Overall input size: 8,47 GB
17:52:59 - Overall output size: 5,59 GB (Ratio 65.98%)
17:52:59 - Overall conversion time: 01:34:45
17:52:59 - Overall conversion media: 1 custom media

wareck 24-06-2025 20:29

Ancestors The Humankind Odyssey
 
Ancestors The Humankind Odyssey:

data1 (pak) :
xtool:mzlib:mreflate:dd3+lolz
data2 (.wem)
msc_tak+srep_new:490g+LZMA-MT
data3 (.bik)
xtool:c1792mb:mbik
data4 (other)
xtool:dd3+lolz

=> arround 3.32gb compressed + installer

JustFun 03-07-2025 08:29

Ben and Ed

Code:

Selected ARC/DS method for Data1a-01.bin was:
srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 6.55 GB
Overall output size: 1.07 GB (Ratio 16.36%)


panker1992 04-07-2025 10:10

Dedup, WIP 2
 
1 Attachment(s)
This is Dedup.exe
part of my archiver i plan to release soon
Please test it

[External compressor:dedup]
header = 0
packcmd = dedup.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = dedup.exe d - - <stdin> <stdout>

shazzla 04-07-2025 13:48

# arc a -i2 -mdedup p:\ded.arc e:\shdw.xt
FreeArc 0.67 (March 15 2014) creating archive: p:\ded.arc
Compressing 1 file, 31,837,733,330 bytes
Compressing shdw.xt 99.9%
Compressing 31,837,733,330 bytes with dedup.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
100.0%=== DEDUPLICATION (XXH3-128) ===
Pass 1: Scanning for block positions (minimal RAM)
Processing 29.65 GB input file...
Progress: 100.00%
Starting collision-free frequency-based deduplication...
Pass 1 complete: 44006478 blocks, 37991985 unique XXH3 hashes
100.0%Collision-Free Analysis: 6014493 duplicate blocks, 2938568 unique hashes cached
Pass 1.75: Loading ONLY duplicate block data into RAM...
100.0%Loaded 2938568 unique duplicate blocks into RAM
100.0%RAM usage: ~5.3% of original data (duplicates only)
Pass 2: Writing all blocks with XXH3 collision-free hints...
100.0%=== DEDUPLICATION RESULTS (XXH3-128) ===
Original Size: 29.65 GB (31837733330 bytes)
Compressed Size: 26.74 GB (28713904996 bytes)
Compression Ratio: 90.19% (1.11:1)
Space Saved: 2.91 GB (9.8%)
Total Blocks: 44006478
Unique Blocks: 37991985 (86.3%)
Duplicate Blocks: 6014493 (13.7%)
Cached Blocks: 2938568 (7.7% of unique)
Hash Algorithm: XXH3-128 (Collision-Free)
RAM breakdown: Tabs=1.31 GB, FreqMap=869.57 MB, Duplicates=1.58 GB, Total=3.74 GB (12.6% of input)
Collision-free deduplication complete!
100.0%
Errorlevel=0

Compressed 1 file, 31,837,733,330 => 28,713,904,996 bytes. Ratio 90.19%
Compression time: cpu 8.25 sec/real 1506.37 sec = 1%. Speed 21.14 mB/s
All OK

# arc t p:\ded.arc
FreeArc 0.67 (March 15 2014) testing archive: p:\ded.arc
Testing 1 file, 31,837,733,330 bytes. Processed 36.9%
ERROR: write error (disk full?) in compression algorithm dedup

#

[External compressor:dedup]
header = 0
packcmd = dedup.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = dedup.exe d - - <stdin> <stdout>


Win10 Enterprise 21H2
Ryzen 7 9700x
32 GB RAM

ATM no more free time to testing.
Thanks for the program! :)
(Maybe later...)

Interesting (another dataset):
Microsoft Windows [Version 10.0.19044.5965]
(c) Microsoft Corporation. All rights reserved.

arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 8.3%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.8%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.2%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.2%
ERROR: write error (disk full?) in compression algorithm dedup

Dragonis40 05-07-2025 03:29

Good morning! I'm trying to compress "Banishers Ghosts Of New Eden" videogame, but i'm having some troubles with .bk2 files. How can i compress thwm? Bpk and Xtool bk2 plugin don't compress at all. What can i compress them? Thank you for you reply!

KaktoR 05-07-2025 06:44

They are kb2n, you cannot compress them.

panker1992 05-07-2025 15:05

Quote:

Originally Posted by shazzla (Post 508151)
# arc a -i2 -mdedup p:\ded.arc e:\shdw.xt
FreeArc 0.67 (March 15 2014) creating archive: p:\ded.arc
Compressing 1 file, 31,837,733,330 bytes
Compressing shdw.xt 99.9%
Compressing 31,837,733,330 bytes with dedup.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
100.0%=== DEDUPLICATION (XXH3-128) ===
Pass 1: Scanning for block positions (minimal RAM)
Processing 29.65 GB input file...
Progress: 100.00%
Starting collision-free frequency-based deduplication...
Pass 1 complete: 44006478 blocks, 37991985 unique XXH3 hashes
100.0%Collision-Free Analysis: 6014493 duplicate blocks, 2938568 unique hashes cached
Pass 1.75: Loading ONLY duplicate block data into RAM...
100.0%Loaded 2938568 unique duplicate blocks into RAM
100.0%RAM usage: ~5.3% of original data (duplicates only)
Pass 2: Writing all blocks with XXH3 collision-free hints...
100.0%=== DEDUPLICATION RESULTS (XXH3-128) ===
Original Size: 29.65 GB (31837733330 bytes)
Compressed Size: 26.74 GB (28713904996 bytes)
Compression Ratio: 90.19% (1.11:1)
Space Saved: 2.91 GB (9.8%)
Total Blocks: 44006478
Unique Blocks: 37991985 (86.3%)
Duplicate Blocks: 6014493 (13.7%)
Cached Blocks: 2938568 (7.7% of unique)
Hash Algorithm: XXH3-128 (Collision-Free)
RAM breakdown: Tabs=1.31 GB, FreqMap=869.57 MB, Duplicates=1.58 GB, Total=3.74 GB (12.6% of input)
Collision-free deduplication complete!
100.0%
Errorlevel=0

Compressed 1 file, 31,837,733,330 => 28,713,904,996 bytes. Ratio 90.19%
Compression time: cpu 8.25 sec/real 1506.37 sec = 1%. Speed 21.14 mB/s
All OK

# arc t p:\ded.arc
FreeArc 0.67 (March 15 2014) testing archive: p:\ded.arc
Testing 1 file, 31,837,733,330 bytes. Processed 36.9%
ERROR: write error (disk full?) in compression algorithm dedup

#

[External compressor:dedup]
header = 0
packcmd = dedup.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = dedup.exe d - - <stdin> <stdout>


Win10 Enterprise 21H2
Ryzen 7 9700x
32 GB RAM

ATM no more free time to testing.
Thanks for the program! :)
(Maybe later...)

Interesting (another dataset):
Microsoft Windows [Version 10.0.19044.5965]
(c) Microsoft Corporation. All rights reserved.

arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 8.3%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.8%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.2%
ERROR: write error (disk full?) in compression algorithm dedup


arc t e:\ixi-dedup.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\ixi-dedup.arc
Testing 178 files, 12,435,931,640 bytes. Processed 15.2%
ERROR: write error (disk full?) in compression algorithm dedup

Hello, i am trying to understand what you are doing ...it's working fine here

shazzla 05-07-2025 23:49

As you see,decompression doesnt work for me.
What can i do?!

panker1992 06-07-2025 07:39

1 Attachment(s)
Try this one and tell me

shazzla 06-07-2025 12:15

Its bit-identical with first version.

wareck 06-07-2025 15:31

Quote:

Originally Posted by shazzla (Post 508166)
Its bit-identical with first version.

It works for me
Don't forget to update arc.ini
And you need to have dedup.exe both for compress and decompression.
you may also need to have admin rights to uncompress

panker1992 08-07-2025 06:33

i think when i wrote this i had shared memory technology activated in one module :/ i will check and recompile one more version that doesnt use shared memory as this isnt needed here

L0v3craft 08-07-2025 21:54

Someone checked "Tony Hawk's Pro Skater 3 + 4" ? The pak files are encrypted. I got the key and tried with "unrealengine_R13" and after zlib/reflate and kraken too. The precompression doesn't work. I guess we need of an updated plugin?

Masquerade 09-07-2025 04:08

^^
Tony Hawk 1+2 used a custom algo.
You could always rebuild the paks like I did - but that's not lossless + CRC perfect

Jarmen_Kell 05-08-2025 09:57

Any suggestion for "Le Mans Ultimate"?

KaktoR 05-08-2025 10:14

Sample?

Edit: Waste of time. Use reflate on *.mas files...
Code:

Streams: 105 / 105
Time: 00:00:13 (CPU 00:01:02)

Size: 101 MB >> 749 MB


trillestdwane 14-08-2025 19:10

I am having a bit of trouble trying to compress certain Unity games. When I try to use xtool:munity,lz4hc,l12:lz4_193 specifically, it compresses just fine. When I go to test unpacking in both DiskSpanGUI and an installer (using the Setup.dll generated from DiskSpanGUI post-compression) it always fails. I do not have these issues when I use xtool:c32mb:munity,lz4hc,l12, but the former command can get a better ratio.

KaktoR 14-08-2025 20:27

Try another lz4 library version, such as lz4_190 up to lz4_194.

If nothing works send a sample file where extracting fails.

Himanshu 15-08-2025 08:41

Red Dead Redemption
 
1 Attachment(s)
Quote:

Originally Posted by ZAZA4EVER (Post 506461)
Red Dead Redemption (Test)

Files Test (381MB)

Code:

"mapres.rpf"
"blackwater.rpf"

Method ZSTD (ZSTD file attached)

Code:

-mzstd,l19
Result

Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,213,731,228 bytes. Ratio 303.58%
Compression time: cpu 0.61 sec/real 44.09 sec = 1%. Speed 9.07 mB/s
All OK

You can use Zlib+Zstd and get some streams

Code:

-mzlib+zstd,l19
Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,224,533,926 bytes. Ratio 306.28%
Compression time: cpu 0.61 sec/real 46.82 sec = 1%. Speed 8.54 mB/s
All OK

You can use Xtool Ver 0.4.3 to Ver 0.6.0 all works fine with ZSTD

Special thanks to Masquerade , Fitgirl and Kaktor

Dear ZAZA4EVER
I tried all versions of Xtool from 0.4.0 to 0.6.9 , with zstd but none of them gave me a ratio of more than 100% , I also tried with the libzstd.dll shared by you , but that didn't also work.
Kindly help me out.
Or can you share the Xtool version that you used , to compress the game with it's arc.ini file 🙏

Masquerade 16-08-2025 14:10

^^
It's all in the levels, re-read ZAZA4EVER's post and you will see that you need to lock XTool's precompressor at ZSTD level 19.

-mzstd,l19

wareck 16-08-2025 22:57

Quote:

Originally Posted by L0v3craft (Post 508172)
Someone checked "Tony Hawk's Pro Skater 3 + 4" ? The pak files are encrypted. I got the key and tried with "unrealengine_R13" and after zlib/reflate and kraken too. The precompression doesn't work. I guess we need of an updated plugin?

Hello
does someone progress on it ?

Himanshu 17-08-2025 10:32

Red Dead Redemption
 
Quote:

Originally Posted by Masquerade (Post 508357)
^^
It's all in the levels, re-read ZAZA4EVER's post and you will see that you need to lock XTool's precompressor at ZSTD level 19.

-mzstd,l19

Thanks Masquerade , it's working now
Earlier I was using -mzstd,l19 but it didn't work
Then I tried -mzlib+zstd:l19 & it worked

JustFun 25-08-2025 06:52

Assassins Creed: Mirage (v1.06 - No Denuvo Version) (English Only)
Ripped: ACMirage_plus.exe since it's not needed

For .FORGE files use:
Code:

xtool:manvil:core_2.8.4:dd3+lolz
For Sound files use:
Code:

xtool:dd3+bcm
For rest of the files use:
Code:

srep_new:490g+lolz
------------------------------------------------------------------------------------------

Game size: 37.5 GB
Compressed Game Size: 18.5 GB

wareck 26-08-2025 02:08

Mafia The Old Country
 
I'm working on "Mafia The Old Country"

I found that:
For *.pak oodle kraken core_2.9.10 lib_2 and maybe ue4 (better ratio with)
but for *.ucas seems to be encrypted => no compression ratio

here a sample
https://mega.nz/folder/3kwS2LaA#7q_kcF1WkgZYFgn8RxJ5YQ

KaktoR 26-08-2025 02:35

Quote:

Originally Posted by wareck (Post 508398)
I'm working on "Mafia The Old Country"

I found that:
For *.pak oodle kraken core_2.9.10 lib_2 and maybe ue4 (better ratio with)
but for *.ucas seems to be encrypted => no compression ratio

here a sample
https://mega.nz/folder/3kwS2LaA#7q_kcF1WkgZYFgn8RxJ5YQ

Code:

0x79CC9CCE12698C8D04B7C69FC8D82691270C75BE28A55F01EE95F62AC37BDF25

trillestdwane 27-08-2025 13:49

What's the method for Metal Gear Solid Delta? Tried Kraken, only got a 70.9 GB result, I think I did it wrong. :eek:

Code:

xtool:d1:mue5:mkraken:core_2.9.5+srep+lolz

ScOOt3r 27-08-2025 13:51

Metal Gear Solid Delta is OODLE-Kraken

dixen 27-08-2025 21:47

What about *.war, *.upk & *.tfc from GoW Reloaded?

KaktoR 28-08-2025 00:25

https://www.patreon.com/posts/gears-of-war-137428559

JustFun 28-08-2025 15:38

Watchmen: The End Is Nigh Part 2

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 907,15 MB
Overall output size: 649,43 MB (Ratio 71.59%)


JustFun 28-08-2025 16:05

Watchmen: The End Is Nigh

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,44 GB
Overall output size: 869,67 MB (Ratio 58.88%)


JustFun 29-08-2025 16:26

Marine Sharpshooter II: Jungle Warfare

Code:

Selected ARC/DS method for Data1a-01.bin was:
msc_tak+srep_new:490g+razor_mtx
------------------------------------------------------------------------------------------
Overall input size: 1,10 GB
Overall output size: 367,44 MB (Ratio 32.53%)



All times are GMT -7. The time now is 02:57.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com