Best Compression Methods for 'Specific' Games. Q&A

[KaktoR]

Originally Posted by Kitsune1982

Hi all, just wanted to post a brief comment about the old XT.exe b in the _Tools (2019-01-22).7z vs the one in _Tools (2019-03-28).7z, XT.exe

I experimented a little bit on some test files and this is what i discovered.

The game I compressed, Agarest Generations of War took 30 minutes to decompress on a dual core AMD cpu with 8 gb ram and 4 gb vram/1 gb dedicated video. while this might sound normal, i found out something interesting.

the original compression method i used with UltraArc 2900 R6 Full/_Diskpan.bat folder from the Tools (2019-01-22).7z for the second and third set of .bra files was xZ1+srep:m5f+msc+lolz:d256m:mc1023. for the 1st folder which was all videos i just used srep:m5f+LZMA. the third set of files i used the same method as the second set, and the total decompression time was 30 minutes using the old XT/xtool from that package i specified above.

now heres the interesting part. i downloaded and extracted the compression tools from kaktors _Tools (2019-03-28).7z file on his CIUV3 thread, and copy/pasted the largest .bra file which just has .ogg music files stored in it to a seperate test folder, and after modifiying my ciuv3 script a bit to just process the test.bin.001 file i made with diskspan, i recreated the same installer with just the one fille then ran the setup again. this time the installer unpacked the data from that new text .bin file in 1 minute!! the same file that it got stuck on for like 15 minutes from my first compression test unpacked in only one minute. :O the only change i made to the fa.ini script in the ultraarc 2900 r5 FULL/resources directory was changing the decomp setting under xzlib from 100p to t40.

what i dont understand is that i did the same experiment with the compression using the same settings with the old xtool program from the older _tools archive from 1-2019 and it was 3x longer, it took 30 min to install the game where just the newe one unpacked almost a gb of data in under a minute, and on a dual core desktop to boot. moreover when i was on my quad core laptop with 8 gb ram and a 4 gb vram integrated video card, the decompression took about 8 minutes less but it still was almost 20 nin to fully install a 4.7 gb game.

as a final comment, i did use lolz to compress the .bra files along with cxlib but even the lolz compressor processed quickly, it wasnt until the old xtool started prcoessing the last of the files that it hung for like 15 min on a 90 mb file.

needless to same, im pleased with the speed increase. any thoughts or ideas about why an older one was slower than the newer one? only thing that comes to mind is the xtool in the new package was a 64 bit version and the older one was 32 bit. let me know what you all think.

-Kitsune1982

"They couldnt hit an elephant at this dis---" last words of general custer during the civil war.

Addendum Note - 4/11/19

Ok, i tried another experiment. i repacked the game's .bra files that had been giving me trouble
with the new tools using the default settings UltraArc r2900 Full's diskspan folder has, and let it compress. it took a bout an hour or so, i dont reallyknow how long because i started the compression then went to sleep. when i woke up, i edited the setup.ini file in the ciuv3 folder and attempted to reinstall
the game again. this time the whole game took 10 minutes to decompress!!! So for whatever reason, it seems the updated XT.exe file in the recent tools package
Kaktor put together is much faster and more stable than the old one. this is the method i used.

XZ1+srep:m5f+msc+lolz:d512m:mc1023, and left packcmd at t75p and chunk size at 32mb. I have confirmed the newer one works faster and doesnt freeze up slower computers
as much for some reason. In any case, I know which one im using now. leave any feedback below this post.

thanks, have a good one.

-Kitsune1982

"they couldnt hit an elephant at this dis---" famous last words of general custer from custer's last stand during the civil war.
guess those musket rifles can hit even an elephant at that distance.

This is not surprising.

Packing a file, which contains only audio (ogg) files with XTool doesn't make sense. XTool has nothing to work during unpacking audio files (recompression) because XTool had nothing to work on precompression, and therefore it is fast.

You should test your theories with zips, png, pdf or other files which contains streams (as large as possible) to get a accurate time.

[Kitsune1982]

Hi Kaktor. I forgot to mention I also used this same method on the other files which contain data and dds texture as well as other assets such as text, models and so on. the ogg file test was simply that. just a test to see how fast it would extract. after applying the recompression to the other files, the game installed much faster than before. before the whole game took 30 min to install, and after i recompressed the .bra files which contain data textures and other assets with lolz and msc which can compress dds wav and mp3 files, then recompiled the installer, the game installed completely in 10 minutes. just thought i would mention that.

-Kitsune