Newbie Question Thread

[Masquerade]

To all new members, welcome to FileForums!

This thread is for newbies to ask questions regarding learning how to compress their own games. The idea behind this thread is to have an informal pool of knowledge where new members can ask questions and more experienced people can answer without polluting the forum with single question topics.

Anything goes, just please be respectful and follow the Forum Rules (https://www.fileforums.com/showthread.php?t=54193).

If you have a question about compressing a specific game, please ask here (https://www.fileforums.com/showthread.php?t=99554).

[adrianskiloses]

1. What are usually the steps for a repacker to start working on repacking a game? (Assuming all gamefiles are not already packed or encrypted and are "open")
2. What is an easy Inno Setup "template" can you suggest for beginners (and has enough documentation to get started easily) and tutorials for making new ones?
3. Is AutoIt scripts a suitable replacement for batch files?
4. How do I adopt my repacking setup to be suitable and convenient for making xdelta patches?

Thank you.

[Masquerade]

^^
1. If there's no compression or encryption to look for, then jump straight to compressing your archive. Tools like XTool ([1] (https://fileforums.com/showthread.php?t=102832), [2] (https://www.fileforums.com/showthread.php?t=103876)) can act as precompressors for certain algorithms so you don't have to manually decompress the data when you find a compressed game. XTool handles it whilst making your archive.

2. Try this (https://www.fileforums.com/showthread.php?t=104745). All the info you need is there and it is equipped with the required libraries to unpack FreeArc archives.

3. Never used it, batch files are just easier and don't trigger AntiVirus.

4. Making patches is a bit outside the scope of our forum since this is for compressing game backups. However, there are two AIO utilities I recommend to make patches - Anadius' tool (https://github.com/anadius/patcher) and ISXPM. Anadius' tool is easier to use but offers only XDelta whereas ISXPM offers more customisation and choice of patch engine (XDelta, HDiffZ or JDiff).

[adrianskiloses]

Thank you for the reply.
In a previous post you have recommended DiskSpan_GUI (https://fileforums.com/showthread.php?t=104507) for compression, and unfortunately not enough documentation is provided on how to split into several bin files, and how to seperate decompression method for that particular bin files. It is a useful tool, but not much information for now.
I was essentially looking for templates that work with DiskSpan_GUI, but SIS is excellent and simple.

As for precomp, is it necessary to first go through one type of files on the game, running a command/script to compress all of them, then run another command, for example, for the soundtracks, etc? And after being done with all that, what would be the best compression (for .bin files) post-precomping?

Is it preferable to put the decomp scripts and utilities on the bin files, and after extraction, run the scripts to decompress the assets, or some other way is preferred?

I also do not understand the compression method command thingy, an example is this:
srep+LOLZ_NORMAL/$mscpack=msc/$jpgpack=xtool:mbrunsli/$oggrepack=oggre/$mp3pack=mpz/$ddspack=srep+LOLZ_NORMAL/$deflatepack=xt_zlib+srep+LOLZ_NORMAL/$storepack=FA-MIN/$textpack=srep+LZMA

Thank you again.

[KaktoR]

Thank you for the reply.
In a previous post you have recommended DiskSpan_GUI (https://fileforums.com/showthread.php?t=104507) for compression, and unfortunately not enough documentation is provided on how to split into several bin files, and how to seperate decompression method for that particular bin files. It is a useful tool, but not much information for now.
I was essentially looking for templates that work with DiskSpan_GUI, but SIS is excellent and simple.

I wrote a quick guide here: https://fileforums.com/showpost.php?p=500756&postcount=13
In the next days I will write even more informations about how to properly set-up all the settings.

[adrianskiloses]

I wrote a quick guide here: https://fileforums.com/showpost.php?p=500756&postcount=13
In the next days I will write even more informations about how to properly set-up all the settings.

Thank you very much for writing the guide and also for choosing a game that I was working on as well! :D

Just a smol issue (literally :p) some of the screenshots are too small to be able to zoom-in in browser and see clearly, so a little fix pls

But other than that, it is very informational.

[KaktoR]

I have added direct links to imgur.com above the images. They are in the original measurements there.

[adrianskiloses]

Alright, now that the files are compressed, its now time to create the actual setup.exe program. How does one go about that? I noticed that DiskSpan_GUI makes a Setup.dll
which I assume holds all of the decomp program for installing (?) .
However, I might be wrong, so how DO you use the Setup.dll file?
Thanks.

[KaktoR]

Setup.dll is a freearc archive actually. It contains all files needed to unpack the compressed files.

You can use one of the supported installers.
https://www.fileforums.com/showpost.php?p=492987&postcount=3

However you can also use DiskSpan_GUI.exe for installing. Just copy DiskSpan_GUI.exe to the same folder where Setup.dll is and run it.

[adrianskiloses]

You can use one of the supported installers.
https://www.fileforums.com/showpost.php?p=492987&postcount=3

However you can also use DiskSpan_GUI.exe for installing. Just copy DiskSpan_GUI.exe to the same folder where Setup.dll is and run it.

I see. It is quite useful; however I would like to use the supported installers, which, I don't know what to use and how to use them with Setup.dll, etc.

A few pointers would be appreciated :)

Thank you!

[KaktoR]

For all the installers, you have just to make the settings you want, like setting the game name, size and so on, then compile the script. After that you got a setup.exe file which you copy to the same folder where setup.dll and the compressed archives are located and then run setup.exe and install. Simple :)

[adrianskiloses]

Another question:
What if I convert all the mp3 files in a game to ogg beforehand, and then attach a ogg to mp3 converter and a batch script to revert them back to mp3? Will it improve compression ratio, even a little bit depending on how many mp3 files are there?

[KaktoR]

Why would you do this anyways? You have some tools to compress mp3 files. Another thing is, that in most cases you wouldn't be able to convert the ogg files back to mp3 with the correct hashsum, which means you have to create some diff files to patch the converted files back to original.

[Ra7eN]

I have several games that rely on wav files. what would be the best way to go about this. precomp that I know of cannot handle wav files.

thanks!!

Example, I like RIPPING. Skyrim has gigs of wav files that i can lame down to about 10%-25%, but I need to get them back into the bsa. How would I do this?

[Masquerade]

For WAV audio, use MSC with Tak plugin. You can rebuild the BSA archives using BSAOpt.
https://www.nexusmods.com/skyrim/mods/247/

Please note if you recode the wavs to mp3 with lame, the game might not recognise them anymore.

[kj911]

Masquerade: The recompressed BSA archives, betterly compressibble or not?? Or better ways recompressing in store mode (supported?) and packing with any better methods? (ex: msc+srep+lolz)

[Ra7eN]

For WAV audio, use MSC with Tak plugin. You can rebuild the BSA archives using BSAOpt.
https://www.nexusmods.com/skyrim/mods/247/

Please note if you recode the wavs to mp3 with lame, the game might not recognise them anymore.

Hi, thanks for the fast reply!
I have BSAOPT I'll have to take another look at it. I used to do this with other games that use wav and use lame to crush them down. so far no issues, except one game. I had a wav compressor at one time, however, my main issue was getting the files BACK INTO the bsa file LOL

Currently I did the usual precomp+srep basic and got 27GB skyrim to 11GB, but I am sure if I can at least get the wav files down to size, I should be able to get it under 10GB.

[Ra7eN]

Another question:
What if I convert all the mp3 files in a game to ogg beforehand, and then attach a ogg to mp3 converter and a batch script to revert them back to mp3? Will it improve compression ratio, even a little bit depending on how many mp3 files are there?

why are you converting mp3 to ogg?
It appears that you do not have the LAME encoder based on your post. Instead, it seems that you have a decOGG compressor. If this is correct, I recommend replacing it with LAME, which can be easily found online. I'm not sure if I'm allowed to share a link, but you can look for it in various places. LAME will effectively compress your MP3 files, so there's no need to use OGG in this scenario.

[Masquerade]

Masquerade: The recompressed BSA archives, betterly compressibble or not?? Or better ways recompressing in store mode (supported?) and packing with any better methods? (ex: msc+srep+lolz)

BSAopt is not suitable for repacking since every time you make an archive with it, the CRC is different.

I'm not up to scratch with BSA format, but I don't think the media files are compressed. Beyond that, I believe LZ4 compression is used.

[KaktoR]

SkyrimSE using lz4f with level 10 if I remember correct. And if I am not mistaken you have to use lz4 library 1.8.0 or something like that to get a decent amount of streams.

[Ra7eN]

BSAopt is not suitable for repacking since every time you make an archive with it, the CRC is different.

I'm not up to scratch with BSA format, but I don't think the media files are compressed. Beyond that, I believe LZ4 compression is used.

confirmed, they are not compressed. I forgot about the CRC. Does - skyrim(and/or bethesda) make use of that.

[allgravy]

Howdy! Apologies for my grammer and thanks in advance for your help.

How would i add ztool to diskspan please...

[Ra7eN]

OK, I am from the old BBS days (Myth, Paradigm etc..) and have been fascinated by extreme compression. These days I learn it as a hobby. So wanted to thank you all for this site.

Sorry I have a noob question.

I know this is part of an arc bat file (I love working with batch), however what is this part

-mprecomp+srep+lzma:a1:mfbt4:d256m:fb128:mc1000:lc8

Because when I run it,
arc a -lc8 -ep1 -ed -r -w.\ game-1.bin -mprecomp+srep+lzma:a1:mfbt4:d256m:fb128:mc1000:lc8 "D:\GAMES\my_game_file\*"

gives me an error
recent one was general (de)compression error in precomp
other times it looks like it skips both precomp and srep and just finishes susccessfully.

How am I using this incorrectly?

Thanks!

[Masquerade]

^^

The -m parameter in FreeArc specifies the compression method to be used when making your archives.

Methods can be internal methods built into FreeArc or they can be defined in arc.ini as external compressors.

In your case, it looks like precomp and srep are external compressors whereas you are using FreeArc's internal lzma compressor.

Secondly, when you have methods with colons after their name, this is a way of defining parameters for that compressor at runtime.

For example, the oggre compressor by ProFrager, arc.ini would look like this:

[External compressor:oggre]
header = 0
packcmd = {compressor} {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

FreeArc has many internal constants, two of which I have shown above.

{compressor} instructs FA to look for oggre.exe. It simply fills the {compressor} with the name defined at the start of the entry.

{options} is where we pass our arguments at runtime. For example, if I used -moggre:s3, FA would translate the s3 to -s3 and use that parameter for oggre.exe.

[Ra7eN]

@Masquerade

thanks.. easy to follow.
I noticed when reading the "specific game" section they put up a series of commands. Using an example where in the command line is that inserted?

Is there a thread here that goes into more detail. I would like to explore this more in depth.

Thanks!

[Ra7eN]

hI
OK, One final Noob question

regarding the "specific game" section ??

What criteria is needed to determine which method to use.

Lets say for example I have a ton of wav files in a game, or lots of DDS files. What would I need to look for?

Thanks!

[Masquerade]

@Masquerade

thanks.. easy to follow.
I noticed when reading the "specific game" section they put up a series of commands. Using an example where in the command line is that inserted?

Is there a thread here that goes into more detail. I would like to explore this more in depth.

Thanks!
Please may you give an example of what you mean?
hI
OK, One final Noob question

regarding the "specific game" section ??

What criteria is needed to determine which method to use.

Lets say for example I have a ton of wav files in a game, or lots of DDS files. What would I need to look for?

Thanks!
for wavs, use msc with tak plugin. For DDS, use srep then lolz.

[Ra7eN]

^^^
Example: Yes
Here is a good page I think covers the information I am looking for

https://fileforums.com/showpost.php?p=462631&postcount=11

Which goes hand-in-hand with my question for the following you suggested:
for wavs, use msc with tak plugin. For DDS, use srep then lolz

How are these and what you suggested implemented?

Please keep in mind, that I am from the old BBS days where we ripped games in order to get them to make for a decent download on 9,600 - 14.4bps modems. So this method with arc is a little new to me after some 30yrs :-) (ok, Im an old fart)

[KaktoR]

^
You cannot use more then one -m option in command line for a single archive. So either combine them or make two archives.

arc.exe a -s -ep1 -r -ed -w.\game1.arc -mmsc ["path_to_game_folder\*.wav"]
arc.exe a -s -ep1 -r -ed -w.\game2.arc -msrep+lolz ["path_to_game_folder\*.*"] -x["path_to_game_folder\*.wav"]

As for the TAK plugin for MSC, you just have to place "pack_TAK.exe" in the same folder where "msc.exe" is located for compression and use in arc.ini like this
[External compressor:msc]
header = 0
default = -raw=1 -bmf=9 -bmp=1 -wav=1 -mp3=1
packcmd = "MSC.exe" c -f -v {options} InFile OutFile
unpackcmd = "FAZip32.exe" -i0 decompress:msc_d <stdin> <stdout>
datafile = InFile
packedfile = OutFile

[Ra7eN]

I caught that right as I sent that. I am working with too many bat files, so cleaned it up. I'll take another look again

Thanks!

[adrianskiloses]

Best compression method for text files?

[shazzla]

Razor,ppmd.

[kj911]

Special question: This minimal software requirement, prerequisities have installed on freshly installed Win7 x64 machine???

Tested few compressors in pure Win7 x64 installations and get very slow compression speed's. Not installed VC++, NETFX and more...

LOLZ in get around 100-150kB/s speed and not utilize more core/threads..
Chechked in LZTurbo v1.2 its slow.
Checked Razor 1.03.7 its slooow.
Checked Nanozip 0.09 x64 its finely goodly packing via optimum1 mode utilize 8 threads and get avg. 13MB/s compression speed! optimum2 or cc mode its slow. (Tested in highly compressibble data sample, compression ratio higher than 20:1)
Tested WinRAR 5.60 x64 and packing in very goodly avg. 25MB/s speed use, Best/Solid mode and 1GB dictionary. CPU usage 30-60% percentage, probably bottlenecked via 17 years old HDD.

Installed VC++ 2012 packages taken from Catalyst 15.7.1 package and LOLZ compress its faster! Any idea?

Any members I have Xeon CPU??

My PC in has Xeon E5-2665 CPU + 16GB RAM. Will hope compression speed via 1 threads around probably really getting avg. 400-800kB/s or higher. (LDMF mode) Or higher than ~2MB/s via MTT mode and 8+ threads.

[FitGirl]

lolz don't have dependencies.
It's compression speed for 1 thread is slow, you can use faster params or turn on one of two MT modes.

On your Xeon you can utilize all 16 threads, it's also the maximum lolz can use anyway.

Also, ldmf mode is very slow compared to non-ldmf, while giving not much better results than srep+lolz

[sailwhist]

Hello. I reviewed literally everything on this forum which is considered as a "guide", and I still can't understand what I should do and how should I apply all that compression methods and scripts on different kind of files (audio, movies etc.) and moreover how should I determine what is what and which files should I compress with which method?
I reviewed best compression methods index for the game I wanted to compress (gta4) and I have no clue what should I do with that information:
gta iv complete edition 12.9GB
audio file srep+lolz...
you can use msc but i didn't see improve ratio...
movie bkp...
rest of the file zlib:max:c32m+srep:m3f:a2+lolz...
I downloaded DiskSpanGUI and all I understood for now is how to select the input game folder. How should I learn about game compression if there is no comprehensible clear guide for begginers, only some general info like "use XTool and compress your games", but not a single example about how to apply all that scary configs and scripts for a certain files.
Please, if I missed something on this forum what potentially could've helped me - point me on that.

[sailwhist]

There is literally not a single comprehensive step-by-step instruction for beginners on this forum, all I see as a beginner is methods and troubleshooting discussions. Sorry for that, but as a beginner who spent 2 days on this forum I understood nothing, even though I really wanted. Thank you.

[KaktoR]

@sailwhist

I will take your example for GTA4.

Sometimes you already see what a game file could include just by file name or folder name.

For example the following file
GTAIV\pc\audio\sfx\ambient_streams.rpf

"sfx" just means "sound effects" in the audio terminology. This means, you already know what is inside this file or what the file will contain mostly - namely sound/audio files of some sorts.

I don't have the game so I have to check steamdb depot (https://steamdb.info/depot/12211/) for this game and sort the file list by type. What I can see is that literally all *.rpf files contain sound/audio files, just by looking at the file name and/or folder where the files are located.

Another example for this type is
GTAIV\pc\audio\sfx\radio_*.rpf

Just looking at all the file names beginning with radio_ I am pretty sure all these files contain all the music for the radio stations in game, just because I know the word "radio" and the name of the radio station in the file name.

But because I don't have the game nor any of the files, I can just guess that the files contain riff-wave files, mp3 files or raw files or whatever else, because I know msc can handle them mostly. And the post creator you have quoted said "you can use msc but i didn't see improve ratio...", so I think my guess is correct here lol.

Next one: movie files
For bink movies (BIK or BK2) you can mostly use BinkPack (bpk), which will compress this files lossless. Just note that sometimes it will not work, either because the bink files using some sort of alpha channel (https://en.wikipedia.org/wiki/Alpha_compositing), using a codec which bpk cannot handle correctly, or - which applies to BK2 files - have a newer codec which BinkPack cannot handle at all. If you have Bink video files, you can do the following:

Use HxD (https://mh-nexus.de/en/hxd/) and see how the beginning of the file looks like. If the file has a "KB2n" (image (https://imgur.com/p6eizb9.png)) or anything greater then the "n" letter, BinkPack will fail on these files (this is mostly the case in new games, especially games which are build with unreal engine 4/5).
Copy all the Bink video files to a test folder and use bpk on them, testing compression and decompression. Or use bpk file-by-file instead of all at once. This way you can sort out faulty ones of the list and note them somewhere, so you know what files making trouble to exclude them from bpk compression.


Next one: zlib etc.
Again back to steamdb depot for the game GTA4 (https://steamdb.info/depot/12211/), I also see "*img" files, which are 6.91 GiB in size. I would guess that these files contain all sorts of other files and textures. So now, the post you quoted sais "rest of the file zlib:max:c32m+srep:m3f:a2+lolz... ", which is including this *.img files. In other words, it seems that these *.img files are compressed with either zlib or deflate I would say. Devs simply do this to save space. To be sure, you can follow this little guide (https://fileforums.com/showpost.php?p=500772&postcount=18) (to be fair this is for oodle compression, but just ignoring oodle option and using zlib option isn't that complicated).

We always want to extract all the data/streams in the game files if possible, in other words we want to decompress them (xtool/other tools) to have fully possibility of deduplication (srep) and compression (lolz/lzma/whatever else) in order to compress them even better than the original.

[sailwhist]

@sailwhist

I will take your example for GTA4.

Sometimes you already see what a game file could include just by file name or folder name.

For example the following file
GTAIV\pc\audio\sfx\ambient_streams.rpf

"sfx" just means "sound effects" in the audio terminology. This means, you already know what is inside this file or what the file will contain mostly - namely sound/audio files of some sorts.

I don't have the game so I have to check steamdb depot (https://steamdb.info/depot/12211/) for this game and sort the file list by type. What I can see is that literally all *.rpf files contain sound/audio files, just by looking at the file name and/or folder where the files are located.

Another example for this type is
GTAIV\pc\audio\sfx\radio_*.rpf

Just looking at all the file names beginning with radio_ I am pretty sure all these files contain all the music for the radio stations in game, just because I know the word "radio" and the name of the radio station in the file name.

But because I don't have the game nor any of the files, I can just guess that the files contain riff-wave files, mp3 files or raw files or whatever else, because I know msc can handle them mostly. And the post creator you have quoted said "you can use msc but i didn't see improve ratio...", so I think my guess is correct here lol.

Next one: movie files
For bink movies (BIK or BK2) you can mostly use BinkPack (bpk), which will compress this files lossless. Just note that sometimes it will not work, either because the bink files using some sort of alpha channel (https://en.wikipedia.org/wiki/Alpha_compositing), using a codec which bpk cannot handle correctly, or - which applies to BK2 files - have a newer codec which BinkPack cannot handle at all. If you have Bink video files, you can do the following:

Use HxD (https://mh-nexus.de/en/hxd/) and see how the beginning of the file looks like. If the file has a "KB2n" (image (https://imgur.com/p6eizb9.png)) or anything greater then the "n" letter, BinkPack will fail on these files (this is mostly the case in new games, especially games which are build with unreal engine 4/5).
Copy all the Bink video files to a test folder and use bpk on them, testing compression and decompression. Or use bpk file-by-file instead of all at once. This way you can sort out faulty ones of the list and note them somewhere, so you know what files making trouble to exclude them from bpk compression.


Next one: zlib etc.
Again back to steamdb depot for the game GTA4 (https://steamdb.info/depot/12211/), I also see "*img" files, which are 6.91 GiB in size. I would guess that these files contain all sorts of other files and textures. So now, the post you quoted sais "rest of the file zlib:max:c32m+srep:m3f:a2+lolz... ", which is including this *.img files. In other words, it seems that these *.img files are compressed with either zlib or deflate I would say. Devs simply do this to save space. To be sure, you can follow this little guide (https://fileforums.com/showpost.php?p=500772&postcount=18) (to be fair this is for oodle compression, but just ignoring oodle option and using zlib option isn't that complicated).

We always want to extract all the data/streams in the game files if possible, in other words we want to decompress them (xtool/other tools) to have fully possibility of deduplication (srep) and compression (lolz/lzma/whatever else) in order to compress them even better than the original.

Thank you for your assistance, but still I have no clue how to operate with all of these in DiskSpanGUI and how to apply all these methods, select files by files and make everything combined work altogether in a final compressed game.
Talking about audio/movies - I'm beginning to understand where they stored, but I'm not sure how to find every single audio/movie file in game folder to make sure I compressed all of these files with the right method.
Also, I don't understand how to choose exact settings for that method for gta4 (zlib and etc.). In diskspangui it only shows me presets for that but without numbers like in those guy method. There are a lot of settings in that software which scares me.
I literally don't know everything and I would like to see a detailed guide with an example on some game about how to apply all of these so me and others could use that as a complete FAQ on compressing games.
Thank you.

[KaktoR]

Ah, I forget to explain a small but important detail about all the numbers in the method "those guys" sometimes share.

First, you have to keep in mind that tools are changing over time. Old tools like "zlib:max:c32m" (which should be this tool (https://fileforums.com/showthread.php?t=98486) or maybe this (https://fileforums.com/showthread.php?t=99379)) are not used anymore nowadays, because some tools are "evolving" over time, new tools are comming and old ones are obsolete.

So, the guy you quoted posted this method on 14-03-2018, which was over 5 years ago. On this time there was no "xtool". Means, methods and tools can (and most likely will) change over time. The method "zlib:max:c32m" belongs to an old tool. No one is using this tool anymore. "zlib" is the codec, ":max" is some other option I don't remember (I guess it has something to do with stream detection) and ":c32m" is the chunk size used for this tool.

Basically you have to "transfer old methods into new methods". So the old method "zlib:max:c32m" is something like "xtool:c32mb:mzlib" in diskspangui.

Edit:
Again I forgot to tell you "how to use this method on GTA4 for your needs" lol, sorry.

Personally I do all of this configuration stuff through the INI file "Resources\DSG_Settings.ini", just because I find it more easier.

So for your example of GTA4 it could look like this:
[Game1Settings]
Name=Grand Theft Auto 4
Exec=
EGPath=C:\Program Files (x86)\Rockstar Games\Grand Theft Auto 4
MethodState=0
Data1.Name=Data<game><data>-01.bin
Data1.Method1=msc|0
Data1.InclText=*.rpf

Data2.Name=Data<game><data>-01.bin
Data2.Method1=xtool:mzlib+srep+lolz|0
Data2.InclText=*.img

Data3.Name=Data<game><data>-01.bin
Data3.Method1=bpk|0
Data3.InclText=*.bik

Data4.Name=Data<game><data>-01.bin
Data4.Method1=srep+lolz|0
Data4.ExclText=*.img|*.rpf|*.bik

Data1 only includes *.rpf (audio) files, so I would suggest msc over srep+lolz, just because for the fact that lolz is not meant to used with audio files at all. lolz is meant to be used especially for texture files. So I would use msc, which is faster and should have near the same ratio. Sometimes it is a good idea to use even msc+srep, but I cannot say for sure here in this case because I don't have this game to test it.
Data2 includes only *.img files, which probably using some compression like zlib or deflate, so you have to use xtool here. If xtool:mzlib doesn't work, try with another codec like xtool:mpreflate or xtool:mreflate.
Data3 includes only *.bik files. I guess it ok to use bpk method only.
Data4 includes all other files, excluding *.img, *.rpf and *.bik, so you can just use srep+lolz if you want.

|0 indicates that this method belongs to freearc, while |1 is 7-zip (.7z) and |2 is RAR (.rar) (freearc methods cannot be used with |1 or |2). I guess using |1 or |2 are special and absolutely rare cases of some sorts.
InclText and ExclText telling diskspangui/freearc, that only given file types/files with paths should be included in given archive, or exclude them of the given archive. In the example above we have 3 archives which includes specific file types. So we need a last archive which includes all other files except the 3 previous archives, otherwise we would missing all other files on installation which are not exclusively included in other archives. Makes sense, does it? PS: In InclText and ExclText you can (or have to) separate file types/files with paths with a | character.
<game><data> are just placeholders or variables. <game> is actually [Game#Settings] (it takes the number # from [Game1Settings] section, so 1), <data> is actually Data#, which takes the number # from Data#. However you can name the archives to whatever you want. Just make sure to not use any super special characters.

[kj911]

FitGirl: Thanks from reply!

Two MT mode? Or MTT only?? LDMF its supporting multithreaded compression, with lower ratio! Special commands trick to usage.

https://i.kek.sh/bNhofYWbNzw.png
https://i.kek.sh/2udWoqik8Ut.png

Xeon 16 threads with 16GB RAM from LOLZ --> Use 128MB dictionary size. Memory its fully loaded.

srep:m3f:a0:l512+LOLZ:mtt1:mt2/4+ worser ratio than srep:m3f:a0:l512+LOLZ:ldmf1 mode.

[sailwhist]

Ah, I forget to explain a small but important detail about all the numbers in the method "those guys" sometimes share.

First, you have to keep in mind that tools are changing over time. Old tools like "zlib:max:c32m" (which should be this tool (https://fileforums.com/showthread.php?t=98486) or maybe this (https://fileforums.com/showthread.php?t=99379)) are not used anymore nowadays, because some tools are "evolving" over time, new tools are comming and old ones are obsolete.

So, the guy you quoted posted this method on 14-03-2018, which was over 5 years ago. On this time there was no "xtool". Means, methods and tools can (and most likely will) change over time. The method "zlib:max:c32m" belongs to an old tool. No one is using this tool anymore. "zlib" is the codec, ":max" is some other option I don't remember (I guess it has something to do with stream detection) and ":c32m" is the chunk size used for this tool.

Basically you have to "transfer old methods into new methods". So the old method "zlib:max:c32m" is something like "xtool:c32mb:mzlib" in diskspangui.

Edit:
Again I forgot to tell you "how to use this method on GTA4 for your needs" lol, sorry.

Personally I do all of this configuration stuff through the INI file "Resources\DSG_Settings.ini", just because I find it more easier.

So for your example of GTA4 it could look like this:
[Game1Settings]
Name=Grand Theft Auto 4
Exec=
EGPath=C:\Program Files (x86)\Rockstar Games\Grand Theft Auto 4
MethodState=0
Data1.Name=Data<game><data>-01.bin
Data1.Method1=msc|0
Data1.InclText=*.rpf

Data2.Name=Data<game><data>-01.bin
Data2.Method1=xtool:mzlib+srep+lolz|0
Data2.InclText=*.img

Data3.Name=Data<game><data>-01.bin
Data3.Method1=bpk|0
Data3.InclText=*.bik

Data4.Name=Data<game><data>-01.bin
Data4.Method1=srep+lolz|0
Data4.ExclText=*.img|*.rpf|*.bik

Data1 only includes *.rpf (audio) files, so I would suggest msc over srep+lolz, just because for the fact that lolz is not meant to used with audio files at all. lolz is meant to be used especially for texture files. So I would use msc, which is faster and should have near the same ratio. Sometimes it is a good idea to use even msc+srep, but I cannot say for sure here in this case because I don't have this game to test it.
Data2 includes only *.img files, which probably using some compression like zlib or deflate, so you have to use xtool here. If xtool:mzlib doesn't work, try with another codec like xtool:mpreflate or xtool:mreflate.
Data3 includes only *.bik files. I guess it ok to use bpk method only.
Data4 includes all other files, excluding *.img, *.rpf and *.bik, so you can just use srep+lolz if you want.

|0 indicates that this method belongs to freearc, while |1 is 7-zip (.7z) and |2 is RAR (.rar) (freearc methods cannot be used with |1 or |2). I guess using |1 or |2 are special and absolutely rare cases of some sorts.
InclText and ExclText telling diskspangui/freearc, that only given file types/files with paths should be included in given archive, or exclude them of the given archive. In the example above we have 3 archives which includes specific file types. So we need a last archive which includes all other files except the 3 previous archives, otherwise we would missing all other files on installation which are not exclusively included in other archives. Makes sense, does it? PS: In InclText and ExclText you can (or have to) separate file types/files with paths with a | character.
<game><data> are just placeholders or variables. <game> is actually [Game#Settings] (it takes the number # from [Game1Settings] section, so 1), <data> is actually Data#, which takes the number # from Data#. However you can name the archives to whatever you want. Just make sure to not use any super special characters.

Hello again. Thank you, now there is some more insight into this for me. I copied preset you provided to .ini file and currently it is in a process of conversion. But is it okay that it going that slow, I mean it shows me 6 hours and it's gradually ascending. https://pasteboard.co/U1kdlXT9Qr0b.png

Also I don't understand how to pack it all in installer after compression, I wouldn't mind a detailed explanation of how to create an installer after compressing all the files :) Thank you

[KaktoR]

lolz compressor is slow, yes. However you can speed up the process by using some options for it, for example utilize all your cpu cores.

Example
lolz:mtt1:mt12
12 = number of cpu threads. If you have more or less cpu threads, change the number.

As for the installer, you have several choices here. There are some installers which are compatible with diskspangui. You can find supported installers in the diskspangui topic, I think it is the second or third post in the topic. However you can also use diskspangui itself to unpack the archives. Just copy diskspan_gui.exe in the same folder where all the created archives are and run it.

[sailwhist]

lolz compressor is slow, yes. However you can speed up the process by using some options for it, for example utilize all your cpu cores.

Example
lolz:mtt1:mt12
12 = number of cpu threads. If you have more or less cpu threads, change the number.

As for the installer, you have several choices here. There are some installers which are compatible with diskspangui. You can find supported installers in the diskspangui topic, I think it is the second or third post in the topic. However you can also use diskspangui itself to unpack the archives. Just copy diskspan_gui.exe in the same folder where all the created archives are and run it.

But, where do I set options for threads? I had to abort the process since it shown me 10 hours estimated time and it was ascending more and more. I just copypasted these and started compression: https://pasteboard.co/4orsQiqonK2O.png
I don't think 24 GB game should be compressing to a 13 GB for 10+ hours:)

[KaktoR]

It will look like this in the settings:
Data2.Method1=xtool:mzlib+srep+lolz:mtt1:mt12|0

However you can set it directly even in the GUI aswell by type it manually or select one of the presets for lolz:
https://i.imgur.com/xMp4E29.png
https://i.imgur.com/xMp4E29.png

* All possible options for the lolz compressor can be found here: "Resources\FINAL\LOLZ\Options.txt"

Time for compression mostly depends on your hardware, especially how fast your cpu and ram is, and of course of the input data and used settings. So there is no way to telling you how long compression will take if no one write the time and his pc specs.

[sailwhist]

It will look like this in the settings:
Data2.Method1=xtool:mzlib+srep+lolz:mtt1:mt12|0

However you can set it directly even in the GUI aswell by type it manually or select one of the presets for lolz:
https://i.imgur.com/xMp4E29.png
https://i.imgur.com/xMp4E29.png

* All possible options for the lolz compressor can be found here: "Resources\FINAL\LOLZ\Options.txt"

Time for compression mostly depends on your hardware, especially how fast your cpu and ram is, and of course of the input data and used settings. So there is no way to telling you how long compression will take if no one write the time and his pc specs.

After adding these settings to .ini file, now it is compressing as it should and percentage of progress goes up, last time it was just stuck at 10% and time showed 10 hours, now its 1:30 hours estimate and it looks like it is working. https://pasteboard.co/QPptA0Q6bKIw.png
Thank you, I hope it'll work.
BTW, what's wrong with Imgur? It's not opening any links, just shows it like that: https://pasteboard.co/rE3HCJIzwHaM.png

[sailwhist]

It will look like this in the settings:
Data2.Method1=xtool:mzlib+srep+lolz:mtt1:mt12|0

However you can set it directly even in the GUI aswell by type it manually or select one of the presets for lolz:
https://i.imgur.com/xMp4E29.png
https://i.imgur.com/xMp4E29.png

* All possible options for the lolz compressor can be found here: "Resources\FINAL\LOLZ\Options.txt"

Time for compression mostly depends on your hardware, especially how fast your cpu and ram is, and of course of the input data and used settings. So there is no way to telling you how long compression will take if no one write the time and his pc specs.

So, compression finished and now game size is 16 GB.
https://pasteboard.co/5nJo8MUxm2vF.png
https://pasteboard.co/DGBaecrwauDc.png
But why? It should've been 13 GB.
It also splitted in some weird directories with a lot of files, is that the way it should be?
https://pasteboard.co/X9L5I0DVeYRe.png
I tried making an installer via your utility ASIS and I didn't understand how it works, I putted these directories in DECOMRESSOR folder and ran script Create_setup.dll and it just started a whole process of compressing files again and I don't know if I did everything right.

[KaktoR]

Imgur is working here. Check if the site is blocked by your AV, ISP, browser extensions or whatever.

PS: GTA4 compression test from me
15:56:44 - Overall input size: 30.93 GB
15:56:44 - Overall output size: 13.48 GB (Ratio 43.58%)
15:56:44 - Overall conversion time: 00:58:27

16:08:51 - Overall decompression time: 00:11:52

I used LZMA-MT instead of lolz.

Edit:
Here is my configuration. With this you should get 13GB with LZMA-MT or less if you use lolz (probably).
Data1.Name=Data<game><data>-01.bin
Data1.Method1=msc+srep+LZMA-MT|0
Data1.InclText=*.rpf

Data2.Name=Data<game><data>-01.bin
Data2.Method1=xtool:mzlib+srep+LZMA-MT|0
Data2.InclText=*.img|*.wtd

Data3.Name=Data<game><data>-01.bin
Data3.Method1=bpk|0
Data3.InclText=*.bik

Data4.Name=Data<game><data>-01.bin
Data4.Method1=srep+LZMA-MT|0
Data4.ExclText=*.img|*.rpf|*.bik|*.wtd

As for the first archive I applied srep+LZMA-MT after msc because I found out that all of the files contain raw audio (msc do a good job here) and some other files aswell, so use srep and a compressor of your choice afterwards.

As for the file splitting: You have to select the "~ Unlimited ~" tab in the options on the bottom.
https://pasteboard.co/Zqx0UiosFqJ1.png

Edit 2:
As for the ASIS installer: you don't have to copy anything. Open Settings.ini and make your settings there, like game name, game size and so on, disable or enable options you want (see included help file located in "Help\Settings.txt"). After this you just open "Script.iss" with Inno Setup 6+ (https://jrsoftware.org/isdl.php) and compile the script (maybe you can just run "Compile_Script.bat" if you don't want to download and install Inno Setup). Then copy Setup.exe to the compressed game files which diskspangui created and then you can unpack/install the game with the Setup.exe.

[kj911]

The DEMO test package from my multithreading DLZ compressors, using MTX/clsw.exe combo.... (See pictures from package.)

Compression use MTX, within more threads without any problems.

Decompression/testing don't work with MTX-only usage, required CLS wrapper, and user interactions, pressed many "clsw.exe" crashed messages before starting any decomp/test process..
Any idea from fixing? Try it PMT? Any?

My precalculation in has fast CPU's probably capable 200+ MB/s decomp/testing speeds. (SSD or NVME drive only with more RAM.)

[Masquerade]

^^
But why? LOLZ is better and natively multithreaded.

[Yee]

Can someone please help. when using

[External compressor:lolz]
header = 0
packcmd = "lz\lolz_x64.exe" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

I get

WARNING: bad section [External compressor:lolz] in C:\Program Files (x86)\FreeArc\bin\arc.iniarc: user error (Unsupported compression method or error in parameters: lolz)

What am I doing wrong?

[kj911]

Noob questions: Some various "Disk Spanning" methods, will make spiltted DVD5/any size setup*.bin files, streamable exctracting during installation progress, whole get next disk sequence, without copying splitted *.bin files to is-xxxx.tmp dir joining to one big archive before extraction process?? Aka no need extra temp space from installations?

Example: setup1.bin.001-003 files have 10GB size. (contains 30GB data.)
aka1: extract from directly the archives to installdir, req. 30GB hd space or
aka2: joined to one big 10GB file extra hd space + 30GB installed --> 40GB hd space required?

Three disk spannig methods/cls-filters available. The best? (From stabilty.)

Another questions: Previous years downloaded from forums and looose in last time, not found the F-Done 0.01 applications the site. Available or deleted?

[awxcigin]

1. How do I get into repacking games, like I have a game downloaded, where do I start form?

2. What tools do I use as a beginner?

Sorry for asking this noob of a question.

[Masquerade]

^^
You can use DiskspanGUI to compress your game and extract the archives you make.

[Tmills]

Hey, i'm new and trying to learn. Is there a guide to the methods in DiskSpan gui?
SREP
LOLZ
Zlib
Xtool 2020 reflate
Xtool 2020 preflate
LZMA2
what are they best at and what are the pros & cons of each?
what file types are they used for?
how do i know what file types a game uses etc?

[-tara]

Hi Tmills

Heres what the methods listed are:
SREP (LD Deduplicator)
LOLZ (Final Compressor)
Zlib (I'm guessing this is XTool's implemintation of zlib, that is a Precompressor)
Xtool 2020 reflate (Precompressor)
Xtool 2020 preflate (Precompressor)
LZMA2 (Final Compressor)

I'm going to shortly explaing what these do.

SRep removes duplicates in files you feed it, it's very quick.
Lolz and LZMA2 are final compressors and they always go at the end of a method chain, they compress data that was previously fed to Precompressors or SRep.
Lolz is very slow on compression, but it's upside is faster decompression compared to LZMA or LMZA2, while LZMA/2 are a lot faster in compression but a little slower when it comes to decompression. (Lolz gives better ratio compared to LZMA/2 aswell)

Typical chain of methods when it comes to game data (excluding media) is SRep+Lolz/LZMA/LZMA2.

However sometimes you can encounter game data that is already compressed, this is where Precompresser come in, during compression you place them in the beginging of the chain, what Precompressor you use depends on what algo the game is compressed with.
For example if the game is compressed with zlib, you use a Precompressor that utilizes zlib decompression (like XTool zlib/reflate/preflate) which one of those you use depends on how they preform, they have tiny differences in speed/ratio, however they all decompress zlib.

Typical chain with precompression is XTool+srep+lolz

I skipped over loads of info in order to keep my replu consise, but this should be good enough for a beginer

Hope I Helped

[ravikant]

Hey, i'm new and trying to learn. Is there a guide to the methods in DiskSpan gui?
SREP
LOLZ
Zlib
Xtool 2020 reflate
Xtool 2020 preflate
LZMA2
what are they best at and what are the pros & cons of each?
what file types are they used for?
how do i know what file types a game uses etc?

Use this tool (https://fileforums.com/showthread.php?t=99270) to find which method of xtool to use

Generally use Xtool(zlib is also included) + srep + lolz/lzma [LZMA is faster but usually makes the output size a bit larger]

[ravikant]

I need to know more about xtool reflate/preflate and what it actually means

Why lzma compresses my files quickly while lolz takes lot of time

why do precomp gives better results than xtool on my old games

[Tmills]

Thank you both for the replies, very well put and easy to understand. Now I need to learn the inno setup script stuff :)

[shazzla]

Guys ,it extracts/creates nothing. Only correct CPU load ,process finishes and nothing.

xtool.exe generate -mkraken -t3 -bd_libraries "x\*" "p:\ARMORED CORE VI FIRES OF RUBICON-\Game\Data3.bdt" ac6.xtl

whats wrong ?

Edit :

Solved. Sandboxie interfered somehow...

[murphy78]

Hi, I've been very curious about repacking games. I'm not interested in spreading my own repacks or anything like that. I am just obsessive about archiving the latest versions of my games when I download them and I want to take up very little space.

I've played with a couple tools like xtool and diskspan_GUI and they work a bit but I don't really grasp everything very well. I would like to know if it is generally a good idea to reflate, srep, and then compress the files. Could someone help me with some advice and maybe a small basic packer and unpacker script? I saw another person's example but it only had the arc/lolz combination. I admit I don't really know much about lolz. I've read a comment from someone saying that the tool was old and that people mostly stopped using it.

Thanks for any help people could give in advance.

[Masquerade]

Lolz is still in use, it's still the best compressor for game data. But, it's still old. Most tools that we use are ancient.

About inflate + srep + compress - this correct. Inflation, or precompression is the step where the compression already on the game is reversed (decompressed) so that you can apply a stronger algorithm (srep+lolz/lzma). You need to use the correct codec for the game since not every game uses the same compression algorithm. When you decompress your archive, those resources are recompressed identically.

[murphy78]

Does anyone have a compilation of commandline and ini file structure for xtool.exe, freearc 0.67's arc.exe ?

I looked around to the best of my ability but I'm having a hard time locating information. I can't find any command line information on xtool and what little information I can find on freearc doesn't seem to apply to the way people are using it with the ini files and the -m single argument passing method.

I have checked the chm file that comes with xtool and it seems to only discuss precomp. There are a lot more features that the exe uses and I imagine that there's gotta be more information somewhere.

I'm sorry if this all seems trivial to everyone. It's hard to write myself a script when I don't really have any example or documentation to go on.

Thanks again in advance.

[reXy]

Are there video tutorials you would recommend for beginners?

[Masquerade]

^^
Unfortunately there are none.

[reXy]

Hellos again, so i'm not sure how I am getting this error in DiskSpan GUI, but my hdd is not full in the slightest.


-------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------- CONVERSION DATE 27/09/2023 -------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
09:37:06 - Deleting files from the output folder.
09:37:06 - Files have been deleted.
-------------------------------------------------------------------------------------------------------------------------------------------
09:37:06 - You have chosen the DVD5 type for the media size.
09:37:06 - ISO images will not be created when conversion is complete.
09:37:06 - The computer will remain on when the conversion is complete.
09:37:07 - Selected ARC/DS method for reXy-01.bin was: xtool:mpreflate+srep_new+lolz:dtb1:d23:mtt1:mt8:mc 1023
-------------------------------------------------------------------------------------------------------------------------------------------
09:37:13 - Compression initialized in 27/09/2023 at 09:37:13.
09:37:13 - Starting reXy-01.bin archive creation.
10:29:33 - Archive reXy-01.bin creation error.




-------------------------------------------------------------------------------------------------------------------------------------------
Creating reXy-01.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\reXy-01.bin
Compressing 1,265 files, 6,484,935,585 bytes. Processed 11.8%
Compressing 6,485,919,563 bytes with "OTHERS\SREP\srep.exe" -m3f -a0/0 InFile OutFile
11.8%SREP 3.93 beta (September 30, 2014): input size 6185 mb, memory used 370 mb, -m3f -l512 -c512 -a0/0 -hash=vmac -b8mb
100%: 6,485,919,563 -> 3,098,642,643: 47.77%. Cpu 86 mb/s (71.828 sec), real 52 mb/s (119.957 sec) = 60%. Remains 00:00
Decompression memory is 1286 mb. 1,404,531 matches = 22,472,496 bytes = 0.73% of file
Errorlevel=0
11.8%
Compressing 3,098,651,475 bytes with "FINAL\LOLZ\lolz.exe" -dtb1 -d23 -mtt1 -mt8 -mc1023 InFile OutFile
11.8%
lolz v22c4b [Dec 30 2018 16:44:58]
Settings:
----------------------------- detection settings -----------------------------
detection is enabled(-dt)
use probs propagate(-dtp[0..1]): 1 use brute detect(-dtb[0..1]): 1
o1 detect(-dto[0..1]): 1
raw graphic mm detect(-dtm[0..1]): 1 width detect algo(-dtw[0..1]): 1
dxt detect(-dtd[0..1]): 1
--------------------------- multithreading settings --------------------------
mt type(-mtt[0..1]): 1
mt threads(-mt[1..16]): 8 mt block size MB(-mtb[2..512]): 32m
---------------------------- lz & parser settings ----------------------------
dict size in MB(-d[0..2047]): 32m num parser arrivals(-tt[1..256]): 4
parser hard lim(-oh[8..14]): 12 parser soft lim(-os[0..hard_lim]): 8
fb type A(-fba[0..4096,0=off]): 256 fb type B(-fbb[0..4096,0=off]): 0
calc all literals(-al[0..1]): 1 slow parser mode (mlr)(-x[0..2]): 0
move some of calculations to lz matchfinder thread(-ac[0..1]): 0
---------------------------- matchfinder settings ----------------------------
use only lz matchfinder (-rt[0..2]: 0)
lz mf cycles(-mc[2..1023]): 1023
use long distance mf(-ldmf[0..1]): 0
------------------------------- models settings ------------------------------
general models :
use simple cm(-cm[0..1]): 1
context mixer o1 ctx (-bc[0..6]): 4
use only o1 nibble literal model (-lm[0..4]): 0
literal hi o1 ctx (-blo[0..8]): 8 literal lo o1 ctx (-bll[0..8]): 8
literal hi rep0lit ctx (-blr[0..6]): 4
match flag rep0lit ctx (-bm[0..6]): 4
all -p* settings are ignored due to using detect
dxt models :
dxt color(X) and alpha(Y) model (-dmXY[X[0..3], Y[0..4]]: 34)
raw graphic models :
color mdl options (-gmXY[X[0..2], Y[0..1]]): 00
100.00% ldmf dec_mem=0m 0:38:17 /s
o1 model : 313'004 kb
raw graphic model 8 bit : 37'418 kb
raw graphic model 16 bit : 1'046'332 kb
raw graphic model 24 bit : 23'545 kb
raw graphic model 32 bit : 224'074 kb
dxt1 model : 340'555 kb
dxt5 model : 150'058 kb
o1 model pos mod 2 : 48'588 kb
o1 model pos mod 4 : 314'824 kb
o1 model pos mod 8 : 114'372 kb
o1 model pos mod 16 : 413'251 kb
total size : 3'026'026 kb
decode mem usage per thread = 66mb
11.8%
Errorlevel=0
Compressed 1,265 files, 6,484,935,585 => 1,483,642,757 bytes. Ratio 22.88%
Compression time: cpu 16.53 sec/real 3123.64 sec = 1%. Speed 2.08 mB/s
Testing 1,265 files, 6,484,935,585 bytes. Processed 0.0%
ERROR: write error (disk full?) in compression algorithm lolz:dtb1:d23:mtt1:mt8:mc1023




pls help! :rolleyes: Am i perhaps using the wrong method?

[KaktoR]

Try unpacking with the dsg unpacker ("Tools -> Unpacker").

[reXy]

Try unpacking with the dsg unpacker ("Tools -> Unpacker").

I got it eventually with another method (xtool:mpreflate+srep+LZMA-MT).
I don't exactly know how to use the dsg unpacker. (I don't exactly know how any of these tools work just yet)

[IRAQIGHOST]

Hi
some games I used choose best method to compress the game
but the result not the same
like Cyberpunk
I used xtool:c32mb:mcp2077+srep_new+4x4:b64mb:lzma:ultra: 64m:bt4:fb273:lc8:mc1000000
I read it the game size is 66Gb after compress will be 43 Gb
but when I do it I got final size 56 Gb
is there are something wrong that I did?
I used Diskspain GUI
I have Ryzen 5 3600
Ram 32 Gb

[Masquerade]

^^
Because, no doubt, you are compressing v2.0 with Phantom Liberty DLC?

Ofc the compressed size will be larger... because their is simply more content... :confused:

[IRAQIGHOST]

^^
Because, no doubt, you are compressing v2.0 with Phantom Liberty DLC?

Ofc the compressed size will be larger... because their is simply more content... :confused:

even the version 1.3 the old one this happened

[kj911]

Good methodology compress via MP2 audio files? (except Ocarina, still very slow.)

My horrible methods: unpackmp2+srep:m3f:a0:l32+msc:raw=1:tak=9+delta+dl z

DLZ replaced via NZ/LOLZ its probably better. (and/or without delta filtering.) Tested one 192MBs MP2 file packed to 152MBs size. Final ratio minmally smaller than 80%*

DLZ replaced to Ocarina probably the best and slooow results. This MSC will found ~35-36 million bytes raw audio from processing and packed via TAK. Before used or not SREP precompression.

*Three step compression: 201 348 000 -> 335 766 244 -> 314 530 933 -> 159 583 312 byte

DLZ replaced to MPZX (64MB block size) results: 173 752 545 byte

NanoZip0.09/cO/768m, before not used delta filtering: 152 764 729 byte (~76% final ratio. Typically MJPG/MP3 recompression ratio.)

Archivers info: SREP 3.92; DLZ 0.2.2; MSC 0.0.6.5a2/TAK 2.3.3; unpackmp2 v22.03.16.1

[Masquerade]

^^
DLZ was only useful for DDS images. So using it eveywhere won't help.

unpackmp2+razor is my method of choice.

[kj911]

Tested again, different Win7 x64 PC (Previously used PC: XP SP3, DLZ 64MB dict size....)

Its winner?? LOLZ!

unpackmp2 only archive: 335 766 506 byte
unpackmp2+srep:m3f:a0:l32+msc:raw=1:tak=9 (masked comp. named in v2) archive: 314 510 792 byte

Tested Ocarina 0.6 compressor:
unpackmp2 compressed archive: Failing in compress
v2 (unpackmp2+srep+msc): 172 074 214 byte

DLZ with 512MB Dict. size:
unpackmp2: 159 505 095 byte
v2: 159 618 382 byte

Razor 1.03.7 x64 with 512MB Dict. size:
unpackmp2: 157 403 746 byte
v2: 157 526 068 byte

LOLZ v22c4b:
unpackmp2: 150 686 438 byte
v2: 153 138 005 byte

Ocarina 0.6: level 12 chunksize 256MB
DLZ: -slp -d512m -lp0 -fb337 -mc8192 -cu3 -cm43
LOLZ settings: lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:fb a4096:tt16:mc1023

Goodly winner: unpackmp2+lolz

Directly compress pure MP2 file.
Size: 201 348 000 byte

Ocarina 0.6: Failing!
NZ0.09/cO/768m: 184 169 837 byte
LOLZ: 189 974 702 byte
DLZ: 187 631 951 byte
RZ: 185 784 860 byte

Any ways, trick from modify "unpackmp2" from preprocessing any files? BIK, MP3, WAV?

[JayJiang]

I have already installed the game, but I encountered a mistake when selecting the language before downloading the game, I forgot to select Polish in addition to English.

But I don't want to reinstall the game all over again because it's very large, and it took me a long time to download it.

Can someone tell me what I should do?

[Masquerade]

This is not a repack support forum, if you don't have a question about making backups of games you own, then leave! :mad:

[Non Volatile Human]

I'm using DiskSpan GUI, is there a preferred setting to use here, regarding DVDs or BDs?

*the image link is refusing to load so I'll type it down*:

At the bottom of the page (non-developer mode) it gives me 2 DVD sizes, 3 BD sizes, and an "~unlimited" option, is there a reason to pick one over the others?

Added: in the compression tab on DiskSpan GUI, is the timer in the title bar the "remaining time"?

I'm compressing Death Stranding DC on a Core i5 6400 and the timer says "34:15:45" lmao.

[mausschieber]

I'm using DiskSpan GUI, is there a preferred setting to use here, regarding DVDs or BDs?

*the image link is refusing to load so I'll type it down*:

At the bottom of the page (non-developer mode) it gives me 2 DVD sizes, 3 BD sizes, and an "~unlimited" option, is there a reason to pick one over the others?

Added: in the compression tab on DiskSpan GUI, is the timer in the title bar the "remaining time"?

I'm compressing Death Stranding DC on a Core i5 6400 and the timer says "34:15:45" lmao.

You can read Here https://www.fileforums.com/showthread.php?t=104533

[Non Volatile Human]

I read it, but I feel like it's missing something. It means that I need to have an idea of the data present in each *.bin archive and as such pick the appropriate disk size (or take into account the size of the whole game). e.g. if I know that all the archives/*.bin files will be at most 45GB big, I should choose the BD 50 disk size?

why not choose ~unlimited so that the need to merge is made irrelevant, as all the data would be on one disk, and that disk would contain all of the archives/*.bin files?

And if in "developer mode" I choose "Disk spanning using CLS-Diskspan.dll" in the "Game1" tab, would I need to adjust the disk sizes in the "General" tab?

And if I add the game from the Database, would I need to touch anything at all afterward? Or does using the Database automate everything so that I would just have to save my preset, re-launch DiskSpan GUI in its normal mode, and press "compress"?

[KaktoR]

The size tabs are just for people who still burn their crap on discs for backup purpose (honestly drives cost nothing these days) or for some shady black market stuff. Just choose unlimited tab if you don't care.

If you load a preset from database, you don't need to touch anything except for game directory. Some presets require some user action however, like language selection and such.

Added: in the compression tab on DiskSpan GUI, is the timer in the title bar the "remaining time"?

I'm compressing Death Stranding DC on a Core i5 6400 and the timer says "34:15:45" lmao.

Yes, but do not take this times seriously. You simply cannot say how long compression takes if you didn't do it in the first place.

[KaktoR]

test

[Non Volatile Human]

Games are missing from the DiskSpan GUI database which are in the index, any idea why? 388 entries vs 1300+

Edit: I guess the server saw how I copy-pasted an entire paragraph rather quickly suspicious and decided to discard it completely, lesson learned

[KaktoR]

Because I simply have no time to download, compress, test and add 1300+ games to database, that's it, and I am the only one who shares updates for DSG database.

[Non Volatile Human]

Wow! I had no idea methods had to be manually tested!
That's a monumental task! Thanks for taking the time and making the database in the first place!

[kj911]

Oodle codecs/library: Is it possible to use the Oodle library to achieve even 0.5-1GB/s decompression speed on NVME/M.2 SSD-drives, using existing tools?? (ex: Oodle MT Compressor R1, Oodle MT 1.3, Oodle compressors library by Shelwien, any...)

[nordi]

VCL STYLE IN UNINSTALLER - Solved

Hello there,

.. maybe someone can help me with this...

In the attached test script, the VCL style plugin is used for the uninstaller.
Everything works fine so far as long as it is installed in the C:\ drive.

But if I change the drive to D:\, the dll file is always left over during
uninstallation and is not deleted.

(All files should be moved into the tmp folder during uninstall
to let it clean up from InnoSetup itself. Tried this, too. (https://stackoverflow.com/questions/36940506/load-external-dll-for-uninstall-process-in-inno-setup))

Maybe someone has a better solution for this?!
:confused:

Best regards
Nordi

https://i.postimg.cc/5Xq4LD1F/VCL.png (https://postimg.cc/5Xq4LD1F)

[BLACKFIRE69]

Hello there,

.. maybe someone can help me with this...

In the attached test script, the VCL style plugin is used for the uninstaller.
Everything works fine so far as long as it is installed in the C:\ drive.

But if I change the drive to D:\, the dll file is always left over during
uninstallation and is not deleted.

(All files should be moved into the tmp folder during uninstall
to let it clean up from InnoSetup itself. Tried this, too. (https://stackoverflow.com/questions/36940506/load-external-dll-for-uninstall-process-in-inno-setup))

Maybe someone has a better solution for this?!
:confused:

Best regards
Nordi

https://i.postimg.cc/5Xq4LD1F/VCL.png (https://postimg.cc/5Xq4LD1F)




this one functions here without any issues.


.

[nordi]

Hello BLACKFIRE69,

I'm so grateful to you, it works perfectly.
Thank you for that. :)

Kind regards
Nordi

[ozerune]

Hi, I'm brand new to this forum and to repacking as a whole. I've gotten FreeArc setup with a few external compressors but am running into an issue with MSC. Whenever I try to extract what I've compressed with it, the progress goes up indefinitely.

packed with: arc a -mmsc+srep:m3f game.bin .\game\hl2\hl2_sound_*.vpk

arc.ini:

[External compressor:msc]
packcmd = msc.exe c -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

Does anyone have any idea as to what could be causing this?

[kj911]

The unpackers files located in total different folders than compressor files??

Ex: Compressors files located in "Pack" folder. Decompressor files located in "Unpack" folder.

Following files need from compressing: Arc.exe; MSC.exe; pack_TAK.exe; pack_FROG.exe; pack_BMF.exe (optional, freeze Win7!); another compressors, properly configured arc.ini files.

Following files need from decompressing: Arc.exe or unarc.exe; cls-msc.dll; another (de)compressors and properly configured arc.ini files. MSC not need arc.ini configuration from decompress. More tools required configured cls.ini file.

WAV files not compressing? Try add -tak=9 options. Safer way switch off BMP/DDSDXT/DDSRAW compression using BMF. (any=0)

[ozerune]

The unpackers files located in total different folders than compressor files??

Ex: Compressors files located in "Pack" folder. Decompressor files located in "Unpack" folder.

Following files need from compressing: Arc.exe; MSC.exe; pack_TAK.exe; pack_FROG.exe; pack_BMF.exe (optional, freeze Win7!); another compressors, properly configured arc.ini files.

Following files need from decompressing: Arc.exe or unarc.exe; cls-msc.dll; another (de)compressors and properly configured arc.ini files. MSC not need arc.ini configuration from decompress. More tools required configured cls.ini file.

WAV files not compressing? Try add -tak=9 options. Safer way switch off BMP/DDSDXT/DDSRAW compression using BMF. (any=0)

FreeArc doesn't seem to like it when I put files into other folders, I just get a "The system cannot find the path specified." error. I did not know about cls-msc.dll and just tried to add it to the bin folder, but now when I try to archive with msc, it tells me "ERROR: user error (Unsupported compression method or error in parameters: cls-msc.dll)"

It looks like most of my problems are coming from something causing CRC errors. I just tried it on a folder of just basic wav files, and when extracting, all of them give a CRC error.

I just tried with lolz and that is also giving me CRC errors and infinitely looping as well now

[KaktoR]

compress with msc.exe
decompress with cls-msc.dll

[ozerune]

compress with msc.exe
decompress with cls-msc.dll

I'm fairly certain I am doing that, but the progress still goes up indefinitely, even when I try to extract a single wav file. This also happens when using lolz, and srep works, but gets stuck at 99.9% and I have to forcefully close it.

[ozerune]

I'm getting a CRC error on every file with lolz and msc, but I have no idea what could be causing this.

[ozerune]

I'm getting a CRC error on every file with lolz and msc, but I have no idea what could be causing this.

Does anyone have any thoughts on what I could try? I'm happy to send any configuration files I have if anyone needs it to help me with this.

[ozerune]

I'm trying to compress an existing .bin file in a game, but I can't figure out how to do it. I saw people saying that .bin files are most likely compressed data, so I tried out Xtool. I turned on all of the Methods xtool supports, and it seems to be finding zlib and lzo data, but the majority of the output in verbose mode says "processing ... stream at ... has failed" with an occasional success. I was finally left with "Size: 6.65 GB >> 6.65 GB" Is there any better way I can get this data processed?

[Masquerade]

Please may you upload a sample file?

[ozerune]

Please may you upload a sample file?

This was for Anonymous;Code which you helped me out with in the specific games thread :)

I'll let you know how it goes once I get back around to that one

[Tmills]

Quick Questions.
1.How do i get my setup.dll merged into my game.exe when using DiskSpan Gui?

2. Is there a list of file extension/types and the method to use on them?

for example:
.wav,.rpf,.mp3 are audio files. Would MSC be best to compress them? What other methods are good on audio files etc?
.img,.png are image files. what method is good for these? mzlib?

3. What are the differences between the xtool options and what files would they be used for:
xtool 0.12 xzlib
xtool 2020 mzlib
xtool 2020 mreflate
xtool 2020 mpreflate
xCrilayla
xlz4
xlzo
xzstd

I'm assuming that xlz4,xlzo,xzstd are the best method exclusively for those file types.

I have everything working to repack my games and now I'm trying to learn about all the different file types and how best to compress them.
Is there anywhere I can read up on this kind of information to learn?
Any advice would be appreciated.

[ozerune]

1. If you are using the ASIS installer, you can use DiskSpan to unpack the Setup.dll contents to the DECOMPRESSORS folder in the installer's main folder

2. https://fileforums.com/showthread.php?t=104866

3. As far as I can tell you only need to use the 2020 ones, someone might have to correct me on that though.

[Tmills]

From my understanding.

Reflate: is decompression. It takes a compressed data stream and reconstructs the original uncompressed data before feeding it to another compression tool.

Preflate: is precompression. It takes uncompressed data and pre-processes it before feeding it to another compression tool.

Xtool 2020:mzlib is also precompression. It takes uncompressed data and pre-processes it before feeding it to another compression tool.

Reflate:
Input: Compressed data
Output: Uncompressed data (original data before compression)
Example is using reflate on .ftex file. mzlib does nothing to it as it's a compressed file, however reflate will uncompress it so you can recompress it with LOLZ/LZMA2 for a smaller file size.

Mzlib & Preflate:
Input: Uncompressed data
Output: Pre-compressed data (prepared for further compression)


Is that correct? If so, what is the actual difference from mzlib and preflate. Is just the algorithm they use?

[KaktoR]

All of them a precompressors. Actually they decompress compressed streams.

[KaktoR]

Quick Questions.
1.How do i get my setup.dll merged into my game.exe when using DiskSpan Gui?

2. Is there a list of file extension/types and the method to use on them?

for example:
.wav,.rpf,.mp3 are audio files. Would MSC be best to compress them? What other methods are good on audio files etc?
.img,.png are image files. what method is good for these? mzlib?

3. What are the differences between the xtool options and what files would they be used for:
xtool 0.12 xzlib
xtool 2020 mzlib
xtool 2020 mreflate
xtool 2020 mpreflate
xCrilayla
xlz4
xlzo
xzstd

I'm assuming that xlz4,xlzo,xzstd are the best method exclusively for those file types.

I have everything working to repack my games and now I'm trying to learn about all the different file types and how best to compress them.
Is there anywhere I can read up on this kind of information to learn?
Any advice would be appreciated.

1. Game.exe? You mean installer Setup.exe? You have to use compatible installers which allow to merge setup.dll into installer file.

2. If you use diskspangui, yes. I have written something about it somewhere.
For mp3 I would use 7z with mp3 plugin (7z-dll in diskspangui somewhere). msc for wav/raw.
For png you can use xtool with png plugin + d1 option + reflate/preflate. I guess you can use precomp aswell.

3. Precompressors. In some cases you cannot predefine what method to use on a specific file type.
However in most cases you can just presume by used game engine. For example Unreal Engine 4+5 mostly uses zlib and oodle.

[ozerune]

What I usually do in Diskspan is go through all of the main xtool methods one at a time and see if the size increases. Is that a good way to go about it?

[KaktoR]

No, use xtoolui.

[Masquerade]

You can use a hex editor to look for headers / magic numbers.

[ozerune]

After giving up on configuring FreeArc itself, I switched to DiskSpan GUI for a little bit. Now that I have a better understanding of repacking, I want to go back to FreeArc, but am running into some issues.

1) It won't let me put the compressors into different folders
e.g. unpackcmd = "compressors\lolz_64.exe" {options} InFile OutFile
gives me an error, where not using subfolders works just fine

"The system cannot find the path specified."

2) cls-lolz refuses to unpack anything. I have all 4 necessary cls-* files, and am running an unpack.bat file as administrator, and nothing is working



3) When I have the cls files in the FreeArc bin folder, it tries to compress the files using the cls instead of the actual compressor. How can I stop this from happening?

arc.ini at defining external compressors:

[External compressor:lolz]
packcmd = "compressors\lolz_x64.exe" {options} InFile OutFile
datafile = InFile
packedfile = OutFile

[Masquerade]

unpackcmd = "compressors\lolz_64.exe" {options} InFile OutFile
lolz_x64.exe cannot be used to unpack data, only compress.

As for file not found, don't try and use a FreeArc installation for your repacking. Make a separate folder with arc.exe, arc.ini and your tools. Make sure arc.exe from your FreeArc installation isn't in the PATH variable because things get messy this way.

cls-lolz refuses to unpack anything. I have all 4 necessary cls-* files, and am running an unpack.bat file as administrator, and nothing is working
Please may you explain what error you face? You only really need:
cls-lolz.dll
cls-lolz_x64.exe
And the appropriate configuration in cls.ini.

When I have the cls files in the FreeArc bin folder, it tries to compress the files using the cls instead of the actual compressor. How can I stop this from happening?
You can't. FreeArc will always load and prefer cls plugins over external compressors configured in arc.ini. you will need to remove cls-lolz components and other cls based decompressors from your Arc compressor.

You don't need to use datafile = / packedfile = in arc.ini either, you can simply use $$arcdatafile$$.tmp $$arcpackedfile$$.tmp in the command (this could be why your archives won't unpack with lolz cls).

datafile/packedfile is reserved for awkward tools that require a specific in/out file extension to work and has no filename override.


[External compressor:lolz]
packcmd = "compressors\lolz_x64.exe" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[ozerune]

I did everything you said and this is the error I am getting when extracting:

available options:
transfer data options: -Bufsize, -transfer_ReadBufSize, -transfer_WriteBufSize
for mtt1: -MaxThreadsUsage, -MaxMemoryUsage
for ldmf1: -ldmfTempPath, -ldmfMaxMemoryUsage, -ldmfDeleteTmp

ldmf mode
temp path = .\
ldmf memory per thread = 8'192kb

ERROR: general (de)compression error in lolz

I am using the CLS.ini that is provided in the lolz 7z folder you linked to in your tools thread. Should I be making modifications to that to get it to work? Also, quick correction to your post earlier, I am not using unpackcmd, only packcmd.

After trying a few more things, it would seem that its the archives I make in FreeArc that are being corrupted, as unpacking one I created with DiskSpan GUI worked just fine.

[ozerune]

I feel so stupid now. For anyone else running into these issues, first, keep header = 0 at the top of the external compressors definition, and also run in cmd and not the windows 11 terminal. That was all it took and I feel like an idiot for spending 3 days on this lol

[weednbeer]

Hello beautiful community :) I have a question about DiskSpan and I am on the latest version. I would like to know why when I go to arc compression and then I select FA-ULTRA custom compression, 50% of the time the compression remains at the beginning and the time only increases and without results after several hours and on other games it works ...I find that really strange. If anyone could enlighten me on this it would be really appreciated.

[KaktoR]

Most probably because of the use of rep dedup. However I don't recommend to use any of the predefined freearc templates except for fa-store and fa-min inside of diskspangui because settings will not be the same as if you use them by freearc directly.

Example
https://i.imgur.com/6VNXG6y.png
https://i.imgur.com/H7RbVbq.png

Both archives were made with freearc ultra template (left: freearc, right: diskspangui)

Freearc templates inside diskspangui will not be using predefined arc.ini settings from freearc installation folder and will not apply on different file types, hence the ratio difference.

[Cesar82]

Although not identical, the predefined modes FA-BEST_SP, FA-MAX_SP and FA-ULTRA_SP work with FreeArc's internal masks.
https://i.imgur.com/Hvk4fEg.png
https://i.imgur.com/FbujOlL.png
But to use these methods you can NOT choose DiskSpan mode (You can use it together with SPLIT mode or WITHOUT any split mode).
I don't know why these 3 methods don't give us the same results, taking into account that the final method formed on the command line is something like:
FA-BEST_SP >> -m9x -ld192m
FA-MAX_SP >> -mx -ld800m
FA-ULTRA_SP >> -mx -ld1600m
The other settings of the FA-??? presets use fixed methods.
See below the presets used (They can be changed in "DSG_Arc.ini" whenever you want).
FA-STORE=0
FA-MIN=1
FA-LOW=rep:1kb:256+exe+4x4:tor:1kb:h4kb
FA-FAST=rep:1kb+exe+delta+4x4:lzma:32kb:fast:32:mc4
FA-NORMAL=rep:1kb+exe+delta+4x4:lzma:32kb:normal:16:m c8
FA-HIGH=rep:1kb+exe+delta+4x4:lzma:32kb:normal:32:mc3 2
FA-BEST=exe+delta+lzma:32kb:normal:bt4:128
FA-MAX=rep:1kb+exe+delta+lzma:64m:ultra:bt4:128
FA-ULTRA=rep:1kb+exe+delta+lzma:128m:ultra:bt4:128

[wareck]

Hello
I would like to know how to process these king of fille
(RIFF/FSB5)
They take lot of space in the conversion.
I think they could be converted before and after compression
but I'm a beginner and I don't understand certain things...

best regards

[Masquerade]

^^
Extract all FSB5 banks from the bank/data with this quickbms script:
https://www.fileforums.com/showthread.php?p=502141#post502141

Then, extract those new FSB files with this script:
https://github.com/HearthSim/python-fsb5

For the audio extracted, concatenate the files by file type with bincat by Aluigi.

Bin files? Reextract the original FSB with fsbext and compress the extracted wav with msc
WAV files? Compress with msc.
Ogg? Compress with oggre.

So you can have up to 3 different files at this point.

Use oggvorbis2fsb5 to convert your decompressed ogg data into FMOD ogg audio. Concatenate your (up to) 3 files and make a patch to the original bank with hdiffz.

During unpacking/installation use oggvorbis2fsb5 again, concatenate and then apply the hdiffz patch you made earlier to restore the original bank.

[wareck]

Thanks !
it was exactly what I needed :)

[nordi]

Hi there,

UPDATE : Solution found and attached !!!

I created an installer with different titles for each page.

I managed that it work without VCLstyles (or when I use ISSkin instead).

But with VCLstyles it only changes the titles when i press e.g. the cancel button.

https://i.postimg.cc/Rh94BHmz/Unbenannt.gif (https://postimages.org/)

>> My test script is attached

Thank you in advance...
Nordi


UPDATE : Solution found and attached !!! ( >> Procedure to hide + show WizardForm )
Solution 1 = managed with WinAPI function >> SendWindowsText
Solution 2 = managed with WizardForm.Caption
>> works without the styles too

SOLUTION3 => no flickering Wizard : https://fileforums.com/showpost.php?p=504155&postcount=121

[KaktoR]

Library bug. Maybe you can try to refresh the WizardForm.Caption on each page change, but not sure.

[Cesar82]

Try

[nordi]

@ KaktoR - thx for replay and tip ;)

@ Cesar82 - thx for quick solution, titles are correct now :) :)


> but now the buttons and text are not shown correctly... very strange behavior (using the WinAPI function)

Solution (https://fileforums.com/showpost.php?p=504150&postcount=120)

https://i.postimg.cc/fyxpJrPy/1.gif (https://postimages.org/)
https://i.postimg.cc/T2CHHnXZ/2.gif (https://postimages.org/)

[nordi]

Hello,

first of all thank you for the help and tips.
I have found a solution to my problem.

I have attached my solution here and in the first post:
https://fileforums.com/showpost.php?p=504132&postcount=116

Maybe it's just a simple workaround, but it's ok for me... :)
( >> Procedure to hide + show WizardForm )

Now all buttons + text are shown correctly (https://fileforums.com/showpost.php?p=504138&postcount=119) too.

Best regards
Nordi

UPDATE : Solution found and attached !!!
Solution 1 = managed with WinAPI function > SendWindowsText
Solution 2 = managed with WizardForm.Caption
>> works without the styles too

SOLUTION3 => no flickering Wizard : https://fileforums.com/showpost.php?...&postcount=121

[Cesar82]

I think this solves the problem with less design blinking (Solution 3 attached).

EDIT:
Taking advantage of the topic, if anyone has a solution to my problem, let me know.
The caption 'Информация' of secondary windows using VCLStyles + Botva2 are not displayed correctly in languages that use Cyrillic characters such as Russian.
P.S: If you use TForm, the VCL design is not applied to the title bar.
https://i.imgur.com/0A4ldoP.png
This is an old bug that CIU has, and despite many attempts I still haven't been able to solve it.
If anyone has a solution, please share.

[KaktoR]

I'm pretty sure you have to use encoded unicode characters (#$xxxx), but I am not sure if you can use them in a CustomForm title.

[Cesar82]

I'm pretty sure you have to use encoded unicode characters (#$xxxx), but I am not sure if you can use them in a CustomForm title.
Thanks for the suggestion, but it doesn't work (I tested directly: #$0420#$0443#$0441#$0441#$043A#$0438#$0439).
It is some internal bug in VCLStylesInno.dll that causes conflict with the Botva2.dll library (GDI PLUS).
To prove this you can just comment the line "Img := ImgLoad(...." (Disable GDI+ loading) and recompile the script and the caption will be displayed correctly.

[nordi]

@Cesar82

In order to practice, I try to implement the SHEstyle from your Light Installer Script (LIS).

Is this the same behavior that you (or KaktoR) explained with VCLstyle...
>>> if I use StyleSHE with "Russian", so I have to do it with language codes... (#$xxxx)?

> Sample attached.

https://i.postimg.cc/K8GVQb5K/Unbenannt.gif (https://postimages.org/)

[Cesar82]

@Cesar82

In order to practice, I try to implement the SHEstyle from your Light Installer Script (LIS).

Is this the same behavior that you (or KaktoR) explained with VCLstyle...
>>> if I use StyleSHE with "Russian", so I have to do it with language codes... (#$xxxx)?

> Sample attached.

https://i.postimg.cc/K8GVQb5K/Unbenannt.gif (https://postimages.org/)
Replace two files and try russian language

[nordi]

:) it`s working, thx :)

It's nice to see that the more experienced members don't give up on still helping other users like me.

[Cesar82]

:) it`s working, thx :)

It's nice to see that the more experienced members don't give up on still helping other users like me.

I only went to see after sharing that the wizardform background doesn't load color, it only works partially.
This library is from Githhub, maybe someone can adapt and correct it so that it works correctly in inno setup.

[nordi]

oh, you're right.
Because of my anticipation, I overlooked this... :D
Anyway, first of all, thank you very much.


After research, some interesting links (when someone will look for solution):

https://www.skinsharp.com/htdocs/en/ (API Documentation on Website)
https://github.com/CodyGuo/skinh/blob/master/SkinHelp.txt

www . autohotkey . com/boards/viewtopic.php?t=5841 (#2 SkinSharp)
www . autohotkey . com/boards/viewtopic.php?t=116251
www . github . com/samfisherirl/autohotkey-v2-GUI-skins

[nathan.stone]

I would like to ask for help, advice, or something similar.

Until now, I used IsTool to create the installer, I know how to use it. IsTool was enough for me, until the size of the games was around 10-20 GB, but for today's 80-100-150+ GB games, IsTool's LZMA2 compression is not enough.

How can I, - if at all - create a setup file (Setup.exe) for my compressed game using the DiskSpan GUI, if my goal is to make the installer as simple as possible? I don't need a fancy installer, I don't want background music, pictures, etc. just a simple Setup.exe that I can combine with the .bin files created by the DiskSpan GUI to install the game.

[Cesar82]

I would like to ask for help, advice, or something similar.

Until now, I used IsTool to create the installer, I know how to use it. IsTool was enough for me, until the size of the games was around 10-20 GB, but for today's 80-100-150+ GB games, IsTool's LZMA2 compression is not enough.

How can I, - if at all - create a setup file (Setup.exe) for my compressed game using the DiskSpan GUI, if my goal is to make the installer as simple as possible? I don't need a fancy installer, I don't want background music, pictures, etc. just a simple Setup.exe that I can combine with the .bin files created by the DiskSpan GUI to install the game.

https://fileforums.com/showthread.php?t=103408

[kj911]

A mystical question: Does anyone have any idea where to find the old "Netopsystems FEAD Optimizer/Dynamizer" or "NOSSO/iNOSSO" compression programs? Hacked "NOSSO" is only suitable for compression benchmarking and repackaging of "Adobe AIR Installer" installers. If the LZMA part could be replaced with LOLZ, what would it be capable of?

The hacked "NOSSO" is available at encode.su

UPDATE: Added few old Netopsystems documents from old my and web archives, many PDFs still not available from encode.su site uploads.

And uploaded demonstration package from repackaged Adobe AIR Installer. (v50.2.4.1)

Original installer: https://airsdk.harman.com/assets/downloads/AdobeAIR.exe
Size: 5.85MB

NOSSO package: https://www.upload.ee/files/16755387/AdobeAIR_NOS_SFX.zip.html
Size: 4.45MB (zipped: 4.43MB)

NanoZip 0.09a with nz_cm results: 4.66MB
WinRK3.1.2 PWCM/800MB -> 4340kB, with 2047MB and many max. settings resulted in size 4329kB

And added hacked NOSSO package.

[brispuss]

Is there any guide on what the command syntax is for lolz?

There seems to be nothing that I could find that shows the required syntax for using lolz for compression and decompression (of single or a few files) manually (instead of running scripts/batch files).

I'm guessing the command syntax is something like -

lolz [compress|decompress or, encode|decode or, something similar] -options input output

There is documentation for the options available for lolz, but no apparent documentation on the commands and on the order of inputs and outputs?

Can someone please clarify what are the available compression and decompression commands, and confirm the order of file inputs and outputs (is it input output, or is it output input?).

Thank you.

[kj911]

Using FreeArc, this is the syntax, for LOLZ, in the "arc.ini" file:

[Compression Methods]
CUSTOM = xZLib+srep+lzma:ultra:64mb:bt4:fb273:lc8:mc1000000
MASK = xZLib+srep+lolz:dt1:dtb1:dtw1:dtm1:dto1:dtd1:mtt1: mt4:d160m:tt16:fba4096:mc1023/$binkpack=bpk/$OGGRE=oggre/$MP3Pack=packMP3/$Void=srep
LZMA = lzma:a1:ultra:256mb:bt4:fb273:lc0:pb0:lc8:mc4096
LOLZSLOW = lolz:dt:dtb1:d160m:fba4096:mc1023:tt16:mt1:ldmf1:l dc0:ldl5
LOLZ = lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:fb a4096:tt16:mc1023
LOLZFAST1 = lolz:dt:d160m:mc32:mtt1:mt2
LOLZFAST2 = lolz:dt:d320m:fba4096:mc32:ldmf1:mt1
LZP = lzp:8mb:3:h22:d64mb

[External compressor:lolz]
header = 0
packcmd = "Resources\lolz_x64" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

To unpack, you need this (arc.ini):

[External compressor:lolz]
header = 0
unpackcmd = "cls-lolz_x86" {options} $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

+ cls.ini:

[lolz]
Bufsize=512k
transfer_ReadBufSize=512k
transfer_WriteBufSize=512k
MaxThreadsUsage=100%
MaxMemoryUsage=25%
TempPath=.\
ldmfMaxMemoryUsage=32m
ldmfDeleteTmp=0

This is the default usage mode as shown above.

If you want to compress manually, you basically have to copy the above command from the "Compression Methods" line and modify it a bit.

For example: lolz:dt:d320m:fba4096:mc32:ldmf1:mt1
Rewrite to: "lolz_x64.exe" -dt -d320m -fba4096 -mc32 -ldmf1 -mt1 datafile.arc packedfile.lolz

To unpack: "cls-lolz_x86/x64.exe" packedfile.lolz datafile.arc

The use of the "cls.ini" file may be omitted in this case.
I hope I got the question right...

[brispuss]

Thank you very much for your detailed reply!

Your information is very useful and basically answers my question.

Thanks again!

[Masquerade]

^^
The official lolz documentaiton is here:
https://krinkels.org/resources/lolz.264/

[brispuss]

Trying to precompress a png image using xtool (2020 version) using the command -

xtool precomp -mpng+preflate -d1 Hint.png hint_png_preflate

and trying -

xtool precomp -mpng+reflate -d1 Hint.png hint_png_reflate

In both cases the outputs were only slightly larger than the original!?

For preflate option, 3642461 bytes to 3642579 bytes

and for reflate option, 3642461 bytes to 3642578 bytes.

Why only a slight increase in file sizes? Is there something I'm missing in the commands?

png image here (https://u.pcloud.link/publink/show?code=XZmYlS0ZzUeN9LTYx18lXCH6aiCYdbS7i4yk)

[KaktoR]

You do something wrong for sure.

Streams: 2 / 2
Time: 00:00:01 (CPU 00:00:00)

Size: 3.47 MB >> 12.7 MB

Done!!!

[brispuss]

You do something wrong for sure.

Streams: 2 / 2
Time: 00:00:01 (CPU 00:00:00)

Size: 3.47 MB >> 12.7 MB

Done!!!

Thanks. But what is the correct commands to expand the png file using xtool (2020)?

Note that this is a manual input via command line and not part of a script (yet).

[Masquerade]

Your command line looks all correct.

[brispuss]

Thanks for the verification of my command line command syntax.

However, how does Kaktor get a larger file output, whereas my outputs are only slightly larger than the original file?

Kaktor using different version of xtool? Using different settings/options?

Can someone please explain?

Thank you.

[KaktoR]

Here is what I use

xtool:reflate:png:d1

Compressed 1 file, 3,642,461 => 13,362,043 bytes. Ratio 366.84%
Compression time: cpu 0.00 sec/real 1.24 sec = 0%. Speed 2.94 mB/s
All OK

[brispuss]

Thanks for the details, but maybe I'm still doing something wrong?

I'm using xtool version 0.8.7.

My manual translation of your script format command and options for manual input within an elevated command box were now -

xtool precomp -mreflate+png -d1 Hint.png hint_png.xtool

Again, to be clear, I'm inputting the commands and options directly/manually within a command box and executing the commands immediately by pressing the enter key. I am not running a script, and I'm not creating/adding anything to a (*.ini or *.bat) script at this time either.

Unfortunately I'm still getting hardly any improvement in output file size.

Input 3,642,461 bytes, output 3,642,578 bytes.

So how is it possible to get output of 13,362,043 bytes?

[Masquerade]

No idea, I use command line too and it works fine for me:

xtool precomp -mpng+preflate -d1 Hint.png Hint.png.xtool
XTool is created by Razor12911

Version 0.8.7

Streams: 2 / 2
Time: 00:00:01 (CPU 00:00:00)

Size: 3.47 MB >> 12.3 MB

I have attached my working files below + a bat file if you want to have a go:
https://www.upload.ee/files/16621022/Test.7z.html

[KaktoR]

I tried but I cannot make it to work with normal cmd too. Only dsg is working lol. I will look tomorrow. Most likely I just made something wrong.

[Masquerade]

I have xtool.exe in my PATH so I can use it from any command line window. Maybe that affects something.

[brispuss]

No idea, I use command line too and it works fine for me:

xtool precomp -mpng+preflate -d1 Hint.png Hint.png.xtool
XTool is created by Razor12911

Version 0.8.7

Streams: 2 / 2
Time: 00:00:01 (CPU 00:00:00)

Size: 3.47 MB >> 12.3 MB

I have attached my working files below + a bat file if you want to have a go:
https://www.upload.ee/files/16621022/Test.7z.html

Thanks for this!

I downloaded and installed your copy of xtool.exe and preflate_dll.dll to a temporary directory for testing (D:\test).

My original xtool.exe and associated files are located within the C:\xtool directory.

PATH to the original installation of xtool (C:\xtool) was added to the Environment Variables.

PATH to D:\xtool was NOT done to avoid conflict between my original version of xtool and this downloaded version of xtool.

I compared both xtool.exe and preflate_dll.dll files from my original installation (C:\xtool) to the same files within the test directory (D:\test). Both files match precisely.

I ran a manual command line test (xtool precomp -mpng+preflate -d1 Hint.png Hint.png.xtool) for the test directory version of xtool, and it successfully expanded the Hint.png file to Hint.png.xtool to 12,858,367 bytes!

I then ran another manual command line test for xtool.exe in the D:\test directory. Followed by a manual command line test for xtool.exe in the C:\xtool directory. Outputs from this test was different. Refer to enclosed screenshots for details.

This is the contents of the temporary test directory D:\test

test contents (https://u.pcloud.link/publink/show?code=XZM05j0ZV0lQhVBxxkm5w2EynpzlOmLx31ck)

This is the contents of the original installation directory C:\xtool

installation contents (https://u.pcloud.link/publink/show?code=XZ855j0ZmmHT499gxV7TgC1SPXkcRmfOxsSV)

This is the result of xtool.exe test from the temporary test directory D:\test

xtool test from test directory (https://u.pcloud.link/publink/show?code=XZd55j0Z2a74fv7a0WjTLCziF0PLfROagzEy)

This is the result of xtool.exe test from the original installation directory C:\xtool

xtool test from installation directory (https://u.pcloud.link/publink/show?code=XZv55j0ZDWYL4I6IGtQfYiDU0FMHkFI1246y)

Tried to attach screenshots directly, but result was very messy layout! So had to upload screenshots to image server instead (sorry).

The results for the xtool.exe tests are different as can be seen in the screenshots. So despite the two xtool executables (xtool.exe) being exactly the same, their outputs do not match!?

It should be noted that the preflate_dll.dll file for the original installation is located within the _libraries sub folder of C:\xtool ie located here - C:\xtool\_libraries.

Further comments, please?

[KaktoR]

It should be noted that the preflate_dll.dll file for the original installation is located within the _libraries sub folder of C:\xtool ie located here - C:\xtool\_libraries.

Further comments, please?

Do you have set the xtool option for custom path in arc.ini? I don't think you can set this option directly from command line.

[brispuss]

Do you have set the xtool option for custom path in arc.ini? I don't think you can set this option directly from command line.

If you are referring to the arc.ini file located in Freearc here - C:\Program Files (x86)\FreeArc\bin , then no I haven't changed anything in arc.ini.

In any case, I rarely use Freearc or Arc at the moment.

How does changing the arc.ini content help with running xtool properly from the command line?

[Masquerade]

@brispuss
Ah, there's your issue.

XTool is inherently a command line tool, Razor made xtoolui.dll for quick testing.

So, in your C:\XTool folder, you will see the ui load because xtoolui.dll is present. This dll is not a dependency to xtool.exe so you can move/delete it and then xtool.exe will function only in console mode.

Furthermore, you need the library for precompression to be in the same folder as xtool.exe in order for precompression to work. XTool won't read/search the _libraries folder for compression libraries.

So, by having xtool.exe and preflate_dll.dll in a single folder, you will have working precompression. Same goes for zlibwapi.dll for zlib, hif2raw_dll.dll+raw2hif_dll.dll for reflate, libzstd.dll, liblz4.dll etc..... all need to be in the same folder as xtool.exe for those codecs to work.

[brispuss]

@brispuss
Ah, there's your issue.

XTool is inherently a command line tool, Razor made xtoolui.dll for quick testing.

So, in your C:\XTool folder, you will see the ui load because xtoolui.dll is present. This dll is not a dependency to xtool.exe so you can move/delete it and then xtool.exe will function only in console mode.

Furthermore, you need the library for precompression to be in the same folder as xtool.exe in order for precompression to work. XTool won't read/search the _libraries folder for compression libraries.

So, by having xtool.exe and preflate_dll.dll in a single folder, you will have working precompression. Same goes for zlibwapi.dll for zlib, hif2raw_dll.dll+raw2hif_dll.dll for reflate, libzstd.dll, liblz4.dll etc..... all need to be in the same folder as xtool.exe for those codecs to work.

Thank you very much!!

I was coming to the same or at least very similar conclusions as you have after carrying out more tests. I was about to post my findings and comments something similar to yours, but you bet me to it!! :)

I have deleted the GUI files xtoolui.dll and xtoolui.ini. Moved all the codecs from the _libraries sub folder to the main folder with xtool.exe (C:\xtool\).

So, for now, xtool seems to be working as expected. I just have to be extra careful about inputting the correct parameters for xtool to function properly!

[brispuss]

As per the title, I'm trying to compress MP4 files, but I'm having difficulty in getting any compression of the files so far.

From searches here, it appears that SREP should be used first followed by LZMA for final compression. SREP settings as per posts at this forum could be -a2 -m5f -l512. Unknown what setting(s) used for the LZMA compression.

How to achieve best possible compression of MP4 files please?

Using SREP, LZMA, and/or other precompressors/compressors?

And what settings should be used (in general) for compressing MP4 files?

Process MP4 files via Freearc? 7zip? or what?

Thank you.

Sample MP4 video enclosed for testing.

Test Video (https://u.pcloud.link/publink/show?code=XZQA5f0ZB9br3Lr24Uz7Car5s86koy6fSdUk)

[Masquerade]

^^
Videos don't compress well, so just use srep. Anything more is wasting time.

The only video type that compresses only kind of well is bink videos (bik, bk2) because there is a specialised tool for it (bink_pack).

Other video formats, mp4, webm, avi etc etc etc all just use srep.

[KaktoR]

Uhm, MP4 videos are already compressed, at least the video stream.

[brispuss]

Thank you for the comments.

Until a specific compression tool for MP4 comes along, I'll have to use SREP.

Any particular or best options to set for SREP like -m3f or -m5f etc etc? Or do the options depend on the actual encoder used for the MP4 container?

[Masquerade]

^^
There will never be a specific tool for mp4, as kaktor said the video is already compressed.

[brispuss]

OK.

Suggested SREP settings?

[Masquerade]

^^
-m3f is fine.

[kj911]

Try it find FAILSAFE methods? (ignored many srep errors during decompression.)

Trying use internal LZP compression faster than lzma and betterly compress, incompressibble data or use very strong compressors. Likely LOLZ or RAZOR. This two compressors very goodly compress hardly compressibble data than others. (avg. 0.5-1% to upto 2-3% or better.) Default srep settings = bigger results this sample file. Try l32-64 settings need more memory from biggest datasets and not guaranteed failsafe decompress, except minimally better ratio. MSC/BPK/OGGRE not enhance ratio/compress the data.

Few quick test results added in TXT-file. (zipped)

[brispuss]

Try it find FAILSAFE methods? (ignored many srep errors during decompression.)

Trying use internal LZP compression faster than lzma and betterly compress, incompressibble data or use very strong compressors. Likely LOLZ or RAZOR. This two compressors very goodly compress hardly compressibble data than others. (avg. 0.5-1% to upto 2-3% or better.) Default srep settings = bigger results this sample file. Try l32-64 settings need more memory from biggest datasets and not guaranteed failsafe decompress, except minimally better ratio. MSC/BPK/OGGRE not enhance ratio/compress the data.

Few quick test results added in TXT-file. (zipped)

Thanks for the detailed tests!

Yes, it seems at this time from doing some testing myself now, that Razor compressor seems to give the best compression result so far!

[brispuss]

Partly in relation to compressing MP4 files, I'm experimenting with the command line version of FreeARC (Arc.exe) and using additional (pre)compressors in association with Arc.

For example I've tried this manually entered command -

Arc a -m=srep wigo.mp4 wigo.mp4.srep

but get error message -

"Arc: user error (Unsupported compression method or error in parameters: srep)"

Srep executable (srep.exe) is located next to arc.exe within the same directory.

And arc.ini (located within same directory as arc.exe and srep.exe) has an entry for srep as follows -

[External compressor:srep]
packcmd = srep {options} $$arcdatafile$$.tmp - <stdout>
unpackcmd = srep -d {options} - - <stdin> <stdout>

So what is wrong with the command to run srep via freearc?

[kj911]

Try it: -msrep or -msrep+lolz/razor or -msrep:m3f:l512(or smaller l# values):a0
This CMD window navigated to FreeArc\bin directory? Or use FreeArc GUI? Win7 x64 differentially resolve cmd/path related commands than XP.*

See the 'mp4.txt' file used commands...

*Very strong example:

WinXP: move data.bin data.7z --> equal to "ren" console commands.
Win7: move data.bin data.7z --> worser scenario, destroying the whole file! Better scenario, workly rename.

[brispuss]

Thanks for the comments.

I'm suspecting the issue of arc not working properly may be because I have several other programs that also have the arc.exe included. So when I attempt to run the arc command, the command may be selecting another location of arc.exe and not the arc.exe within the Freearc directory.

Presently trying to remove paths to the other programs containing arc.exe and see how things go. And trying to run arc.exe from the current working directory of Freearc\bin.

EDIT: I managed to get Arc.exe working with methods by copying Arc.exe and Arc.ini and srep to the current working directory which contained the test mp4 file.

The initial problem of Arc not working properly was most likely due to the presence of multiple copies of Arc.exe and Arc.ini within several other programs which were causing conflicts with each other.

EDIT2: Did some compression tests on 70 MP4 files ranging in size from 547 kB to 11,351 kB.

Compressor Commands/Options Compressed Size Compression Time
Original 70 files - - - - - - > 171,978,453

srep -m3f -a2 157,392,930 1.979
mcm -x13 156,938,895 296.611
nz a -co -m4g 156,859,691 20.767
nz a -cO -m4g 156,823,772 25.322
nz a -cc -m4g 156,766,025 171.793
lolzslow as per arc.ini 156,693,448 207.330
arc a -m=razorx 156,681,739 87.046
fp8 v6 -8 156,625,221 1038.185
lolz per arc.ini + lm4 156,546,454 318.421
lolzslow per arc.ini + lm4 154,768,632 320.855

NOTE: for compression options using lm4 for files, files fail to decompress! Do not use lm4 option!

There is not a great difference in final compressed file sizes regardless of compression method.

Not really the best ideal final compressed sizes, but MP4 files are difficult to compress.

[kj911]

brispuss: LOLZ is the right choice. (especially if x86 compatibility is also a consideration, when unpacking.)

Can someone share the very first LOLZ version? (2016.12. release.) Is this version "compressor" 32 or 64 bit? (cls-lolz.dll found only x86 version.)

brispuss: Don't use different configuration than lm0 compression settings! Other lm1-2-3-4 its used (not totally removed) any special/older methods from LOLZ code, its works only from compression, will create not decompresseable archives. Try it testing lm4 compressed archives, still crashing now! (Yes, this ratios better than lm0 mode only...)

[brispuss]

Yes, thanks for this caution!

I was about to make a post as to why files compressed using the lm4 option failed to decompress!

It is now my understanding that the option lm is not fully/properly setup/coded, so that is why files fail to decompress with lm compression options lm1 ~ lm4. lm0 option seems to be fine though.

[brispuss]

I was reading a text document about a week ago that describes in some detail the file types, and/or the (pre)compressors used in compression. I didn't finish reading this document which I thought I had on my computer, but I can't seem to find the document now!? I think I originally downloaded this document from this forum, but on searching this forum, I can't seem to find this document (again)?

The document author was I think someone fairly well known on this forum (Razor?, Masquerade?, Panker?, Blackfire?, or ??), but I'm not sure.

Can anyone please locate this document?

Thank you.

[KaktoR]

Why not just go to your browser history and search there all links from this forum? I mean you know that it was a week ago, so just search by links which are visited least 7 days ago.

[brispuss]

The document I was reading I believe it was on my computer. I think I downloaded this document quite some time ago, so I would have to look on my browsing history maybe quite a few months back, maybe a year ago(?) and not look just for a week.

[Masquerade]

^^
Could you mean how to spot compression algorithms with your eyes? That's an archived link in the Tools for Repacking thread.

[brispuss]

^^
Could you mean how to spot compression algorithms with your eyes? That's an archived link in the Tools for Repacking thread.

Thanks for your suggestion. But this was not the content of the document.

I believe the document discussed the various types of (pre)compressors and related software for use with certain file types. It went into some detail as to which was the most appropriate software to use for the file types to be processed and perhaps suggested software settings as well.

[brispuss]

xtool supposedly is capable of handling (all) files compressed with zlib. JPEG files are usually compressed via zlib I believe, but xtool seems incapable processing the JPEG files?

Precomp, by Christian Schneider, is capable of processing JPEG files, but xtool is not for some reason(s).

Can someone please explain why xtool can't process JPEG files (properly)?

Thank you.

[KaktoR]

There are xtool plugins for JPG images

brunsli
Streams: 42 / 42
Time: 00:00:00 (CPU 00:00:01)

Size: 20.9 MB >> 16.0 MB

Done!!!

packjpg
Streams: 42 / 42
Time: 00:00:02 (CPU 00:00:06)

Size: 20.9 MB >> 15.3 MB

Done!!!

jojpeg
Streams: 42 / 42
Time: 00:00:16 (CPU 00:00:36)

Size: 20.9 MB >> 14.2 MB

Done!!!

precomp intense
Compressed 43 files, 21,960,949 => 17,766,251 bytes. Ratio 80.90%
Compression time: cpu 0.02 sec/real 2.01 sec = 1%. Speed 10.91 mB/s
All OK

16.94 MB

[brispuss]

Thanks for the detailed reply.

I'm looking into this now and will run some compression tests.

EDIT: Compression test results -

(Pre)Compressors Commands/Options Compressed Size Compression Time

Original 183 jpg files size - - - - - - - - - > 32,620,858

razorx xtool precomp -mrazorx 32,687,179 31.001
xtool + lolz(std) -mzlib+reflate 32,526,487 135.073
xtool + lolz(std) -mzlib+preflate 32,526,388 132.572
xtool + lolz(std) -mreflate 32,525,512 136.298
xtool + lolz(std) -mpreflate 32,525,336 132.941
xtool + lolz(std) -mzlib 32,524,854 132.349
precomp + mcm -cn -brute + -x13 26,888,771 226.509
precomp + razorx -cn -brute + -mrazorx 26,817,498 32.449
brunsli 26,740,253 4.243
precomp + nanozip -cn -brute + -co 26,669,251 14.942
precomp + nanozip -cn -brute + -cO 26,668,630 14.602
precomp + nanozip -cn -brute + -cc 26,662,387 169.879
xtool + brunsli -mbrunsli 26,633,426 100.867
precomp -cl -brute 26,631,038 9.895
precomp + lolz(slow) -cn -brute 26,624,723 31.563
precomp + lolz(std) -cn -brute 26,622,527 33.496
lepton 26,350,606 14.776
lepton-slow-best-ratio 25,920,088 17.905
xtool + packjpg -mpackjpg 25,850,839 100.766
packjpg -np -p 25,826,134 10.584
xtool + jojpeg -mjojpeg 25,678,421 105.765
jojpeg c 25,016,673 129.041
fp8 v6 -8 24,445,243 255.514


Note: lolz(std) means options for lolz as defined in arc.ini file. And lolz(slow) means options for lolzslow as defined in arc.ini file.

[CrownRepack]

Hey guys, im new to repacking and compressing, i learned quite a few stuff here myself by reading, i just want to know what is the most used/relevant compressor tool now? is it DiskpanGUI or etc etc...? Please suggest the most updated and best compression tool for that does FreeArc, Thanks

[Masquerade]

^^
DiskspanGUI is probably the best bet.

[kj911]

Still available special offset-shifted file compare tool??? (File sizes differ smaller than 1KB.)

Read two file started in predefined text or hexa code to file ends.. (Ex: Two WAV files compares, compressed non-standardized tools or etc...)

Quick example (not available the offset-functions):

"fc /b start=data file1.wav file2.wav" or "fc /b start=64617461 file1.wav file2.wav"

start -> start offset reading from two comparable files, reading to file ends, ignoring previous bytes before the offset value. With(in/out) added extra switch, "end=0" aka read files to file endings or "end=X" read files to "X" offset points. (reading two files before file ends.)

A similar question was asked here, almost ~20 years ago: https://www.ghisler.ch/board/viewtopic.php?t=8523

[KakiDev]

Where should I get started if I want to compress a game and have no indea what the best tools for it are? Looking at the Tools for Repacking thread there are LOTS of tools to try, are there any I should prioritize trying, or just take the few hours to go through all of them and see what does the best?
Thanks in advance :D

[KaktoR]

First you have to know what data you are dealing with.

From there on you have to know what tools you have to use.

However you can always ask here what game you are working with, then users will probably tell you something about it.

[KakiDev]

Well, I'm mostly dealing with renpy games (renpy is an engine for building visual novels), looking at one game right now its 90% jpg, mp3 and webm files, webm files make up nearly 50% of the game size here. I want to avoid lossy compression, so no reencoding.

So I guess I'd need to know what the best tools for webm, jpg and mp3 are as they make up 98.8% of the game together.

[KaktoR]

webm are already compressed files. Nothing you can do except for reencoding.

mp3 are already compressed aswell, but for this there are several tools to compress them further without reencoding (packmp3, mpz, 7z mp3 plugin (power archiver)).

jpg the same, but here aswell we have some tools like packjpg, brunsli, jojpg, precomp, lepton.

[zuryki]

I started a few days ago, and I'm using DiskSpan_GUI with the Light Installer Script. I've done some experiments, and everything is perfect.

While searching for an efficient method to compress the game Mad Max:
https://fileforums.com/showpost.php?p=462574&postcount=5

Upon inserting the data from the above link:
-------------------------------------------------------- CONVERSION DATE 2024/07/20 --------------------------------------------------------
02:00:48 - You have chosen the DVD5 type for the media size.
02:00:48 - ISO images will not be created when conversion is complete.
02:00:48 - The computer will remain on when the conversion is complete.
02:00:48 - Selected ARC/DS method for Data1a-01.bin was: xtool
,t4
+srep_new+lzma
02:00:48 - Selected ARC/DS method for Data1b-01.bin was: srep_new+lzma2
02:00:50 - Compression initialized on 20/07/2024 at 02:00:50.
02:00:51 - Starting Data1a-01.bin archive creation.
02:00:53 - Archive Data1a-01.bin creation error.



Summary:
FreeArc 0.67 (March 15, 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressing 79 files, 34,125,862,912 bytes. Processed 0.0%
ERROR: write error (disk full?) in compression algorithm xtool
,t4

Data1a-01.bin
List to filter included files:
*.arc

Data1b-01.bin
List to filter included files:
*.*

This site is a goldmine. Can you tell me what I'm doing wrong?

[KaktoR]

This method belongs to a very old xtool version which is not included in DSG.

Just use xtool 2020 with zlib codec instead.

[demon964]

This method belongs to a very old xtool version which is not included in DSG.

Just use xtool 2020 with zlib codec instead.


https://i.imgur.com/TURifGi.png

Hi. I am working with Light Installer Script

I have 2 question

1) One of game folder need unpack to Documents folder. Is this settings true?

2) can i disable this settings? i dont use winrar part system like DVD 4.7gb or other

[KaktoR]

Yes.

You can just use "Unlimited" tab outside of developer mode.

[demon964]

Yes.

You can just use "Unlimited" tab outside of developer mode.


why next button not active

https://i.imgur.com/eju1h47.png



https://i.imgur.com/dheCGzU.png

[KaktoR]

Because you have no next tab (no tab after Game 1)

[demon964]

Because you have no next tab (no tab after Game 1)

i want finish making installer. Next button not clickable

[KaktoR]

Then click save button

[mausschieber]

i want finish making installer. Next button not clickable

you better read here How to use Diskspan GUI v2+ (https://fileforums.com/showthread.php?t=104533) there is all Information for you

[demon964]

Then click save button

thanks worked. how i can make created .bin files optional ? like fitgirl

if i delete bin files, setup.exe still search file and give error

but, fitgirl repacks, if bin file not in the folder, setup.exe skips this file

[kj911]

Has anyone encountered such an installer? (Is their compression technique beatable? ex: FitGirl could do it?)

https://i.kek.sh/WV7u2qf3qR3.png

Is it possible to find such an installation package?? (Yes, bitWrapper(tm) compressed.)

[Doomman2008]

Hi.

I have a question about Diskspan GUI / Inno Setup.
I'm actually getting on quite well with the software so far.

Now I have a new situation:
I want to pack a game in one archive and a mod for this game in a second archive.

In the finished Inno Setup installer there should be a checkbox.

If the checkbox is enabled the installer should extract archive 1 and then archive 2.
If the checkbox is disabled the installer should only extract archive 1.

Is this even possible? Or another way to reach the same goal?

Maybe I'm thinking too complicated? :confused:

[KaktoR]

Any installer which allows components and is compatible with DSG is suitable for this

[Doomman2008]

Good to hear so far..

Can you give me an example how i can do that?

I use Diskspan GUI mainly for compressing.

I use the script from "Light Installer" as template but i do changes for every game i compress manually in Inno Setup editor.

[KaktoR]

If you use LIS try changing that

[Components]
Name: comp1; Description: Game 1; Types: full; ExtraDiskSpaceRequired: 10000000000
Name: comp2; Description: Game 2; Types: full; ExtraDiskSpaceRequired: 20000000000

[Doomman2008]

I already saw that lines.

But how could i define what data is "comp1" for example.
I don't understand this.

[Doomman2008]

Ok, my problem is solved:

Components are defined in records.ini, too.

in case anyone else has this problem at some point:

[Record1]
Type=FreeArc_Original
Source={src}\Data1a-01.bin
Output={app}\
Disk=1
Size=386.315.748 bytes

[Record2]
Type=FreeArc_Original
Source={src}\Data1b-01.bin
Output={app}\
Disk=1
Component=1
Size=712.924.910 bytes

[InstallerSettings]
SingleDisk=1

[audy]

Hello, I have been registered for years, and this is my first post.
I have yet conceptual doubts.

1. It is assumed that if I use a codec (or method) from a compressor (example: zlib of xtool), it is because the group of files that I want to compress, most of them have the codec zlib (right?), But in practice, how can I recognize what codecs are used in those files? maybe a scanner?

2. I have seen statements of compression method like:
5rzm = rep:300mb+delta+rzm/$wav=tta/$bmp=bmp/$jpg=jpg/$jpgsolid=precomp+rep:32

What do the characters /$ mean, what function does it meet? I have an idea, but I prefer an explanation.

3. How do we know what size of chunk set in the parameters of a precompressor (or SREP)? or what is the most advisable to use? For example, I want to compress 100 files and each of them weigh between 8 and 64 MB.

[wareck]

Hello, I'm not a very good repacker, but I know some basis...

1 for finding codec:
-experience (after a while you will know that game made by unreal, or by this other kind of editor use the same compression template)
-hexeditor (you can find some signature or some indications like ogg, wave, mpeg ect)
-game scanner tool (you can find it on this forum)
-"monkey test" test this parameter or this and compare results. For some game it takes me days to find the right setup to gain some %

2 masked compression:
$xxx is mean a group
for example $bmp is for "all file in the bmp group"
It can be .bmp .gif . bitmap (you can setup this group in arc.group)
then the method
in you example $wav=tta it means that all file in the "$wav" group will be processed with TTA method
at the begining of the mask you have rzm
It mean that the files who are not in a group or if the method for this file will fail, it will processed by the main method rzm

3. srep
if you use a normal process like diskspangui or similar
the process will first do a solid chunk (like is you use 7zip with store option and 0 compression)
file1+file2+file3+file4... => bigfile
the srep will work on the bigfille
file1+file2+file3+file4=>bigfile=>srep>=bigfile.srep
then bigfile processed by srep will be processed by the final compressor

file1+fille2+file3+file4...=> bifile=>srep => bigfile.srep=>lzma2= > archive
In your question, 64mb is the chunk srep will work with.
it's not the size of the final files, its how many partial chunk of "bigfile" it will work in the same time.
big chunk give better results, but slower compression/decompression.
and less computer compatibility (if you decompress this file on a slow computer with low memory it may fail)
lower chunk size lower the deduplication results, faster compression and decompression but bigger final files.
it's more or less a compromise...
this is very linked to the file.
if you try to srep audio or video files you will not find deduplication (same words, same octet combinasion)
but for texture, in some games there is a lot of same structure, same files, you will have a lot of redundancy and it will reduce the file a lot...


the best method is to experiment, test, try and try again

[jakepaulie]

Hello, I have a question... I want to do Repacks using Inno. But I dont know any tutorials, and some like that... Please help me, or say what tutorials should i watch

[dyren]

^^
Hi, there aren't any specific tutorials for making repacks. Inno is used for making installers, not the repack itself. Making repacks is based on doing research and gaining experience.

Here's a few tools I would recommend for starting out:

DiskSpan GUI - for compressing, aka making the repacks: https://fileforums.com/showthread.php?t=104507
ASIS - for making the installer, so you can unpack the archive DiskSpan creates: https://fileforums.com/showthread.php?t=99481

Additionally, you can browse the 'Best Compression Methods for 'Specific' Games INDEX' thread, to find out what method works for certain games (https://fileforums.com/showthread.php?t=101639)

[jakepaulie]

Okay. I try'd to use the DiskPan, but i have a small problem. It's a bit weird. When i normally create a compressed file, i cant put it in some program. Or can I? Can you please help me?

[Masquerade]

^^
What do you mean?

[jakepaulie]

Hello. I was looking at Inno Setup Compiler. I dont know why, but when i do some wizard and install it, the file of the game dont want to open. Maybe some decompress error. But can you please help me with it?

[Ra7eN]

HI not sure if this is the right area..
Im old school back in the days of MYTH, Paradigm etc... So I know now, how they got those files down to size. ripping. Back then is was crucial due to the 2400b - 1440b.. now a days with fiber optics no biggy. So I still like to compress all my games. I don't rip them anymore. but I found this forum and here is what I use:

I do all this in a bat. keep in mind I've used this for some time, so forgive me if it is ancient

This was done with skyrim with all the lang taken out but english. saved about 10gb. Leaving a size of approx 20gb

compress: (all the user input var and other features including escape codes for color and locations have been omited to show just compression)
yes this is line-by-line, I like it that way).
7z a -ttar "%temp_tar%" "%input_folder%\*"
precomp_047.exe -o"%temp_pcf%" "%temp_tar%"
srep64.exe -m3f -a0 "%temp_pcf%" "%temp_srep%"
7z a "%output%" "%temp_srep%" -mx9 -m0=lzma2 -md=512m -mfb=273 -ms=on -bsp1
Result: 13gb


Another compress from here (chain)
arc a -lc8 -ep1 -ed -r -w.\ %archive% -msrep+delta+lzma:a1:mfbt4:d158m:fb273:mc1000:lc8 "%gamepath%\*"

surprisingly 13gb actually 40mb smaller

Now this works great with just using a .bat. the iss are cool but I love bat stuff. And yes this is just a basic run of the mill I use and does a really good job for the most part.

so getting up to date, are there newer methods that are standard for .bat file compression? Or are these still the tried and true holy trinity :-), I'm old so be gentle on the corrections. I am always looking for new ways of shrinking without ripping.

[L33THAK0R]

I'd say check out xtool for potentially better pre-compression, can't hurt to experiment with other final compressors like razor or lolz, other than that it's difficult for general advice as the level of research and experimentation for a given title varies greatly I've found.

[Ra7eN]

Hi, thanks for the reply!
I’ve tried a few different tools, and in my experience, when it comes to compressing (not ripping), there’s usually only a minor reduction in size. Using methods like Precomp and SREP with either ARC or 7z, the results typically fall within a 10–20% range, depending on the game and file structure. Of course, larger games might show more variance, but that’s not always the case.

I haven’t used "lolz" yet—I actually thought it was an emoji at first, haha! I’ll definitely have to check that one out.

[Masquerade]

Lolz is an archiver designed specifically for game texture data such as DDS (DXT#) images. It's slower than lzma but is multithreaded and will yield smaller ratios. Out of all tools made by ProFrager, lolz is the most stable.

You may see some overlap between the effectiveness of precomp vs xtool on games compressed with deflate, as both tools support this algorithm. XTool is superior due to having multithreading and a diverse range of supported algorithms. By default there's zlib/deflate, oodle, lz4 and zstd but you can add further codecs and add plug in other algorithms. Over the years I've plugged in doboz and yaz0, for example.

It's all about finding the right algo. If you try to compress a unity lz4 game without the unity lz4 XTool plugin, you'll get an severely larger result, the same result if you used precomp.

[phoenix1of1]

Hello, I am yet another newbie prostrating myself before the alter of knowledge :p
Anyway, I've been doing my research, getting my feet wet and I think I have a reasonable understanding of the processes as far as a newbie can be.

So as it stands, I understand that some of the games I may want to repack, require unpacking and to do this, I need to find out what form of compression was initially used.

In one particular case, I found that a game uses some form of ARC compression but it seems to be custom on that particular game. Is there a way of overcoming forms of custom compression in order to unpack files from a .bin? Am I sniffing up the wrong tree?

Thanks in advance for any clarity on this.

[KaktoR]

What makes you think that the particular game uses "ARC" compression?

You can always share some small samples here if possible so experienced users can trake a look.

[phoenix1of1]

What makes you think that the particular game uses "ARC" compression?

You can always share some small samples here if possible so experienced users can trake a look.

I read that it was suggested to check for "magic numbers" via a hex editor as this should give a fairly strong indication as to what compression may be used.

41 72 43 01 00 00 06 07 41 72 43 01 02 73 74 6F
72 69 6E 67 00 10 10 D7 C8 DE A3 45 45 C1 4E 00
00 00 00 00 00 10 00 E9 FE 04 00 01 F2 DB 00 00

Above is the first three lines of a .bin file from the game I am looking at. It's 41 72 43 that outputs text of "ArC".

If I am sniffing up the wrong tree, happy to be pointed in the right direction and undertake more learning.

Thanks again for your time.

[KaktoR]

This is a standard FreeArc archive.
https://i.imgur.com/AR9liWW.png

You should be able to extract it with FreeArc as long as it is not encrypted. Other then that there are no tools you can use to precompress it.

PS: The "storing" has nothing to do with the actual compression settings used.

[Masquerade]

What game is this?

[phoenix1of1]

This is a standard FreeArc archive.
https://i.imgur.com/AR9liWW.png

You should be able to extract it with FreeArc as long as it is not encrypted. Other then that there are no tools you can use to precompress it.

PS: The "storing" has nothing to do with the actual compression settings used.

Appreciate the clarity, thank you :)

[phoenix1of1]

What game is this?

This is "Graviteam Tactics: Mius Front". I literally just took a game from my collection at random to try and "cut my teeth on" and put what I read in to actionable experience.

[KaktoR]

Could you share a sample file in question from this game?

[phoenix1of1]

Could you share a sample file in question from this game?

My games are from certain groups if you catch my meaning.
The .bin in question is 1.49gb in size.

Happy to share, not sure on the best method to do so as I'll assume that sharing links of that nature would be against forum rules?

I presume that the games being from certain groups wouldn't add any complexity to the process?

[Joe Forster/STA]

Happy to share, not sure on the best method to do so as I'll assume that sharing links of that nature would be against forum rules?

It would; please, don't continue in this direction.

[KaktoR]

Oh ok, so this is actually warez (we aren't allowed to talk about warez, nor sharing it here apparently).

In this case I missunderstood you. I thought the bin files are actually part of the game itself. I looked on steamdb depots for this game and there aren't any file types in this case so I was confused. Now I understand.

Ok so, then this bin/arc archive should be a part of an installer I guess, which contains the actual game files just compressed (like we do if we creating repacks).

The problem now is, that you can't compress this already compressed files further to achive a better ratio. First you have to extract the archive (the game files) and then apply better compression on them to gain a better ratio, simple said.

[phoenix1of1]

Oh ok, so this is actually warez (we aren't allowed to talk about warez, nor sharing it here apparently).

In this case I missunderstood you. I thought the bin files are actually part of the game itself. I looked on steamdb depots for this game and there aren't any file types in this case so I was confused. Now I understand.

Ok so, then this bin/arc archive should be a part of an installer I guess, which contains the actual game files just compressed (like we do if we creating repacks).

The problem now is, that you can't compress this already compressed files further to achive a better ratio. First you have to extract the archive (the game files) and then apply better compression on them to gain a better ratio, simple said.

No worries, appreciate the clarity and the information.
I will figure this out in time :)

[shazzla]

Hi all!

In SREP 4.90 WIP there is the -M5R parameter.
Anybody knows what is it?
When and how recommended its usage?
Did some tests ,but i have no valuable results.

[mastercheff]

Hi Guys,

I really am sorry for the noob question, but I am an absolute beginner concerning compression.

I am backing up my Steam Library and thus far have used the Inno setup wizard to create the Installers. What bothers me most is that max limit of roughly 2.2 GB per file.

I would love to know how to create for example one 40 GB *.bin file instead of 17 - 19 of them (like Fit girl's installers).

I think that Freearc is involved, but apart from that i have no idea how to go about it.

Do you compress the Game Directory with free arc and then somehow integrate it into Inno-setup?

I have seen all these custom installing scripts on here but at first glance nothing that specifically helps me with my problem.

I tried the search function but i could not really find anything . But again it could be that i do not know what to search/look for in all of this data.

Thanks in advance and sorry again

[KaktoR]

Yes we use freearc here as a pipe for other compressors.

If you really just want to get rid of the 2GB limitation of those bin files, there is also the possibility to use just 7zip and make a SFX archive, but then without inno setup.

If you want to have inno setup as an installer, use DSG (https://fileforums.com/showthread.php?t=104507) and use 7zip lzma2 compression or go with something other methods if you want to achieve better ratios. Then get a installer script which is compatible with DSG, like this https://www.fileforums.com/showthread.php?t=103408

[mastercheff]

If you want to have inno setup as an installer, use DSG (https://fileforums.com/showthread.php?t=104507) and use 7zip lzma2 compression or go with something other methods if you want to achieve better ratios. Then get a installer script which is compatible with DSG, like this https://www.fileforums.com/showthread.php?t=103408

Hey, thanks for the quick answer.

Well to be honest, if I could, I would love to get a whole game into a bin, like most of the fit girls installers I tried out (apart from the optional languages/Redistributable and extras bin).

I will take a look at DSG and the script and try to find tutorials or experiment a bit with it.

Thank you very much for the help!

[Tmills]

Hello everyone,

I'm currently using mini compressor tool to compress my games into SFX.exe files. I'm looking for an easy way to quickly identify a good compression method for my games. I'm not looking for maximum compression, just a good balance of compression and decompression/extraction speed that can handedly beat just using 7zip.

is there any kind of tool that can scan a game folder and give you a report of file types found?
is there a common method that will be good enough to just throw at most games?
mini compressor has presets for example:
mxc = xprecomp+lolz1/$bitmapz=bmpz/$bink=bink/$void=void/$photo=jpg/$ogg=ogg/$wavz=wvp/$mp3=mpz

any help would be appreciated. i just want a simple way to store my games and save storage space.

[dyren]

Hello everyone,

I'm currently using mini compressor tool to compress my games into SFX.exe files. I'm looking for an easy way to quickly identify a good compression method for my games. I'm not looking for maximum compression, just a good balance of compression and decompression/extraction speed that can handedly beat just using 7zip.

is there any kind of tool that can scan a game folder and give you a report of file types found?
is there a common method that will be good enough to just throw at most games?
mini compressor has presets for example:
mxc = xprecomp+lolz1/$bitmapz=bmpz/$bink=bink/$void=void/$photo=jpg/$ogg=ogg/$wavz=wvp/$mp3=mpz

any help would be appreciated. i just want a simple way to store my games and save storage space.

wiztree gives you each file types in great detail, i use it myself. there are also plenty of examples in these threads for methods:
https://fileforums.com/showthread.php?t=99554
https://fileforums.com/showthread.php?t=101639

its quite easy to learn the best methods for unity or unreal games, for example. i also suggest that you switch to DiskSpan GUI: https://fileforums.com/showthread.php?t=104507

i used both mini compressor and DiskSpan, and i can tell you that DiskSpan will be much better once you learn how to use it.

[felixible11]

Friends. I have a great interest in repacking and am always fascinated by how blatantly you can compress some games. I can program, but here in the forum you are really overwhelmed with tools and programs.

Theoretically it works like this:

1) Precomp the data with xtool (there are already so many plugins here...)
a. Here it already fails. If I want to read in an entire folder, I always get 0 / 0 streams and the precomp file is exactly the same size as the source (that shouldn't be the case?)

%INPUT% = Sourcedir
xtool.exe precomp -mzlib+reflate -d3 -c64mb -t16 "%INPUT%"

b. In addition, there are countless possibilities and every game reacts differently to the combination of precomp algorithms. Can't I read in a game folder and it automatically returns the best combination? Somehow I'm at a loss here.

c. The precomp files must then be compressed (ideally with FreeArc?).
- If I try
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -mxtool:c32mb:mzlib data.arc “pack\*”
the data.arc is 1.5x too large as the source file?

I would love your help - somehow I'm at a loss.

Many thanks!

I will take care of the unpacking/installing afterwards. That should be easier...

[KaktoR]

You have to know what compression a game is using.

Your archive is larger because you did not deduplicated the data (srep or xtool:dd1..5) and you did not compress the data at the end. However if you precompress the data with xtool zlib and the archive is larger than the original input, you have possibly found out that the input is zlib compressed.

[felixible11]

You have to know what compression a game is using.

And how do you do that? Or is that exactly what the precompress is for?

Your archive is larger because you did not deduplicated the data (srep or xtool:dd1..5) and you did not compress the data at the end. However if you precompress the data with xtool zlib and the archive is larger than the original input, you have possibly found out that the input is zlib compressed.

To understand: Do I need to know what compression the games already have so that
a. to remove/optimize the compression during precompress and (better) compress again later or
b. so that I can pack it even better (over it)?

I'm (still) at a loss here.

[KaktoR]

Experience, knowledge, repetitions.

I would say you can go with game engine repetitions most of the time.

I can tell you that nearly 99.9% of all unreal engine games using either zlib or oodle compression. Older engine versions like UE3 used lzo algo mainly back then.

Most of the unity engine games (if they have bundle files) using lz4hc compression with level 9.

RE_Engine (Resident Evil and all these games) were compressed with either zlib or zstd, but to be honest I didn't worked with this engine since years so things might have changed.

Precompression just means that already compressed streams will be decompressed, that's why the output will be larger than the input, so you can compress the decompressed streams with a better compression than the original used compression. Like if you compress some random folder on your pc with 7z. The compressed files (7z archive) will be smaller then the original input. If you extract the compressed 7z archive, the output will be larger again. That's how precompression works to keep it simple.

I think a and b are the same, or I am really confused.

To keep it simple, these steps in this order:
Precompress -> remove duplicated data -> apply stronger compression

[felixible11]

Thanks for the explanation!

Let's take this as an example:

__________________
Ripped-encoded Files : Nothing (Multi-11)

Original Size : 7.5G

Compressed Size : 2.7G

xZLib+srep+exe+delta+lzma2
__________________


With this command
arc.exe a archiv.arc "C:\Bionic Commando" -m xZlib+srep+exe+delta+lzma2

I arrive at 6.21 GB. I don't understand that.
Since it is not compromised according to the quote, does precompress not apply?

Apparently I still don't quite get it.

[KaktoR]

xZLib belongs to an old XTool version. If you use current XTool version (0.8.9 I guess) use xtool:zlib instead (maybe reflate is a possible option aswell, I don't have the game).

srep is there for deduplication. Actually you just need srep.exe (64bit version) and then use xtool option :dd1..5 (where 1 is fastest and 5 is slowest but might get a few more duplicates. Default is 3 and in my humble opinion this is peak most of the time).
exe and delta are freearc built-in compressors (honestly I don't really know what they do actually).
lzma2 is from 7z archiver so you need 7z.exe and 7z.dll files aswell.

arc.exe a archiv.arc "C:\Bionic Commando" -mxtool:zlib:dd3+lzma2
You can also go with freearc built-in lzma in addition with 4x4 instead of lzma2 to use up to 16 cpu threads.
arc.exe a archiv.arc "C:\Bionic Commando" -mxtool:zlib:dd3+4x4:lzma

[KaktoR]

I downloaded that game and investigated it real quick. Here are my results
https://i.imgur.com/saojnEi.png

The *.bundle files contain WAVE audio. You can use xtool flac plugin or msc on them.
https://i.imgur.com/PRvRCMB.png

The movie files are Bink1 videos. You can use xtool bink plugin on them.
https://i.imgur.com/UsjPcPO.png

[felixible11]

Thank you!

Where i can find xtool flac and bink plugin?

[L33THAK0R]

Thank you!

Where i can find xtool flac and bink plugin?

FLAC is built-in, bink and the majority if not all plugins can be found here (https://fileforums.com/showthread.php?t=103876)

[zioboh]

Hello everyone, yesterday i tried to repack one of my favourite games, i tried looking for some guides but i wasn't lucky at all. Here are the things i did:
1. Got my game folder and packed it into a .arc file (without compression)
2. Ran the file in precomp and got a .pcf file

After that, i wanted to make an installer, but i really can't find anyway to do it.
Could someone help me? Thanks :D

https://imgur.com/a/Wt2o3LZ

[kj911]

Has anyone managed to write or create a working lossless h264 recompressor in recent years from existing losslessh264 (https://github.com/danielrh/losslessh264) and or avrecode (https://github.com/dropbox/avrecode) sources?? (Not in all games, but the presence of x264 videos, next to/instead of Bink2, is becoming more common.) Compression ratio avg ~15% better than 7z/any ~1-3% size reduction. Leaked executables used from DropBox??

Shelwien preserved (https://encode.su/threads/3213-Is-there-an-MP4-optimizer?p=61950&viewfull=1#post61950) few years ago halfly working H264 recompressor from encode.su site. The file, not available.

[TellASpray]

Hi, after reading this thread I think I have some knowledge but I wanna be sure of what I know :

1. Deduplication is the process of deleting identical files ?
2. Precompression is for decompressing already compressed files to compress them better afterwards ?

Now I have some questions about things I don't understand :

3. What are streams ?
4. What do changing chunks, dictionary size and depth do ?
5. If DiskSpanGUI is a compressor why can't I directly extract files with WinRar for example ?

[wareck]

dedup it's not only files, but stucture of file
for example, a .zip file full of GIFs.
Each GIF file will start the same (header), and dedup will also take this into account to reduce the final size.

Precompression => yes decompress file and some kind of file (gif, png ,mp3) to let them compressed later with a better compression system

Chunks/dictionary and depth :
They will impact performances (better speed, better compression, better detection)
these values are a compromise between performance and speed.
values that are too high can also lead to CRC errors.
like dictionary, it's a way to reduce a file size if inside they is several time the same words, same structure, or same hex data
depth is to find stream deeper inside file
like if you have a zip file with inside a movie with a wav file. depth 0 => zip file , depth1 =>zip file with something inside=>movie , depth3 => zip file with something inside=>movie => wav
but with a high depth, soft may need to read file several times => more time to compress/uncompress
also going deep may add some errors and you will have CRC error during decompression
again => these values are a compromise between performance and speed.

streams...
They are in files
For example, to put it simply, a large game file can contain images, sound, texture, text...
during the analysis process, software can indentify a sound stream and decode/decompress/recompress

diskspangui is a compressor, but to be fair, is a collection of diffèrent programs
in the same file you can have several type of data

like if you mix oil and vinegar to make a salad sauce
then you want to put back oil and vinegar in their bottle
you wll need a tool to separate them and work on them each on their own.

diskspangui it's a GUI for several freearc function.
you can open files with freearc (not to unpack, just read them, if you want to unpack, you will need to add soft/config in freearc software)

[TellASpray]

Thank you very much for your answer, do you mind sharing config files (arc.ini I guess ?) or softwares (if that's what you mean by "soft") or even threads about those :]

[Kriogenik]

I have been having great success using DiskSpan GUI and the Installer Script example that comes with it here https://www.fileforums.com/showthread.php?t=104507

However, I have recently come across a game that has an extremely long path name that almost exceeds 260 characters in length (245), which heavily limits where it can be installed, because the installer errors out when trying to write to a path that exceeds 260 characters in length. It will install without error when the installation path is less than 15 characters (260-245)

Does anyone know what steps I should take to avoid the problem? I am very new to this, thanks!

[Masquerade]

^^
Disable PATH limit:
https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/

[brispuss]

I have several games where the assets are contained within compressed and encrypted *.pak files.

What I am wanting to do is to minimize (compress) these *.pak files to save space as some of these assets take up a huge amount of disc space.

I have tried xtool, srep, lolz, precomp and a few other final step compressors but have found the *.pak files don't compress well at all (not surprising as they are already compressed).

However, I understand from reading some posts on the internet that it may still be possible to compress these *.pak files using specialized/other tools(?)

How can I compress these *.pak files?

Sample asset *.pak file - here. (https://u.pcloud.link/publink/show?code=XZs18f5ZDt8WU7ob8Kuz2Xg1Yfy1gpPM6YDV)

[KaktoR]

What game?

[brispuss]

What game?

Whispered Secrets - Cruise of Misfortune Collectors Edition

[wareck]

hello guys
i'm looking for a way to recombine two files.
i have a big .pak file that contains all the game data.
i've duplicated it and processed each part.

one part contains the data compressible by lolz and the other compressible by another compressor.
after decompression i'd like to recombine these two files to make one.

I'm trying to figure out how to do this, with SFK, RAW or equivalent, but I'm drying out...

[wareck]

I've found a solution, maybe not the most beautiful, but it works.
I use xtool (7.9) to generate a database, extract stream from file2 and inject in file1

it works, it's fast, the only problem is to need to have 3x the space on drive (file1, file2, extracted stream) before recombine.

[Masquerade]

Is the data in the big pak compressed beforr you use lolz?

It could be much easier to just extract all raw data and concatenate the data into two blocks.

Compress each block as required, then during unpacking stage, concatenate the two blocks and xdelta patch it to the original pak.

[wareck]

Hello yes I think about this but it's 30gb files
the xdelta file is big and patch is low speed

the pak is aes crypted.

What i've done:
take original pak and uncrypt it
extract "compressible" parts who lolz can have a raisonnable gain
extract low compression parts , compressed later with razor
extract small fast compression files (like txt, json, xml)
then pack each chunk block
(I've got a better compression ratio than compress extracted files)
during install => rebuild the big pak
recrypt pak
small hdiffz patch to recover a good crc

[audy]

Is there a precompressor compatible with the following algorithms?
(https://en64.shoutwiki.com/wiki/n64_compression).
The vast majority of the pack that I thought compress use Yay and Yaz (maybe 50%) (and Gzip/Zlib to a lesser extent)

[Masquerade]

^^
Yaz0 can be handled with wszst - however don't expect lossless 1:1 recompression when compared to original streams. It is a proprietary algorithm and all available implementations come from research and reverse engineering.

Best you can do is recompress the entire game and hope it works, I recommend using a very much older version of wszst as newer version cause incompatibilities when the executable file belonging to the game tries to load the data.

You also need to be aware of the Yaz0 data structure - especially the rotational byte which can cause load errors if you fail to reimplement the byte in the compressed data.

I won't endorse this discussion further since it does not relate to PC games and the company that developed the Yaz0 algorithm is well known to hunt down anyone who doesn't play by their rules. You know where to find me if you want to discuss such Yaz0 more privately.