new to this

[Neuromancer815]

hello, im trying to get into repacking games for the first time ever, for my own private use. i have a very slow wifi where i live and its always a pain to download anything larger than a few gigs. so what i wanna do is compress the games im currently not playing for later use. ive been directed to this forum as a great resource to help me get started, and ive been looking through the sticky for a few days now and id be lying if i said it wasnt very overwhelming, especially since i know absolutely nothing about compression and repacking. i am willing to put the time and effort to learn but im really not sure where to even start? id very grateful if i could get some guidance as to where i need to start this process. currently all i have done besides reading the stickies is downloaded this <http://freearc.org/download/0.666/FreeArc-0.666-win32.exe> and this <https://jrsoftware.org/isdl.php#stable>. i have tried compressing a game, sniper elite 4 to be exact, since i dont have any smaller games to test on but after seeing how much time it was gonna take i aborted it for the sole reason of not being able to use my pc for that duration and not knowing whether or not it was gonna work

[KaktoR]

Depends on how much you want to size down your games. Generally spoken: The stronger the compression, the more time will it take and also CPU/RAM resources, to be simple.

To get the "quick and easy way" you can use DiskSpan_GUI to compress your files.
You can also unpack your compressed archives with it too (Tools -> Unpacker).

In your example for "Sniper Elite 4" I guess you can just use srep+lolz if you want it strong, with cost of speed and pc resources, or srep+lzma with cost of compression ratio, but requires way less pc resources. But I didn't checked the game so I cannot tell you for sure.

Some games requires that you first unpack the files and then compress them. In most cases you use xtool to unpack the game files, but you should know what compression method the game is compressed with. For example "Sniper Elite 5" is compressed with zlib (if I remember well), so you have to use xtool first to unpack the files, then apply srep to get rid of duplicates, then use lolz or lzma to use final compression.

Maybe you have noticed the problem for "newbies" here. That's nothing to explain in a few sentences.

[Neuromancer815]

thanks kaktor for the reply, helped understand a few things i was held up on, i got "srep+7zip multi volume" and "lzma" from the stickies as well as diskspan GUI, but i followed the link to lolz to krinkles.org and it seems i need to post stuff there before i can get download righs which sucks because i dont know anything so what would i post about. also diskpan gui is being flagged as malware, is that a false flag?

[Cesar82]

Originally Posted by Neuromancer815

thanks kaktor for the reply, helped understand a few things i was held up on, i got "srep+7zip multi volume" and "lzma" from the stickies as well as diskspan GUI, but i followed the link to lolz to krinkles.org and it seems i need to post stuff there before i can get download righs which sucks because i dont know anything so what would i post about. also diskpan gui is being flagged as malware, is that a false flag?

If DiskSpan GUI was flagged as a virus it is a false positive.
You can download without worries.
If you need to, disable your antivirus and after extracting the zip file from DiskSpan GUI, enable your antivirus again and check the extracted folder from DiskSpan GUI. Let me know exactly which file was captured by your antivirus so that if there is an alternative version maybe I can replace the captured file.

NOTES: DiskSpan GUI already includes all compressors reported by KaktoR, such as LOLZ.

EDIT: DiskSpan GUI is only False positive for viruses.
No virus detection using Kasperky Internet Security (See attached video below).

[Neuromancer815]

Hey Cesar82 thanks for the reply, i did what you said; Affected files:

containerfile: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z

file: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z->COMPRESSOR/Resources/TOOLS/WemTool/WemTool.exe

webfile: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z|https://fileforums.com/attachment.ph...30245021635373

EDIT: after using the program for a bit i just wanna say great work i love the UI and the program is vey easy to use

[Cesar82]

Originally Posted by Neuromancer815

Hey Cesar82 thanks for the reply, i did what you said; Affected files:

containerfile: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z

file: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z->COMPRESSOR/Resources/TOOLS/WemTool/WemTool.exe

webfile: C:\Users\Neuromancer\Downloads\DiskSpan_GUI v2.0.1.3 (x64).7z|https://fileforums.com/attachment.ph...30245021635373

EDIT: after using the program for a bit i just wanna say great work i love the UI and the program is vey easy to use

Thanks for the test.
If possible download the WemTool file attached in the post HERE and let me know if your anti-virus also captures this updated version without UPX compression.

[Masquerade]

Sorry about the WemTool. It's probably going to trigger AV anyway because it's lazily built with bat2exe.

[Neuromancer815]

Originally Posted by Cesar82

Thanks for the test.
If possible download the WemTool file attached in the post HERE and let me know if your anti-virus also captures this updated version without UPX compression.

yes this one also gets flagged

[shazzla]

Try compress using ASPack or MPress. Maybe it works.

[Masquerade]

Originally Posted by shazzla

ASPack

ASPack will be even worse for detections.

[shazzla]

Originally Posted by Masquerade

ASPack will be even worse for detections.

You mean ASProtect ?!

[Cesar82]

Originally Posted by Masquerade

Sorry about the WemTool. It's probably going to trigger AV anyway because it's lazily built with bat2exe.

Thanks for info!
Is there no other way or program you can use to convert your BAT that doesn't create false positives?

[Masquerade]

Probably could re-write the entire thing in C# but I'm currently so busy trying to balance a lot of IRL stuff I will not get time. It's something I will consider but it won;t be soon.

[Crimzan]

Hello! Thank you for these explanations!

I'm also really new to this and I have read a lot but it is a lot of arbitrary seeming detailed information in these forums that make it incredibly hard to categorize and make sense of, so I have a lot of questions and I hope it's ok to just dump them all at once

I downloaded DiskSpan GUI, it seems like it's one of the more customizeable, yet very easy to use ones. Do you think this is a good tool to use or would I be better off using these exe's separately, or even a completely different software?

Originally Posted by KaktoR

In your example for "Sniper Elite 4" I guess you can just use srep+lolz if you want it strong, with cost of speed and pc resources, or srep+lzma with cost of compression ratio, but requires way less pc resources.

You said "srep+lolz" and "srep+lzma", does this mean, in DiskSpan GUI (if you are familiar with the software), I can select both srep first and then lolz / lzma and then it is basically what you had described?

To understand the logic behind this, when I select a folder to compress with DiskSpanGUI, "srep+lolz" would basically mean using srep first to deduplicate, and then compress the srep output with lolz?

I don't have Sniper Elite 4, but this is interesting -- how do you know srep+lolz is strong? Is it one of the strongest general combinations? Or how do you know, or in other words, how can I find out what compressors and thingies work the best?

Originally Posted by KaktoR

Some games requires that you first unpack the files and then compress them. In most cases you use xtool to unpack the game files, but you should know what compression method the game is compressed with. For example "Sniper Elite 5" is compressed with zlib (if I remember well), so you have to use xtool first to unpack the files, then apply srep to get rid of duplicates, then use lolz or lzma to use final compression.

Super interesting!
So, to make sure I understand what you said at the end, in DiskSpanGUI I would basically have xTool_zlib+srep+lolz (or lzma), is that correct?

Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?

One final question, which is hopefully not too broad: What is your workflow finding 'the best' compression combinations (that is, not as slow as paq compression for instance but still really good compression ratio)? Is it Trial and Error? Like, do you just select a couple of files to compress, and try each and every combination? Because that's what I'm doing currently and this kinda sounds super inefficient so I suspect there's a way to find out what works with scanners or anything

Anyways, I'm eager to learning more about compression but I don't think I'm getting far without asking some dedicated beginner questions. I feel like I know some stuff but they are absolutely random bits of information in different fields and it's incredibly hard and really confusing trying to make sense of them

Thanks!
- Crimzan

[KaktoR]

Originally Posted by Crimzan

I downloaded DiskSpan GUI, it seems like it's one of the more customizeable, yet very easy to use ones. Do you think this is a good tool to use or would I be better off using these exe's separately, or even a completely different software?

I would say this tool is handy, but have it's limitations aswell. Most repackers use their own tool set anyway, but in the end all of them are using the same compressors, with some additions and techniques.

Originally Posted by Crimzan

You said "srep+lolz" and "srep+lzma", does this mean, in DiskSpan GUI (if you are familiar with the software), I can select both srep first and then lolz / lzma and then it is basically what you had described?

To understand the logic behind this, when I select a folder to compress with DiskSpanGUI, "srep+lolz" would basically mean using srep first to deduplicate, and then compress the srep output with lolz?

Yes.

Originally Posted by Crimzan

I don't have Sniper Elite 4, but this is interesting -- how do you know srep+lolz is strong? Is it one of the strongest general combinations? Or how do you know, or in other words, how can I find out what compressors and thingies work the best?

lolz is mainly good for texture compression. You know, games use textures. Save your time and don't try to use lolz for audio/video files and such.

For instance, took some DDS image files, and compare ratios between srep+lolz and srep+lzma.

Spoiler: lolz will be superior on these files in terms of ratio. It's just a matter on how long you want to run the process and how small the output you want.

Originally Posted by Crimzan

Super interesting!
So, to make sure I understand what you said at the end, in DiskSpanGUI I would basically have xTool_zlib+srep+lolz (or lzma), is that correct?

Yes.

Originally Posted by Crimzan

Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?

It's like common sense sometimes. If you are in the process for years, you will get a "sense" of it. For example take any Unreal Engine 4 game. Most of these games are compressed with zlib/reflate, many others with oodle, and a few even with lz4 or zstd.

To check a file, you can use xtool's verbose mode, or manually check the file with HxD or similar for magic numbers and such (this requires that you know about these things).

I'm not a magican or something, I still have questions sometimes too

Originally Posted by Crimzan

One final question, which is hopefully not too broad: What is your workflow finding 'the best' compression combinations (that is, not as slow as paq compression for instance but still really good compression ratio)? Is it Trial and Error? Like, do you just select a couple of files to compress, and try each and every combination? Because that's what I'm doing currently and this kinda sounds super inefficient so I suspect there's a way to find out what works with scanners or anything

On larger archives (like in UE4 games the *.pak files) use xtool's verbose mode to find out. After you find out, pick the file and compress it with different combinations until you are happy with the outcome (ratio/time). There is no "magic tool" to give you all answers about a file in general.

Personally I don't use lolz because it is time consuming af and it tooks literally all PC resources while running, and I want it to be as fast as possible with a decent outcome in ratio, therefore for me my choice is in most cases 4x4:lzma.