MagesPack DiskSpan integration?

[ozerune]

I am very new to this whole repacking scene and am trying to understand as much as I can about all of this. The problem I am running into right now is with the game Chaos;Child, and will likely run into this issue again when I do Steins;Gate. About half the game is stored in .mpk files, and the only way that I can find to unpack and repack them is with MagesPack, which from what I can tell doesn't work with FreeArc. Is there anyone who could explain to me what I would need to do to get these files properly repacked? From what I can tell, existing FreeArc compatible compressors and preprocessors cannot see too well inside of the mpk files, with a ratio of 90% when I try. As of right now, I primarily use DiskSpan GUI. I would prefer not to write my own code at this point in time, but if I absolutely have to and there is no other way, knowing the steps to take next would still be helpful.

[KaktoR]

As long as the tools have a command line, you can integrade them into diskspan gui relatively easy using callbefore/runbefore/callafter/runafter. It looks like MagesPack tool has gui only, so no integration into diskspan gui is possible.

However there is a bms script (http://aluigi.altervista.org/bms/chaos_child.bms) which you could use with quickbms, and quickbms you can integrate with diskspan gui

Edit: literally 90% of the game files contain just PNG images. Use xtool:d1:mpng:mreflate

chara.mpk

Streams: 1706 / 1706
Time: 00:02:36 (CPU 00:14:37)

Size: 833 MB >> 3.02 GB

se.mpk and voice.mpk contain OGG files. Use oggre on them.

[ozerune]

Awesome thank you! I'll have to play around with this and see if I can get a better ratio if I use the bms first or not. I have not heard of bms before, is it something that repackers use often that I should look more into?

[KaktoR]

Yes, repackers use quicmbms sometimes

http://aluigi.altervista.org/quickbms.htm

However, for this particular game, you really don't need it.

[Masquerade]

Just to add a comment - the mpk files in Chaos Child and Steins Gate are all non-compressed so you don't need to unpack them.

[ozerune]

Originally Posted by KaktoR

Yes, repackers use quicmbms sometimes

http://aluigi.altervista.org/quickbms.htm

However, for this particular game, you really don't need it.

Trying to figure out how to use QuickBMS now, how do I recreate a file extracted using it? It seems to want the exact file which was extracted from in order to reimport in the first place, but that would be incredibly counterproductive considering I would have to put both in the archive.

[Masquerade]

^^
QuickBMS cannot repack a file traditionally, it's not that kind of tool. Reimport mode can work but it gets messy very quickly as you need to import empty files into the original archive and then reimport your files during installation. Many archive repackaging tools for games are game engine specfic or even game specific.

As for Steins;Gate, since the archives aren't compressed, you don't need to extract them. If you unpack the raw data, you can simply concatenate that data using bincat and then use xdelta or hdiffz to restore the missing archive information.

[ozerune]

I've spent the last couple days going through these and have managed to get the ratio down to 80%, but the install takes 15 minutes for an 8GB game, which I feel is a bit much. The majority of this time comes from the png mpk files which take (in my opinion) absurd amounts of time to both compress and decompress. For these files I am using

xtool:d1:mpng:mreflate:dd3+LOLZ_NORMAL

Even with just using lolz normal, it takes a solid 1.5 hours to compress 3GB of PNGs. Is there some other compressor I should be using for PNGs? And is there any way to add more compression after using bink 2 and oggre without costing too much install time?

I am a massive visual novel player and those are the main things that I want to back up so I imagine all of these will be used often.

[Masquerade]

^^
Welcome to repacking
Say goodbye to all of your hours

Use a separate archive for both bink videos and oggre. There's no way to speed oggre up since it runs at single digit mb/s. You can increase bink speed by using XTool bink video plugin.

[ozerune]

I've already been doing that lol. Guess I'll have to just either choose 15 minute install time for 80% ratio or 1 minute install time for 90% ratio. Might be a better idea to just do the latter.

[kj911]

ozerune: Compress them 7.88GB's game version? (newer im has ~9.6GB's size.)

Compression results only "chara.mpk" file and separately/completely any other files? 90% compress = total game packed to 7.1GB's size? Betterly avg. 6.3GB?

UPD: Quick testing the whole "chara.mpk" file. Better idea trying to use precomp?

Test1: xZLib+srep:m3f:a0+xlzma --> ~604MB (~434sec.)
Test2: precomp+srep:m3f:a0+xlzma --> ~450MB (~686sec.)

Note: xZLib = xtool 0.12 use zlib and 32MB chunk size. Precomp use v0.4.7 "-intense -cn" switches. SREP used v3.92beta x64 version. Last v0.4.8 precomp better? Combined with xtool? delta filtering increase the ratio? Probably Blizzard 0.2.4 betterly compress than any LZMA algos. (kick ass LOLZ?) Harder testings results coming soon.

[Masquerade]

^^

xZLib = xtool 0.12 use zlib and 32MB chunk size

...but why?
This ancient XTool has no png preprocessor.

[kj911]

...but unpacking.

832MB --> 3.01GB

Precomp 0.4.8 -intense0 -cn --> 3.43GB! (Pack it RZ with d1023M -> 328MB.)

xzlibbed or precompressed data compress via srep+bliz/xlzma. Compressed results?? No difference! Smaller than 100kB's. Without srep compress = loose ~15% final ratio.

Tested blizzard = better than faster xlzma compression. Precomp precompressed results equal the any tiny png-set results, see it: https://encode.su/threads/3964-Best-...cartoon-images

precomp+srep:m3f:a0+bliz --> 358MB (simple "-intense -cn" mode, not brute'd.) LOLZ/DLZ not tested, wait results...