Best Compression Methods for 'Specific' Games. Q&A

[KaktoR]

Originally Posted by Danziel123

I followed your compress method but the final result was 47 gb maybe you know what's wrong with my compression method? I use the compress method: xtool:mkraken:mmermaid:dd3+LZMA-MT

Bro, what about set a library to use?

[Danziel123]

Originally Posted by KaktoR

Bro, what about set a library to use?

sorry, I don't know which library to use. Can you tell me the correct compression method or which library I should use? Thank you

[KaktoR]

I already told what library to use. Heck I even did a test which you had quoted.

[Danziel123]

Originally Posted by KaktoR

I already told what library to use. Heck I even did a test which you had quoted.

Sorry, this one? v2.9.5?

[KaktoR]

Yes

[KaktoR]

Originally Posted by dixen

Decompression test for Lies of P

*.pak

38 gb > 73 gb
ue4d by Shegorat + xtool v0.85 + zlib (without reflate because decompression errors).
-d1 for xtool parameter is required

Use preflate then

Streams: 1631192 / 1692861
Time: 00:09:04 (CPU 00:51:31)

Size: 37.3 GB >> 72.9 GB

Compressed 15 files, 40,045,269,380 => 78,279,906,040 bytes. Ratio 195.48%
Compression time: cpu 32.39 sec/real 1182.25 sec = 3%. Speed 33.87 mB/s
All OK

Extracted 15 files, 78,279,906,040 => 40,045,269,380 bytes. Ratio 195.48%
Extraction time: cpu 57.06 sec/real 1409.74 sec = 4%. Speed 28.41 mB/s
All OK

[L0v3craft]

Originally Posted by dixen

Decompression test for Lies of P

*.pak

38 gb > 73 gb
ue4d by Shegorat + xtool v0.85 + zlib (without reflate because decompression errors).
-d1 for xtool parameter is required

For which reason did you use "ue4d"? Also if the game looks encrypted the decryption tool is not needed, I tried directly with zlib+reflate and works.

[ozerune]

Anonymous;Code help

I applied Precomp+Srep+Lolz on the whole thing and I got a ratio of 93.3%. I have tried it on individual file types as well and can't figure this out. The two main file types that are taking up space are .bin and .mzv. Any ideas on what I could use for this game?

[Masquerade]

^^
Anonymous;Code is an encrypted game and the only way you can repack it is by unpacking the archives using FreeMote and rebuilding them during installation.

https://github.com/UlyssesWu/FreeMote/releases

The key for the game is in plaintext inside the executable (visible with hex editor).

The key is "5fWhAHt4zVn2X" (game.exe, 0x5CEFCC)

[Masquerade]

ANONYMOUS;CODE bin Decrypt+Repack

The archives for ANONYMOUS;CODE are encrypted+compressed. The first 3 bytes are mdf.

The archives are a combination of .bin and .psb.m files.

sound_body.bin and voice_en_body.bin are not encrypted or compressed.

Step one is finding the key in the executable file. This is easy since the key is stored in plaintext (idiot gamedevs).

Search for ".bin" and search until you see the key:



The key for ANONYMOUS;CODE is:

5fWhAHt4zVn2X

Second step is to download Freemote:
https://github.com/UlyssesWu/FreeMote/releases/latest

You can unpack the archive using this command:

PsbDecompile info-psb -k 5fWhAHt4zVn2X <.psb.m file>

You can repack the archive from the unpacked data using this command:

PsBuild info-pdb <.psb.m.json file>

Repackaged files are CRC correct to original archives so no need to use HDiffZ or XDelta to patch any bytes (probably wouldn't work anyway due to encryption).

[Hexagon123]

using System;
using System.IO;
using System.Security.Cryptography;
using System.Text;

namespace Dantora
{
	// Token: 0x02000253 RID: 595
	public class Crypt
	{
		// Token: 0x06000E77 RID: 3703 RVA: 0x0007A8A4 File Offset: 0x00078CA4
		public static string Encrypt(string text)
		{
			RijndaelManaged rijndaelManaged = new RijndaelManaged
			{
				BlockSize = 128,
				KeySize = 128,
				Padding = PaddingMode.Zeros,
				Mode = CipherMode.CBC,
				Key = Encoding.UTF8.GetBytes("giVJrbHRlWBDIggF"),
				IV = Encoding.UTF8.GetBytes("jCddaOybW3zEh0Kl")
			};
			ICryptoTransform transform = rijndaelManaged.CreateEncryptor();
			MemoryStream memoryStream = new MemoryStream();
			CryptoStream cryptoStream = new CryptoStream(memoryStream, transform, CryptoStreamMode.Write);
			byte[] bytes = Encoding.UTF8.GetBytes(text);
			cryptoStream.Write(bytes, 0, bytes.Length);
			cryptoStream.FlushFinalBlock();
			byte[] inArray = memoryStream.ToArray();
			return Convert.ToBase64String(inArray);
		}

		// Token: 0x06000E78 RID: 3704 RVA: 0x0007A950 File Offset: 0x00078D50
		public static byte[] Encrypt(byte[] data)
		{
			RijndaelManaged rijndaelManaged = new RijndaelManaged
			{
				BlockSize = 128,
				KeySize = 128,
				Padding = PaddingMode.Zeros,
				Mode = CipherMode.CBC,
				Key = Encoding.UTF8.GetBytes("giVJrbHRlWBDIggF"),
				IV = Encoding.UTF8.GetBytes("jCddaOybW3zEh0Kl")
			};
			ICryptoTransform transform = rijndaelManaged.CreateEncryptor();
			MemoryStream memoryStream = new MemoryStream();
			CryptoStream cryptoStream = new CryptoStream(memoryStream, transform, CryptoStreamMode.Write);
			cryptoStream.Write(data, 0, data.Length);
			cryptoStream.FlushFinalBlock();
			return memoryStream.ToArray();
		}

		// Token: 0x06000E79 RID: 3705 RVA: 0x0007A9E4 File Offset: 0x00078DE4
		public static string Decrypt(string cryptText)
		{
			RijndaelManaged rijndaelManaged = new RijndaelManaged
			{
				BlockSize = 128,
				KeySize = 128,
				Padding = PaddingMode.Zeros,
				Mode = CipherMode.CBC,
				Key = Encoding.UTF8.GetBytes("giVJrbHRlWBDIggF"),
				IV = Encoding.UTF8.GetBytes("jCddaOybW3zEh0Kl")
			};
			ICryptoTransform transform = rijndaelManaged.CreateDecryptor();
			byte[] array = Convert.FromBase64String(cryptText);
			byte[] array2 = new byte[array.Length];
			MemoryStream stream = new MemoryStream(array);
			CryptoStream cryptoStream = new CryptoStream(stream, transform, CryptoStreamMode.Read);
			cryptoStream.Read(array2, 0, array2.Length);
			return Encoding.UTF8.GetString(array2);
		}

		// Token: 0x06000E7A RID: 3706 RVA: 0x0007AA90 File Offset: 0x00078E90
		public static byte[] Decrypt(byte[] data)
		{
			RijndaelManaged rijndaelManaged = new RijndaelManaged
			{
				BlockSize = 128,
				KeySize = 128,
				Padding = PaddingMode.Zeros,
				Mode = CipherMode.CBC,
				Key = Encoding.UTF8.GetBytes("giVJrbHRlWBDIggF"),
				IV = Encoding.UTF8.GetBytes("jCddaOybW3zEh0Kl")
			};
			ICryptoTransform transform = rijndaelManaged.CreateDecryptor();
			byte[] array = new byte[data.Length];
			MemoryStream stream = new MemoryStream(data);
			CryptoStream cryptoStream = new CryptoStream(stream, transform, CryptoStreamMode.Read);
			cryptoStream.Read(array, 0, array.Length);
			return array;
		}

		// Token: 0x04000BC6 RID: 3014
		private const string AesIV = "jCddaOybW3zEh0Kl";

		// Token: 0x04000BC7 RID: 3015
		private const string AesKey = "giVJrbHRlWBDIggF";
	}
}

Heroland (2019) Game is using Unity and it is encrypted with AES IV and compressed with LZ4.

using System;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using UnityEngine;

// Token: 0x02000B1A RID: 2842
public static class CryptoUtility
{
	// Token: 0x060049E1 RID: 18913 RVA: 0x00156E24 File Offset: 0x00155224
	public static byte[] Crypto(string key, byte[] src, bool encode = false)
	{
		string fileNameWithoutExtension = Path.GetFileNameWithoutExtension(key);
		byte[] bytes = Encoding.UTF8.GetBytes(CryptoUtility.salt);
		Rfc2898DeriveBytes rfc2898DeriveBytes = new Rfc2898DeriveBytes(fileNameWithoutExtension + CryptoUtility.pw, bytes);
		rfc2898DeriveBytes.IterationCount = 1103;
		RijndaelManaged rijndaelManaged = new RijndaelManaged();
		rijndaelManaged.Padding = PaddingMode.Zeros;
		rijndaelManaged.Mode = CipherMode.CBC;
		rijndaelManaged.KeySize = CryptoUtility.KeySize;
		rijndaelManaged.BlockSize = CryptoUtility.BlockSize;
		rijndaelManaged.Key = rfc2898DeriveBytes.GetBytes(rijndaelManaged.KeySize / 8);
		rijndaelManaged.IV = rfc2898DeriveBytes.GetBytes(rijndaelManaged.BlockSize / 8);
		ICryptoTransform cryptoTransform;
		if (encode)
		{
			cryptoTransform = rijndaelManaged.CreateEncryptor();
		}
		else
		{
			cryptoTransform = rijndaelManaged.CreateDecryptor();
		}
		byte[] result = cryptoTransform.TransformFinalBlock(src, 0, src.Length);
		cryptoTransform.Dispose();
		return result;
	}

	// Token: 0x060049E2 RID: 18914 RVA: 0x00156EEC File Offset: 0x001552EC
	public static bool CryptoFile(string srcPath, string dstPath, bool encode = false, bool checkFS = false)
	{
		bool result;
		using (FileStream fileStream = File.OpenRead(srcPath))
		{
			if (fileStream == null)
			{
				result = false;
			}
			else
			{
				byte[] array = new byte[fileStream.Length];
				fileStream.Read(array, 0, (int)fileStream.Length);
				if (checkFS)
				{
					string text = Encoding.ASCII.GetString(array).Substring(0, 7);
					if (text != "UnityFS")
					{
						fileStream.Close();
						Debug.LogError("Not Asset Bunddle File[" + text + "]" + srcPath);
						return false;
					}
				}
				byte[] array2;
				if (encode)
				{
					Debug.Log("Encode\n" + srcPath + "\n" + dstPath);
					array2 = CryptoUtility.Crypto(srcPath, array, true);
				}
				else
				{
					Debug.Log("Decode\n" + srcPath + "\n" + dstPath);
					array2 = CryptoUtility.Crypto(srcPath, array, false);
				}
				fileStream.Close();
				using (FileStream fileStream2 = File.OpenWrite(dstPath))
				{
					if (fileStream2 == null)
					{
						result = false;
					}
					else
					{
						fileStream2.Write(array2, 0, array2.Length);
						fileStream2.Close();
						result = true;
					}
				}
			}
		}
		return result;
	}

	// Token: 0x0400401D RID: 16413
	private static string pw = "c4GXA8FT";

	// Token: 0x0400401E RID: 16414
	private static string salt = "t4ChYLcH";

	// Token: 0x0400401F RID: 16415
	private static int KeySize = 128;

	// Token: 0x04004020 RID: 16416
	private static int BlockSize = 128;
}

Super Bomberman R (2017) game is using Unity and it is encrypted with Salt.

[KaktoR]

No Man's Sky
v4.52

 11:57:59 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd5+4x4:lzma
 11:57:59 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd5+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 12:42:55 - Overall input size: 16.06 GB
 12:42:55 - Overall output size: 9.40 GB (Ratio 58.55%)
 12:42:55 - Overall conversion time: 00:44:54

NMSARC.audio.pak was unpacked and wwise files were processed, hence the big input. Saves you about 150mb.

[Danziel123]

Maybe someone knows a compress method for Brothers - A Tale of Two Sons - Remake? Sorry, I can't upload the test file, because the file is too large to upload. Thank you

[ScOOt3r]

Originally Posted by Danziel123

Maybe someone knows a compress method for Brothers - A Tale of Two Sons - Remake? Sorry, I can't upload the test file, because the file is too large to upload. Thank you

i did this game and had 3 data files

data1- i used xtool:mermaid+srep+lolz
data2- movies (xbk2+srep)
data3- all miscellaneous files srep+lolz


total repack size was 13.0 im sure you can get better on the movie files a but xbk2 sucks for compression for me

ScOOt3r

[Masquerade]

^^
Almost spot on, I would like to make some additions.

The 3GB pak file contains raw WWise audio. This can be packed with msc(tak)+razor.

Most of the game data is stored uncompressed, there is only an incredibly tiny amount of mermaid streams so using any precompressor will have little impact on the end result. These handful of mermaid streams and the start of UCAS archives are becoming more common in Unreal Engine 5 games.

The director's commentary video is KB2i so it can be packed / unpacked with cls-bpk.dll. Little point in using XTool for this task since bpk unpacks the video without data errors and there is no opportunity for multithreading.

The remaining videos are KB2n which is unsupported by bpk (the XTool configuration sutomatically filters these videos). I used the latest BCM (v2.03 at time of writing) by Ilya Muravyov after srep to squeeze a slight bit extra compression.

My size was 12.4GB for:

Primary: srep+lolz
Secondary: msc+razor
Tertiary: srep+bcm
Quaternary: bpk

And the Director's Commentary can be separated to give 10.5GB minimum size.