How to create XTool plugins (configuration based)

[KaktoR]

Introduction
This post is about the frostbite 3 engine, how to deal with lz4, zlib, zstd and oodle kraken streams.
It is recommended to read the previous posts, about how to search for offsets and such.

In frostbite3 engine the structure is like this commonly for lz4:

DSize(#),Signature(#),CSize(#),Stream

Lets start
Beginning with lz4, first make a batch file next to xtool.exe with the following command

xtool.exe precomp -mlz4:s64k -t100p -v -s - - < %1 > %1.out

and drag&drop a sample onto this batch file. You will see something like this with potentially detected streams.

XTool is created by Razor12911

[0] Performing scan from block 0000000000000000 to 0000000000703A01 (7354882)
[0] Actual lz4 stream found at 00000000000BF1EA (21353 >> 65536)
[0] Actual lz4 stream found at 000000000035142B (20177 >> 65536)
[0] Actual lz4 stream found at 00000000004193CE (23693 >> 65536)
[0] Actual lz4 stream found at 000000000046D953 (12877 >> 65536)
[0] Actual lz4 stream found at 000000000052B0B2 (12330 >> 65536)
[0] Actual lz4 stream found at 000000000060A5D6 (26312 >> 65536)
[0] Actual lz4 stream found at 0000000000681013 (23435 >> 65536)

[0] Processing streams on block 0000000000000000 to 0000000000703A01 (7354882)

Streams: 7 / 7
Time: 00:00:01 (CPU 00:00:01)

Size: 7.01 MB >> 7.32 MB

Why "-mlz4:s64k"? Because sizes like 65536 are somewhat common (64 * 1024 = 65536). This way you can speed up the stream detection process, because xtool is just searching for stream sizes which are 65536.

However, go to the first offset (in this case BF1EA). Unlike in the previous examples, this one has no specific header, but we don't need one here.

https://i.imgur.com/BFEKAsp.png


Now we have to go just 2 bytes back before the offset starts and we will see the CSize (you might want to change the endianness).

https://i.imgur.com/R2nZnYo.png


DSize and Signature:

https://i.imgur.com/9RrSGGd.png


We use BigEndian and Streamoffset is set to 0. Now the plugin is nearly complete.

[Stream1]
Name=lz4
Codec=lz4
BigEndian=1
Signature=0x7009
Structure=DSize(3),Signature(2),CSize(2),Stream
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize

Testing:

xtool:testplugin
Compressed 1 file, 7,354,882 => 14,408,711 bytes. Ratio 195.91%
Compression time: cpu 0.02 sec/real 1.61 sec = 1%. Speed 4.56 mB/s
All OK