View Single Post
  #4  
Old 13-09-2023, 07:16
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,496
Thanks: 1,085
Thanked 7,121 Times in 2,704 Posts
KaktoR is on a distinguished road
Introduction
This post is about the frostbite 3 engine, how to deal with lz4, zlib, zstd and oodle kraken streams.
It is recommended to read the previous posts, about how to search for offsets and such.

In frostbite3 engine the structure is like this commonly for lz4:
Code:
DSize(#),Signature(#),CSize(#),Stream
Lets start
Beginning with lz4, first make a batch file next to xtool.exe with the following command
Code:
xtool.exe precomp -mlz4:s64k -t100p -v -s - - < %1 > %1.out
and drag&drop a sample onto this batch file. You will see something like this with potentially detected streams.
Code:
XTool is created by Razor12911

[0] Performing scan from block 0000000000000000 to 0000000000703A01 (7354882)
[0] Actual lz4 stream found at 00000000000BF1EA (21353 >> 65536)
[0] Actual lz4 stream found at 000000000035142B (20177 >> 65536)
[0] Actual lz4 stream found at 00000000004193CE (23693 >> 65536)
[0] Actual lz4 stream found at 000000000046D953 (12877 >> 65536)
[0] Actual lz4 stream found at 000000000052B0B2 (12330 >> 65536)
[0] Actual lz4 stream found at 000000000060A5D6 (26312 >> 65536)
[0] Actual lz4 stream found at 0000000000681013 (23435 >> 65536)

[0] Processing streams on block 0000000000000000 to 0000000000703A01 (7354882)

Streams: 7 / 7
Time: 00:00:01 (CPU 00:00:01)

Size: 7.01 MB >> 7.32 MB
Why "-mlz4:s64k"? Because sizes like 65536 are somewhat common (64 * 1024 = 65536). This way you can speed up the stream detection process, because xtool is just searching for stream sizes which are 65536.

However, go to the first offset (in this case BF1EA). Unlike in the previous examples, this one has no specific header, but we don't need one here.

https://i.imgur.com/BFEKAsp.png


Now we have to go just 2 bytes back before the offset starts and we will see the CSize (you might want to change the endianness).

https://i.imgur.com/R2nZnYo.png


DSize and Signature:

https://i.imgur.com/9RrSGGd.png


We use BigEndian and Streamoffset is set to 0. Now the plugin is nearly complete.

Code:
[Stream1]
Name=lz4
Codec=lz4
BigEndian=1
Signature=0x7009
Structure=DSize(3),Signature(2),CSize(2),Stream
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize

Testing:
Code:
xtool:testplugin
Compressed 1 file, 7,354,882 => 14,408,711 bytes. Ratio 195.91%
Compression time: cpu 0.02 sec/real 1.61 sec = 1%. Speed 4.56 mB/s
All OK
__________________
Haters gonna hate
Reply With Quote
The Following 5 Users Say Thank You to KaktoR For This Useful Post:
Junior53 (14-09-2023), L0v3craft (14-09-2023), Lord.Freddy (13-09-2023), mausschieber (13-09-2023), Razor12911 (14-09-2023)