How to create XTool plugins (configuration based)
 

Index
Oodle kraken
Oodle kraken with size byte difference
frostbite3 engine (lz4)

Introduction
Sometimes it is better to have a plugin for XTool to catch all the streams, or even speed up the precompression. Here I will show you how to write it and where to start. In this example I will choose a easy one, namely "LEGO Star Wars The Skywalker Saga", which is compressed with oodle kraken. XTool/oo2reck already performs good, but with a plugin it is even better.

Side Notes
Theoretically it is possible to make a plugin for majority of games, but only the minority of them are interestingly enough to make a plugin anyways. Other compression algorithms are a bit harder to understand, and for others it's totally useless to make one (example zlib and zstd). Maybe someone else can make a quick "how to" for algos like lz4/lz4hc.

What do you need?
HxD - https://mh-nexus.de/en/downloads.php?product=HxD20
Notepad (or notepad++)
XTool UI verbose mode

Lets start
First lets take a look on a easy plugin sample:

Open XTool UI, select input file, select method, check "verbose" checkbox and start.
Simultaneously open the same file in HxD.

https://i.imgur.com/Z5xmjgt.png
https://i.imgur.com/Z5xmjgt.png

Copy the first offset 135674 and go to it in HxD (CTRL+G). You will see kraken header "8C 06".

So now we have our kraken header "8C 06" at offset 135674.

Again looking at XTool verbose information on the first stream:
We have two more important informations here which will help us alot: CompressedSize (CSize) and DecompressedSize (DSize)

https://i.imgur.com/juCSHtr.png
https://i.imgur.com/juCSHtr.png

Now we have to know how such streams are constructed. For the oodle compression family it is mostly a shemata like this (at least for kraken and mermaid):
Where CSize and DSize are 4 bytes long, and the header 2 bytes, which are in total 10 bytes.

https://i.imgur.com/AzO89a7.png
https://i.imgur.com/AzO89a7.png

Now we can check if our thoughts are correct. Again we look at our first stream in XTool verbose information:
We see CSize and DSize, and we know the structure is like this:
https://i.imgur.com/xtfzlRh.png
https://i.imgur.com/xtfzlRh.png

The same with CSize

https://i.imgur.com/Y5usWSO.png
https://i.imgur.com/Y5usWSO.png

So basically we know now everything what XTool is looking for. Now comes the part what makes a plugin usefull.

But first we note what informations we got until now:
We know the oodle kraken header, which is 2 bytes
We know the DSize for the stream which is 4 bytes
We know the CSize for the stream which is 4 bytes
Which for the Structure key it is like this:
And we know the compression method is kraken (8C 06).

So lets write them right away in our configuration plugin as follow:
Now comes the tricky part. We need a signature, so the plugin is usefull at all. In cases like this I pick the first 20 streams starting from the oodle kraken header and paste them in notepad like this:
Personally I copy between 14 and 20 bytes long for each stream, but that's up to you. Just keep in mind that the first 10 bytes includes CSize, DSize and Header.

In the above table of streams we see a repetitive pattern, our signature. The Signature in this case is "4F 4F 44 4C", so 4 bytes long.

https://i.imgur.com/jRBnaMA.png
https://i.imgur.com/jRBnaMA.png

Now that we have our repetitive pattern (Signature), we fill in the information again:
In this case you could even use just 2 or 3 bytes as signature, but personally I would recommend to use as much as possible, for reasons.

Also we have to fill the Signature= key with our pattern. Because of endianness and the inclusion of "0x", we have to write the signature "backwards".
"4F 4F 44 4C" will become "4C 44 4F 4F" and adding "0x" at the start will finally become this:
So actually we read it from right to left.

Again we fill in the informations:
Now we need just 2 keys to finalize our plugin: BigEndian and StreamOffset.

The BigEndian key is important because it will tell XTool if a file is Little- or BigEndian. To find out the endianness just mark the CSize or DSize and change the endianness with the radio buttons. If the numbers you see there don't make any sense, then you know that the endianness is wrong.

https://i.imgur.com/py1hfSR.png
https://i.imgur.com/py1hfSR.png

We know in this case that it is little endianness, so it should be BigEndian=0 in our plugin.

The other key is StreamOffset. Since our oodle kraken header "8C 06" is part of the entire stream, the offset should be set to "-2" because of the 2 bytes difference. Theoretically you can skip this part and just set is as "0", but XTool have to ensure that this is really a kraken stream, otherwise XTool would think anything beginning from the signature is a kraken stream, even if the signature is something else and there is nothing following. So basically this is just a safety measurement, if you say so.

Now our configuration is complete and it should look like this:
Save it as INI file and name it like whatever you want. Here I name it "legotsws.ini" and place it next to xtool.exe

So finally lets test our plugin.

We see the plugin performs better in case of speed and ratio.

-----------------------------------------------

Here is another example for oodle kraken for the game "Shadow of War". In this case making a plugin is useless, because there are many different signatures.

Here are the first 20 streams as 20 bytes long:
Did you see it? Many signatures are 2 bytes "58 58" or just 1 byte "58". And the other half are just random bytes, which don't follow any pattern.

Testing:
However, maybe you have a game which has for example just 2 different signatures which you can use. In this case you make [Stream2] inside the same configuration file, like this:

-----------------------------------------------

Did you remember, in the beginning I said that plugins for zlib or zstd are useless? However some time ago I made one for zstd, just for fun and for learning stuff.

The Signature was 4 bytes, followed by 4 bytes DSize (DecompressedSize), 8 bytes of unknown trash and 4 bytes zstd header. Maybe some of you would think now, where is CSize? CompressedSize is set to 0, just because the zstd library has a function for it to read the value and tells XTool about this value. But the interesting part here is just the "Unknown(8)" part I wanted to show you, because sometimes you can have something like the above example, were you have a Signature, followed by some unknown bytes, then CSize/DSize and the header.

Oodle kraken with size byte difference

Introduction
Sometimes it could be the case that either CSize or DSize has a byte difference. In this case I will show you how to deal with such things.

Lets start
First lets take a look on the data we have.

https://i.imgur.com/pJAVOGK.png
https://i.imgur.com/pJAVOGK.png

We have a repetitive pattern (Signature) 7 bytes long, some unknown trash 10 bytes long, DSize 4 bytes, CSize 4 bytes and our kraken header 2 bytes.

Now lets look for the sizes. I can say here that DSize matches, but what happened to CSize?

https://i.imgur.com/THt7MkJ.png
https://i.imgur.com/THt7MkJ.png

Next stream, same thing.

https://i.imgur.com/0ZYM8ey.png
https://i.imgur.com/0ZYM8ey.png

And again, same thing on the next stream.

https://i.imgur.com/Y1nEpCR.png
https://i.imgur.com/Y1nEpCR.png

So the CSize does not match here. In fact on every stream you have a 12 byte difference. I cannot tell you how that comes, or what happened here technically, but we can fix it within the plugin configuration. All we have to do now is to subtract the 12 bytes from the CSize.

And here is the finished plugin:
Testing:
Fairly the ratio did not increase here (just 0.02%), but this example was just a showcase on what to do if you encounter byte differences.

Introduction
This post is about the frostbite 3 engine, how to deal with lz4, zlib, zstd and oodle kraken streams.
It is recommended to read the previous posts, about how to search for offsets and such.

In frostbite3 engine the structure is like this commonly for lz4:
Lets start
Beginning with lz4, first make a batch file next to xtool.exe with the following command
and drag&drop a sample onto this batch file. You will see something like this with potentially detected streams.
Why "-mlz4:s64k"? Because sizes like 65536 are somewhat common (64 * 1024 = 65536). This way you can speed up the stream detection process, because xtool is just searching for stream sizes which are 65536.

However, go to the first offset (in this case BF1EA). Unlike in the previous examples, this one has no specific header, but we don't need one here.

https://i.imgur.com/BFEKAsp.png
https://i.imgur.com/BFEKAsp.png

Now we have to go just 2 bytes back before the offset starts and we will see the CSize (you might want to change the endianness).

https://i.imgur.com/R2nZnYo.png
https://i.imgur.com/R2nZnYo.png

DSize and Signature:

https://i.imgur.com/9RrSGGd.png
https://i.imgur.com/9RrSGGd.png

We use BigEndian and Streamoffset is set to 0. Now the plugin is nearly complete.


Testing:

i have tried your method of creating ini based xtool plugins the idea in its core is nice to be able with simple adjustments to gain 2%~ 3% ratio and about 10%~15% in speed. speed only comes in compression mode where compress once decompress many applies!!


generally i have tried 2 games to create with your guide above one is Sekiro which uses kraken and the second one is zstd only for resident evil 2 and resident evil 3 series!! and failed

an average user will fail this easily i will upload samples and an explanation. maybe this idea is very nice if we get it as per engine because as it is, it cant be used!!

Of course this is not the same case for every game or engine. Initially I wanted to point it out on second post, but unfortunatelly I cannot edit it because the forum will kick me back to front page of fileforums.com. Most like it is some bug in the software this forum is using.

As for the game Sekiro, I guess it is not possible here to create ini based plugins, because most likely some of the information is stored in the bhd files. I once looked on the game Armored Core 6, which is using the same engine. All I found was header and csize, nothing more. It looked like this
But you can still try to find something usefull.

i didnt have time last night to post an explanation of my findings so here we go

sample = a chunk of resident evil 2 with 2 zstd streams.

re2.ini
[Stream1]
Name=zstd
Codec=zstd:l11:f0:b0
BigEndian=1
Signature=0xA7DA
Structure=Dict(1),DSize(3),Signature(2),CSize(2),Z StdHdr(4),Stream
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize
Condition1=ZStdHdr = 0x28B52FFD


attachment1= ZStdHdr
attachment2= xtool verbose
attachment3= HEX csize