FileForums - XTool 2020 (Plugins)

Page 10 of 11

« First

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - XTool 2020 (Plugins) (https://fileforums.com/showthread.php?t=103876)

kuyhaa

16-05-2023 18:56

Quote:

Originally Posted by Masquerade (Post 501112)

In that case, you will need to make a new database.

You will need VGMToolbox, which you can download here.

Copy this text and save it as PPKG.XML inside the "plugins\AdvancedCutter" folder.

Open the tool and go to Misc Tools ---> Extraction Tools ---> Generic ---> Advanced Cutter / Offset Finder.

In the presets drop down box, select PPKG from the list and then click "Load".

Then, drag the three PKG files into the drag'n'drop box to the left of the preset box.

After all the streams have been extracted, rename the extracted folders to remove the _CUT suffix.

To generate the database, just open a command prompt window in the folder containing the pkg files and do:

Code:

for %f in (*.PKG) do xtool generate -c128mb -t100p -mlz4f "%~nf" "%f" TTIoM3.xtl

Now you can delete the folders from VGMToolbox.

works, thanks :D

Code:

Streams: 19102 / 19113

Time: 00:02:28 (CPU 00:00:55)



Size: 8.23 GB >> 13.4 GB

soulfit15

01-06-2023 08:55

looks like a bug in xtool GUI with unreal plugin

Quote:

XTool is created by Razor12911

Streams: 1430889 / 1675133
Time: 00:21:48 (CPU 00:16:56)

Size: 5.25 GB >> 5.03 GB

unpacking a game with ue5:key and it gets stuck "used the -d1" and cpu keeps spooling at 16%

Masquerade

06-06-2023 06:29

Orcs Must Die! 3 database updated for game version Build 11248349 / v1.2.1.0 (June 5th 2023).

kj911

17-06-2023 08:33

Xtool 0.71 dont compress Bink video

1 Attachment(s)

Quote:

Originally Posted by Masquerade (Post 500233)

Bink Video

Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Whats wrong or am i lame?? (From testing. See two attached pictures.)

Directly use Binkpack.exe or Freearc usaged whole Bink*.exe file and or CLS-based (de)compressors its works in (de)compress in vice versa... Packed 3.07GB huge BIK file correctly to 2.4GB size with Binkpack.
Use Xtool 0.71 directly or Arc with configurations, don't compress smaller.

KaktoR

23-06-2023 06:43

Letting you know here that the playstation archive plugin doesn't work with xtool 0.7.2(HF) on decompression step.

It works with xtool 0.7.1

Masquerade

23-06-2023 14:26

1 Attachment(s)

[Proof of Concept / Work In Progress] Yakuza SLLZ v1

This is a little project I've been working on in an attempt to implement the SLLZ algorithm found in Yakuza games as an XTool plugin. The code used for compression and decompression was written by Kaplas80. The attempt was to make it into a standalone executable. I wanted to share it here despite being unfinished. Requires .NET Framework v4.8.

There are a few problems with the plugin in its current state. Decompression of sllz streams seems to work completely fine. However, compression is slow and is missing the 2x 4 bytes of header information containing the compressed size and decompressed size (because for some reason Visual Studio insists on this being 8 byte rather than 4 byte arrays). They are filled as 00 bytes instead for the time being. The compressor is also bugged in other ways when I tested on samples from Yakuza Kiwami. Reading and writing to stdin/stdout hasn't been added yet so it's even slower since data needs to be extracted to disk.

Here's some information about this algorithm from its header:

Code:

First 4 bytes, Magic: SLLZ

Next byte, endianness: 00 (little)

Next byte, version: 01 or 02 (compression version - v2 is zlib I think)

Next 2 bytes is header size (from what I've seen, always 0x1000 because the header is always 16 bytes in my findings)

Next 4 bytes is decompressed size.

Next 4 bytes is compressed size.



Total: 16 bytes

Code:

[Stream1]

Name=yakuza

Codec=sllz

BigEndian=0

Signature=0x5A4C4C53

Structure=Signature(4),Endianness(1),Version(1),HeaderSize(2),CSize(4),DSize(4),Stream

StreamOffset=-16

CompressedSize=CSize

DecompressedSize=DSize

Code:

xtool precomp -myakuza -df20p -c128mb -t12 hact.par

XTool is created by Razor12911



Streams: 873 / 3397

Time: 00:11:34 (CPU 00:00:17)



Size: 398 MB >> 404 MB

I will probably drop this project for a bit and return to it sometime in the future :D

dixen

01-07-2023 04:10

1 Attachment(s)

Witcher 3: Complete

Used xtool v0.7.3 hotfix

-mzlib+reflate

Quote:

Streams: 2786 / 2786
Time: 00:01:17 (CPU 00:04:24)
Duplicates: 300 (1.58 MB) [1.80 MB >> 19.1 MB]
Srep decompression memory: 3.00 MB [6.34 MB*]

Size: 508 MB >> 658 MB >> 638 MB >> 607 MB >> 400 MB

100%
Errorlevel=0

Compressed 1 file, 532,191,280 => 419,520,814 bytes. Ratio 78.83%
Compression time: cpu 0.53 sec/real 79.37 sec = 1%. Speed 6.71 mB/s
All OK

-mw3+zlib+reflate

Quote:

Streams: 3490 / 3854
Time: 00:01:35 (CPU 00:05:40)
Duplicates: 476 (2.20 MB) [3.18 MB >> 226 MB]
Srep decompression memory: 7.00 MB [23.1 MB*]

Size: 508 MB >> 1.12 GB >> 916 MB >> 814 MB >> 303 MB

100%
Errorlevel=0

Compressed 1 file, 532,191,280 => 318,049,218 bytes. Ratio 59.76%
Compression time: cpu 0.47 sec/real 97.06 sec = 0%. Speed 5.48 mB/s
All OK

Needed libliz4.dll (v1.9.3) and doboz.dll near xtool.exe

Razor12911

08-07-2023 21:30

Plugin update available

Changes

- Improved stream detection in unreal engine plugin's ue5 codec (ucas/utoc format)

Notes

I just wished the format of unreal engine file system would stop changing because it's a headache trying to write one library that supports all current and future games and even this plugin has a caveat, mainly the first and last stream left behind and not decrypted because if I tried to grab them using the current method that would mean false positives, I've left R9 just in case R10's detection is worse but good luck.

pakchunk0optional-Windows.ucas (I don't know what game is this but it was a sample I was sent a while ago)

Unreal Engine ucas database maker doesn't work on this file... because they changed they format...

Code:

-mue5:k0xCD353CBF6E250AF8ADF4B2B0F79649A27D2BC743AB0AAAEB202DEF7A7C19B2DB+kraken -d1

Code:

Tested 1 file, 437,937,960 => 290,461,744 bytes. Ratio 150.77%

Testing time: cpu 0.19 sec/real 11.54 sec = 2%. Speed 25.17 mB/s

R10

Code:

Tested 1 file, 578,518,003 => 290,461,744 bytes. Ratio 199.17%

Testing time: cpu 0.25 sec/real 16.30 sec = 2%. Speed 17.82 mB/s

Masquerade

09-07-2023 07:26

1 Attachment(s)

The Legend of Heroes Trails into Reverie

Extract the below archive into XTool folder and use -mReverie.
This database is for assets.pka. The other pkg files are not compressed.
Requires liblz4.dll. I included one that gave a good ratio.
You can discard Reverie.xtl for decoding, but liblz4.dll must be retained.

This database took 13 hours to generate :p

Razor12911

18-07-2023 01:42

Plugin update available

Changes

- Improved stream detection in unreal engine plugin's ue5 codec (ucas/utoc format)

Notes

Using ue5 codec even on pak files yielded streams which should not have been detected meaning ratio is affected one way or another and this release tackles false positive detections.

Razor12911

26-07-2023 09:01

Plugin update available

Changes

- ue3 codec no longer requires m# parameter to work
- ue3 codec supports lzopro streams (requires lzopro executable plugin)
- ue4 codec speed improvement
- ue4 codec improved detection
- ue5 codec minor bug fixes

Notes

This update is primarily made for the next xtool release (0.7.7) but it should work fine for older versions.

Link

Razor12911

26-07-2023 17:49

1 Attachment(s)

Insomniac Engine

Supported games
Marvel's Spider-Man Remastered
Marvel's Spider-Man: Miles Morales
Ratchet & Clank: Rift Apart

Usage
Place insomniac.ini near xtool.exe along with required libraries and use -minsomniac

Notes
Tested on small sample provided by dixen for testing, not sure if it works for the whole game. This is the first advanced configuration plugin so also not sure if this feature in xtool works without bugs.

Cesar82

26-07-2023 18:20

@Razor12911, Thanks for the updates.
As always, once again I fill the topic of questions, but here we go..

1) Can I continue to use the m1, m2, and m3 parameters so the DiskSpan GUI knows what to copy into the xtool folder?

Code:

Unreal_Engine3_MK11;         Method: xtool:mue3,m1;

Unreal_Engine3_MELE;         Method: xtool:mue3,m2;

Unreal_Engine3_LZO;          Method: xtool:mue3,m3;

2) Could you share which library and version is used for each of the 3 games supported by the new "Insomniac Engine" plugin?

Razor12911

26-07-2023 21:38

Plugin update available

Changes

- Added support for DirectStorage (gdeflate) streams in Insomniac plugin

Notes

Turns out, not only lz4 was used in this game, after investigations made by Fitgirl, she pointed out the plugin was not working in the initial release and after a few mins of research I managed to create a plugin (gdeflate) to add support for the other streams compressed using the Direct Storage technology and as a result of that, to precompress Ratchet & Clank: Rift Apart, you need dstorage.dll, dstoragecore.dll, gdeflate.dll and liblz4.dll.

tex_char_ratchet0 (sample compressed with gdeflate)
-mxtool:insomniac:rcrads

Code:

Compressed 1 file, 135,778,928 => 325,814,638 bytes. Ratio 239.96%

Compression time: cpu 0.11 sec/real 1.36 sec = 8%. Speed 99.86 mB/s

IMPORTANT
With the use of DirectStorage, compression and/or installation will only work on Windows 10 or later, Anything older will not work.

@Cesar82

1) Yes
2) It's the same one that comes with xtool (v1.8.3), a lot of games use this version.

dixen

26-07-2023 23:40

Decompression results

Insomniac R1
39 gb > 48 gb

Insomniac R2
39 gb > 63 gb

KaktoR

26-07-2023 23:56

Quote:

Originally Posted by Razor12911 (Post 501718)

Thanks for the update, although it seems that some streems were skipped for ue3+lzopro

Code:

Streams: 373 / 1776

Time: 00:00:04 (CPU 00:00:01)



Size: 67.2 MB >> 88.9 MB

DODI	27-07-2023 21:48

Quote:

Originally Posted by Razor12911 (Post 501722)

Thanks :)

Precompression result

Insomniac R2
38.3 gb to 64.5 gb

Razor12911

22-08-2023 14:39

Plugin update available

Changes

- Updated Unreal Engine plugin
- Fixed hanging issue in PlayStation archive plugin

Razor12911

01-09-2023 01:11

1 Attachment(s)

Starfield

Supported games
Starfield (requires liblz4.dll)

Usage
Place starfield.dll near xtool.exe along with required libraries and use -mstarfield+reflate (plugin only precompresses lz4 streams)

Notes

liblz4.dll that comes with xtool seems to work just fine (v183).
You will need to use reflate/preflate, the game uses optimised zlib, zlib-ng to be specific and that makes the zlib method not to work.

Razor12911

06-09-2023 01:34

1 Attachment(s)

lzopro

Supported games

To be used by old anvil and unreal engine 3 games

Notes

I forgot to post this a while ago but here anyways,

Edit xtool.ini and use -anvil for anvil games and -unreal for unreal engine 3 games while specifying compression level, where level 0 uses lzopro_lzo1x_1_14_compress and level 1..10 uses lzopro_lzo1x_99_compress function for compression.

If level 0 and mostly 6..10 does not work on precompression then you should use uelr as the streams are compressed in such a way that they rely on the work memory buffer left behind by the previously compressed stream (to improve on compression). The streams are compressed in sequence, however it's this improvement that makes it impossible for xtool to work with such streams as because xtool breaks this sequence when it precompresses the streams using multiple threads and if you tried to brute force the work memory to work or if you use xdelta to patch the streams, you will get crc errors.

Masquerade

08-09-2023 01:12

Updated Distant Worlds 2 database for game version v1.1.6.7.

KaktoR

09-09-2023 08:31

1 Attachment(s)

Quote:

Originally Posted by Masquerade (Post 500233)

Bink Video

XTool plugin adding support for Bink Video compression.
This plugin uses Bink Pack library by ProFrager.
Thanks to Razor12911 for the little help.

Advantages of using this plugin versus vanilla Bink Pack

Multithreading support.
No CRC check fail since XTool will patch the videos as they are unpacked.
Incompatible Bink 2 videos (KB2n) are automatically ignored.

Special Notes

For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.
You may need to use a large chunk size to ensure all videos are processed. See example below.

Example Usage

Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Updated plugin.

Basically the 'MessageBoxA' was patched out and replaced with some other function from User32.dll. On some bink2 inputs you had encountered some error messages like 'MotionData_warn' or 'err'. I have included two samples for which it is the case. You shouldn't receive error messages anymore with this patched plugin. Inputs which would give these types of errors are just ignored but processed still, but not compressed with binkpack.

Thanks to masquerade and Razor12911.

Masquerade

05-10-2023 00:18

1 Attachment(s)

Forza Motorsport 2023

This game uses XMemCompress in its paks (as denoted by Method: 21).

I didn't download the full game, but here's how to make your own database:

1. Download the attached archive. This contains bms2xtl config and required script as well as XCompress plugin for XTool.

2. Extract the archive, and place the following files into the Forza Motorsport root folder:

bms2xtl.dll
bms2xtl.exe
bms2xtl.ini
quickbms_4gb_files.exe
zip.bms

3. Open command prompt in the root of the Forza Motorsport game folder and paste this command:

Code:

for /r %f in (*.zip) do bms2xtl zip.bms "%f" ForzaMotorsport.xtl

4. Once the database is made, use that along with the xcompress plugin supplied to precompress the game data:

Code:

CMD> xtool precomp -mForzaMotorsport body01_forzafaithful.zip



XTool is created by Razor12911



Streams: 4 / 4

Time: 00:00:05 (CPU 00:00:00)



Size: 17.0 MB >> 32.0 MB

If you make a full database, consider sharing it here.

Razor12911

05-10-2023 09:39

1 Attachment(s)

The config version for the plugin above

I've included 64-bit version of xmemcompress, I compiled it a while ago but I didn't post it here.

Forza Motorsport\media\base\Cinematics.zip

xmemcompress -mfmzip:

Code:

Compressed 1 file, 260,317,756 => 1,146,247,029 bytes. Ratio 440.33%

Compression time: cpu 0.08 sec/real 37.12 sec = 0%. Speed 7.01 mB/s

xmemcompress64 -mfmzip:

Code:

Compressed 1 file, 260,317,756 => 1,146,247,031 bytes. Ratio 440.33%

Compression time: cpu 0.00 sec/real 28.70 sec = 0%. Speed 9.07 mB/s

It should also work for Forza Motorsport 7,

the older side project can be found here

Razor12911

08-10-2023 07:44

1 Attachment(s)

Tony Hawk's Pro Skater 1+2

This only removes the custom encryption used in the game (Not tested)

Drag and drop the game folder and xtool will remove the encryption and give you the decrypted pak files in output folder which will be your new game files, use -mzlib on them for precompression.

Base\Content\Paks\pakchunk1-WindowsNoEditor.pak

Code:

Compressed 1 file, 291,695,622 => 503,712,687 bytes. Ratio 172.68%

Compression time: cpu 0.05 sec/real 2.30 sec = 2%. Speed 126.91 mB/s

This is not tested and will all depend on your luck.

dixen

08-10-2023 08:36

Test pakchunk6-WindowsNoEditor.pak

Quote:

Compressing 1 file, 630,211,718 bytes
Compressing pakchunk6-WindowsNoEditor.pak 58%
Compressing 630,211,718 bytes with xtool1.exe precomp -mthps12+zlib -d3 -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 46582 / 47129
Time: 00:00:14 (CPU 00:01:53)

Size: 601 MB >> 1.40 GB

100%
Errorlevel=0

Compressed 1 file, 630,211,718 => 1,503,292,444 bytes. Ratio 238.54%
Compression time: cpu 0.28 sec/real 20.72 sec = 1%. Speed 30.41 mB/s
All OK

Quote:

E:\RePack_tools\xtool_0.7.7\Freearc>arc t data.arc
FreeArc 0.67 (March 15 2014) testing archive: data.arc
Testing 1 file, 630,211,718 bytes. Processed 99%WARNING: CRC failed in "pakchunk6-WindowsNoEditor.pak". File is broken.
Tested 1 file, 1,503,292,444 => 630,211,718 bytes. Ratio 238.54%
Testing time: cpu 0.20 sec/real 6.08 sec = 3%. Speed 103.62 mB/s
There were 1 warning(s)

With decrypting PAK - game doesn't launch

Razor12911

08-10-2023 09:00

Welp
https://media.giphy.com/media/qN8mwU...e3W1/giphy.gif

Masquerade

08-10-2023 13:32

Quote:

Originally Posted by dixen (Post 502766)

Test pakchunk6-WindowsNoEditor.pak

ofc CRC error because this library decrypts paks only, not re-encrypts them. Re-encryption of this game is not possible.

hwang4

15-10-2023 23:27

Asterigos - Anniversary Update

Quote:

Originally Posted by Masquerade (Post 498675)

Asterigos: Curse of the Stars

Place asterigos.xtl and asterigos.key in the same folder as XTool, use "-masterigos+preflate -d1".

Masquerade , tried with this one , but cannot get a good compression ratio !thx for help!:)

Hexagon123

16-10-2023 09:03

Quote:

Originally Posted by hwang4 (Post 502849)

Masquerade , tried with this one , but cannot get a good compression ratio !thx for help!:)

Looks like this needs an update?

Masquerade

16-10-2023 10:04

DB generator no longer works.

Masquerade

03-03-2024 04:13

XNB

Code:

[Stream1]

Name=xnb

Codec=xmemcompress

BigEndian=0

Signature=0x77424E58

Structure=Signature(4),XNBFormatVersion(1),Compressed(1),FileSize(4),DSize(4),Stream

CompressedSize=FileSize-14

DecompressedSize=DSize

Condition1=Compressed = 0x80

Requires XCompress plugin + correct configuration in xtool.ini.

Redmist2013

03-04-2024 21:59

Quote:

Originally Posted by Razor12911 (Post 502762)

Code:

Compressed 1 file, 291,695,622 => 503,712,687 bytes. Ratio 172.68%

Compression time: cpu 0.05 sec/real 2.30 sec = 2%. Speed 126.91 mB/s

This is not tested and will all depend on your luck.

Interesting. It seems to decrypt okay but re-encryption only seems to partially work.

Code:

xtool decode pakchunk1-WindowsNoEditor.pak.xt pakchunk1-WindowsNoEditor.pak

...results in about 20% of the bytes in the file matching those in the original file using a binary file comparer.
Fitgirl seems to have made a custom THPS12 encryption tool which gets close and then uses hdiff to fix things up.

Masquerade

04-04-2024 13:18

^^
Did you even read Razor's post?

"This only removes the custom encryption used in the game (Not tested)"

Ofc decode isn't sopprted!

kuyhaa

28-04-2024 19:14

hi, anyone have plugin or other method for Megaton Musashi W ? i try use unity plugin for *.bundle file but not work

thanks

KaktoR

28-04-2024 22:29

Sample

kuyhaa

28-04-2024 22:32

Quote:

Originally Posted by KaktoR (Post 504340)

Sample

this is bro

Code:

https://pixeldrain.com/u/Ng8j3XAs

KaktoR

29-04-2024 05:04

Either not compressed, crypted or something else. There is no unityfs header, so maybe this is the problem.

kuyhaa

07-07-2024 00:05

hi, anyone have plugin or database creator The Legend of Heroes: Trails through Daybreak

.p3a files

sample

Code:

https://pixeldrain.com/u/sFp3YTHU

KaktoR

07-07-2024 00:49

I will quote masquerade, maybe he will tell you how to process this.

Quote:

A game came out called The Legend of Heroes Trails through Daybreak. They used ZSTD compression on the archives yet because they used dictionary based zstd, xtool can't detect the streams let alone process them.

All times are GMT -7. The time now is 13:42.

Page 10 of 11

« First

Show 40 post(s) from this thread on one page