FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   XTool 2020 (Plugins) (https://fileforums.com/showthread.php?t=103876)

Razor12911 25-06-2021 02:15

3 Attachment(s)
@Cesar82

I guess everything got glitched, reuploaded :D

Attachments for this post as it's currently glitched and I can't make changes to it.

@admin/mod please check what the problem is :o

Grumpy 25-06-2021 06:53

Quote:

Originally Posted by Razor12911 (Post 493000)
@Cesar82

I guess everything got glitched, reuploaded :D

Attachments for this post as it's currently glitched and I can't make changes to it.

@admin/mod please check what the problem is :o

Just so I understand exactly what you want to do, do you want the attachments in 'THIS' post (Post #161) to be moved to Post #4?

Cesar82 25-06-2021 08:34

Quote:

Originally Posted by Razor12911 (Post 493000)
@Cesar82

I guess everything got glitched, reuploaded :D

Attachments for this post as it's currently glitched and I can't make changes to it.

@admin/mod please check what the problem is :o

Thanks for the updates.
1) Is there still another plugin "Ubisoft Dunia 2 Engine" to be updated, or with these post files all plugins are already working with XTool 0.3.20+?
2) These plugins are new plugins for "Far Cry 3 R1" and "Far Cry 4 R1" and
how should they be used (-mfc3 and -mfc4)?
3) This plugin "tc2" is "The Crew 2 R3" (Before "The Crew 2" was always with full name in the zip, so the doubt)?
4) Which library LZ4 (or other) should be used together with fc3 and fc4? The same liblz4.dll version used with the fc5 plugin?

L0v3craft 25-06-2021 09:54

Quote:

Originally Posted by Masquerade (Post 493003)
The encryption key given for the UCAS files is incorrect, they key is for the PAK files (this is a UE4 game). The UCAS key will be different.

The key is the same for both pak and ucas.

Hexagon123 25-06-2021 17:23

Quote:

Originally Posted by Masquerade (Post 493003)
The encryption key given for the UCAS files is incorrect, they key is for the PAK files (this is a UE4 game). The UCAS key will be different.

This format is also seen in Fortnite?

Razor12911 26-06-2021 00:23

1 Attachment(s)
Scarlet Nexus

Usage
Place scarletnexus.xtl and scarletnexus.key near xtool.exe along with required libraries and use -mscarletnexus+zlib (can replace with reflate or preflate)

Notes

You can discard xtl and key file when decoding
-d1 is required

__________________

@Grumpy

Not necessarily, there was something wrong with that post and I wanted you to check what was wrong with it because it didn't allow edits at all, but thanks to Cesar's advice about deleting all the contents and start afresh fixed the issue.

@Cesar82

1) There is still more to be updated
2) Yes, they should be used like that
3) When it comes to xtl/ini files, they can be renamed to whatever you please just know that for them to be used in xtool, the name has to match. The same cannot be said with dlls, dlls can be renamed too but the internal codec name does not change.
4) Far Cry 3 and Far Cry 4 both require lzo2.dll, doesn't matter which version. As lzo isn't like lz4 whereby data compressed with different version also differs.

Grumpy 26-06-2021 05:27

Quote:

Originally Posted by Razor12911 (Post 493013)

@Grumpy

Not necessarily, there was something wrong with that post and I wanted you to check what was wrong with it because it didn't allow edits at all, but thanks to Cesar's advice about deleting all the contents and start afresh fixed the issue.

Ahh he is a man of many talents. :)

I couldn't see an issue with why you couldn't edit the post, must have just been some type of glitch

L0v3craft 26-06-2021 09:02

Quote:

Originally Posted by Razor12911 (Post 493013)
Scarlet Nexus

Usage
Place scarletnexus.xtl and scarletnexus.key near xtool.exe along with required libraries and use -mscarletnexus+zlib (can replace with reflate or preflate)

Notes

You can discard xtl and key file when decoding
-d1 is required

Thanks! However I have generated the two files using the "Unreal Engine ucas Database maker" + XTool 0.3.21 and these commands:

xtool.exe precomp -mscarletnexus+zlib+reflate -d1 -c128mb -t100p --dbase $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

XTool has precompressed all the ucas files (10.0GB) to 14.2GB. Someone said to me that all the ucas files unpacked are 22GB or so. So XTool is not detecting all the streams?

Razor12911 26-06-2021 09:15

Plugin(s) updated

Changes

- Remade Scarlet Nexus plugin using unreal_db_R2

pakchunk29-WindowsNoEditor.ucas

R1
Code:

Compressed 1 file, 355,386,352 => 493,475,542 bytes. Ratio 138.86%
Compression time: cpu 0.45 sec/real 182.46 sec = 0%. Speed 1.95 mB/s

R2
Code:

Compressed 1 file, 355,386,352 => 784,570,925 bytes. Ratio 220.77%
Compression time: cpu 0.58 sec/real 76.31 sec = 1%. Speed 4.66 mB/s

ScarletNexus\Content\Paks\*
Code:

Compressed 250 files, 17,638,403,034 => 28,178,138,151 bytes. Ratio 159.75%
Compression time: cpu 22.95 sec/real 3084.21 sec = 1%. Speed 5.72 mB/s


L0v3craft 26-06-2021 09:49

Quote:

Originally Posted by Razor12911 (Post 493020)
Plugin(s) updated

Changes

- Remade Scarlet Nexus plugin using unreal_db_R2

Now precompressed from 10GB to 22.3GB. Thank you very much :)

DODI 26-06-2021 18:22

Quote:

Originally Posted by Razor12911 (Post 493020)
Plugin(s) updated

Changes

- Remade Scarlet Nexus plugin using unreal_db_R2

Precompressed well (29 GB) and decompress was fast (5 mnins)

Thanks

dixen 27-06-2021 11:11

XTool 0.3.21 + ScarletNexus db R2

17 607 628 673 > 29,065,967,205

biotec2k8 28-06-2021 01:08

The Fitgirl Repack uses IDW Tools and AES for the reconstruction.

Is it possible with the database files to get better game compression?

mixtro 10-07-2021 04:00

I attempted to use the unity plugin for the game "The Fall". 0/0 streams, file comes out the same size.

Will test other unity games and report back.

Edit:
Unless I am using the plugin incorrectly but I will write down what I used to attempt to precompress the game:
xtool version 0.3.21
liblz4.dll and unity.dll are right next to xtool.exe.

The Fall

Code:

xtool precomp -munity+preflate -c32mb -t100p-1 -d1 --dbase --dedup "packed\data.arc" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:06 (00:00:26)
Memory: 128 MB (128 MB)


xtool precomp -munity -c32mb -t100p-1 -d1 --dbase --dedup "packed\data.arc" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:03 (00:00:06)
Memory: 128 MB (128 MB)


xtool precomp -munity -c32mb -t100p-1 -d1 --dbase --dedup "game_files\TheFall_Data\level6" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)


xtool precomp -munity -c32mb -t100p-1 --dbase --dedup "game_files\TheFall_Data\level6" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)


xtool precomp -munity -c32mb -t100p-1 "game_files\TheFall_Data\level6" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)

xtool precomp -munity+reflate -c32mb -t100p-1 "packed\data.arc" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:00:05 (00:00:25)
Memory: 128 MB (128 MB)

Greenhell:
xtool precomp -munity
Code:

xtool precomp -munity "Green Hell\GH_Data\resources.resource" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:06:59 (00:00:14)
Memory: 128 MB (128 MB)

Input file: resources.resource (2,666,519,775 bytes)
Output file: packed.xt (2,666,519,775 bytes)

Entire game stored inside arc file (no compression) (no exe, dll etc only unity files)
Code:

xtool precomp -munity "packed\data.arc" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:03:19 (00:00:30)
Memory: 128 MB (128 MB)
input file: data.arc 5.65 GB (6,075,222,393 bytes)
output file: packed.xt 5.65 GB (6,075,228,288 bytes)

Code:

xtool precomp -munity+lz4hc "packed\data.arc" packed.xt
XTool is created by Razor12911

Streams: 0/0
Time: 00:03:27 (00:00:31)
Memory: 128 MB (128 MB)

Am I testing this correctly?

Masquerade 10-07-2021 07:49

mixtro
The games you mentioned - "The Fall", "Green Hell" do not contain files that are compressed by lz4. The unity plugin is for things like Unity Asset Bundles and Data.unity3d files which are compressed.

Files such as .resource, *.assets and .resS are 99% of the time uncompressed.

mixtro 10-07-2021 13:41

Thank you for letting me know this. My heart rests at ease. But do you know which unity games are compressed with lz4?

Masquerade 10-07-2021 14:21

Quote:

Originally Posted by mixtro (Post 493193)
Thank you for letting me know this. My heart rests at ease. But do you know which unity games are compressed with lz4?

I know XTool works with Minute Of Islands. Games with data.unity3d such as Skul The Hero Slayer are also compressed but XTool doesn't support such versions of unity yet.

Hoping Razor12911 will and add more support to the plugin :)

KaktoR 10-07-2021 14:44

Quote:

Originally Posted by Razor12911 (Post 492646)
Madness Engine

Games supported
Automobilista 2
Project Cars 1?? (someone test this plugin on this game and if it's not working, please upload a few samples)
Project Cars 2 (wait for R2)
Project Cars 3 (since bdt is buggy on this game, you can use this as an alternative) [also wait for R2]

Usage
Place madness.dll near xtool.exe along with required libraries and use -mmadness

Notes

This only decrypts streams, you have to involve zlib method and set depth to 1

zlibwapi.dll is required

Does not work with PCars 1

Samples / Some more samples

Harsh ojha 10-07-2021 20:06

Quote:

Originally Posted by DODI (Post 493026)
Precompressed well (29 GB) and decompress was fast (5 mnins)

Thanks

dodi repack is here
nice 😅

Carldric Clement 10-07-2021 20:53

Nothing could change when I'm using the latest of XTool v0.3.21. Try to repack the games "Saiko no Sutoka"
.assets, .resS, and .resource

using -munity with the plugin unity.dll and lz4.dll next to xtool.exe

Code:

Compressed 18 files, 621,696,943 => 621,697,130 bytes. Ratio 100.00%
Compression time: cpu 0.80 sec/real 14.58 sec = 5%. Speed 42.65 mB/s
All OK

:(

Masquerade 11-07-2021 01:33

Quote:

Originally Posted by Carldric Clement (Post 493198)
.assets, .resS, and .resource

https://www.fileforums.com/showpost....&postcount=175

Even look at game on SteamDB, you can see it compresses to under 200mb with Steam compression alone:
. https://steamdb.info/depot/1546861/

Carldric Clement 11-07-2021 23:01

Quote:

Originally Posted by Masquerade (Post 493200)
https://www.fileforums.com/showpost....&postcount=175

Even look at game on SteamDB, you can see it compresses to under 200mb with Steam compression alone:
. https://steamdb.info/depot/1546861/

I figure it out myself. Only works with unity asset bundle. Try repack with Broforce to using -munity with unity plugin next to xtool

Code:

Compressed 192 files, 478,851,648 => 879,219,222 bytes. Ratio 100.64%
Compression time: cpu 0.58 sec/real 32.47 sec = 2%. Speed 14.75 mB/s

somehow it works a little bit. :D

:( Sad8669 22-07-2021 10:05

@Razor12911

Any chances that you will work on WDL?

GFS won't detect any streams, but the plugin actually works.
The problem is that only two files are able to be precompressed, which is barely a 2 GB gain.

L0v3craft 23-07-2021 09:17

@Razor12911

Hi. Please can you take a look at the new game "Orcs Must Die! 3" ? It uses Unreal Engine and the pak files are compressed with LZ4. XTool doesn't work.

Sample 1
Code:

https://drive.google.com/file/d/1KlE5rLaVugr4t-t_FefUEH46Fzw2MuIu/view?usp=sharing
Sample 2
Code:

https://drive.google.com/file/d/18i9wbdCNQ0a0nUUDx9Aydzp4J3h0ofw5/view?usp=sharing

infovs 24-07-2021 03:19

Scarlet Nexus R2 plugin does not work on latest build of the game (build 6979246, 7 July 2021). Tested multiple times - it works on original release but does not work on all .pak .ucas files from latest.
(obvious when tested with same file Razor12911 used for example, pakchunk29-WindowsNoEditor.ucas, it will expand on original release as expected. Same file from latest build will not expand at all)

Razor12911 24-07-2021 07:41

Quote:

Originally Posted by infovs (Post 493397)
Scarlet Nexus R2 plugin does not work on latest build of the game (build 6979246, 7 July 2021). Tested multiple times - it works on original release but does not work on all .pak .ucas files from latest.
(obvious when tested with same file Razor12911 used for example, pakchunk29-WindowsNoEditor.ucas, it will expand on original release as expected. Same file from latest build will not expand at all)

https://fileforums.com/showpost.php?...2&postcount=14
generate a new database file and use that

infovs 24-07-2021 11:30

Quote:

Originally Posted by Razor12911 (Post 493399)
https://fileforums.com/showpost.php?...2&postcount=14
generate a new database file and use that

It works, thanks :) (I saw that Database Maker before but thought it is only for testing or something like that... :o )

Razor12911 28-07-2021 19:40

1 Attachment(s)
CryEngine

Games supported
Crysis 3 (for now)

Usage
Place cryengine.dll and the ced file (cryengine database) near xtool.exe along with required libraries and use -mcrysis3

Notes
This only decrypts streams, you have to involve zlib method and set depth to 1 (I recommend the method -mcrysis3+zlib+preflate)
zlibwapi.dll (preflate_dll.dll optional) is required
Credits to these guys here

Some tests
C3\ObjectsWeap.pak
Code:

Tested 1 file, 1,048,043,508 => 531,745,904 bytes. Ratio 197.09%
Testing time: cpu 0.42 sec/real 15.69 sec = 3%. Speed 33.90 mB/s

C3\Textures.pak
Code:

Tested 1 file, 972,835,170 => 600,518,988 bytes. Ratio 162.00%
Testing time: cpu 0.41 sec/real 14.68 sec = 3%. Speed 40.91 mB/s

Announcement

This is the last release in a long time as I have become extremely busy with stuff going on in my life and have very little time if any to continue with the development of this project, this plugin itself took about 4 days to do, not because it was difficult but I barely have time. I have tried my best to make xtool open as possible to the community for anyone of you to continue its development by creating additional plugins for it. I will only update the documentation so you can educate yourselves and only upload the necessary tools that will also assist some of you to making your very own plugin.

Sorry members of ff :(

L33THAK0R 28-07-2021 20:28

Quote:

Originally Posted by Razor12911 (Post 493470)
CryEngineGames supported
Crysis 3 (for now)

This is incredible! It'll be very interesting to see if this CryEngine method can be applied to other titles utilising different versions of CryEngine!

dixen 29-07-2021 03:07

Test Crysis3 plugin

Quote:

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressing 262 files, 11,667,584,710 bytes. Processed 100.0%
ERROR: general (de)compression error in xtool
Quote:

Streams: 105099/105099
Time: 00:00:15 (00:01:51)
Memory: 149 MB (149 MB)

100.0%EAccessViolation: Access violation at address 00000000029E3574 in module 'cryengine.dll'. Read of address 0000000000000008
UPD.
with -d1 -c256m - no errors

KaktoR 29-07-2021 06:35

Quote:

Originally Posted by Razor12911 (Post 493470)
CryEngine

Games supported
Crysis 3 (for now)

Usage
Place cryengine.dll and the ced file (cryengine database) near xtool.exe along with required libraries and use -mcrysis3

Notes
This only decrypts streams, you have to involve zlib method and set depth to 1 (I recommend the method -mcrysis3+zlib+preflate)
zlibwapi.dll (preflate_dll.dll optional) is required
Credits to these guys here

Some tests
C3\ObjectsWeap.pak
Code:

Tested 1 file, 1,048,043,508 => 531,745,904 bytes. Ratio 197.09%
Testing time: cpu 0.42 sec/real 15.69 sec = 3%. Speed 33.90 mB/s

C3\Textures.pak
Code:

Tested 1 file, 972,835,170 => 600,518,988 bytes. Ratio 162.00%
Testing time: cpu 0.41 sec/real 14.68 sec = 3%. Speed 40.91 mB/s

Announcement

This is the last release in a long time as I have become extremely busy with stuff going on in my life and have very little time if any to continue with the development of this project, this plugin itself took about 4 days to do, not because it was difficult but I barely have time. I have tried my best to make xtool open as possible to the community for anyone of you to continue its development by creating additional plugins for it. I will only update the documentation so you can educate yourselves and only upload the necessary tools that will also assist some of you to making your very own plugin.

Sorry members of ff :(

Thanks and can't wait for it to try stuff out myself ;)

And good luck with all what's ongoing onto your live atm and hope to see you back sometime next year maybe :cool:

Grumpy 30-07-2021 01:04

Quote:

Originally Posted by KaktoR (Post 493480)
And good luck with all what's ongoing onto your live atm and hope to see you back sometime next year maybe :cool:

Yes agree,

@Razor12911
all the best and take care, thanks for everything you have done and do. Go live life for awhile whilst you are young enough to enjoy it ;)

infovs 30-07-2021 04:40

Thank you Razor12911 for everything, hope that at least you'll have time to read some posts occasionally to point us in right direction in using your tools.
Regards

P.S. if you ever get some time for advice, I still can't figure out if I'm doing something wrong...can't use xtool 0.3.21 + unreal.dll R5 on StreetFighter5.. :confused:
https://www.fileforums.com/showpost....&postcount=394

HunterEmrah 30-07-2021 17:03

When I use the xtool:zstd method for "Star Wars Battlefront 2", I get an "archive corrupt" error when extracting the archive. When using the frostbite3 plugin with the xtool:frostbite3:zstd method, the compression stays at 0% and there was no progress even though I waited all day, but the processor is constantly running around 70%. When I try it on an 800 megabyte .cas file, there is no problem. When I start compressing the 20 gigabyte Patch folder, it stays at 0.1% and does not progress.

Can someone with knowledge on the subject help?

https://i5.imageban.ru/out/2021/07/3...72ff93d94d.png

Using latest xtool and libraries.

arc.ini
Code:

[External compressor:xtool]
header    = 0
packcmd  = xtool2020\xtool.exe precomp { -moption} -c32mb -t100p --dbase --dedup - - <stdin> <stdout>
unpackcmd = xtool2020\xtool.exe decode -t100p - - <stdin> <stdout>

pack.bat
Code:

arc a -s; -dsgerpn -lc- -di -hp13213 -cfg"C:\Program Files (x86)\FreeArc\bin\arc.ini" -wD:\Temp -mxtool:frostbite3:zstd+srep:m3f:l512:m512+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000 ..\data1.bin * -r
https://i6.imageban.ru/out/2021/07/3...a9c0b2137f.png

:( Sad8669 30-07-2021 22:06

Try using older version of XTool or zstd.

Try these threads here.

KaktoR 31-07-2021 04:05

According to your screenshots you are not using latest version of xtool.

Try without --dbase --dedup

HunterEmrah 31-07-2021 09:05

Quote:

Originally Posted by KaktoR (Post 493519)
According to your screenshots you are not using latest version of xtool.

Try without --dbase --dedup

now it's definitely latest version, trying without --dbase --dedup, nothing changed.

Code:

[External compressor:xtool]
header    = 0
packcmd  = xtool2020\xtool.exe precomp { -moption} -c32mb -t100p - - <stdin> <stdout>
unpackcmd = xtool2020\xtool.exe decode -t100p - - <stdin> <stdout>

https://i2.imageban.ru/out/2021/07/3...3a4b5724cc.png

KaktoR 31-07-2021 12:23

Quote:

Originally Posted by HunterEmrah (Post 493520)
now it's definitely latest version, trying without --dbase --dedup, nothing changed.

Code:

[External compressor:xtool]
header    = 0
packcmd  = xtool2020\xtool.exe precomp { -moption} -c32mb -t100p - - <stdin> <stdout>
unpackcmd = xtool2020\xtool.exe decode -t100p - - <stdin> <stdout>

https://i2.imageban.ru/out/2021/07/3...3a4b5724cc.png

I don't know, works fine here

https://i.imgur.com/rKoANLN.png

Maybe it's because of your arc settings

HunterEmrah 31-07-2021 12:40

Quote:

Originally Posted by KaktoR (Post 493523)
I don't know, works fine here

https://i.imgur.com/rKoANLN.png

Maybe it's because of your arc settings

you are using only zstd, try it with frostbite3 plugin. zstd gives error when extracting.

KaktoR 31-07-2021 12:53

https://i.imgur.com/TujGmba.png

Works

Edit: Stucks at 4%. I guess you have to edit frostbite3.ini then.


All times are GMT -7. The time now is 06:08.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com