FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   XTool 2020 (Main Project) (https://fileforums.com/showthread.php?t=102832)

Simorq 11-08-2019 14:02

reflate (XTool 1908_R2_x86)
 
#############################################
# reflate (XTool 1908_R2_x86)
#############################################

Test File : bigfile.004.tiger 1.99 GB

Code:

XTool x86 Command Line

xtool.exe precomp:reflate:c16mb,t100p - - < %1 > %1.out
xtool.exe decode:t100p - - < %1.out > %1.res
fc /b %1 %1.res

Test 1 = OK
Test 2 = OK
Test 3 = OK
Test 4 = OK

Code:

XTool x86 Command Line

xtool.exe precomp:reflate:c32mb,t100p - - < %1 > %1.out
xtool.exe decode:t100p - - < %1.out > %1.res
fc /b %1 %1.res

Test 1

P: OK
D: OK (FC: no differences encountered)

Test 2

P: EAccessViolation: Access violation at address 5378614D in module 'xtool.exe'. Read of address 5378614D
XTool is created by Razor12911

D: EReadError: Stream read error


Test 3

P: Exception EAccessViolation in module xtool.exe at 000BCEC7.
Access violation at address 004BCEC7 in module 'xtool.exe'. Read of address 00611D34.
D: EReadError: Stream read error

reflate works well with c16m in x86.:rolleyes:

Simorq 11-08-2019 14:11

zlib (XTool 1908_R2_x86)
 
#############################################
# zlib (XTool 1908_R2_x86)
#############################################

Test File : bigfile.004.tiger 1.99 GB

Code:

XTool x86 Command Line

zlib
xtool.exe precomp:zlib:c32mb and c16mb,t100p - - < %1 > %1.out
xtool.exe decode:t100p - - < %1.out > %1.res
fc /b %1 %1.res


TEST 1
P: OK
D: CRC Error (0F2C892C: 09 D7)

TEST 2
P: Error (EReadError: Stream read error)

TEST 3
P: Error (EReadError: Stream read error)

TEST 4
P: EAccessViolation: Access violation at address 00000000 in module 'xtool.exe'. Read of address 00000000

D: EReadError: Stream read error

zlib is very unstable, once successful, failed several times:(

Simorq 11-08-2019 14:15

rzlib (XTool 1908_R2_x86)
 
#############################################
# rzlib (XTool 1908_R2_x86)
#############################################

Test File : bigfile.004.tiger 1.99 GB

Code:

XTool x86 Command Line

rzlib
xtool.exe precomp:rzlib:c16mb/c32mb/c64mb/c128mb,t100p - - < %1 > %1.out
xtool.exe decode:t100p - - < %1.out > %1.res
fc /b %1 %1.res

TEST 1
P: Error (EReadError: Stream read error)

TEST 2
P: Error (EReadError: Stream read error)

TEST 3
P: Error (EReadError: Stream read error)

TEST 4
P: Error (EReadError: Stream read error)

TEST 5
P: Error (EReadError: Stream read error)

rzlib always fails.:(

Razor12911 12-08-2019 15:26

@Simorq

Looks like I'll have to go through debugging road once more :D

@everyone

Update available

1908_R3
- Compatiblity broken (old history db and inputs cannot work in this version)
- Added plugin support
- Added LZ4 codec placeholders (lz4,lz4hc)
- Added LZO codec placeholders (lzo1c,lzo1x,lzo2a)
- Added ZSTD codec placeholders (zstd)
- Added Oodle codec placeholders (lzna,kraken,mermaid,selkie,leviathan)
- Added LZX codec placeholders (lzx)
- rzlib codec removed (use zlib)
- Added x86 memory limit

Notes
Placeholder does not mean the codec is ready to be used universally, it's intended for plugins which can be found here

ZAZA4EVER 12-08-2019 17:52

xtool_1908_R3 X64 ZLIB CODEC
eFootball Pes2020 Demo ... dt00_4K_x64.cpk
Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 1 file, 15,252,601 => 72,893,019 bytes. Ratio 477.91%
Compression time: cpu 0.05 sec/real 37.50 sec = 0%. Speed 0.41 mB/s
All OK

Zlib Codec is Stable .. i tested it Three Times ...:D
/////////////////////////////////////////////////////////////
xtool_1908_R3 X64 oodle Codec
Wolfenstein Youngblood ... chunkbase_9_pc.resources

Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 1 file, 100,552,642 => 100,552,714 bytes. Ratio 100.00%
Compression time: cpu 0.33 sec/real 3.38 sec = 10%. Speed 29.79 mB/s
All OK

oodle leviathan Stream Dont Work :o

FitGirl 12-08-2019 19:18

Quote:

Originally Posted by ZAZA4EVER (Post 482067)
oodle leviathan Stream Dont Work :o

They have custom chunks in there, no size in headers, so the decompressor should guess the right size, which is not a good idea :) Also it's slow af in Oodle 2.6.* .

Razor12911 12-08-2019 19:25

Quote:

Originally Posted by ZAZA4EVER (Post 482067)
oodle leviathan Stream Dont Work :o

Quote:

Originally Posted by Razor12911 (Post 482063)
Notes
Placeholder does not mean the codec is ready to be used universally, it's intended for plugins

Wolfenstein Youngblood ... chunkbase_9_pc.resources

- Could you upload a sample, I need to check something


Quote:

Originally Posted by FitGirl (Post 482068)
They have custom chunks in there, no size in headers, so the decompressor should guess the right size, which is not a good idea :) Also it's slow af in Oodle 2.6.* .

Last time I checked the headers did contain size, it was a while ago but I didn't add support because no game used this codec back then

Simorq 16-08-2019 07:49

Hi Razor12911

Assassin's Creed: Origins
Test File: DataPC_extra.forge / oodle4

Code:

Creating archive: data.arc using oodle
Compressed 1 file, 1,049,427,968 => 1,049,428,104 bytes. Ratio 100.00%   
Compression time: cpu 1.47 sec/real 4.11 sec = 36%. Speed 255.62 mB/s
All OK

__________________________________________________ _____________________________________________
DataPC_SharedGroup_02.forge Sample File
Oodle_DLL_x86/x64

Oodle precompressor (Side project)
Code:

Creating archive: data.arc using oo2recm
Compressed 1 file, 105,512,960 => 167,404,655 bytes. Ratio 158.66%       
Compression time: cpu 0.17 sec/real 15.59 sec = 1%. Speed 6.77 mB/s
All OK

XTool 1908_R3
Code:

Creating archive: data2.arc using oodle
Compressed 1 file, 105,512,960 => 105,513,032 bytes. Ratio 100.00%       
Compression time: cpu 0.09 sec/real 0.53 sec = 18%. Speed 199.34 mB/s
All OK


______________________________________

The problem was solved thanks to KaktoR.

doofoo24 16-08-2019 08:34

Quote:

Originally Posted by Razor12911 (Post 482063)
- Added LZ4 codec placeholders (lz4,lz4hc)
- Added LZO codec placeholders (lzo1c,lzo1x,lzo2a)
- Added ZSTD codec placeholders (zstd)
- Added Oodle codec placeholders (lzna,kraken,mermaid,selkie,leviathan)
- Added LZX codec placeholders (lzx)

"placeholders" :)

KaktoR 16-08-2019 08:50

@Simorq:

Oodle is not inside xtool 2019 yet (they are only "placeholders" for later use). Only zlib and unity lz4 are inside.

Razor12911 16-08-2019 15:13

Yes they are placeholders meant to be used by the plugins, they will get universal support eventually. But the plugins should give the best output since they are specific and not universal and as we know the universal tools I made (side projects) couldn't handle some games properly so I added placeholders to try and give the best precompression via plugins

@everyone, update was delayed. Because I noticed several issues in xtool thanks to Panker, FitGirl and of course you guys, plus I was busy making plugins for some games.

The changelog for next update

- Updated plugin support [DONE]
- Fixed plugin stream detection issues [DONE]
- Added delta/diff support for imperfect streams [0%]
- zlib codec renamed to deflate [DONE]
- Fixed several deflate related issues [DONE]
- Added png stream support [90%]
- Fixed future stream issues [DONE]
- Added low memory usage mode [0%]
- Added zstd universal stream support [DONE]
- Added lzo1x universal stream support (alpha) [DONE]

Plugins done so far:

haemimont, frostbite and anvil and updated unity plugin

K4miKaZe 16-08-2019 15:25

This is looking good.

Many thanks Razor12911 !!!

ansi 19-08-2019 02:48

When packing CS:GO vpk files, xtool 1908 R3 freezes when got to 5%.
Parameters: zlib,lz4:c32mb,t100p
However, there are no freezes on the xtool 1908 R2 version.

Andu21 21-08-2019 08:17

Remove lz4 from the parameters and try again.

Razor12911 21-08-2019 15:10

Update available

1908_R4
- Updated plugin support
- Fixed plugin stream detection issues
- Added delta/diff support for imperfect streams
- zlib codec renamed to deflate
- Fixed several deflate related issues
- Added png stream support
- Fixed future stream issues
- Added low memory usage mode (slower)

Notes

If this update does not fix most issues then my PC crashing has really ruined the development of this project. :(
If you take a look at the change log, it says I renamed zlib codec to deflate, well the reason for that is because zlib codec will be reintroduced later on but it will be mostly use by plugins that find zlib streams in games so as a normal user, use deflate


All times are GMT -7. The time now is 23:33.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com