Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #31  
Old 21-07-2019, 13:20
shazzla shazzla is offline
Registered User
 
Join Date: Nov 2010
Location: Hunnia
Posts: 132
Thanks: 223
Thanked 50 Times in 30 Posts
shazzla is on a distinguished road
If game uses oodle,then you can find its DLLs in game folder somewhere.
Imho : oodle stream scanner is obsolete ?!
Reply With Quote
Sponsored Links
  #32  
Old 21-07-2019, 15:29
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by Simorq View Post
Hi Razor12911
Code:
v: skip verification
This option works for oodle, crilayla, lzo.
Please do not delete this option in the new version.
The option will be there

Quote:
Originally Posted by dixen View Post
oodle detect, for example
I remember including an oodle scanner, but fine I'll think of something for the scanner.

Quote:
Originally Posted by KaktoR View Post
This is great.
how about this?

Quote:
There will be a new feature in xtool called history data, what it does is simply store processed stream information and be able to recall it at the later stage.

If you are wondering how this helps, well there are certain games made by lazy developers who repeat game resources instead of just reusing them, resulting in bigger game size for no absolute reason, apart from that, if they are compressed streams it means that xtool will have to process the same stream over and over again, storing history data allows it to prevent the same stream from being reprocessed instead, it will just look at the history and find out how the previous copy was processed then automatically assume it's the same stream and then apply the same settings, resulting in more speed.

I ran a test on a file I repeated on purpose and these were the results.

old xtool took 196.671 seconds and used 335MB ram
new xtool took 154.174 seconds and used 319MB ram
new xtool with history feature took 72.223 seconds and used 439MB ram

the cost of the feature depends on how much history you want to be stored but with default settings, it should use an additonal 192mb ram usage than without, you can always disable it or order it to store more history, depends on how much memory you have to spare.
Note:
In a practical situation, you can shave off a few seconds but then again, depends on the input
__________________
Looking for something?
Visit the
Tutorial Index


Last edited by Razor12911; 21-07-2019 at 15:35.
Reply With Quote
The Following 9 Users Say Thank You to Razor12911 For This Useful Post:
78372 (23-07-2019), Andu21 (22-07-2019), Carldric Clement (25-07-2019), ffmla (22-07-2019), Harsh ojha (22-07-2019), KaktoR (21-07-2019), PsYcHo_RaGE (22-07-2019), shiyamrrc (23-07-2019), Simorq (22-07-2019)
  #33  
Old 23-07-2019, 07:18
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 2,778
Thanks: 712
Thanked 4,139 Times in 1,495 Posts
KaktoR is on a distinguished road
This is also good

Did you have to make changes to the libraries too? Or just the source of xtool executable? Just corious
__________________
Haters gonna hate
RIP GOD
Reply With Quote
  #34  
Old 23-07-2019, 15:04
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
Only the source of xtool, all I did was optimize optimize and optimize, the CPU is forever doing something, I tried to minimize the gaps between reads and writes by making xtool process stuff while it is reading and writing so yeah, the threads hardly get rests now.
__________________
Looking for something?
Visit the
Tutorial Index


Last edited by Razor12911; 23-07-2019 at 15:12.
Reply With Quote
The Following 8 Users Say Thank You to Razor12911 For This Useful Post:
78372 (23-07-2019), Carldric Clement (25-07-2019), Cesar82 (24-07-2019), Harsh ojha (23-07-2019), KaktoR (24-07-2019), shazzla (23-07-2019), Simorq (24-07-2019), ZAZA4EVER (25-07-2019)
  #35  
Old 26-07-2019, 16:55
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
Project status = Welp...

I hope this isn't strike 2 in terms losing my sources, my PC died about 2 days ago, motherboard and GPU got fried this time XD. As for my sources well they were stored on my main PC (the dead one) and I don't know if they are safe since I don't have a spare PC to boot up the drive to grab important files and store them to my backup drive, so I can't really tell if the only components that died were only the board and GPU and not my HDD, but I guess that's a story for some other time. If you are wondering what this means in terms of xtool 2019...

Well since I normally code from desktop because I don't need to go far to do things and etc, it was different for xtool somehow... because I was coding from a backup drive only for that project like I knew some shit like this would happen so yes the source is safe (lucky af), if you are wondering about the development now that my main PC is busted.

Well I just had to dust an old laptop and continued development there, issue is it's incredibly slow but it will have to do. If you are wondering why I only provided x86 binaries, well. OS is 32-bit , somehow can't compile for x64.

Anyways enough about that, this was suppose to be the last test before the first release, but before I uploaded, I noticed some shitty error in made in the reflate code and I have to rewrite some parts of it so after fixing that, I'll post the final test and if all is good, I'll be making a release.

there are 3 codecs in xtool right now (zlib, reflate, rzlib). The bug was in reflate in multi threaded mode when decoding, but if you want to try it out anyways, open the test bat file. replace zlib with reflate or rzlib but use 1 thread to decode.

zlib = xtool uses zlib1.dll/zlibwapi.dll to process data, fails on some streams
reflate = xtool uses reflate dlls to process data, processes all streams but slow
rzlib = xtool blends zlib and reflate dlls to process data, processes all streams but with better size and speed.

PS: Even the archive is zip, don't have 7-Zip installed for my normal 7z attachments
PPS: Nothing is stopping the development of this project XD... I hope cause you never know
Attached Files
File Type: zip xtool_1907_T3.zip (552.9 KB, 59 views)
__________________
Looking for something?
Visit the
Tutorial Index


Last edited by Razor12911; 26-07-2019 at 16:59.
Reply With Quote
The Following 11 Users Say Thank You to Razor12911 For This Useful Post:
78372 (26-07-2019), Andu21 (27-07-2019), FitGirl (26-07-2019), GTX590 (26-07-2019), Harsh ojha (26-07-2019), KaktoR (27-07-2019), oltjon (27-07-2019), Sergey3695 (27-07-2019), shazzla (26-07-2019), Simorq (26-07-2019), ZAZA4EVER (29-07-2019)
  #36  
Old 26-07-2019, 18:33
FitGirl FitGirl is offline
Registered User
 
Join Date: Dec 2014
Location: Riga
Posts: 147
Thanks: 11
Thanked 167 Times in 93 Posts
FitGirl is on a distinguished road
Shit, mate! You're lucky to have not loosing sources this time! Gotta use some repo soft next time for more security, no?

Anyways, it's nice to see the project is coming to release.
Reply With Quote
The Following 3 Users Say Thank You to FitGirl For This Useful Post:
Harsh ojha (26-07-2019), Razor12911 (28-07-2019), Sergey3695 (27-07-2019)
  #37  
Old 26-07-2019, 21:01
Harsh ojha's Avatar
Harsh ojha Harsh ojha is offline
Registered User
 
Join Date: May 2019
Location: INDIA
Posts: 47
Thanks: 263
Thanked 35 Times in 18 Posts
Harsh ojha is on a distinguished road
Smile

@Razor12911
Download Windows 10 64 bit Archived file and install

https://youtu.be/zGQ9xXYgVJM
Since i also lost my data

Well we are all depends our pc and laptops
__________________
🎵🎶Video Creator
Reply With Quote
The Following User Says Thank You to Harsh ojha For This Useful Post:
Razor12911 (28-07-2019)
  #38  
Old 28-07-2019, 15:14
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
First release

Notes

- No idea whether x64 works or not, I just somehow managed to compile. That's up to you to find out.
- Reflate currently is set to use level 6, if you are looking for better output, best to use the old xtool. The reason it's like this is because I'm still working on a way to improve the horrible slow code I used before.
- Skip verification option is currently unavailable.

What I can say, run a lot of tests on this release. I've written some dangerous memory saving code and it may cause errors but if all is well then I can move on to add other codecs such as oodle, lzo, lz4 and the rest. BTW, crilayla will NOT be added, that piece of crap is incredibly slow
__________________
Looking for something?
Visit the
Tutorial Index

Reply With Quote
The Following 4 Users Say Thank You to Razor12911 For This Useful Post:
Harsh ojha (29-07-2019), shazzla (29-07-2019), Simorq (28-07-2019), ZAZA4EVER (29-07-2019)
  #39  
Old 28-07-2019, 16:11
Simorq's Avatar
Simorq Simorq is offline
Registered User
 
Join Date: Mar 2014
Location: Iran
Posts: 634
Thanks: 3,451
Thanked 1,112 Times in 427 Posts
Simorq is on a distinguished road
Image003.png

xtool_1907_R1 64Bit
Code:
Creating archive: data.arc using zlib
Compressed 1 file, 668,113 => 668,225 bytes. Ratio 100.02%                 
Compression time: cpu 0.02 sec/real 0.18 sec = 9%. Speed 3.71 mB/s
All OK

Testing archive: data.arc
Tested 1 file, 668,225 => 668,113 bytes. Ratio 100.02%                     
Testing time: cpu 0.00 sec/real 0.06 sec = 0%. Speed 10.77 mB/s
All OK
Code:
Creating archive: data.arc using reflate
Compressed 1 file, 668,113 => 1,767,491 bytes. Ratio 264.55%               
Compression time: cpu 0.00 sec/real 0.19 sec = 0%. Speed 3.44 mB/s
All OK

Testing archive: data.arc
Tested 1 file, 1,767,491 => 668,113 bytes. Ratio 264.55%                   
Testing time: cpu 0.00 sec/real 0.16 sec = 0%. Speed 4.31 mB/s
All OK
Code:
Creating archive: data.arc using rzlib
Compressed 1 file, 668,113 => 1,767,491 bytes. Ratio 264.55%               
Compression time: cpu 0.00 sec/real 0.28 sec = 0%. Speed 2.40 mB/s
All OK

Testing archive: data.arc
Tested 1 file, 1,767,491 => 668,113 bytes. Ratio 264.55%                   
Testing time: cpu 0.00 sec/real 0.16 sec = 0%. Speed 4.25 mB/s
All OK
Reply With Quote
The Following User Says Thank You to Simorq For This Useful Post:
Razor12911 (28-07-2019)
  #40  
Old 28-07-2019, 16:30
Simorq's Avatar
Simorq Simorq is offline
Registered User
 
Join Date: Mar 2014
Location: Iran
Posts: 634
Thanks: 3,451
Thanked 1,112 Times in 427 Posts
Simorq is on a distinguished road
xtool_1907_R1 Test

Does not work with ba2/csg files.
Code:
Fallout4 - Textures7.ba2 / DLCCoast - Geometry.csg
32bit= ERROR: write error (disk full?) in compression algorithm zlib/reflate/rzlib
64bit= ERROR: write error (disk full?) in compression algorithm zlib/reflate/rzlib or stuck cpu


-------------------------------------------------------------------

Works with this setting.
Code:
[External compressor:rzlib,zlib,reflate]
header    = 0
packcmd   = xtool.exe precomp:rzlib:c32mb,t1 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = xtool.exe decode:t50p - - <stdin> <stdout>
Code:
Fallout4 - Textures7.ba2 520 MB
ZLIB
Code:
Creating archive: data.arc using zlib
Compressed 1 file, 545,959,220 => 1,237,654,873 bytes. Ratio 226.69%       
Compression time: cpu 0.52 sec/real 84.11 sec = 1%. Speed 6.49 mB/s
All OK

zlib: decode:t100p
Testing archive: data.arc
Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           
Testing time: cpu 0.45 sec/real 9.53 sec = 5%. Speed 57.29 mB/s
All OK
REFLATE
Code:
Creating archive: data.arc using reflate
Compressed 1 file, 545,959,220 => 1,237,783,515 bytes. Ratio 226.72%       
Compression time: cpu 0.42 sec/real 113.43 sec = 0%. Speed 4.81 mB/s
All OK

reflate: decode:t100p
Testing archive: data.arc
Tested 1 file, 1,237,783,515 => 545,959,220 bytes. Ratio 226.72%           
Testing time: cpu 0.36 sec/real 12.23 sec = 3%. Speed 44.62 mB/s
All OK
RZLIB
Code:
Creating archive: data.arc using rzlib
Compressed 1 file, 545,959,220 => 1,237,654,873 bytes. Ratio 226.69%       
Compression time: cpu 0.53 sec/real 85.22 sec = 1%. Speed 6.41 mB/s
All OK

rzlib: decode:t2
Testing archive: data.arc
Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           
Testing time: cpu 0.98 sec/real 73.38 sec = 1%. Speed 7.44 mB/s
All OK

rzlib: decode:t100p
Testing archive: data.arc
Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           
Testing time: cpu 0.77 sec/real 73.54 sec = 1%. Speed 7.42 mB/s
All OK

Last edited by Simorq; 28-07-2019 at 17:16.
Reply With Quote
The Following User Says Thank You to Simorq For This Useful Post:
Razor12911 (28-07-2019)
  #41  
Old 28-07-2019, 17:14
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
Can you run xtool outside freearc, like via console.

Code:
@echo off
xtool.exe precomp:rzlib:c16mb,t100p %1 %1.out
pause
I'm interested in the exception/error message
__________________
Looking for something?
Visit the
Tutorial Index

Reply With Quote
The Following User Says Thank You to Razor12911 For This Useful Post:
Simorq (28-07-2019)
  #42  
Old 28-07-2019, 17:21
Simorq's Avatar
Simorq Simorq is offline
Registered User
 
Join Date: Mar 2014
Location: Iran
Posts: 634
Thanks: 3,451
Thanked 1,112 Times in 427 Posts
Simorq is on a distinguished road
@Razor12911
good


-------------------------
I think the main problem is - -.
- - <stdin> <stdout>

Last edited by Simorq; 28-07-2019 at 17:25.
Reply With Quote
The Following User Says Thank You to Simorq For This Useful Post:
Razor12911 (28-07-2019)
  #43  
Old 28-07-2019, 17:31
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
try this then

Code:
@echo off
xtool.exe precomp:rzlib:c32mb,t100p - - < %1 > %1.out
xtool.exe decode:t100p - - < %1.out > %1.res
fc /b %1 %1.res
pause
__________________
Looking for something?
Visit the
Tutorial Index

Reply With Quote
The Following User Says Thank You to Razor12911 For This Useful Post:
Simorq (28-07-2019)
  #44  
Old 28-07-2019, 17:38
Simorq's Avatar
Simorq Simorq is offline
Registered User
 
Join Date: Mar 2014
Location: Iran
Posts: 634
Thanks: 3,451
Thanked 1,112 Times in 427 Posts
Simorq is on a distinguished road
@Razor12911
no work
stuck cpu
Reply With Quote
The Following 2 Users Say Thank You to Simorq For This Useful Post:
Razor12911 (28-07-2019), ZAZA4EVER (29-07-2019)
  #45  
Old 28-07-2019, 17:42
Razor12911's Avatar
Razor12911 Razor12911 is offline
Coder
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,293
Thanks: 1,786
Thanked 8,300 Times in 1,838 Posts
Razor12911 is on a distinguished road
Nice ghost bugs, I'll check what I missed in the source and thanks for the tests.
__________________
Looking for something?
Visit the
Tutorial Index

Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
Simorq (28-07-2019), ZAZA4EVER (29-07-2019)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[CIUv3] Custom Installer Ultimate v3 KaktoR Conversion Tutorials 282 14-10-2019 19:57
[CIUv2] Custom Installer Ultimate v2 Project Source Codes KaktoR Conversion Tutorials 606 26-12-2018 07:28
Project Cars Digital Edition (3xDVD5) (srep+lzma) GTX590 PC Games - CD/DVD Conversions 10 28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039 mausschieber PC Games - CD/DVD Conversions 0 24-07-2017 15:12
Space Channel 5 Part 2 Translation Project Christuserloeser DC Games 0 21-06-2004 18:16



All times are GMT -7. The time now is 05:50.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.
Copyright 2000-2018, FileForums @ https://fileforums.com