FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   pZLib v3 (Final release) (https://fileforums.com/showthread.php?t=98672)

FitGirl 10-05-2017 12:41

A friend of mine did more tests of cls vs generic pzlib. Decoding speed.

Code:

          |                      t1 |                      t2 |                      t4 |                        t7 |
----------*--------------------------*--------------------------*--------------------------*---------------------------*
pzlib_x86 |  192 sec. / 2613 kB/sec. |  118 sec. / 4249 kB/sec. | 78.3 sec. / 6419 kB/sec. | 61.7 sec.  / 8144 kB/sec. |
pzlib_x64 |  141 sec. / 3556 kB/sec. | 87.5 sec. / 5745 kB/sec. | 57.9 sec. / 8680 kB/sec. | 44.1 sec. / 11386 kB/sec. |
cls-pzlib |  198 sec. / 2537 kB/sec. |  119 sec. / 4223 kB/sec. | 78.3 sec. / 6422 kB/sec. | 61.5 sec.  / 8169 kB/sec. |


enc opts: pzlib:m2

win7_sp1_x64
i7-4700MQ / 8 GB RAM
RAM-Disk

Razor12911 11-05-2017 21:22

1 Attachment(s)
CLS updated

-Cancel bug fixed
-Added multi threading user control example

Razor12911 12-05-2017 16:37

Quote:

Originally Posted by FitGirl (Post 458709)
A friend of mine did more tests of cls vs generic pzlib. Decoding speed.

Code:

          |                      t1 |                      t2 |                      t4 |                        t7 |
----------*--------------------------*--------------------------*--------------------------*---------------------------*
pzlib_x86 |  192 sec. / 2613 kB/sec. |  118 sec. / 4249 kB/sec. | 78.3 sec. / 6419 kB/sec. | 61.7 sec.  / 8144 kB/sec. |
pzlib_x64 |  141 sec. / 3556 kB/sec. | 87.5 sec. / 5745 kB/sec. | 57.9 sec. / 8680 kB/sec. | 44.1 sec. / 11386 kB/sec. |
cls-pzlib |  198 sec. / 2537 kB/sec. |  119 sec. / 4223 kB/sec. | 78.3 sec. / 6422 kB/sec. | 61.5 sec.  / 8169 kB/sec. |


enc opts: pzlib:m2

win7_sp1_x64
i7-4700MQ / 8 GB RAM
RAM-Disk

Well it's not surprising, cls is actually x86, it should produce roughly the same speed as pzlib x86.

danswano 14-05-2017 07:14

Quote:

Originally Posted by Simorq (Post 458697)
Mad Max
Code:

pZLibv2:c32m+srep+Lzma2 4.36 GB
Code:

pZLibv3:c256m+srep+Lzma2 6.4 GB
Code:

pZLibv3:x:c32m+srep+Lzma2 4.26 GB

What are your srep/lzma2 parameters you got 4.36GB with?
Please provide .bat and arc.ini parameters.
Thanks.

Simorq 14-05-2017 12:44

Quote:

Originally Posted by danswano (Post 458788)
What are your srep/lzma2 parameters you got 4.36GB with?
Please provide .bat and arc.ini parameters.
Thanks.


pzlib+srep+lzma2 4.26 GB
Arc.ini
Code:

[External compressor:pzlib3]
header = 0
packcmd  = pZLib e -m2 -x -c32m -t25p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>

[External compressor:pzlib2]
header = 0
packcmd  = pZLib e -m2 -c32m -t25p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>


[External compressor:srep]
header    = 0
;packcmd  = srep {options} -a1 -m5f -l256 -t8 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
packcmd  = srep {options} -m5f $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:lzma2, Lzma2, LZMA2, XZ, xz]
header = 0
packcmd  = 7z a -txz -an -mcrc=0 -m1=lzma2:d25:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=on -mx9 -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -mmt=on -an -y -si -so <stdin> <stdout>

Test pzlib2
http://s8.picofile.com/file/82949510..._Max_pZLib.png

FitGirl 15-05-2017 00:22

Razor, thank you for the latest update - now the installer closes properly!
You're a gift to this community!

pakrat2k2 16-05-2017 09:25

Quote:

Originally Posted by KaktoR (Post 458631)
Someone on the previous pages telling not to use <stdin> - <stdout>. Then you should use all settings without crc error. But honestly, i don't know how to use this.

change arc.ini to use the commands listed ( parts without crc error )

LuzhinE 04-06-2017 22:57

With t25p and t75p works well until now free of CRC, before there was that problem to cancel and did not, do not comment because Razor12911 is busy in classes, but now if it works well.

But waiting for an improvement so that the CPU can be used to maximum when compressing t8.

Thank you very much

Razor12911 05-06-2017 03:35

Version 3.2 now available

Changes

-Tried to fix crc errors, no idea if they are fixed or not.
-Removed several options because there are just to many for some people to understand what they do.
-Tried to fix decoding termination, in case you want to cancel decoding process, it must now work without killing process, not sure either.
-method setting removed, from now it just searches for deflate streams which is m2 by default.
-by default reflate and zlib are blended.
-special stream option "s" is removed, found a way to process them.

Simorq 05-06-2017 10:36

Deflate Streams Better.

Code:

Test (Act Of Aggression Reboot Edition Deflate Streams)
pZLib version 2 and 3 Not Support
pZLib2017 Support


felice2011 05-06-2017 11:59

More than an Update seems to be a Downgrade, I enjoy working on pzlib, great Razor.

Mini 05-06-2017 12:44

thx.

unpack with CLS x64 gives an error message:
ERROR: unsupported compression method pzlib

x86 - OK

Razor12911 05-06-2017 17:28

Quote:

Originally Posted by felice2011 (Post 459510)
More than an Update seems to be a Downgrade, I enjoy working on pzlib, great Razor.

Downgrade in what way? I just removed options that you guys have no knowledge of how they are used then complain afterwards. For example, I've seen most people using 's' when I specified that the option is only for the game DOOM. Otherwise it messes up ratio. I've seen people setting things like b4096m or even c4096m. Pzlib will obviously crash. Maximum chunk size of program is 1792mb. And even that is very high because there are no streams in games that big. The changes now allow you to use 400mb chunk size for DOOM rather than 600mb or 700mb to capture all streams. So the only thing I did was make program decide by itself for all the options I removed.

Razor12911 05-06-2017 17:31

Quote:

Originally Posted by Mini (Post 459512)
thx.

unpack with CLS x64 gives an error message:
ERROR: unsupported compression method pzlib

x86 - OK

X64 DLLs can only be imported by x64 executables -_- CLS x64 can only work with x64 arc.exe and unarc.DLL x64

This is where this comes in http://fileforums.com/showthread.php?t=99263

@everyone
If CRC errors are gone, then bug was caused by multi threaded scanning and/or processing. Threads tend to disturb each other since they are using the same data and info to function. Problem isn't related to I/O

felice2011 05-06-2017 22:57

Quote:

Originally Posted by Razor12911 (Post 459515)
Downgrade in what way? I just removed options that you guys have no knowledge of how they are used then complain afterwards. For example, I've seen most people using 's' when I specified that the option is only for the game DOOM. Otherwise it messes up ratio. I've seen people setting things like b4096m or even c4096m. Pzlib will obviously crash. Maximum chunk size of program is 1792mb. And even that is very high because there are no streams in games that big. The changes now allow you to use 400mb chunk size for DOOM rather than 600mb or 700mb to capture all streams. So the only thing I did was make program decide by itself for all the options I removed.

It was a sarcastic joke, obviously for the options you've taken away, and not added.;)

LuzhinE 06-06-2017 20:58

It works like a charm

shazzla 07-06-2017 23:23

Is any chance for a pzlib like preprocessor which unpacks ZSTD streams ?

Gupta 08-06-2017 03:16

Quote:

Originally Posted by shazzla (Post 459585)
Is any chance for a pzlib like preprocessor which unpacks ZSTD streams ?

I assure you a lot of names have tried
And "also you have to be religious and ask your god if u need it even" - Deadsec

shazzla 08-06-2017 03:18

I didnt know that.
Thanks!

doofoo24 21-06-2017 20:53

can't set number of thread for the new cls-pzlib.dll ?
using pzlib with arc.ini set t100p is much faster...
is there cls-pzlib.ini option ???

MCWRX 25-06-2017 06:03

Hi everyone! Thanks Razor again for all the work done.

I've got a question regarding the CLS. I've taken a 3GB data and compressed/decompressed via .exe and .dll. So, CLS decompresses data 2 times slower.
  • I've used x64 executable, x64 cls and x64 lib. dlls for compression.
  • During decompression I am using x86 cls.

Using x64 cls gives me CRC error at start and afaik it is not possible to use it with x86 unarc. What suggestions do you have to get nearly (not same) decompression time as in case with .exe?

And am I using the correct files for decompression process?

Simorq 29-06-2017 14:20

Hi Razor12911
How do we know what is best chunk size of each game?

!
Mad Max pZLib:c32+srep+lzma 1hour Extract
Mad Max pZLib:c256+srep+lzma 35Min Extract

harshlohaan 17-07-2017 03:45

Quote:

Originally Posted by ZAZA4EVER (Post 454941)
Not Solve , Thanks For Reply And Help





Thanks Very Much ... Extract Perfect ... Finally

Can you please provide the code & resource files you used in your script that you used to decompress the pzlib based archive. I'm suffering from the same issue, my installer can't extract. I'm just asking for the required code and resource files that instruct the installer to extract the pzlib archives.
Please..:(

ZakirAhmad 23-07-2017 01:05

Ztool related question
 
What is arc.ini setting for ztool posted by razor in another thread.
Please help me.

EzzEldin16 23-07-2017 01:50

don't you feel like the man who made this tool is gone because it was stolen and you are asking for an arc.ini setting give the man some respect

ZakirAhmad 23-07-2017 08:59

I am not that selfish. I already thanked him. BTW he posted it in anpther thread, what is the point of posting it, if he is not going to tell anyone how to make it work.

EzzEldin16 23-07-2017 14:32

okay i didn't say that you are selfish but someone in his place for example me would do the same as he did maybe more

linggar17x 01-10-2017 20:07

why error disk full when i am compressing?

darkwolves 03-10-2017 23:51

Quote:

Originally Posted by ZakirAhmad (Post 460971)
What is arc.ini setting for ztool posted by razor in another thread.
Please help me.

Ztool is no longer under production anyways better off using pzlib or precomp :rolleyes:
i am not going to play innocent and claim i have not tried it thats how i know..
it won't work the way you want it to it is an abandoned project by razor wich
also led to his retirement due to it being leaked and he worked very hard on that tool..
it was not supposed to be public

felice2011 04-10-2017 04:54

Quote:

Originally Posted by darkwolves (Post 462852)
Ztool is no longer under production anyways better off using pzlib or precomp :rolleyes:
i am not going to play innocent and claim i have not tried it thats how i know..
it won't work the way you want it to it is an abandoned project by razor wich
also led to his retirement due to it being leaked and he worked very hard on that tool..
it was not supposed to be public

Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

78372 04-10-2017 05:45

The stable(v1) version was just not leaked, but exists. Unstable version was leaked. And yep it's not updated anymore with this name.

darkwolves 04-10-2017 09:23

Quote:

Originally Posted by 78372 (Post 462859)
The stable(v1) version was just not leaked, but exists. Unstable version was leaked. And yep it's not updated anymore with this name.

i don't need Ztool lol i can compress things just fine without it lol:D

darkwolves 04-10-2017 10:36

Quote:

Originally Posted by felice2011 (Post 462857)
Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

given all the free he has given us it's his right to keep things private
if he wants to he never had to make any of the free tools i don't need it i compress just fine without it lol

GBT repack 05-10-2017 04:18

How is the program used? I hope someone will provide a way to thank everyone

GBT repack 05-10-2017 04:23

help me, thanks.....

darkwolves 05-10-2017 08:51

Quote:

Originally Posted by GBT repack (Post 462882)
How is the program used? I hope someone will provide a way to thank everyone

as i said its pointless to use it its not supported anymore and doesn't work
except for two functions on certain games pzlib3 does the same thing

oltjon 05-10-2017 12:23

Quote:

Originally Posted by darkwolves (Post 462889)
as i said its pointless to use it its not supported anymore and doesn't work
except for two functions on certain games pzlib3 does the same thing

is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..

Razor12911 05-10-2017 19:34

Quote:

Originally Posted by felice2011 (Post 462857)
Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

I program far less than you think now, slowly depleting in DNA and no, no improved version of ztool exists, all that I created after that are just scattered unfinished projects like pUnity and pOodle which were just created and put away :(

Quote:

Originally Posted by oltjon (Post 462891)
is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..

It's actually true, pzlib in ztool is terrible, even sometimes using pzlib v3 is better due to the fact that the person who leaked it saved me a lot of trouble by leaking the most unstable release, the release that came before 0.0.19.8 is even better so jokes on him.
plz4 only works on Frostbite 3 engine, nothing much to offer there since lz4 is a weak algorithm, you gain 1gb diff at most.
plzo only works on anvil engine games but not all
pzstd works on Frostbite 3 engine games which happens to be only Mass Effect Andromeda and doesn't work on FIFA 17 while nor in any upcoming games because game devs decided to switch to Oodle for FIFA 18 and any other upcoming game probably so he's right by saying it's not good, the leaker leaked tool before it was worth it :)

darkwolves 05-10-2017 22:08

Quote:

Originally Posted by oltjon (Post 462891)
is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..

if that is so they are modfiying it themselves doesn't change the fact it was stolen

KaktoR 06-10-2017 04:58

Quote:

Originally Posted by Razor12911 (Post 462895)
[...]because game devs decided to switch to Oodle for FIFA 18 and any other upcoming game[...]

How you know that? Serious question.:D


All times are GMT -7. The time now is 22:43.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com