FileForums - pZLib version 2 (Fastest zlib/deflate precompressor)

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - pZLib version 2 (Fastest zlib/deflate precompressor) (https://fileforums.com/showthread.php?t=98486)

pZLib version 2 (Fastest zlib/deflate precompressor)

Works on Mad Max, GTAV, DiRT Rally, GRID Autosport and pretty much every game that uses zlib except DOOM.

Benchmarks? Broke my last record of compressing Mad Max in 8 hours and decompressing it in 25 minutes using reflate. pZLib compresses game in 2 hours and decompresses in 19 minutes.

Took me months to create this so do mention Razor if used in your repacks/conversions else just don't use it at all.

Enjoy.

Mad Max speed showcase.
pzLib alone reached 70mb/s restoration speed.

you gave it first to dzrepack team then releasing it here BTW thnx bro :( I got this one 2 days ago :)
I hope you Just release something like the reflate used by dzrepack that you made

7 views already in like, 9 minutes and no thanks. Haha, rejoice people. This may be my last project.

@aswadd
this is faster than reflate and delivers the same output.

Quote:

Originally Posted by Razor12911 (Post 453526)

7 views already in like, 9 minutes and no thanks. Haha, rejoice people. This may be my last project.

@aswadd
this is faster than reflate and delivers the same output.

Great so why dzteam used reflate instead on this pzlib in pes17 :D

what bugs ?
do you mean it doesn't work on pes17 ? so we have to use reflate again

No, the program is so inconsistent, it raises CRC errors out of nowhere and when you precompress the game once again, the error is gone, but errors are rare.

thank's Razor nice work pefect, your projects are all very useful!
do not stop to amaze us

More showcase of Ride 2 and GTAIV Complete Edition

Quote:

Originally Posted by Razor12911 (Post 453526)

7 views already in like, 9 minutes and no thanks. Haha, rejoice people. This may be my last project.

lately I have difficulty understanding what you write, let the job is tested to thank.

Quote:

Originally Posted by felice2011 (Post 453533)

lately I have difficulty understanding what you write, let the job is tested to thank.

I just found it shocking :p, as if people knew I was going to post this.

Quote:

Originally Posted by Razor12911 (Post 453534)

I just found it shocking :p, as if people knew I was going to post this.

Personally a some things that you write do not make sense.;)

Thanks to Razor12911:)

Test
x64q.rpf (2.62GB) reflate+Srep+Lzma (1.14GB)
x64q.rpf (2.62GB) pZLib2+Srep+Lzma (1.14GB)

No Work Act of Aggression - Reboot Edition .dat file

Quote:

Originally Posted by Razor12911 (Post 453530)

No, the program is so inconsistent, it raises CRC errors out of nowhere and when you precompress the game once again, the error is gone, but errors are rare.

yes, that's the RAM. BTW works really fast.
Offtopic: i don't like to see any "memes" in this forum.

Please don't stop.

Quote:

Originally Posted by Razor12911 (Post 453526)

7 views already in like, 9 minutes and no thanks. Haha, rejoice people. This may be my last project.

@aswadd
this is faster than reflate and delivers the same output.

You are the best. Please don't close your projects. Your projects are the most useful.

Thank you very much Razor for pzlib! I tested on this file: x64f.rpf (981mb)

Only Shelwien's reflate in level 4:
Compression:
FreeArc 0.67 (March 15 2014) creating archive: data\datareflate.arc
Compressed 1 file, 1,028,757,504 => 2,304,942,259 bytes. Ratio 224.05%
Compression time: cpu 1.92 sec/real 471.78 sec = 0%. Speed 2.18 mB/s
All OK
Decompression:
FreeArc 0.67 (March 15 2014) extracting archive: datareflate.arc
Extracted 1 file, 2,304,942,259 => 1,028,757,504 bytes. Ratio 224.05%
Extraction time: cpu 6.44 sec/real 205.69 sec = 3%. Speed 5.00 mB/s
All OK

Only pzlib version2:
Compression:
FreeArc 0.67 (March 15 2014) creating archive: data\datapzlib.arc
Compressed 1 file, 1,028,757,504 => 2,226,261,282 bytes. Ratio 216.40%
Compression time: cpu 0.84 sec/real 108.17 sec = 1%. Speed 9.51 mB/s
All OK
Decompression:
FreeArc 0.67 (March 15 2014) extracting archive: datapzlib.arc
Extracted 1 file, 2,226,261,282 => 1,028,757,504 bytes. Ratio 216.40%
Extraction time: cpu 0.92 sec/real 86.33 sec = 1%. Speed 11.92 mB/s
All OK

Reflate maybe find more stream but pzlib much faster!
Thanks again and good work!

I had problems with file.ttarch2 Telltale Games

does not work

earn to die 2

Code:

C:\Users\oLtJoN\Desktop\reflate_1500>arc.exe a -ep1 -r -ed -w.\temp -s4096m -mreflate data.arc "pack\*"

FreeArc 0.67 (March 15 2014) creating archive: data.arc

Compressing 86 files, 200,374,281 bytes. Processed  54%

Compressing 200,374,281 bytes with reflate -process $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

Reflate (Free Edition) created by Razor12911

Searching for deflate streams

Splitting streams 91/91

Processing extracted deflate streams

Gathering processed streams

Merging stream 183/183

Removing temporary files



Processed: 44.0 MB/191 MB (23%)

New size: 961 MB instead of 191 MB (503%)



Errorlevel=0

100%

Code:

C:\Users\oLtJoN\Desktop\reflate_1500>del /q unpacked\



C:\Users\oLtJoN\Desktop\reflate_1500>arc.exe x -w.\ -dpunpacked\ data.arc

FreeArc 0.67 (March 15 2014) extracting archive: data.arc

Extracting 86 files, 200,374,281 bytes. Processed   0%SSEInv             S├             T├             s          Extracted 86 files, 1,007,971,708 => 200,374,281 bytes. Ratio 503.04%

Extraction time: cpu 0.73 sec/real 32.87 sec = 2%. Speed 6.10 mB/s

All OK

Code:

C:\Users\oLtJoN\Desktop\pZLib_v2>del /q data.arc

Impossibile trovare C:\Users\oLtJoN\Desktop\pZLib_v2\data.arc



C:\Users\oLtJoN\Desktop\pZLib_v2>arc.exe a -ep1 -r -ed -s; -w.\temp -mpzlib data.arc "pack\*"

FreeArc 0.67 (March 15 2014) creating archive: data.arc

Compressed 86 files, 200,374,281 => 577,169,086 bytes. Ratio 288.05%

Compression time: cpu 0.27 sec/real 13.66 sec = 2%. Speed 14.67 mB/s

All OK

Code:

C:\Users\oLtJoN\Desktop\pZLib_v2>del /q unpacked\



C:\Users\oLtJoN\Desktop\pZLib_v2>arc.exe x -w.\ -dpunpacked\ data.arc

FreeArc 0.67 (March 15 2014) extracting archive: data.arc

Extracting 86 files, 200,374,281 bytes. Processed   0%SSEInv             S├             T├             s          Extracted 86 files, 577,169,086 => 200,374,281 bytes. Ratio 288.05%

Extraction time: cpu 0.44 sec/real 6.59 sec = 7%. Speed 30.39 mB/s

All OK

Thanks, mate. Will test it and if it's as good as you describe will use it with credits given, ofc.

Quote:

Originally Posted by oltjon (Post 453563)

I had problems with file.ttarch2 Telltale Games

does not work

Cause they are encrypted + new games (Batman i.e.) use Oodle, not zlib

Pzlib_v2 VS ReflateMT...Under-Test *.rpf File:)

Only pzlib_v2

Code:

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 05/11/2016 16:59:09

=============================================================================================================================================

 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○



        pzlib_v2         981,1°MB           2,07°GB       210,99%               1°File    00:01:07:929   e -m2 -st50p -t8 -ct8 -c32m -b512k -v -o 

=============================================================================================================================================



-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Decompressed Archive Completed At -↓- 05/11/2016 17:00:32

=============================================================================================================================================

 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ DECOMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○



        pzlib_v2          2,07°GB          981,1°MB        47,40%               1°File       00:00:53:81                           d -t8 -o 

=============================================================================================================================================

Only ReflateMT

Code:

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 05/11/2016 17:10:22

=============================================================================================================================================

 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○



         reflate         981,1°MB           2,08°GB       212,01%               1°File    00:04:56:956                  -process -c9 -t8  

=============================================================================================================================================



-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Decompressed Archive Completed At -↓- 05/11/2016 17:29:48

=============================================================================================================================================

 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ DECOMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○



         reflate          2,08°GB          981,1°MB        47,17%               1°File       00:16:08:60                  -restore -c9 -t8  

=============================================================================================================================================

Excellent compression and decompression speed compared to reflateMT...
I was very pleased, the double embedded option ,"M1 = zlib streams" and "M2 = deflate streams", with the possibility of avoiding the use of "Antiz & Zlib_Jo" for only zlib streams.
As you see here no one does not abandon you, in 7 hours two pages of thanks, of tests and appreciations on yours work.:)
Personally I think it's the best job of zlib...Bravo!!!;)

Mad Max
Input = 31.9 GB > Pzlib2 = 51.8 GB > Srep = 14 GB > LZMA2 = 9.57 GB

http://uupload.ir/files/2lo5_2016-11-05_211751.jpg

9.57 GB?
http://www.fileforums.com/showpost.p...7&postcount=27

Simorq

all the languages?

Simorq@

set -c256m

there are big streams and a higher chunk size must be set, but this results in a higher ram usage, 256mb x Threads + 256mb

razor, what think about cls? with her speed will fall?

Quote:

Originally Posted by Mini (Post 453580)

razor, what think about cls? with her speed will fall?

According to razor cls is a bad idea because buffer cannot be controlled, pzlib was created to work safe on SSD without damaging them that much with excessive writes.

coliotte

that version did you use to reflate?
PES 2017 DZR
Installation is super fast

Quote:

Originally Posted by LuzhinE (Post 453578)

Simorq

all the languages?

No Only EN

DZ Repack
PES 2017: reflate:m2:l6:t75p:p5:s240+srep+lzma2
Mad Max New Repack (4.2 GB): pzlib+srep+lzma2

Quote:

Originally Posted by Simorq (Post 453571)

Mad Max
Input = 31.9 GB > Pzlib2 = 51.8 GB > Srep = 14 GB > LZMA2 = 9.57 GB

http://uupload.ir/files/2lo5_2016-11-05_211751.jpg

9.57 GB?
http://www.fileforums.com/showpost.p...7&postcount=27

Mad Max
Input = 31.9 GB > Pzlib2 = 58.8 GB > Srep = 8.65 GB > LZMA2 = 4.27 GB

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -c256m -t8 - -o - <stdin> <stdout>
unpackcmd = pZLib d -t8 - -o - <stdin> <stdout>

http://uupload.ir/files/snzw_2016-11-06_152153.png

Very Good

I Test It For Mad Max

From 31 GB To 4.26 GB

Thank You

thank you very much

Simorq

but that version of reflate, PES 2017 is not on the forum, I think it was created to coliotte

Quote:

Originally Posted by LuzhinE (Post 453611)

thank you very much

Simorq

but that version of reflate, PES 2017 is not on the forum, I think it was created to coliotte

cls-reflate.dll Level 2

Shelwien's Reflate

if that's the temp folder, I had but not compression parameters

Why Would You need reflate When You have Pzlib now !

Quote:

Originally Posted by coliotte (Post 453622)

Why Would You need reflate When You have Pzlib now !

pzlib causes crc errors for pes17 BTW use shelwin reflate (7z & dll file) it works fine for files bigger than 2 GB