pZLib version 2 (Fastest zlib/deflate precompressor)

[Simorq]

Thanks to Razor12911

Test
x64q.rpf (2.62GB) reflate+Srep+Lzma (1.14GB)
x64q.rpf (2.62GB) pZLib2+Srep+Lzma (1.14GB)

[Mini]

big thanks!!!

[zouzouni]

Big thank you for this nice piece of coding!

After testing between reflate of yours and this pzlib2, I wonder if this is the unlocked version of your reflate (without the need for -s2000m option in arc)

I get same size (few bytes differences) when using pzlib2 or reflate...

but endeed , compression and decompression is way faster with pzlib2!
no error so far with SkyrimSE, ratio is ... correct...
Thx again Razor!

and ....I already promote your work all around me...you're our Master bro!

[zouzouni]

Big thank you for this nice piece of coding!

After testing between reflate of yours and this pzlib2, I wonder if this is the unlocked version of your reflate (without the need for -s2000m option in arc)

I get same size (few bytes differences) when using pzlib2 or reflate...

but endeed , compression and decompression is way faster with pzlib2!
no error so far with SkyrimSE, ratio is ... correct...
Thx again Razor!

and ....I already promote your work all around me...you're our Master bro!

[Simorq]

No Work Act of Aggression - Reboot Edition .dat file

[LuzhinE]

thank you very much

[Lucas65]

Thank you very much Razor for pzlib! I tested on this file: x64f.rpf (981mb)

Only Shelwien's reflate in level 4:
Compression:
FreeArc 0.67 (March 15 2014) creating archive: data\datareflate.arc
Compressed 1 file, 1,028,757,504 => 2,304,942,259 bytes. Ratio 224.05%
Compression time: cpu 1.92 sec/real 471.78 sec = 0%. Speed 2.18 mB/s
All OK
Decompression:
FreeArc 0.67 (March 15 2014) extracting archive: datareflate.arc
Extracted 1 file, 2,304,942,259 => 1,028,757,504 bytes. Ratio 224.05%
Extraction time: cpu 6.44 sec/real 205.69 sec = 3%. Speed 5.00 mB/s
All OK

Only pzlib version2:
Compression:
FreeArc 0.67 (March 15 2014) creating archive: data\datapzlib.arc
Compressed 1 file, 1,028,757,504 => 2,226,261,282 bytes. Ratio 216.40%
Compression time: cpu 0.84 sec/real 108.17 sec = 1%. Speed 9.51 mB/s
All OK
Decompression:
FreeArc 0.67 (March 15 2014) extracting archive: datapzlib.arc
Extracted 1 file, 2,226,261,282 => 1,028,757,504 bytes. Ratio 216.40%
Extraction time: cpu 0.92 sec/real 86.33 sec = 1%. Speed 11.92 mB/s
All OK

Reflate maybe find more stream but pzlib much faster!
Thanks again and good work!

[oltjon]

I had problems with file.ttarch2 Telltale Games

does not work

[FitGirl]

Originally Posted by oltjon

I had problems with file.ttarch2 Telltale Games

does not work

Cause they are encrypted + new games (Batman i.e.) use Oodle, not zlib

[felice2011]

Pzlib_v2 VS ReflateMT...Under-Test *.rpf File

Only pzlib_v2

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 05/11/2016 16:59:09
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

        pzlib_v2         981,1°MB           2,07°GB       210,99%               1°File    00:01:07:929   e -m2 -st50p -t8 -ct8 -c32m -b512k -v -o 
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Decompressed Archive Completed At -↓- 05/11/2016 17:00:32
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ DECOMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

        pzlib_v2          2,07°GB          981,1°MB        47,40%               1°File       00:00:53:81                           d -t8 -o 
=============================================================================================================================================

Only ReflateMT

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 05/11/2016 17:10:22
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

         reflate         981,1°MB           2,08°GB       212,01%               1°File    00:04:56:956                  -process -c9 -t8  
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Decompressed Archive Completed At -↓- 05/11/2016 17:29:48
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ DECOMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

         reflate          2,08°GB          981,1°MB        47,17%               1°File       00:16:08:60                  -restore -c9 -t8  
=============================================================================================================================================

Excellent compression and decompression speed compared to reflateMT...
I was very pleased, the double embedded option ,"M1 = zlib streams" and "M2 = deflate streams", with the possibility of avoiding the use of "Antiz & Zlib_Jo" for only zlib streams.
As you see here no one does not abandon you, in 7 hours two pages of thanks, of tests and appreciations on yours work.
Personally I think it's the best job of zlib...Bravo!!!

[oltjon]

earn to die 2

C:\Users\oLtJoN\Desktop\reflate_1500>arc.exe a -ep1 -r -ed -w.\temp -s4096m -mreflate data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressing 86 files, 200,374,281 bytes. Processed  54%
Compressing 200,374,281 bytes with reflate -process $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
Reflate (Free Edition) created by Razor12911
Searching for deflate streams
Splitting streams 91/91
Processing extracted deflate streams
Gathering processed streams
Merging stream 183/183
Removing temporary files

Processed: 44.0 MB/191 MB (23%)
New size: 961 MB instead of 191 MB (503%)

Errorlevel=0
100%

C:\Users\oLtJoN\Desktop\reflate_1500>del /q unpacked\

C:\Users\oLtJoN\Desktop\reflate_1500>arc.exe x -w.\ -dpunpacked\ data.arc
FreeArc 0.67 (March 15 2014) extracting archive: data.arc
Extracting 86 files, 200,374,281 bytes. Processed   0%SSEInv             S├             T├             s          Extracted 86 files, 1,007,971,708 => 200,374,281 bytes. Ratio 503.04%
Extraction time: cpu 0.73 sec/real 32.87 sec = 2%. Speed 6.10 mB/s
All OK

C:\Users\oLtJoN\Desktop\pZLib_v2>del /q data.arc
Impossibile trovare C:\Users\oLtJoN\Desktop\pZLib_v2\data.arc

C:\Users\oLtJoN\Desktop\pZLib_v2>arc.exe a -ep1 -r -ed -s; -w.\temp -mpzlib data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 86 files, 200,374,281 => 577,169,086 bytes. Ratio 288.05%
Compression time: cpu 0.27 sec/real 13.66 sec = 2%. Speed 14.67 mB/s
All OK

C:\Users\oLtJoN\Desktop\pZLib_v2>del /q unpacked\

C:\Users\oLtJoN\Desktop\pZLib_v2>arc.exe x -w.\ -dpunpacked\ data.arc
FreeArc 0.67 (March 15 2014) extracting archive: data.arc
Extracting 86 files, 200,374,281 bytes. Processed   0%SSEInv             S├             T├             s          Extracted 86 files, 577,169,086 => 200,374,281 bytes. Ratio 288.05%
Extraction time: cpu 0.44 sec/real 6.59 sec = 7%. Speed 30.39 mB/s
All OK

[FitGirl]

Thanks, mate. Will test it and if it's as good as you describe will use it with credits given, ofc.

[Razor12911]

Originally Posted by FitGirl

Thanks, mate. Will test it and if it's as good as you describe will use it with credits given, ofc.

It's not that good, not proud of this, I want more speed while getting same ratio as reflate.

[Amsal]

Originally Posted by Razor12911

It's not that good, not proud of this, I want more speed while getting same ratio as reflate.

Hmm then what about gpu usage now?

[Simorq]

Mad Max
Input = 31.9 GB > Pzlib2 = 51.8 GB > Srep = 14 GB > LZMA2 = 9.57 GB

9.57 GB?
http://www.fileforums.com/showpost.p...7&postcount=27