Shelwien's Reflate compatible with Freearc
 

Files needed for compression:
cls-reflate.dll
reflate.exe

Files needed for decompression:
cls-reflate.dll
reflate.exe

Drawbacks:
It's actually slower during compression process than normal.
-Haven't checked decompression but it's easy to get 6mb/s.

Advantages:
Shows progress when compressing and decompressing.

where control of level?!

Work Win7 64Bit

Ahh matey, its not even his reflate or he don't even have the source he just tried to make hif and unp files in one file so that freearc could decompress it, because it just support one file input and one file output, I hope you understand, still you should thank him for his try for you all ;)

Progress works fine, its stuck at about 13 MB each time.

Specifically, it stucks when Reflate has I/O Read = Input size. I think that's a problem cause Stdin and Stdout are based upon Read from input stream handle.

Anyways, nice work bro

The problem is from cls, something has to be done for it to work, I know how to fix it but I really don't have time at the moment, my exams are about to start.

yeah! That's right! Good try Razor.

Release 1 uploaded

CLS Fixed

There may be issues
Only ran test on one input

Level 9 is strong compression mate, take zstd, set level 1, how fast is it? set level 22? how fa.. how slow is it?

okay i understood what u r saying, bt it is very fast on Characters.vpp_pc, Rockstar and Deep silver both uses same level i.e. 9

dude, Characters.vpp_pc, just look at the ratio, it can only be a small sector that had deflate streams within the file, let's say the first 20mb only had deflate streams, the rest of the file has nothing to process so it's just copied as it is without any further restoration because it wasn't processed in the first place because it didn't have deflate streams.
When you look at GTAV, firstly, it has maximum compressed streams, it has streams all over the file, just look at inflated ratios as well.

x64w.rpf (893MB) x64w.ref (1.99GB)

FreeArc 0.67 (March 15 2014) Creating Archive: Data.arc
Compressed 1 file, 937,340,928 => 2,140,205,962 bytes. Ratio 228.33%
Compression time: cpu 2.04 sec/real 447.45 sec = 0%. Speed 2.09 mB/s
All OK

FreeArc 0.67 (March 15 2014) Extracting Archive: Data.arc
Extracted 1 file, 2,140,205,962 => 937,340,928 bytes. Ratio 228.33%
Extraction time: cpu 5.51 sec/real 164.90 sec = 3%. Speed 5.68 mB/s
All OK

control of level - very good, thanks! there are plans to add multithreading?

Good good! I have the same size like reflateMT.

I don't plan to add MT feature, it's a lot of work.

OK to the disadvantages, but unlike reflateMT already tested, which advantages? only shows progress when compressing and decompressing?

Well I really don't know, only thing that I did was make it show progress when using freearc, since there were 3 descriptors instead of 2, I just combined them somehow, and that's it.
Plus it gets updates almost everywhere or whenever there is a bug that was detected.
http://nishi.dreamhosters.com/u/rawfilt_v1l.rar
Source: http://encode.ru/threads/1399-reflat...mpressor/page4
This came today, just replace reflate.exe with new one, it will probably work with old one.

OK thanks

Can anybody tell me which relate is perfect to use...
Fast
Great compression
And most reliable...

how it works with inno setup decompression

CLS updated

Changes

-Updated for v1l
-Source included

This is the one to use, very reliable, not sure about speed though. Great compression... nah, what you're looking for is great precompression.

simple straight forward, just add cls and exe in inno, done.

First of all I shall thk you Razor for this awesome work and amazing tools you share with us!

Testing report rawfilt v1l:

I could recompress 2x1.8Gb game files using reflate (v1l)+srep+exe2+delta+lzma... that was pretty fast (30 min)

I could also decompress it ok in asc and it is strangely very fast ...3.6 Gb install in less than 2 min ....!!!! burst speed 200Mb/s and average 30Mb/s! ...very impressive!

backsides/limits:

I could compress one 2.5Gb game file ok but decompression stops at 40% helper for setup still runs for a while and stops as well...no error but does nothing, just stops install

is there any size limitation "per file" being processed? it seems after some testing, that it can compress files bigger than 2Gb but it doesn't decompress them...

thus, I tried to recompress 6 Gb (4x1.5Gb files) and comp went ok but once again decomp/install stops randomly at either 14/15% ...

Any repack test ouptut under 4 Gb, with files smaller than 2Gb installs ok with asc setup,
other than that it just fails...

Reports don't often talk about big files testing... hope it will help other repackers!

Thank You Pro

How To Benchmark I took the ReflateMT version with a 122MB input.
Here I've tested both versions "l & k" with CLS support ...
Between the two versions reflate with CLS "l & k" to understand, there is no substantial difference, a difference between the reflate CLS version and the version ReflateMT, in favor of the latter in terms of speed, and with a higher threshold of reflate.
The reflate version CLS will be continuously updated, hopefully soon in new improvements.

You're on fire with benchmarks, wanna bench something?

This is the one to use, very reliable, not sure about speed though. Great compression... nah, what you're looking for is great precompression.
[QUOTE]
Yeah forgot to type pre
But the speed is not good. Is there any way to decompress after precompressed file with 50MB/s or more..... Brewster last time I used reflate it took allott of time

50mb/s? be realistic.
Take a certain input, compress it using these settings and tell me what speed you're getting.
If the input has a lot of streams or 99% of it is covered by deflate streams, that's roughtly the speed you're going to get.

multi threading is the only way for more speed.

Ok I tried it
On hitman silent assassin's
Game size 497 mb
Used this reflate it makes this to 1.23 gb
But while compressing it with 7zip ultra, winrar with maximum, compression ratio is 90℅ why??
Then why I use this...
I tried gozarck reflate it recompress it to 1.23 gb and compressed it to 185 mb no error........

2. While decompression it starts quickly to 14 or 15℅ and stuck. Again try to decompress it extracts it. But I tried to decompress again but it stucks again same like first time.....

My arc ini
[External compressor:reflate]
header = 0
packcmd = reflate -process $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = reflate -restore - - <stdin> <stdout>

Packing batch
del /q data.arc
arc.exe a -ep1 -r -ed -w.\temp -s4096m -mreflate data.arc "pack\*"
pause

Any body

Did anyone know why it stucks on asc

1-you shouldn't put reflate settings in arc.ini as it has its own cls ,I mentioned that in EGYcompressor
http://i.imgur.com/QP6e3dG.jpg
2- I used this reflate many times on large files (more than 2GB) & compression & decompression was success
3- try from cmd to extract the archive & see if it still stuck at 15%

Yeah if I use batch file there is no problem...
But when I put this in asc in arc extract with resources all required it stuck Everytime.....

did you try to delete reflate from arc.ini for ASC ??
if yes & the same happened try on normal inno script if the same you have to make inno launch a bat file after extracting which will unpack reflated files :(

Hello razor. I tested on .sds file of Mafia 3 (32mb). Win 7 64 bit on fx 8350.

Reflate level 1:
FreeArc 0.67 (March 15 2014) creating archive: data\data1l1.arc
Compressed 1 file, 33,397,949 => 56,319,667 bytes. Ratio 168.63%
Compression time: cpu 0.05 sec/real 14.87 sec = 0%. Speed 2.25 mB/s
All OK

Reflate level 9:
FreeArc 0.67 (March 15 2014) creating archive: data\data1l9.arc
Compressed 1 file, 33,397,949 => 50,587,917 bytes. Ratio 151.47%
Compression time: cpu 0.03 sec/real 17.82 sec = 0%. Speed 1.87 mB/s
All OK

This is ok but with level 9 it is slower and a lower ratio. Have you got an idea?
With other reflate versions there was marked how many stream found.
It seemed useful so I do an idea if the result was good.
I tried on xp 32bit but work only 64bit, pity.
Very thanks for job Razor!

I compresed files successfully with above cls +reflate but i am getting extraction problems... Sometime the extraction stucks at 5 sometimes 50... any suggestions please...
Any other reflate with cls or stdin stdout available please do tell me... I also heared about reflateMt, how can i find this please... I need it badly pleease...

@Lucas65, for your information the level between 1-9 isn't a compression level. In Reflate things work different from the level setting. Mostly you get better compression and time at level 6, sometime level 9 or even level 1. You must try a different level for every different kind of file. God Speed.

I'm without luck with a specific game. Pzlib v3 just crashes during recompression or cant find deflate streams in the game. CLSreflate find the streams and recompress fine with good final ratio after srep+lzma. The problem than during decompression it fails.

Tried reflateMT but it stuck always at 59% when compressing.

This fails during decompression?

CLSreflate fails in decompression, yes. PZlib cant find headerless streams during compression. I think the error for reflate in decompression is a bug of reflate itself with the file type.

Game is from PS3.

ERROR: CRC failed in "unpacked\c0000\c0000_a2x.anibnd". File is broken.

If i try compressing every file from the folder individually then decompression works fine, but when i just compress all folder together with reflate it cause CRC error in decompression.