View Full Version : Helping thread :)
So guys I just registered to this community, but I am a old visitor and old member on encode.ru. I made this thread because as I seen many people spamming the forum for every game, so everything related to compression or decompression will be answered here :)
Razor12911
20-09-2016, 20:14
It's useless if you ask me, we been making many threads for asking questions, even some sticky threads for those but people will start new threads, even asking questions for stuff that has been already asked before.
It's useless if you ask me, we been making many threads for asking questions, even some sticky threads for those but people will start new threads, even asking questions for stuff that has been already asked before.
LOL, so peoples are so dumb in this forum that they lost there one(eyes) of the 5 senses given to them by god?
Razor12911
20-09-2016, 20:28
What do you mean they lost one? they lost all of them.
Just kidding :D, but they make you wonder sometimes.
Anyways, welcome to Fileforums.
What do you mean they lost one? they lost all of them.
Just kidding :D, but they make you wonder sometimes.
Anyways, welcome to Fileforums.
Yeah, I seen some hardworking guys too, Like Gozark and altef_4 and yener90 also and you too, anyways, wherever I join, I always share my knowledge so that I couldn't be a leecher there :)
Razor12911
20-09-2016, 20:36
Maybe change will start with you, maybe people will start asking here.
Maybe change will start with you, maybe people will start asking here.
yeah, hope so, I will try my best ;)
Edit - Currently learning more new things from Shelwien, he is a nice guy too and my old friend.
Razor12911
20-09-2016, 21:32
Ok you and Shelwien are old friends, I got a question then.
Where is he from because it doesn't seem like he is from this planet?
Ok you and Shelwien are old friends, I got a question then.
Where is he from because it doesn't seem like he is from this planet?
LOL hahaha, really sometime, I ask this question myself too :P
Maybe u should add some commonly asked question at the first post..
RamiroCruzo
21-09-2016, 00:59
Yeah..Whenever we ask him how he's so good, he tells someone else's name and say he rarely does the thing...Leaving us with the question, "Is he from future or something? -_-"
Anyways, welcome to the forum matey ^_^
Razor bro, had a conflict with Val and blah blah stuff, so am gone....So don't worry.. ;)
LoL Nice Thread MAK.......good to see you here & You Should add some Basic Questions to your thread also
felice2011
21-09-2016, 05:24
I see more normal opening a thread to call for help, although the answers may already be given in other threads, that the new user who opens a new thread just to chat, so there a dedicated section here....http://www.fileforums.com/forumdisplay.php?f=29;)
Maybe u should add some commonly asked question at the first post..
Yeah..Whenever we ask him how he's so good, he tells someone else's name and say he rarely does the thing...Leaving us with the question, "Is he from future or something? -_-"
Anyways, welcome to the forum matey ^_^
Razor bro, had a conflict with Val and blah blah stuff, so am gone....So don't worry.. ;)
LoL Nice Thread MAK.......good to see you here & You Should add some Basic Questions to your thread also
I see more normal opening a thread to call for help, although the answers may already be given in other threads, that the new user who opens a new thread just to chat, so there a dedicated section here....http://www.fileforums.com/forumdisplay.php?f=29;)
Thanks everyone for the welcome :) I will add some basic questions answer too in my thread soon, actually currently busy with my exams and it will end by next month ;) Anyways, nice to see this community. Have Got any problem? Pm me, I will try to resolve it
ChronoCross
21-09-2016, 11:42
Welcome MAK and thanks for made this thread!.
i have a question.
Some files in games have 98% or 99% of entropy like media or compressed data.
sometimes this data are packed and imposible to extract and rebuild without any errors.
So just for save time, there is any way to sort those files with 98% or 99% of entropy in a list to use a store method of compresion.
I mean like a entropy detector or something like that?
i ask you this because we can easy detect media files by extension (ogg, bik, mp3) but in some cases this files are already packed, so extension doesn´t work here.
thanks
Welcome MAK and thanks for made this thread!.
i have a question.
Some files in games have 98% or 99% of entropy like media or compressed data.
sometimes this data are packed and impossible to extract and rebuild without any errors.
So just for save time, there is any way to sort those files with 98% or 99% of entropy in a list to use a store method of compression.
I mean like a entropy detector or something like that?
i ask you this because we can easy detect media files by extension (ogg, bik, mp3) but in some cases this files are already packed, so extension doesn´t work here.
thanks
hmmm, whenever I get that type of files, then I firstly try Offzip by Luigi to see if their is any raw files or not on that specific files, if their is, then I try reflate with them or you can you precomp also after knowing the offzip location of streams ;) , and compress them, if not and you don't even get any rebuilder or extractor, then you could even give a try to zstd, sometime, its worth on these type files ;)
ChronoCross
21-09-2016, 12:13
Thank you!. I´ll try
ChronoCross
22-09-2016, 18:40
I was download smoke.exe by bulat. This tool can calculate the entropy and also have the source in c .
I need a script in batch to sort the files with 95% and 100% of entropy to txt file list.
This list of files is to using in arc.exe to storing.
So any ideas?
I have my game compressed with -mpzlib+srep+lzma:max data.arc
and i want to implement the compression with asc template. Please help me, what to do next??
I was download smoke.exe by bulat. This tool can calculate the entropy and also have the source in c .
I need a script in batch to sort the files with 95% and 100% of entropy to txt file list.
This list of files is to using in arc.exe to storing.
So any ideas?
Ahh matey, that is nice, but its old too, anyways, am not much good in writing batch files, but I will try to write one, currently having exams also :(
I have my game compressed with
and i want to implement the compression with asc template. Please help me, what to do next??
Go to deployment>>Setup Resources and then click on add pzlib.exe and pzlib64.exe
and add your arc.ini also which you used for compression ;)
I was download smoke.exe by bulat. This tool can calculate the entropy and also have the source in c .
I need a script in batch to sort the files with 95% and 100% of entropy to txt file list.
This list of files is to using in arc.exe to storing.
So any ideas?
Btw, why not you use mask with arc?? Just make a new mask in arc.groups and add that file extension, and add that mask in your command line, and then add a no compression method? :P
Mak I want to ask you what's ur opinion or ur experience of / about
lzma & xz (Lzma2) & 4x4:lzma which do you prefer to use for a good ratio & also good decompression speed
I also see most of the repackers doesn't use huge dictionary / lzma max settings !! However in most of the cases it won't help u much the difference will be couple of megabytes :)
can you explain more about lzma :)
I was download smoke.exe by bulat. This tool can calculate the entropy and also have the source in c .
I need a script in batch to sort the files with 95% and 100% of entropy to txt file list.
This list of files is to using in arc.exe to storing.
So any ideas?
you will need a huge batch file to handle this , however i'm not sure about bat , i don't think it can be done with batch
Mak I want to ask you what's ur opinion or ur experience of / about
lzma & xz (Lzma2) & 4x4:lzma which do you prefer to use for a good ratio & also good decompression speed
I also see most of the repackers doesn't use huge dictionary / lzma max settings !! However in most of the cases it won't help u much the difference will be couple of megabytes :)
can you explain more about lzma :)
Aswadd if you ask my opinion, I mostly prefer xz, the reason is, if its offers a very fast decompression speed and it even gives much better ratio sometimes, and normally it could be bigger by 1-2mb in comparison to lzma, but for 1-2mb, I could go for xz because it has faster installation time.... Books are calling me... Talk to you later ;)
Aswadd if you ask my opinion, I mostly prefer xz, the reason is, if its offers a very fast decompression speed and it even gives much better ratio sometimes, and normally it could be bigger by 1-2mb in comparison to lzma, but for 1-2mb, I could go for xz because it has faster installation time.... Books are calling me... Talk to you later ;) yes i know that :D but what if we edited the comparison to be like this
4x4:lzma vs xz
I think 4x4:lzma will be faster & better but in the end it costs more ram & cpu :)
yes i know that :D but what if we edited the comparison to be like this
4x4:lzma vs xz
I think 4x4:lzma will be faster & better but in the end it costs more ram & cpu :)
Matey, there are varieties in every thing, look you could get better speed also, but in the cost of high CPU usuage, but I normally prefer xz, I don't say lzma is bad, but just I'll like xz speed and ratio with its normal CPU usage, am on mobile and couldn't tell you everything now.
yes i know that :D but what if we edited the comparison to be like this
4x4:lzma vs xz
I think 4x4:lzma will be faster & better but in the end it costs more ram & cpu :)
IMOH, if u r planning to use things with freearc then go with fazip
this will give u better compression speed and u don't even need fazip.exe to decompress and even better compression speed many times
[External compressor:rep,lzma,4x4,delta,exe]
header = 0
mem = 730
packcmd = fazip -i0 compress:{compressor}{:option} <stdin> <stdout>
unpackcmd = fazip -i0 decompress:{compressor}{:option} <stdin> <stdout>
IMOH, if u r planning to use things with freearc then go with fazip
this will give u better compression speed and u don't even need fazip.exe to decompress and even better compression speed many times
Yeah that is princegupta, but he actually asked me the difference, and I was on mobile and still on mobile, BTW thanks for contributing, this thread is made for sharing of knowledge only ;) I am good to see that people are stepping forward to make the forum clean :)
Blade2239
23-09-2016, 10:21
anybody playing GTA V PC game.
I am playing this and my problem is whenever I open GTAlauncher.exe game starts and loads directly into game no main menu..
I tried landing page and turned it on and off many times restarted the PC nothing worked....
now how to do it anyone have knowledge about it
IMOH, if u r planning to use things with freearc then go with fazip
this will give u better compression speed and u don't even need fazip.exe to decompress and even better compression speed many times
yes i'm using fazip for 4x4 :D Anyway it took 100% of my cpu usage & a lot of ram but it's 4/5 time faster than normal lzma & 2x faster than xz :D Eventually it's a multi thread tool :)
I just want to ask can i control the cpu usage of 4x4 ?? or even making it take less cpu & ram :( i can't add more than 120mb dictionary to it :( , any way to limit 4x4 ??
ChronoCross
23-09-2016, 16:26
4x4 doesn't have dict option do you mean blocks options. 4x4:b1g:lzma
ChronoCross
23-09-2016, 20:55
Recently i was compiled zlib_jo for test.
Compressing 1,208,218 bytes with precomp -intense0 -cn +srep+exe+delta+lzma = 1.105.920 bytes
Compressing 1,208,218 bytes with zlib_jo
+srep+exe+delta+lzma = 1.114.112 bytes
Compressing 1,208,218 bytes with 7zR\7zR.exe a -t7z -m0=reflate:x6 +srep+exe+delta+lzma = 1.019.904 bytes
Compressing 1,208,218 bytes with reflateMT.exe -process -c6 -t6 +srep+exe+delta+lzma = 692.224 bytes
so reflateMT is the best! Good work guys!
4x4 doesn't have dict option do you mean blocks options. 4x4:b1g:lzma
no i mean it take too much ram so that I can't add a larg dictionary because there is no much free ram :(
lzma:a1:mfbt4:d300m:fb128:mc999:lc8
4x4:lzma:a1:mfbt4:d100m:fb128:mc999:lc8
no i mean it take too much ram so that I can't add a larg dictionary because there is no much free ram :(
lzma:a1:mfbt4:d300m:fb128:mc999:lc8
4x4:lzma:a1:mfbt4:d100m:fb128:mc999:lc8
Matey, as I told you, Overall if you see by system usage and decompression speed, and multi threaded wise, then I vote for xz. ;) Its my opinion and way to see.
if u going with xz then
i advise u left the Freearc and take 7zip route
Matey, as I told you, Overall if you see by system usage and decompression speed, and multi threaded wise, then I vote for xz. ;) Its my opinion and way to see.
yes u are right :D but if the dictionary won't effect so much i'll go with 4x4 :D
ChronoCross
26-09-2016, 10:36
I need some help to configure pzstd for arc.ini.
felice2011
26-09-2016, 10:43
I need some help to configure pzstd for arc.ini.
:confused: what's the problem. :confused:
ChronoCross
26-09-2016, 12:15
i have this code:
[External compressor:pzstd]
header = 0
packcmd = pzstd.exe $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp {options}
unpackcmd = pzstd.exe -d -f $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp
the thing is to use numbers of threads (-n 6) in arc.exe -mpzstd:n 6:p:u:19
the code in arc.exe does not accept spaces and that`s is the problem because i need to write this option in arc.ini
[External compressor:pzstd]
header = 0
packcmd = pzstd.exe $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp -n 6 {options}
unpackcmd = pzstd.exe -d -f $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp
where i can get the source code only to chage this parameter -n (the space thing)
thanks
----------------------------------------------------------------------
Edited
Problem solved with quotation marks!
-mpzstd:"n 6":p:u:19
sorry for spend your time :o
felice2011
26-09-2016, 13:06
I have not had this problem with Cmd use, used individually or with FA support, awesome speed and good compression.
pzstd -u -22 -f -n 4 -p
[External compressor:pzstd]
;header = 0
packcmd = pzstd --ultra -22 -f -n 8 -p $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pzstd -d -f -n 8 -p $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp
OK you've solved.;)
Have anyone an idea to Multithread lzma ?? I already know 4x4 but it took 100% of the cpu so is their a way to multi thread it ?? :D also i saw something named lzmax64.exe have anyone idea about this ??
lzma64 is hard on system resources if you dont have a decent machine.and it slow as F**K :P not worth looking at rather stick with normal lzma.
If it aint broken dont fix (stick with what works)
but by all means test new things but dont fix what works :P
felice2011
26-09-2016, 14:01
@aswadd ...lzmax64.exe ... is the 64 version should use all the processor cores.
Also uses this scheme, to move in the right direction for optimal settings.
a=[0|1] : ( Defaul 1 ) : Sets compressing mode
a=[0|1] : Sets compression mode: 0 = fast, 1 = normal. Default value is 1.
mf={MF_ID} : ( Defaul 1 ) : Sets Match Finder
mf={MF_ID} : Sets Match Finder for LZMA. Default method is bt4. Algorithms from hc* group don't provide a good compression ratio, but they often work pretty fast in combination with fast mode (a=0). Memory requirements depend on dictionary size (parameter "d" in table below).
MF_ID Memory Description
bt2 d*9.5 + 4 MB Binary Tree with 2 bytes hashing.
bt3 d*11.5 + 4 MB Binary Tree with 3 bytes hashing.
bt4 d*11.5 + 4 MB Binary Tree with 4 bytes hashing.
hc4 d*7.5 + 4 MB Hash Chain with 4 bytes hashing.
Note: Your operation system also needs some amount of physical memory for internal purposes. So keep at least 32MB of physical memory unused.
d={Size}[b|k|m] : ( Defaul 24 ) : Sets Dictionary size
d={Size}[b|k|m] : Sets Dictionary size for LZMA. You must specify the size in bytes, kilobytes, or megabytes. The maximum value for dictionary size is 1 GB = 2^30 bytes. Default values for LZMA are 24 (16 MB) in normal mode, 25 (32 MB) in maximum mode (-mx=7) and 26 (64 MB) in ultra mode (-mx=9). If you do not specify any symbol from the set [b|k|m], the dictionary size will be calculated as DictionarySize = 2^Size bytes. For decompressing a file compressed by LZMA method with dictionary size N, you need about N bytes of memory (RAM) available.
fb={NumFastBytes} : ( Defaul 32 ) Sets number of Fast Bytes for Deflate encoder.
fb={NumFastBytes} : Sets the number of fast bytes for the Deflate/Deflate64 encoder. It can be in the range from 3 to 258 (257 for Deflate64). Usually, a big number gives a little bit better compression ratio and a slower compression process. A large fast bytes parameter can significantly increase the compression ratio for files which contain long identical sequences of bytes.
mc={N} : ( Defaul 32 ) : Sets Number of Cycles for Match Finder
mc={N} : Sets number of cycles (passes) for match finder. It can be in range from 0 to 1000000000. Default value is (16 + number_of_fast_bytes / 2) for BT* match finders and (8 + number_of_fast_bytes / 4) for HC4 match finder. If you specify mc=0, LZMA will use default value. Usually, a big number gives a little bit better compression ratio and slower compression process. For example, mf=HC4 and mc=10000 can provide almost the same compression ratio as mf=BT4.
lc={N} : ( Defaul 32 ): Sets number of Literal Context bits - [0, 8]
lc={N} : Sets the number of literal context bits (high bits of previous literal). It can be in range from 0 to 8. Default value is 3. Sometimes lc=4 gives gain for big files.
:)
lzma64 is hard on system resources if you dont have a decent machine.and it slow as F**K :P not worth looking at rather stick with normal lzma.
If it aint broken dont fix (stick with what works)
but by all means test new things but dont fix what works :P thnx :D now i'm pessimistic :D maybe i'll end up with xz
Best external compressor for *.iwd files for Call of Duty Modern Warfare???
guys I have a little problem with zcm , it doesn't understand the paths like this
I:\Shadow of mordor\Repack\ .. even i addedd the quotation mark like this
"I:\Shadow of mordor\Repack\ttt" or I:"Shadow of mordor"\Repack\ttt
================================================== =======
I:\Shadow of mordor\Repack\ttt>zcmx64 a -m8 archive.me "I:\Shadow of mordor\Repa
ck\ttt"
ZCM Archiver v.0.93
Archive is I:\Shadow of mordor\Repack\ttt\archive.me
WARNING: No files Compressed 0 bytes to 0 bytes
I:\Shadow of mordor\Repack\ttt>zcmx64 a -m8 archive.me I:"Shadow of mordor"\Rep
ack\ttt
ZCM Archiver v.0.93. Demo version.
copyright (C) 2011-2015 By Nania Francesco Antonio
Archive is I:\Shadow of mordor\Repack\ttt\archive.me
WARNING: No files Compressed 0 bytes to 0 bytes
what happens if you put the files in just one folder without quotation marks
eg: c:\shadowofmordor
try not leave any spaces i had a very similar issue a few days ago with a compressor just cant remember which one.
Razor12911
29-09-2016, 16:57
Best external compressor for *.iwd files for Call of Duty Modern Warfare???
Try using this.
http://fileforums.com/showthread.php?t=98317
ChronoCross
29-09-2016, 17:00
First read the help from the zcm.exe.
and then use -r (recursive path)
ZCM a -r -m8 -t6 path\*
what happens if you put the files in just one folder without quotation marks
eg: c:\shadowofmordor
try not leave any spaces i had a very similar issue a few days ago with a compressor just cant remember which one. it works fine only folders which have spaces doesn't work on it :confused:
First read the help from the zcm.exe.
and then use -r (recursive path)
ZCM a -r -m8 -t6 path\*
doesn't work :( & -r is just for compressing files in subdirectories
& another question
zcmx64 a -s -r -v -m8 -t2 shadowofmordor.test I:\Shadow of mordor\Repack\test\
the output archive will be I:\shadow\ :confused::confused::confused:
also I can't extract to a folder which have spaces
doesn't work :( & -r is just for compressing files in subdirectories
& another question
zcmx64 a -s -r -v -m8 -t2 shadowofmordor.test I:\Shadow of mordor\Repack\test\
the output archive will be I:\shadow\ :confused::confused::confused:
also I can't extract to a folder which have spaces
checkout this - http://www.fileforums.com/showthread.php?p=452248
Specially made for you ;)
ChronoCross
01-10-2016, 08:42
I need help here in 7z.exe
i want to know what means this code-mb0s0:1 -mb0s1:2 or -mb1s0:3 -mb1s1:4
what is the benefits to use this code in the chain of methods?
7z.exe a -t7z -bd -mcrc=0 -m0=bcj2 -m1=lzma:d26:fb64 -m2=lzma:d19:fb64 -mb0s0:1 -mb0s1:2 ^-m3=lzma:d19:fb64 -m4=lzma:d19:fb64 -mb1s0:3 -mb1s1:4 ^ "data" test.7z
I have a small question ... what is the fastest program for storing files ? 7z or arc or winrar
& can I use srep on Multi files ? or srep is used only on 1 file ??
Hello Aswadd.
All those tools can store any folder quickly. the speed depends of your HDD.
About srep maybe you already know the answer :)
Unpack Code:
@echo off
for %%f in (*.001,*.7z) do 7z.exe x -bd -y "%%f" -so | srep64.exe -d - - | 7z.exe x -bd -y -si -ttar >nul
pause >NUL
Ichiraku2001
03-10-2016, 07:54
Yeah..Whenever we ask him how he's so good, he tells someone else's name and say he rarely does the thing...Leaving us with the question, "Is he from future or something? -_-"
Anyways, welcome to the forum matey ^_^
Razor bro, had a conflict with Val and blah blah stuff, so am gone....So don't worry.. ;)
starting off with what this thread is xD
how did u compress the crew to 12.3 GB ;_;
Hello Aswadd.
All those tools can store any folder quickly. the speed depends of your HDD.
About srep maybe you already know the answer :)
Unpack Code:
yeb , I know other methods to make srep work on all files but I thought there was an option to do that :D
I used this :)
dir /s /b *.pcf |ppx2 -P 20 -L 1 srep64 -m3f -a2 "{}" not for speed :D Just for rematching is taking much time
:D:D 20X Srep64 // 20 rematching
or
for %%C in (dir /s /b) do (
srep64 -m3f -a2 %%C
)
What is the best method to compress Mafia III, for a smaller size ?
What is the best method to compress Mafia III, for a smaller size ?
I didn't repacked it, but as far as by seeing game data, I think it will use reflate+srep+lzma in simple way. Reflate will work on .sds files and except other audio and other data will be repacked with srep+lzma ;)
for some time that I try to force myself to figure out how to make the Russians repacker to compress so much the flie.ttarch2. tell tale games.
there someone who knows something ??
Thank you in advance
I didn't repacked it, but as far as by seeing game data, I think it will use reflate+srep+lzma in simple way. Reflate will work on .sds files and except other audio and other data will be repacked with srep+lzma ;)
thanks :D
ChronoCross
09-10-2016, 09:59
What about freearc next?. I don't undertand very well lua options but what if we can add some lua options and use razor's split tool?.
you have a good chance to learn Lua and apply your abilities to FA :) if you know programming, learning Lua is just one day. if you don't know it at all, learning Lua may be easiest start
for some time that I try to force myself to figure out how to make the Russians repacker to compress so much the flie.ttarch2. tell tale games.
there someone who knows something ??
Thank you in advance rebuilding game data files :) so basically go and find some software to extract ttarch2 files content
then compress the output then when unpacking rebuild the game data again :D
ChronoCross
09-10-2016, 20:20
Thanks Bulat! I started to learn LUA. I download zerobrane studio is a free IDE for LUA language.
Guys why not making a list about best methods to every Game engine like unreal engine / cry engine 3 / unity3d engine :D It will help a lot
Guys I packed some files using
7zip tar | srep | 7zip_lzma (don't remember exact command)
now the problem is that while i am extracting it via
7za e -txz -y data3.bin -so | srep64 -d -hash- -s -- | 7za x -ttar -y -si -oData3\ *.out *.raw -r
the pipe automatically closes after extracting a file of 4083387777 bytes(3.80Gb)
i try many different commands bt problem still persists
any help please...
Razor12911
11-10-2016, 00:44
If pipe closes automatically then one of the programs either found an EOF early or there was an error, you must check process exit code of each program to find out which one failed.
-Either way, you can just normally unpack everything without the use of standard streams, you'll figure out which program is letting you down.
finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each
this is what 7zip has to say
7za x Data3.tar -oData3\
7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
Extracting archive: Data3.tar
ERRORS:
Headers Error
WARNINGS:
There are data after the end of archive
--
Path = Data3.tar
Type = tar
ERRORS:
Headers Error
WARNINGS:
There are data after the end of archive
Physical Size = 4083388416
Tail Size = 14341385728
Headers Size = 512
Code Page = UTF-8
Archives with Errors: 1
Warnings: 1
Open Errors: 1
i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.
Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?
ChronoCross
12-10-2016, 19:15
--Freearc NEXT
iam a master of LUA code... Naaaaaaa. LOL
in fa.ini add
onArchiveDone(function ()
print("Time Lapsed:",os.clock(),"Sec.")
end)
--Razor's split tool
onArchiveDone(function ()
os.execute('split s data.arc 10') --number 10 is 10mb
end)
need to learn how to add external compressors like srep or nanozip
Razor12911
12-10-2016, 19:22
finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each
this is what 7zip has to say
i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.
Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?
Well srep on default, if you're using stdio, it has a setting of expecting no more than 25GB/25 600MB from stdin.
"-sBYTES: explicitly specify filesize (for compression from stdin), default 25gb"
All you need to do is guess or have an estimation of your input and set a value, to be safe, set a high value
-s50gb or whatever, just know that this increases memory allocation so also make sure you have enough memory to allocate such, plus for big inputs, you're better off using x64 version of srep to avoid x86 memory allocation barrier.
--Freearc NEXT
iam a master of LUA code... Naaaaaaa. LOL
in fa.ini add
need to learn how to add external compressors like srep or nanozip
Is there a need to learn LUA code?
-Never mind.
romulus_ut3
13-10-2016, 02:35
How about a thread dedicated to "secrets of the trade" where people will share their methods to achieve the best possible compression/de-compression methods for specific titles that can be understood even by a dummy? I know Razor12911 shared his methods for GTA V and Mad Max IIRC. Lets start with "The Witcher 3 The Wild Hunt Game Of The Year Edition" perhaps? I'm trying to compress this title into archives that'll fit into DVD5. I don't really require a fancy installer or anything, just a simple executable that'll let me extract the game's archives where I want (I believe Razor12911 has an archiver like this which I've used with great success in the past) I know that finding the best compression method for a certain set of files requires trial and error but we're all here to learn, right?
Vapor028
13-10-2016, 03:26
Hello everyone.
I'm new to this Forum and have one or two Questions.
Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
Razor12911
13-10-2016, 03:47
How about a thread dedicated to "secrets of the trade" where people will share their methods to achieve the best possible compression/de-compression methods for specific titles that can be understood even by a dummy? I know Razor12911 shared his methods for GTA V and Mad Max IIRC. Lets start with "The Witcher 3 The Wild Hunt Game Of The Year Edition" perhaps? I'm trying to compress this title into archives that'll fit into DVD5. I don't really require a fancy installer or anything, just a simple executable that'll let me extract the game's archives where I want (I believe Razor12911 has an archiver like this which I've used with great success in the past) I know that finding the best compression method for a certain set of files requires trial and error but we're all here to learn, right?
Get a deflate precompressor, you can use this one.
http://fileforums.com/showthread.php?t=98317
srep then lzma, I think that's the method that needs to be used to get an output of something like 25-30GB.
Hello everyone.
I'm new to this Forum and have one or two Questions.
Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
-Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
Unfortunately no.
-next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
it sometimes all depends on what you're compressing. 4x4 allows a compressor to run in parallel mode, meaning more speed at a cost of ratio. 4x4 splits data into blocks then runs a chosen compressor, this block splitting loses you ratio, but like I said it all depends on what you're compressing, if its something that can be compressed properly by lzma and is long, you'll see ratio difference.
ChronoCross
13-10-2016, 03:48
Is there a need to learn LUA code?
-Nevermind-Nirvana-
:D my mistake. No the old way works fine.
i want to make somethig new interesting with LUA code in FA Next.
Razor12911
13-10-2016, 03:56
Well I didn't know there was a need, I didn't check fa.ini until this morning, seemed interesting, and definitely something I have no knowledge on so carry on, good luck.
Vapor028
13-10-2016, 04:16
-Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
Unfortunately no.
-next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
it sometimes all depends on what you're compressing. 4x4 allows a compressor to run in parallel mode, meaning more speed at a cost of ratio. 4x4 splits data into blocks then runs a chosen compressor, this block splitting loses you ratio, but like I said it all depends on what you're compressing, if its something that can be compressed properly by lzma and is long, you'll see ratio difference.
Thx for the fast answer.
For the XZ files... is there an IC like program to use?
and for the 4x4 problem i wrote a little wrong... with no difference i mean in terms of speed. its like 4x4 isnt there at all.... it will be subjetive but i think its even slower then lzma alone. maybe something wrong in the bat?
-msrep64+delta+4x4:lzma:a1:mfbt4:d1024m:fb128:mc100 0:lc8
Well srep on default, if you're using stdio, it has a setting of expecting no more than 25GB/25 600MB from stdin.
"-sBYTES: explicitly specify filesize (for compression from stdin), default 25gb"
All you need to do is guess or have an estimation of your input and set a value, to be safe, set a high value
-s50gb or whatever, just know that this increases memory allocation so also make sure you have enough memory to allocate such, plus for big inputs, you're better off using x64 version of srep to avoid x86 memory allocation barrier.
specifying correct size probably more good while decompressing
srep will fail if u specify less and use more ram if u specify more, is little bit slow too
probably the reason why it is failing to restore things data correctly is becoz of the fact that i don't even store checksums while processing with srep(-hash-)
Razor12911
13-10-2016, 04:31
Thx for the fast answer.
For the XZ files... is there an IC like program to use?
and for the 4x4 problem i wrote a little wrong... with no difference i mean in terms of speed. its like 4x4 isnt there at all.... it will be subjetive but i think its even slower then lzma alone. maybe something wrong in the bat?
On second thought, I think it's actually possible for IC update, I wonder why didn't I think of this before.
Making update for XZ support.
specifying correct size probably more good while decompressing
srep will fail if u specify less and use more ram if u specify more, is little bit slow too
probably the reason why it is failing to restore things data correctly is becoz of the fact that i don't even store checksums while processing with srep(-hash-)
Actually, srep is suppose to fail during compression, removing -hash- will not fix this, to be on the safe side, don't pipe srep when compressing, only do that when decompressing.
Hey guys. I am in a bit of a problem right here, this happened to me 2 times since I was compressing game files. I have 5 archives. First 3 unpack without any problems, then when it comes to the 4th one, at very ending of it I get this message - http://funkyimg.com/i/2i9Kc.png The 5th archive unpacks without any errors. I happened to me once before, I couldn`t manage to realize what it was, maybe you could help me?
I was packing with
-msrep:m3f:a16:l512+delta+lzma:a1:bt4:d400m:fb273:m c10000000000:lc8:lp2:pb2
Other archives use pretty much the same algorithms, on some of them I used precomp if it matters.
Hey guys. I am in a bit of a problem right here, this happened to me 2 times since I was compressing game files. I have 5 archives. First 3 unpack without any problems, then when it comes to the 4th one, at very ending of it I get this message - http://funkyimg.com/i/2i9Kc.png The 5th archive unpacks without any errors. I happened to me once before, I couldn`t manage to realize what it was, maybe you could help me?
I was packing with
-msrep:m3f:a16:l512+delta+lzma:a1:bt4:d400m:fb273:m c10000000000:lc8:lp2:pb2
Other archives use pretty much the same algorithms, on some of them I used precomp if it matters. I don't know Much about inno but did you try to extract using cmd to know exactly where the problem is ??
arc x archive.arc --adddir
if specific algorithm is causing this error arc will tell you
if it extracted without any problem so the problem is from inno script itself :)
I don't know Much about inno but did you try to extract using cmd to know exactly where the problem is ??
arc x archive.arc --adddir
if specific algorithm is causing this error arc will tell you
if it extracted without any problem so the problem is from inno script itself :)
Thank you. Now I will know how to test this.
But it seems that the operation succeeded via the CMD, no errors :)
What problem can there actually be in my script, is there anyone who would like to take a look? The other 4 archives are unpacking without any errors. Strange.
Thank you. Now I will know how to test this.
But it seems that the operation succeeded via the CMD, no errors :)
What problem can there actually be in my script, is there anyone who would like to take a look? The other 4 archives are unpacking without any errors. Strange.
try to make archive number 4 and 5 one archive :) so that you'll have only 4 archives :D Just ignore the problem you can't solve
Update:
-msrep:m3f:a16:l512+delta+lzma:a1:bt4:d400m:fb273:m c10000000000:lc8:lp2:pb2
you are using large dictionary in your lzma :D the difference will be a couple of megas
& For Srep you are using a16 why not using a64 it's the fastest & best ratio ~ same ratio as a0/a1 :D ~
Guys , I need to understand something
Precomp+msc
msc+precomp
what is the difference ??
is Precomp+msc better to be sure that msc find all media & compress it ?? or Just leaving it msc+precomp because msc will also find all media but will be faster because input is smaller than precomp input ??
Guys , I need to understand something
Precomp+msc
msc+precomp
what is the difference ??
is Precomp+msc better to be sure that msc find all media & compress it ?? or Just leaving it msc+precomp because msc will also find all media but will be faster because input is smaller than precomp input ??
Hello aswadd :D
I think Precomp+msc do the best :rolleyes: As precomp do decompress so msc piping is easy to find out maximum media files with effective speed of finding:D
Razor12911
15-10-2016, 23:31
^Exactly what he said.
But it's general knowledge, think of it this way.
can deflate streams store media streams? yes, deflate is a compression algo and it can contain media streams.
can media streams store deflate streams? no, media streams is not a compressor or a packer of some sort therefore it cannot contain deflate streams.
ChronoCross
16-10-2016, 12:21
Testing only One idea in fa Next with LUA code.
The first idea:
I take a folder with 62MB into -m0 option "data.arc"
and then split the file with razor's split tool in three parts.
data.part1 30 mb
data.part2 30 mb
data.part3 2 mb
OK well the first idea is said to fa next with LUA code:
that's this three files are "data.arc"(with a trick or patch i don`t know).
well the problem is fa next
search the file in directory and seek all the file.
doesn`t see this three files like into one("data.arc").
The principal idea is not join the files is create an emulation of file or something like that(and you know what?, i don´t know how to do :D ).
but i fail because, fa next needs data.arc present in the folder and with his size(and his correct hashes).
OK this is the LUA code for fa.ini to join the files (without Split.exe).
onProgramStart(function ()
a = "none"
f = command.arcname
f = io.open(f, "a")
f:write()
f = io.open("data.arc", "r+b")
f1 = io.open("data.part1", "rb")
f2 = io.open("data.part2", "rb")
f3 = io.open("data.part3", "rb")
f4 = f1:read("*all")..f2:read("*all")..f3:read("*all")
t = io.output (f:write(f4))
f = assert (io.open("data.arc", "r"))
f:close()
end)
and the code for extract in cmd
fa x -dp.\ data.arc
well another problem here (help) you need to run the bat two times.
the first is an error like data.arc is not a valid fa next file.
because the code first create data.arc with 0b.
but the second time you run the bat, fa next extract correctly because the program found a joining data in data.arc.
If someone have an idea to fix this or help to the first idea please go on!
HI ChronoCross Nice idea and a litle complicated for me :D
the first Idea is awesome, but i can't help you :( .
I started to read LUA and looks easy to undertand.
well another problem here (help) you need to run the bat two times.
the first is an error like data.arc is not a valid fa next file.
because the code first create data.arc with 0b.
but the second time you run the bat, fa next extract correctly because the program found a joining data in data.arc.
If someone have an idea to fix this or help to the first idea please go on!
let me help you in this part.
add some lines in your fa.ini code repeat and until ~=
onProgramStart(function ()
repeat
a = "none"
f = command.arcname
f = io.open(f, "a")
f:write()
f = io.open("data.arc", "r+b")
f1 = io.open("data.part1", "rb")
f2 = io.open("data.part2", "rb")
f3 = io.open("data.part3", "rb")
f4 = f1:read("*all")..f2:read("*all")..f3:read("*all")
t = io.output (f:write(f4))
until f ~= assert ( io.open("data.arc", "r"))
f:close()
end)
Now can join and extract the files correctly. :)
btw, if you not found it - the classic Lua learning book is http://www.lua.org/pil/contents.html
it's third edition is available online ;)
ChronoCross
17-10-2016, 19:00
i cleaning the code
onProgramStart(function ()
f = io.open(command.arcname, "wb")
f1 = io.open("data.part1", "rb")
f2 = io.open("data.part2", "rb")
f3 = io.open("data.part3", "rb")
f4 = f1:read("*all")..f2:read("*all")..f3:read("*all")
t = io.output (f:write(f4))
f:close()
end)
HI @ChronnoCross Thanks for the code. Well i was thinking and reading LUA code and maybe i resolve your request(with minimal knowledge).
--how to read parts without merge those files?
Well the only trick i made is send the data to Buffer.
Example:
FA Next request the file to extract, so the code create the file "data.arc" with 0MB(like a temp file but isn't)
and send the data of data.part1, data.part2 and data.part3 to buffer. FA Next read the collected data from buffer and extract the files.
the file data.arc always have 0MB because with the LUA code "flush" you flush the buffer.
onArchiveStart(function ()
f = io.open(command.arcname, "wb")
f1 = io.open("data.part1", "rb")
f2 = io.open("data.part2", "rb")
f3 = io.open("data.part3", "rb")
f4 = f1:read("*all")..f2:read("*all")..f3:read("*all")
io.output(f:write(f4)):setvbuf("full")
end)
onArchiveDone(function ()
f = io.open(command.arcname, "wb")
f:flush()
end)
maybe we need to check the code or add some more lines :)
ChronoCross
20-10-2016, 17:05
Wow awesome gozarck now we don't need extra space to extract multi parts!. Works great! You emulate the arc file or whats?.
I made a test in my usb flash drive: i was fill it to the limit of space and then i added the files to extract with 10mb of free space in the usb flash drive to test if the file data.arc increase his size but not!!! Works perfect! Also extract correctly.You create multiparts extraction for faNext!!
We need to work a little more with the code to add features like unpack from DVD , remaining time for while collecting data, request part if some part are in diferent folder, make the code for IS. Also we need to add the option exclude full path -ep1 with lua code for fa next.
----Update------
Unpack from DVD or iso DONE!
----------------------
For unpack from dvd you need The
data.parts fa.ini fa.exe and unpack.bat
example:
-------------------
data.part1
data.part2
data.part3
fa.ini
fa.exe
Unpack.bat
------------------
the trick is set data.arc file in temp folder.(%temp%\data.arc)
----Unpack.bat----------
fa x -dp"D:\MyGame" -o+ %temp%\data.arc
ChronoCross
21-10-2016, 18:54
This is nice gozarck!. From DVD - ISO works greats!
Now we need a code to "request part if some part are in diferent folder" or another DVD!.
I don`t know how to add "-ep1" option in LUA code.
If someone knows please help us.
ChronoCross
22-10-2016, 10:31
Oh no... My test are wrong!. Data.arc need extra space!.
Blade2239
24-10-2016, 22:30
http://fileforums.com/attachment.php?attachmentid=15943&d=1477332855
Anyone help me with this........ Please.....
Blade2239
27-10-2016, 11:37
Anyone can tell me more forum link which do work like fileforms
Like conversion tutorial and more.....
pakrat2k2 you may know......
For compression I know encode.ru
Please
Anyone can tell me more forum link which do work like fileforms
Like conversion tutorial and more.....
pakrat2k2 you may know......
For compression I know encode.ru
Please
Most of then are Russian forums I don't remember the names but search google & you will find them
Anyone can tell me more forum link which do work like fileforms
Like conversion tutorial and more.....
pakrat2k2 you may know......
For compression I know encode.ru
Please
http://krinkels.org/ russian forum godluck :D
Blade2239
28-10-2016, 00:41
http://krinkels.org/ russian forum godluck :D
Can I get more option beacuse Google sending me to warz groups.....
And anyone can tell me which is the most beautiful installer is available or made .... Like we made with ic asc and other....
Which one of the latest codex/plaza installer
Is anyone know how fitgirl installer looks......
Edit:
On krinkels most user are not active from a while!!!!!!
Edit:
Why fileforms user are not online. is everyone's exam start exactly together........
Edit:
Can anybody tell me how to extract or get source of wpi.....
There's an innoextract but it just give DLL exe file no code or source
Hey guys. Can someone tell me why ISDone tells me that there is Unsupported compression method when I am using srep+delta+exe2+lzma.
The problem is with exe2, but I don`t understand why. When I use exe instead, it`s ok, but with exe2 I can get rid of some kbs.
Hey guys. Can someone tell me why ISDone tells me that there is Unsupported compression method when I am using srep+delta+exe2+lzma.
The problem is with exe2, but I don`t understand why. When I use exe instead, it`s ok, but with exe2 I can get rid of some kbs.
why even using exe :D for only 1 or 2 megabytes ?? & For delta don't use it unless you make sure it will give u better ratio
BTW this problem is related to arc.ini there is something wrong with it
& if you are using fazip for exe & delta & lzma make sure you are including it's configuration is arc.ini also don't forget to include fazip.exe & dll files
why even using exe :D for only 1 or 2 megabytes ?? & For delta don't use it unless you make sure it will give u better ratio
BTW this problem is related to arc.ini there is something wrong with it
& if you are using fazip for exe & delta & lzma make sure you are including it's configuration is arc.ini also don't forget to include fazip.exe & dll files
I just had one executable and a bunch of dll`s and ran them through 16 algorithms, and chose the one with smallest size.
I just wanted to know what is causing the problem, to know where to dig.
What can be wrong with .ini? Do you have any clue?
Thanks.
p.s - Through console it`s running fine, damn.
Hi everyone. For a long time I`ve been searching how to make the unpacking process faster. There is one guy whose repacks are unpacking very quickly, comparing to mine or others.
http://funkyimg.com/i/2jewZ.png
By looking at this screen we can see the (almost) full CPU load and high load on my SSD (on HDDs the load is also high. It can go higher depending on type of SSD/HDD). The same pattern is seen in CODEX / RLD setups.
I couldn`t come up with such results. I tried to take a look in the temp folder of that setup, and haven`t seen anything new. Just saw facompress_mt.dll which did not give me any good.
Is there anyone who can point me to the right direction or these are some secret techniques? Thank you.
Thx, but is it possible to combine 4x4:lzma with other parameters? Like this -m4x4:lzma:a1:mfbt4:d400m:fb128...
- I tested like this but the process of packing is not starting
The example string works just fine
srep+4x4:t4:b128m:lzma:128m
Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.
On user machine ISDone will require four continuous memory blocks of 400 MB each.
Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.
I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.
as fitgirl said 4x4 is not good to use for public repacks you can use it for your personnel repacks it require 4X what normal lzma require but also faster ,
one more thing RLD didn't use 4x4:lzma they used 4x4:tor :D you don't have to use 4x4 for lzma only ;) but it will be a good option for a fast storing with a light compression ratio :)
I tried this string
srep+4x4:t4:b128m:lzma:128m
but I am not able to unpack it.
I tried this string
srep+4x4:t4:b128m:lzma:128m
but I am not able to unpack it.
why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m :D
if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)
why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m :D
if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)
thanks, I will test.
I use LZMA x64
thanks, I will test.
I use LZMA x64
one here mentioned that don't use lzmax64.exe it's not good i don't remember why but he said that as he tested :D use 7z lzma2 (xz) or fazip for (4x4,lzma)
Hi everyone. What can the problem that after using precomp, the decompression process starts very slow? I press "INSTALL" and it stays at 0% for 2-3 minutes then precomp starts. It is not very good I think. Is there any solution?
Hi everyone. What can the problem that after using precomp, the decompression process starts very slow? I press "INSTALL" and it stays at 0% for 2-3 minutes then precomp starts. It is not very good I think. Is there any solution?
it's ok - precomp is in beginning of your compression string, i.e. smth like "precomp+srep+lzma", and this means that on decompression it's last algo in the line, so all other compressora are done when precomp starts
still, you can use cls_precomp and cls_srep to make all decompressors work simultaneously
it's ok - precomp is in beginning of your compression string, i.e. smth like "precomp+srep+lzma", and this means that on decompression it's last algo in the line, so all other compressora are done when precomp starts
still, you can use cls_precomp and cls_srep to make all decompressors work simultaneously
Thanks for your response!
I've got something else - what can cause this error: Not enough memory. unarc.dll returned error code - 5. It appears for some users. It appears when files compressed with 4x4:lzma start to unpack. Yes, I know, you guys told me it's not very ok to use it but what alternatives do I have? It's really fast comparing to default compression. I personally haven't got this error, it's rare, but the fact that it exists, annoys me and I can't move on without making it clear. Have any of you got ideas?
well, it may be bug in my program (unarc.dll) or yiu use it wrong way. so, let's start with lt command output and computer that fails. can you make some checks on this computer so i can understand the situation?
write me to Bulat.Ziganshin********com since i rarely read this forum
The example string works just fine
srep+4x4:t4:b128m:lzma:128m
Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.
On user machine ISDone will require four continuous memory blocks of 400 MB each.
Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.
I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.
i agree with almost everything you said except for two things:
1. 4x4 doesn't mean "use exactly 4 threads both for compression and decompression". let's see - data are split into independently compressed blocks, so you can use any number of threads at compression stage and, independent of that, any number of threads at decompression stage. both arc and unarc find largest contiguous block available (in 64-bit windows it will be 2047 MB), divide its size by the memory required for single (de)compression thread, and run so many threads. so, on 64-bit windows it will use more threads, on 32-bit windows it will use less threads
2. lzma2 is different from lzma in two things:
- it stores incompressible chunks of data
- like 4x4, it can split input data into independent blocks. unfortunately, it always uses only 1 thread for decompression. When manual says "LZMA2 uses: 1 thread for each chunk in x1 and x3 modes; and 2 threads for each chunk in x5, x7 and x9 modes" it means only comperssion stage
So, compared to 4x4:lzma, lzma2 may give slightly better ratio and single-thread speed, but it can't decompress even in 2 threads. but probably you meant compression w/o splitting into independent blocks, where lzma2 indeed is slightly better, but still single-threaded on decompression side. unfortunately, it's inherent property of lzma algorithm - you can't use morte than 1 thread to decompress single lzma compressed stream
3. that said, inability to decompress data compressed with 4x4:lzma is a bug in my program. I already have a better algorithm for InsertTempfile procedure, but ATM it's not yet ideal and still wrtitten in Lua. But i will fix that sooner or later.
4. these days, most of user computers run Win64, so probably it's time to switch to 64-bit unarc.dll in most repacks? i will try to build one.
if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)
it's opposite - fazip can implement any internal arc.exe algorithm as external one. this means that you can compress with fazip and decompress internally, or vice versa.
in practice, i recommend to use fazip for large compression methods (rep, lzma, ppmd). you may also prefer to use fazip for decompression (even in installer) since it's yet another way to break 2/4 GB barrier of 32-bit programs
Bulat, send me please a PM with your email.
Bulat, let me thank you for great tools you've created. Without them most repacks would be impossible, I guess :)
Now to topics you've mentioned.
1) Yes, now I'm aware of it. Could you please clarify if decompression threads number can be controlled by setting more dmem? Let's say I compress with 4x4:lzma256, what should I do, so users with 8-cores and GBs of RAM could decompress it in all their threads. I specifically put higher dictionary sizes here, cause smaller ones aren't good for compression ratio.
2) Yes, I've been talking about mode without blocks splitting and for sure I use it only on data with lots of already compressed data.
4) That would be cool, if it make decompression faster :)
Thank you again!
Yesterday i had very productive debuggng seesion with MCWRX and he helped me to understand why srep+4x4:lzma is hard to master:
InsertTempfile procedure i already mentioned is fired in unarc.dll to adopt decompression algorithms to the real memory available. But on decompression stage, we don't have too much choices - eithe insert "tempfile" between methods and save intermedirte data to the tempfile or limit 4x4 to run less threads simultaneously
With built-in algos, it works fine (modulo bugs), but for external algos including srep there is no any API to limit their memory usage! so, many years ago i developed alternative approach specially to srep - it can be parameterized like srep:mem75%-600mb and this way it will alloc 75% of RAM minus 600 MB. It was supposed that you have compression line like srep+delta+lzma:512m, computed memory usage by remaining algorithms and make appropriate srep setting
It was fine for all built-in algos except for 4x4, whose decompression memory depends on the num_cores and thus cannot be predicted at compression time. So, we just don't have working instrument for limiting srep+4x4 by using only MY tools
Nevertheless, in MCWRX installer memory usage for srep is set by the installer script. I don't looked into details, but obviously community provided right instrument to deal with srep+4x4 case. Unfortunately, it needs some programming skills to manage - you need to find num_cores, compute memory usage of remaining algos, find RAM size, and then compute memory available for srep. And if it becomes too small (say, less than 500 MB) i recommend to limit number of 4x4 threads (by modifying compression string replacing 4x4 with 4x4:t4 or so) in order to prevent excessive disk swapping in srep
Yet another solution will be improvement of :mem language, adding support for smth like mem75%-100m-132m*t
And ideal solution will be implementation of srep as builtin method or just 64-bit unarc.dll. the second way looks much simpler so i will try :)
1) Yes, now I'm aware of it. Could you please clarify if decompression threads number can be controlled by setting more dmem? Let's say I compress with 4x4:lzma256, what should I do, so users with 8-cores and GBs of RAM could decompress it in all their threads. I specifically put higher dictionary sizes here, cause smaller ones aren't good for compression ratio.
as you can see in arcinfo/lt, each thread of 4x4:lzma:256m will require 516 MB of memory, 8 threads need more than 4 GB. the only way is to use external 4x4, i.e. fazip in stdin-to-stdout mode
please try it and write about results - positive or not
ImmortalArc
25-01-2017, 05:19
Can anyone help me out why all the time srep decompression fails and it says write error disk full?? Why??? Searched forum but nothing helps compressed whole 12 gb in 2 hour but happens again and again I am using -m3f with pzlib and lzma.
Firstly, always remember for decompression u should have CLS-SREP.dll (http://fileforums.com/showthread.php?t=97976) in the same folder where the arc.exe is.
If u got CLS-Srep.dll which is 86 Kb or 54 kb or something which is <100 kb then remember those only works during decompression if they are in the same folder where the arc.exe is during COMPRESSION then the arc simply throw disk full error or something other error
the above CLS-SREP.dll, works during compression too.
ImmortalArc
25-01-2017, 05:48
Thanks PrinceGupta will try it.
Do you know a good GUI version for BMS scripts to open game archives for rebuilding and referring data from them such as music and graphics...
doofoo24
25-01-2017, 05:52
try use
arc a -lc2000 -ed -ep1 -r -ma -md -mx -s; -w.\temp -mpzlib+srep+lzma:a1:mfbt4:d185m:fb273:mc10000:lc8 data.arc "game location"
and use srep 3.2 and freearc 0.67 date 15/3/2014 and in arc.ini use
[External compressor:srep64]
header = 0
packcmd = srep64 -m3f -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -c16m -b256k -t2 - -o - <stdin> <stdout>
and for decompress try to use cls-srep 3.3 by profargo with pzlip
and in arc.ini
[External compressor:pzlib]
header = 0
unpackcmd = pzlib d -t50p - -o - <stdin> <stdout>
try to use isdone original script with inno 5.5.9...
ImmortalArc
25-01-2017, 05:56
Srep 3.3 but I have 3.93 why older is better???
doofoo24
25-01-2017, 06:08
try ? form the original site on freearc by Bulat Ziganshin...
download freearc 0.67 alpha portable and srep 3.2
i found a couple of srep exe file infected.
use only the original file.
i found to be more reliable and virus free...
doofoo24
25-01-2017, 06:17
try to use in inno where ExtractTemporaryFile
ExtractTemporaryFile('unarc.dll');
ExtractTemporaryFile('arc.ini');
ExtractTemporaryFile('pzlib.exe');
ExtractTemporaryFile('cls-srep.dll');
and in ExpandConstan try
if not SrepInit(ExpandConstant('{app}\'),128,0) then break;
if not FileSearchInit(true) then break;
if not ISArcExtract ( 0, 100, ExpandConstant('{src}\file.arc'), ExpandConstant('{app}'), '', false, '', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}'), False) then break;
ISDoneError:=false;
ImmortalArc
25-01-2017, 06:23
But it says it is created by bulat.....
Or I am using wrong one
doofoo24
25-01-2017, 06:27
i am trying to help not to argue is easy to modify file attribute...
so please try first if it work great if not then ask why ?
ImmortalArc
25-01-2017, 07:07
Firstly, always remember for decompression u should have CLS-SREP.dll (http://fileforums.com/showthread.php?t=97976) in the same folder where the arc.exe is.
If u got CLS-Srep.dll which is 86 Kb or 54 kb or something which is <100 kb then remember those only works during decompression if they are in the same folder where the arc.exe is during COMPRESSION then the arc simply throw disk full error or something other error
the above CLS-SREP.dll, works during compression too.
I used cls-srep.dll by razor 12911 size is 979kb but when I decompress which is compressed with pzlib+srep+lzma it continuing decompress after 100% original data is about 2.8 gb (before compression) and it now continue extracting data more than 5 gb why ??
Edit: I compressed Twp folder data and update data have 1953 mb and update have 300 mb before compression and after it have 1.18 gb pzlib+srep+lzma
look at http://freearc.org/Research.aspx
it links TWO srep pages, one with stable version (latest 3.2) and one with experimental version (latest 3.93). use 3.93 and check that your installers work
OTOH, you may use 3.2 if you wish. with maximum compression settings it's no different to 3.2 except for faster _compression_
but i don't provide DLLs, so the best approach imho is to use DLLs from trusted sources such as one by Razor
doofoo24
25-01-2017, 07:37
bulat great in time if you manage to answer me will be great.
is there a way to get constant speed in decompression when using lzma:a1:mfbt4:d185m:fb273:mc10000:lc8 without using fastzip or xz only builtin lzma and without 4x4...
i montor the disk usage is about 20 mb/s is there a better way to get 50 or 60 mb/s.
thank in advance...
ImmortalArc
25-01-2017, 07:40
Yeah I have one what youre saying and cls from razor but the problem is it's extracting data more than original size and only in one file I got 8 Files but it's extracting only in 1 file and size is bigger than original. Why???
doofoo24
25-01-2017, 07:48
i truly try to help you man
try use srep witout cls in compression and with decompression use cls by profargo 0.33
ImmortalArc
25-01-2017, 07:53
@PrinceGupta I tried razor cls decompression failed because it's decompressing data more than actual but when I use cls which is 86 on deck pressing successfull with speed of 20 Mb/s
@doofoo24 OK now I had compressed the whole game with srep without cls if I use cls profeg one will it decompress successfully???
But why razor cls extracting more data than actual
doofoo24
25-01-2017, 07:57
ImmortalArc what the method you using
ImmortalArc
25-01-2017, 07:58
-m3f
doofoo24
25-01-2017, 07:59
not srep but in the batch file what srep + ?
ImmortalArc
25-01-2017, 08:02
default pzlib arc.bat
Method is -mpzlib+srep+lzma
ImmortalArc
25-01-2017, 08:06
But I don't know why cls makes decompress slower
doofoo24
25-01-2017, 08:08
ok now
are you using external lzma or fastzip ....
it do not matter
i use file in the folder
srep.exe
pzlib.exe
arc.ini
.bat file
arc.exe
and the folder you want to compress that all i don't use fastzip or other dll
if you have problem in decompression try it
only this file are necessary
try in bat file
arc a -lc4096 -ed -ep1 -r -ma -md -mx -s; -w.\temp -mpzlib+srep+lzma:a1:mfbt4:d185m:fb273:mc10000:lc8 31.arc "file\*"
and make sure their is no fastzip or other lzma in arc.ini only
[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -c16m -b256k -t8 - -o - <stdin> <stdout>
[External compressor:srep]
header = 0
packcmd = srep -m3f -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
lzma:a1:mfbt4:d185m:fb273:mc10000:lc8 without using fastzip or xz only builtin lzma and without 4x4...
i montor the disk usage is about 20 mb/s is there a better way to get 50 or 60 mb/s
no. lzma is really that slow. lzma2 will be faster, since it just copies incomperssible data instead of trying to compress them
Yeah I have one what youre saying and cls from razor but the problem is it's extracting data more than original size and only in one file I got 8 Files but it's extracting only in 1 file and size is bigger than original. Why???
if you have problem, try to solve it in small steps: remove evrything else from your compression method and try with srep.exe, then with vaioyus dlls that you trust
doofoo24
25-01-2017, 08:19
bulat but in lzbench on github lzma or xz speed around 60 mb/ s,
so shouldn't lzma in freearc be the same ?
1. computers are different
2. speed depends on the data
3. we usually measure speed of decompressed stream i.e. if in 1 second 20 mb are decompressed into 60 mb, we say that the speed is 60 mb/s
try to compare with 7-zip/xz speed and you will find that arc is pretty close
@PrinceGupta I tried razor cls decompression failed because it's decompressing data more than actual but when I use cls which is 86 on deck pressing successfull with speed of 20 Mb/s
I Would say it is an Internal Bug , i already reported it earlier
http://fileforums.com/showthread.php?p=452684#post452684
yes i think its strange because cls by Profager is always manages to deliver correct data,
i think it is because the ClS-Srep by Profrager is kinda old and probably written using older sources of srep which doesn't have that bug, may be it can be possible
on the other hand CLS by Razor doesn't work on the ground level, It is simply like using arc.ini but with some more settings like use of x64 srep
I don't know how razor's CLS actually works its a conjecture nothing more
doofoo24
25-01-2017, 09:02
Bulat you right using 7z
[External compressor:lzma2]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d128m:fb=273:mf=bt4:mc=10000:lc=4:lp=0 -mmt=8 -mx9 -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -an -y -si -so <stdin> <stdout>
with freearc is almost the same as using lzma in freearc and the built-in lzma give better ratio 0.5% to 1%.
but i have other question why ZSTD IS WORST THAN 4X4:LZMA in decompression speed i have I7 6700K 4.7GH OVERCLOCK WITH SSD NVME
4x4:lzma decompress around 1gb/ sec but use more memory and cpu to the full while zstd only use 128mb of memory but limit to 400 to 500 mb /sec.
i find it odd ?
if lzma speed is 20 mb/s and 4x4:lzma speed is 1 gb/s, then you have at least 25 cpu cores :) start with measuring full decompression time from start to finish rather than looking into hdd read speeds
but i have other question why ZSTD IS WORST THAN 4X4:LZMA in decompression speed i have I7 6700K 4.7GH OVERCLOCK WITH SSD NVME
4x4:lzma decompress around 1gb/ sec but use more memory and cpu to the full while zstd only use 128mb of memory but limit to 400 to 500 mb /sec.
i find it odd ?
you have a good pc, only if u have what u says
but u should first read some Forums post and others to understand
How actually stuffs works?
@Bulat can you do something for this pls
http://fileforums.com/showthread.php?p=452681#post452681
http://fileforums.com/showthread.php?p=452684#post452684
http://fileforums.com/showthread.php?p=452762#post452762
doofoo24
25-01-2017, 09:23
bulat
if you don't believe me try it for you self you the expert,
but i am not stupid...
i am not trying to argue just sharing what i encounter.
but try it you self...
the system i used is
I7 6700K
DDR4 2X8GB 3200MHZ TIMING 15-15-30
SSD SAMSUNG 950 PRO
THE HDD IS WD BLACK
AND THE EXTRNAL STORAGE IS Seagate STEA2000400.
doodoo24
your English is worse then mine
i am not stupid
may be you get me wrong, i never said u r stupid nor i think like that, what i have said is that u need some reading first, nothing else
using 4x4:lzma the cpu usage is less because there is less data write to the hdd because hdd speed 175 mb/ sec .
it is because
LZ77 decompression is extremely fast, faster than compression. The compressor
must search for matching strings, typically using a hash table or tree. The decompresser
only needs to maintain an output buffer from which to copy repeated strings, and then
write a copy of its output to the buffer.
and LZMA is LZ77 algorithm. Moreover 4x4 can use more then one lzma instance to unpack files by making them work on seperate chunks since its very difficult for Lzma to do that alone and the most Important one, "IGOR don't want to implement it " or in his word he don't have time
Using Lzma the highest speed u can get is around 25-30 mb/s
but u r saying u r getting 175 mb/s with 4x4
this mean u r using 6 out of 8 threads of ur pc
but still saying 50% cpu usage , strange really strange
doofoo24
25-01-2017, 09:54
my frist post on the forum i said pardon me english :o
if someone have skylake or kabylake it is good to test with fast ram and try to montor cpu usage
first run test on normal hdd
the second run try on ssd
you will see different cpu usage
i will try soon agian just to be sure
i will try high clock like 4.5g and then
i will try to test by lower down the clock on the cpu to 3 ghz and see the difference
and if i have more time i will test by disable 2 core to will bee like i3 then i will test with 4 core but without hyper threading like i5
and report back...
but i can't promess so take it with grain of salt.
well, i said how to check the real speed. if you think that i'm wrong, i can't help youi
specifying correct size probably more good while decompressing
razor is correct - -sSIZE works only at compression stage. it allocs memory required to compress files up to this size. it's ignored at decompression stage. if atcompression you tries to process file larger than specified, it should just copy remaining data without further compression and correctly decompress it, but may i'm not sure whether it really works - try yourself with standalone srep.exe and report if it goes wrong
overall, if it goes wrong with srep+7z+shar, why you don't try srep alone?? do you expect that someone will solve yuor problems if you can't do simple things yourself?
doofoo24
25-01-2017, 10:13
the setting i used for zstd ver 1.1.2 IS
[External compressor:zstd]
header = 0
packcmd = zstd -22 --ultra <stdin> <stdout>
and for 4x4:lzma i used
arc a -ed -ep1 -lc2000 -r -ma -md -mx -s; -w.\temp -m4x4:lzma:a1:mfbt4:d185m:fb273:mc10000:lc8
only arc.exe witout srep or anything else.
if anyone have similar system I7 6700K with fast ddr4 and ssd nvme welcome to try and share the results..:confused:
i simple man trying to making sense of what i encounter
there article on pcper about how flash 3d nand accelerate data like burst write...
i don't know maybe this is the reason ?
if someone care to test and explain.
it does not matter to me like i said before i try to post what i experienced if someone find it useful great if not no need for hostile comments...
overall, if it goes wrong with srep+7z+shar, why you don't try srep alone?? do you expect that someone will solve yuor problems if you can't do simple things yourself?
sir, i tried it many maany timeees, every times the same result
i compress those files with shar|srep|7z
since i am archiving with shar i know the correct input size so i give it to srep via -s while compressing
while decompressing i tried to decompress all the things in pipe but it doesn't work srep gives wrong data,
i tried doing every thing separately, first decompress with 7z, It says Every thing is correct, then srep it doesn't say anything, looking at logs and size fo resultant(decompressed) file from srep i concluded that srep is the culprit and it is not restoring the files correctly
if u say so i will run those tests again, i don't have those archives anymore but at morning, its 12AM here
that's great. so you can't describe what exactly happened nor have files. sorry, i can't help in that case
doofoo24
25-01-2017, 12:54
bulat
how the FreeArc Next development going ?
when are you planing to release it..
it's almost done. now i fix bugs and write docs
doofoo24
25-01-2017, 13:10
great
are you plan to make it shareware like winrar or like 7z GNU source
i plan to release existing freearc as shareware so that those whio want to help fa'next development can do it by buying existing freearc
doofoo24
25-01-2017, 13:15
so you planing to support freearc more add more feature like built in zstd-srep or just convert freearc 0.67 alpha to 1.0 stable
doofoo24
25-01-2017, 13:29
anyway
i wish you best of luck on your journey
yours truly
existing freearc will be released as commercial product. fa'next will be developed as new beta product and finally released as freearc 2.0
doofoo24
25-01-2017, 14:12
i done a couple of test using only 4x4:lzma with different setting '4x4:lzma:a1:mfbt4:d64m:fb273:mc10000:lc8' '4x4:lzma:a1:mfbt4:d128m:fb273:mc10000:lc8' '4x4:lzma:a1:mfbt4:d200m:fb273:mc10000:lc8'...
i found that using 4 core 8 threads any setting other than d:64 only 40% cpu utilized ?
using 4x4:lzma:a1:mfbt4:d64m:fb273:mc10000:lc8 it seem to me the best option in scaling cpu with 2 core only only use 384mb in decompression and use the 2 core to tha max...
using on 2 core with 4 threads and 4 core without hyperthreading use the same 756mb of memory and in 4 core 8 threads it use 1024mb and the cpu 100%...
when try to use 4x4:lzma:a1:mfbt4:d128m:fb273:mc10000:lc8 or 4x4:lzma:a1:mfbt4:d200m:fb273:mc10000:lc8 the cpu usage on 4 core 8 threads only 40% whille on 2 core 4 threads and 4 core without hyperthreading cpu usage is 85%? (in decompression not compression)
i am not an expert so no need for smart remarks.
so the best scaling in decompression (in compression on i7 it use around 57% of cpu with 1684mb of memory) is to use 4x4:lzma:a1:mfbt4:d64m:fb273:mc10000:lc8 it work ( i mean that it use the cpu 100% on all setting like pentium-i3-i5-i7)
if anyone find it useful great.
*using 4x4:t does not work for me in decompression i tried 4x4:t4 but when decompress it use 14% of i7 not 50% of i7 4 core 8 threads...
summary
if you use pzlib or precompmt use it with this
4x4:lzma:a1:mfbt4:d64m:fb273:mc10000:lc8 is super fast in decompression but it use the cpu to the max and require more memory but modern system use 64bit so no problem there, for old system and limited ram don't use 4x4 use lzma:a1:mfbt4:d185m:fb273:mc10000:lc8
if srep only use it with zstd the new version 1.1.2
add this line to arc.ini
[External compressor:zstd]
header = 0
packcmd = zstd -22 --ultra <stdin> <stdout>
in compression it use around limit core and only 769mb in compression
in decompression zstd alone use only 128mb and srep better to limit it in inno to 128mb so the total memory use is 256mb it super fast and less demanding on the cpu...
so the best scaling is to use 4x4:lzma:a1:mfbt4:d64m:fb273:mc10000:lc8 it work ( i mean that it use the CPU 100% on all setting like pentium-i3-i5-i7)
I can't find any written document but judging from this
else switch (*param) { // Ïàðàìåòðû, ñîäåðæàùèå çíà÷åíèÿ
case 'b': p->BlockSize = parseMem (param+1, &error); continue;
case 't': p->NumThreads = parseInt (param+1, &error); continue;
case 'i': p->NumExtraBuffers = parseInt (param+1, &error); continue;
case 'r': p->MinOrder0Percents= parseDouble (param+1, &error); continue;
}
i think 4x4:t%NUMBER_OF_PROCESSORS%:lzma:a1:mfbt4:d64m:fb2 73:mc10000:lc8
is betther idea
remember there is no constraint that you need the same number of cores to compress and decompress, so by using external compressor i.e. fazip you can fully control the CPU usage in both compression and decompression
decompression it use only 128mb + srep you can limit the memory
still, it will write the required memory on hard drive making the process a little bit slower
doofoo24
27-01-2017, 05:29
you the "expert" PrinceGupta2000...:o
in decompession i meant using zstd it use 128mb of memory and limt srep to 128mb not 512mb in inno will not make the proccess slower
i test large file (assassin's creed syndicate) with srep+zstd using '128mb' '256mb' and '512mb' in inno setup on normal hdd wd black 1tb write speed was around 176 mb/s max and the same time it took to install so using more than 128mb for srep on hdd is waste of memory usage...
Razor is the King he just taught me what are the pros and kons of using rep+srep.
doofoo24
29-01-2017, 02:57
disk write when using 4x4:lzma on NVME SSD (CPU IS I7 6700K)
* note using srep+4x4:lzma:a1:mfbt4:d185m:fb273:mc10000:lc8
srep limited to 128mb in inno using isdone script
doofoo24
29-01-2017, 03:15
disk write using zstd
* note using srep+zstd
setting of zstd 1.1.2
[External compressor:zstd]
header = 0
packcmd = zstd -22 --ultra <stdin> <stdout>
**srep limited to 128mb in inno using isdone script
the same data compress with two method
the ratio on zstd is worst but around 1 to 2% different
that's it i done testing.
if anyone find it useful great...
GREAT FORUM
hi all !
im in trouble.... i cant unpack my archives using the default isdone script,although it works fine with precomp/srep/lzma,etc.
if i use any extrenal compressors,it says :
17068
My .ISS looks like this :
#define zstd
[Setup]
[Files]
#ifdef zstd
Source: Include\zstd.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\arc.ini; DestDir: {tmp}; Flags: dontcopy
#endif
[Code]
#ifdef zstd
ExtractTemporaryFile('zstd.exe');
ExtractTemporaryFile('arc.ini');
#endif
if not ISArcExtract ( 0, 0, ExpandConstant('{src}\data1.arc'), ExpandConstant('{app}\'), '', false, '', '', ExpandConstant('{app}'), notPCFonFLY {PCFonFLY}) then break;
end;
Upon execution ,setup unpacks the arc.ini ,zstd.exe ,everything ,but the result is the above picture.
I tried to change the unpack command to :
if not ISArcExtract ( 0, 0, ExpandConstant('{src}\data1.arc'), ExpandConstant('{app}\'), '', false, '', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}\'), notPCFonFLY{PCFonFLY}) then break;
Doesnt help.
The only working method for me is :
if not ISExec(0, 0, 0, ExpandConstant('{tmp}\arc.exe'), ExpandConstant('x -o+ -cfg"{tmp}\arc.ini" -w"{app}" -hp"mypw" -dp"{app}" "{src}\data1.arc"'), ExpandConstant('{tmp}'), '...', False) then break;
But its pretty ugly.
Is there any other way to extract my archives using ISArcExtract function ?
Arc.ini content :
[External compressor:zstd]
header = 0
unpackcmd = zstd -d -f <stdin> <stdout>
Thanks in advance !
try using this
http://fileforums.com/showthread.php?t=98121
CLS doesnt work,still the same error. :/
the file was compressed using this method : (FreeArc 0.67 ,March 15 2014)
arc.exe a -mzstd c:\x\chunk0.arc d:\chunk0.dat
arc.ini :
[External compressor:zstd]
header = 0
packcmd = zstd --ultra -v -22 $$arcdatafile$$.tmp -o $arcpackedfile$$.tmp
Tried with this ,too :
packcmd = zstd --ultra -v -22 <stdin> <stdout>
My script : 17069 (This is the basic ISDone script)
oh ,before i forget : i have the same issue with pzlib.....
maybe im failed at compression procedure... ?
USE
if not ISArcExtract ( 0, 0, ExpandConstant('{src}\data.bin'), ExpandConstant('{app}\'), '', false, '123', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}\'), notPCFonFLY{PCFonFLY}) then break;
doofoo24
04-02-2017, 03:53
try using to
if ISDoneInit(ExpandConstant('{tmp}\records.inf'), $F777, Comps1,Comps2,Comps3, MainForm.Handle, 512, @ProgressCallback) then begin
repeat
ChangeLanguage('english');
if not SrepInit(ExpandConstant('{app}\'),128,0) then break;
if not FileSearchInit(true) then break;
if not ISArcExtract ( 0, 100, ExpandConstant('{src}\setup-1'), ExpandConstant('{app}'), '', false, '', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}'), False) then break;
still the original error.
doesnt matter.... im stay with the uglier unpack-method.
thanks to all for the help...!
doofoo24
04-02-2017, 04:41
are you using zstd 32bit or 64bit in isdone ?
if you using zstd 64bit you can't decompress using zstd 32 you need to use zstd 64bit
are you using zstd 32bit or 64bit in isdone ?
if you using zstd 64bit you can't decompress using zstd 32 you need to use zstd 64bit
Successfully unpacked zstd 64 bit archives with x86 version.
File Type: rar ISDone0.6final zstd.rar (5.42 MB, 1 views)
Many-many thanks...
its working fine now !
I dont know what caused the error ,but its fine now!
im a little bit confused ,because the compression-syntax is pretty easy ,but...... :D
no matters. :)
thanks again !
ImmortalArc
18-02-2017, 01:48
Where I can I get freearc .70 to download ?
correct question is "When". answer is never, i plan to release 1.0
ImmortalArc
18-02-2017, 03:05
And when 1.0 ??
ImmortalArc
18-02-2017, 03:09
And how can I optimize cpu and time both ?????
ImmortalArc
18-02-2017, 03:18
But your site still shows .70 will arrive in Feb still ?? Hahha
ZakirAhmad
21-02-2017, 06:56
How crysis 3 pak files can be decompressed using xdelta...
Viper645
23-02-2017, 09:14
Any way to completely disable external compression in freearc? I mean making it impossible to enable by arc.ini or something else.
How crysis 3 pak files can be decompressed using xdelta...
Dont know ,but if u r trying to repack it.... You cant do that ,because it uses an asymmetric cypher. (AFAIK)
doofoo24
24-02-2017, 14:07
so i tried to use fazip for lzma to get better ratio the limit in freearc lzma:a1:mfbt4:d185m:fb273:mc10000:lc8 using fazip i mange up lzma:a1:mfbt4:d756m:fb273:mc10000:lc8 work with ISDONE more than d756mb IT SAY NOT not enough memory...
but it is so slow compare to 4x4:lzma:64 or zstd
so ratio vs speed.
is there a way to get better ratio with zstd
[External compressor:zstd]
header = 0
packcmd = zstd -22 --ultra <stdin> <stdout>
*can i improve compression more than this setting with zstd ?
any ideas ??
Maybe delta can improve compression a little bit...Give it a shot :
arc.exe a -msrep:a8:m5f:l64+delta+zstd 1.arc 1.dat
Viper645
28-02-2017, 20:29
Which is better?
Precomp 0.38 vs 0.4 vs 0.42
Which is better?
Precomp 0.38 vs 0.4 vs 0.42
Precomp 0.45, and one which is in currently in development is even better, uses only 2 temp file per stream
https://github.com/schnaader/precomp-cpp
Viper645
28-02-2017, 20:55
I don't know why, but freearc mx compression is better than precomp04+srep+lzma for some games, such as GTA San Andreas. precomp04+srep+lzma=1.27GB, freearc mx=1.2GB
if Game doesn't get deflate stream and < 4GB then even 7zip's -mx will beat both
precomp04+srep+lzma
and freearc's -mx
in compression , compression time and decompression time.
So, I see recent posts about 4x4. Let me tell you my experience with it. When users were installing my repacks some of them, got "Not enough memory" error. Srep was given 256mb memory in script
if not SrepInit(ExpandConstant('{app}\'),256,0) then break;
Do you think if I would have given it 128mb, no one had this errors? I just left 4x4 and moved onto lzma2.
And have any of you had such error?
Viper645
01-03-2017, 10:27
Which is better?
Precomp 0.38 vs 0.4 vs 0.42
Precomp 0.45, and one which is in currently in development is even better, uses only 2 temp file per stream
https://github.com/schnaader/precomp-cpp
Well then which precomp is the fastest while compressing? (.38/.4/.42)
Viper645
01-03-2017, 10:32
Another question, PPMonstr compressor shows not enough memory while having more than enough memory in windows 10. Why? Any way to fix it?
Hi all!
I need a little help with SREP.
In command line
Srep64 -a8 -m5f -l512 -d1024 mydata.dat mydata.srep
works fine.
But when i chain the algos like this
Arc.exe a -mpzlib+srep:a8:m5f:l512:d1024+delta+zstd myarc.arc mydata.dat
,it says : invalid option -d1024 (or something like that)
Without -d1024,everythings fine...
-d1024 or -d1024mb,it doesnt matter.
Any idea ,guys ?
Thanks in advance ! :)
Solved : Old SREP version was buggy. V3.93a works fine.
Viper645
10-03-2017, 09:35
Suppose, I added an edit in a wizard page of inno setup using ISenhanced. How to use that input of user in any section of the installer? Please, I need to know this fast.
Inactive thread, no one knows the answer/no one wants other to know it
Suppose, I added an edit in a wizard page of inno setup using ISenhanced. How to use that input of user in any section of the installer? Please, I need to know this fast.
Inactive thread, no one knows the answer/no one wants other to know it
Hi,
if you are a fast learner, Then refer the following topics
http://fileforums.com/showpost.php?p=439298&postcount=1
Under installer topic lot of installer with different functions and procedures are there.All are based on inno setup script.
You may download some script files.Then edit with your knowledge.
You may learn
how to create wizard page,buttons,bevels.panels.checkboxes and many things.:)
But if you are out of patience! ask the Masters who create some installers when they have patience.;)
hi !
I would like to use cls-precompmt with an isdone script.
what do i need to compress files with it ?
I tried many times ,but with no success. (crashes,etc)
somebody can help me ? (arc.ini ,location of required files,etc)
Thanks in advance ,again! :)
ZakirAhmad
11-03-2017, 19:34
Help please..
I can unpack .pak files of crysis 3 with pak decrypt, it converts pak to zip but how can i pack them back to .pak
Cant recrypt .pak files. Crysis3 uses an asymmetric cypher and it needs the encyption key. (Decryption key can be found in one of its file. Thats why you have it)
ZakirAhmad
12-03-2017, 01:53
shazzla: What is point of decrypt a file when u cant encrypt... how can i use xdelta to make them .pak back with .diff
Ok,you want to make a repack,i understand.
But you cant reencrypt .pak,because it needs the encryption key !!!
Whats the point of convert pak to zip ?
U can rip out things. :-)
Imho xdelta is not for messing with cryptographic algos....
If you want to know more , google for XENTAX.
ZakirAhmad
12-03-2017, 05:58
@shazzla, But u still need to encrypt that ripped file before using in crysis 3...
is their a way to compress crysis 3 game files...
Viper645
12-03-2017, 06:10
Hi,
if you are a fast learner, Then refer the following topics
http://fileforums.com/showpost.php?p=439298&postcount=1
Under installer topic lot of installer with different functions and procedures are there.All are based on inno setup script.
You may download some script files.Then edit with your knowledge.
You may learn
how to create wizard page,buttons,bevels.panels.checkboxes and many things.:)
But if you are out of patience! ask the Masters who create some installers when they have patience.;)
Thanks but there are only installers, no tutorials :(
@shazzla, But u still need to encrypt that ripped file before using in crysis 3...
is their a way to compress crysis 3 game files...
N.O. !
OTOH files are encrypted,so they looks like random data. (Pseudo)random data cannot be compressed with good efficiency.
hello is need some help, how do i make my created desktop shortcuts run as administrator by default, currently using "Black_Box_Script_By_Kurutucu_V1.8" but i mean for any script
Dansoft
[Registry]
Root: HKCU; SubKey: Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers; ValueType: string; ValueName: {app}\MyProg.exe; ValueData: RUNASADMIN; Flags: uninsdeletevalue uninsdeletekeyifempty
Dansoft
[Registry]
Root: HKCU; SubKey: Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers; ValueType: string; ValueName: {app}\MyProg.exe; ValueData: RUNASADMIN; Flags: uninsdeletevalue uninsdeletekeyifempty
Thanks, but am not sure where exactly am supposed to put that, can you show me how using the Black_Box_Script_By_Kurutucu_V1.8 or any other script please?
Dansoft
[Registry]
Root: HKCU; SubKey: Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers; ValueType: string; ValueName: {app}\MyProg.exe; ValueData: RUNASADMIN; Flags: uninsdeletevalue uninsdeletekeyifempty
Dansoft
[Registry]
Root: HKCU; SubKey: Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers; ValueType: string; ValueName: {app}\MyProg.exe; ValueData: RUNASADMIN; Flags: uninsdeletevalue uninsdeletekeyifempty
THANK YOU!! it worked :):):)
please help i keep getting this error (in picture)while unpack GTA V or need for speed the run. i used this method -mpzlib+msc+srep+Izma , also tried with -mpzlib+srep+Izma
if not SrepInit (ExpandConstant('{app}'),1024,0) then break;
@shazzla, But u still need to encrypt that ripped file before using in crysis 3...
is their a way to compress crysis 3 game files...
use quickbms to encrypt and decrypt dude
ZakirAhmad
14-03-2017, 10:35
-XCX-
But i didnt find quickbms script for crysis 3... can u please give me one.. i mean upload here...
Viper645
17-03-2017, 08:52
There are latest precomp 0.45 0.46 available now. But why people still uses 0.38, 0.40. 0.42 and 0.43?
yes i want to know that too
ImmortalArc
20-03-2017, 01:27
Hello can somebody post freearc installation file here official website is down please....
Vapor028
20-03-2017, 03:13
@ ImmortalArc
https://2ra5-downloads.phpnuke.org/en/c150482/freearc
Hello can somebody post freearc installation file here official website is down please....
Official Site is not down, but that domain is down. Check this http://freearc2.azurewebsites.net/
how do you compress .minizip files ?
because i tried pzlib, masked data compression all methods but none of them compresses anything :/
I am very new to nsis installer. I was using inno but some nsis features are good, so I want to learn it. My question is, how to change the images and icons of nsis installer and how to design the installer?
(I don't hope to get sufficient answers, people here don't use nsis, they love inno)
There are latest precomp 0.45 0.46 available now. But why people still uses 0.38, 0.40. 0.42 and 0.43?
Because 038 is more stable with almost everything. New precomp versions added some new features which sometimes cause error with some files.
For example recently i compressed some files which 045 is causing error but 038 works fine.
ImmortalArc
14-04-2017, 22:28
@Bulat
It long time since you told about FreeArc 1.0
What now is it coming ?
@Bulat
It long time since you told about FreeArc 1.0
What now is it coming ?
He is going to create a paid version of the software, and it will take time. It will be not a free arc, it's going to be paidarc :p
i'm close to releasing next version of fa'next:
https://github.com/Bulat-Ziganshin/FA/blob/master/0.20/README.md
ImmortalArc
15-04-2017, 08:40
So no update for free arc ?
what you want to see updated?
Abin Antony
15-04-2017, 09:34
For compression I use freearc.But I found that sub folders are not being compressed so the compression ratio is high.What should I do?
And
In a compressed file it showed that it had compression "storing" and "srep:m3f:a1+lzma:69mb:normal:bt4:128:lc8".
I need your help.How can I compress in this method.I am a beginner.
ImmortalArc
15-04-2017, 10:15
Bit more speed and lesser cpu usage
And much mpre which i feels everday but haven't remember now :)
And yeah if you can embed pzlib with it.. ;)
it's exactly what i do in fa'next
ImmortalArc
15-04-2017, 11:50
But when :) ?
ChronoCross
15-04-2017, 13:58
You can make pzstd more faster adding 4x4.
I testded in 4x4
1mb to 1gb block size. And is really faster.
Like 4x4 help to zstd to improve speed compression.
srep+exe+delta+4x4:10mb:pzstd
Abin Antony
17-04-2017, 03:04
For compression I use freearc.But I found that sub folders are not being compressed so the compression ratio is high.What should I do?
And
In a compressed file it showed that it had compression "storing" and "srep:m3f:a1+lzma:69mb:normal:bt4:128:lc8".
I need your help.How can I compress in this method.I am a beginner.
Is someone able to help??
This depends on the file type in your subfolders, but the compression method you are using is not very high as well
>>In a compressed file it showed that it had compression "storing" and "srep:m3f:a1+lzma:69mb:normal:bt4:128:lc8".
"Storing" simply means "no compression"
Are you using MAsked data Compression or something
its probably that files compressed with "storing" are not compressible
at all thus decided to not use any compression at all
here are some of the thread helpful for you
read the post from other user, this is very helpful for all of us
http://fileforums.com/showthread.php?t=99120
http://fileforums.com/showthread.php?t=97530
http://fileforums.com/showthread.php?t=99122
http://fileforums.com/showthread.php?t=93191
Viper645
13-05-2017, 05:51
How to fix it?
the sections look ok. so most probably you are using old arc.exe that doesn't understand them. other possibilities are that i don't see some problems, or there are invisible symbols (f.e. LF instead of CR+LF), or arc.ini read from other place
Viper645
13-05-2017, 18:15
the sections look ok. so most probably you are using old arc.exe that doesn't understand them. other possibilities are that i don't see some problems, or there are invisible symbols (f.e. LF instead of CR+LF), or arc.ini read from other place
You are right. I am using freearc 0.40
Viper645
14-05-2017, 03:54
Used fa 0.67. But when extracting using ISDone, the progressbar stuck at 0% and then immediately goes to 100%. I am using ISARCEXTRACT function and Cls-uharcmt.dll and cls-antiz.dll while compressing and decompressing. Anyone can help?
start with figuring out which method in chain do it, then ask the method author
Viper645
14-05-2017, 07:35
The method is right, but I need to ask Razor12911 because he is the creator of those clses
What is the last version of srep that supports almos every version of windows? The latest srep 3.93a has a decompression bug in windows 10
urstacks
15-05-2017, 04:19
Hey, I have a very simple question
can someone post a step-by-step tutorial on how to unzip ARC files in Inno Setup
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.