FileForums
Page 2 of 9 1 2 34 Last »
Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Helping thread :) (https://fileforums.com/showthread.php?t=98274)

felice2011 26-09-2016 13:06
I have not had this problem with Cmd use, used individually or with FA support, awesome speed and good compression.

Code:
pzstd -u -22 -f -n 4 -p
Code:
[External compressor:pzstd]
;header = 0
packcmd    = pzstd --ultra -22 -f -n 8 -p $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd  = pzstd -d -f -n 8 -p $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp
OK you've solved.;)

aswadd 26-09-2016 13:46
Have anyone an idea to Multithread lzma ?? I already know 4x4 but it took 100% of the cpu so is their a way to multi thread it ?? :D also i saw something named lzmax64.exe have anyone idea about this ??

mikey26 26-09-2016 13:59
lzma64 is hard on system resources if you dont have a decent machine.and it slow as F**K :P not worth looking at rather stick with normal lzma.
If it aint broken dont fix (stick with what works)
but by all means test new things but dont fix what works :P

felice2011 26-09-2016 14:01
@aswadd ...lzmax64.exe ... is the 64 version should use all the processor cores.

Also uses this scheme, to move in the right direction for optimal settings.
Code:
a=[0|1] : ( Defaul 1 ) : Sets compressing mode

a=[0|1] : Sets compression mode: 0 = fast, 1 = normal. Default value is 1.

mf={MF_ID} : ( Defaul 1 ) : Sets Match Finder

mf={MF_ID} : Sets Match Finder for LZMA. Default method is bt4. Algorithms from hc* group don't provide a good compression ratio, but they often work pretty fast in combination with fast mode (a=0). Memory requirements depend on dictionary size (parameter "d" in table below).
           
            MF_ID        Memory        Description
            bt2        d*9.5 + 4 MB        Binary Tree with 2 bytes hashing.
            bt3        d*11.5 + 4 MB        Binary Tree with 3 bytes hashing.
            bt4        d*11.5 + 4 MB        Binary Tree with 4 bytes hashing.
            hc4        d*7.5 + 4 MB        Hash Chain with 4 bytes hashing.

    Note: Your operation system also needs some amount of physical memory for internal purposes. So keep at least 32MB of physical memory unused.

d={Size}[b|k|m] : ( Defaul 24 )        : Sets Dictionary size

d={Size}[b|k|m] : Sets Dictionary size for LZMA. You must specify the size in bytes, kilobytes, or megabytes. The maximum value for dictionary size is 1 GB = 2^30 bytes. Default values for LZMA are 24 (16 MB) in normal mode, 25 (32 MB) in maximum mode (-mx=7) and 26 (64 MB) in ultra mode (-mx=9). If you do not specify any symbol from the set [b|k|m], the dictionary size will be calculated as DictionarySize = 2^Size bytes. For decompressing a file compressed by LZMA method with dictionary size N, you need about N bytes of memory (RAM) available.

fb={NumFastBytes} : ( Defaul 32 ) Sets number of Fast Bytes for Deflate encoder.

fb={NumFastBytes} : Sets the number of fast bytes for the Deflate/Deflate64 encoder. It can be in the range from 3 to 258 (257 for Deflate64). Usually, a big number gives a little bit better compression ratio and a slower compression process. A large fast bytes parameter can significantly increase the compression ratio for files which contain long identical sequences of bytes.

mc={N} : ( Defaul 32 ) : Sets Number of Cycles for Match Finder

mc={N} : Sets number of cycles (passes) for match finder. It can be in range from 0 to 1000000000. Default value is (16 + number_of_fast_bytes / 2) for BT* match finders and (8 + number_of_fast_bytes / 4) for HC4 match finder. If you specify mc=0, LZMA will use default value. Usually, a big number gives a little bit better compression ratio and slower compression process. For example, mf=HC4 and mc=10000 can provide almost the same compression ratio as mf=BT4.

lc={N} : ( Defaul 32 ): Sets number of Literal Context bits - [0, 8]

lc={N} : Sets the number of literal context bits (high bits of previous literal). It can be in range from 0 to 8. Default value is 3. Sometimes lc=4 gives gain for big files.
:)

aswadd 26-09-2016 15:03
Quote:
Originally Posted by mikey26 (Post 452128)
lzma64 is hard on system resources if you dont have a decent machine.and it slow as F**K :P not worth looking at rather stick with normal lzma.
If it aint broken dont fix (stick with what works)
but by all means test new things but dont fix what works :P
thnx :D now i'm pessimistic :D maybe i'll end up with xz

minivj 27-09-2016 00:23
Best external compressor for *.iwd files for Call of Duty Modern Warfare???

aswadd 29-09-2016 13:28
guys I have a little problem with zcm , it doesn't understand the paths like this
I:\Shadow of mordor\Repack\ .. even i addedd the quotation mark like this
"I:\Shadow of mordor\Repack\ttt" or I:"Shadow of mordor"\Repack\ttt
================================================== =======
I:\Shadow of mordor\Repack\ttt>zcmx64 a -m8 archive.me "I:\Shadow of mordor\Repa
ck\ttt"
ZCM Archiver v.0.93

Archive is I:\Shadow of mordor\Repack\ttt\archive.me
WARNING: No files Compressed 0 bytes to 0 bytes

I:\Shadow of mordor\Repack\ttt>zcmx64 a -m8 archive.me I:"Shadow of mordor"\Rep
ack\ttt
ZCM Archiver v.0.93. Demo version.
copyright (C) 2011-2015 By Nania Francesco Antonio
Archive is I:\Shadow of mordor\Repack\ttt\archive.me
WARNING: No files Compressed 0 bytes to 0 bytes

mikey26 29-09-2016 13:49
what happens if you put the files in just one folder without quotation marks

eg: c:\shadowofmordor

try not leave any spaces i had a very similar issue a few days ago with a compressor just cant remember which one.

Razor12911 29-09-2016 16:57
Quote:
Originally Posted by minivj (Post 452146)
Best external compressor for *.iwd files for Call of Duty Modern Warfare???
Try using this.
http://fileforums.com/showthread.php?t=98317

ChronoCross 29-09-2016 17:00
First read the help from the zcm.exe.
and then use -r (recursive path)
ZCM a -r -m8 -t6 path\*

aswadd 29-09-2016 20:13
Quote:
Originally Posted by mikey26 (Post 452225)
what happens if you put the files in just one folder without quotation marks

eg: c:\shadowofmordor

try not leave any spaces i had a very similar issue a few days ago with a compressor just cant remember which one.
it works fine only folders which have spaces doesn't work on it :confused:

aswadd 29-09-2016 20:19
Quote:
Originally Posted by ChronoCross (Post 452242)
First read the help from the zcm.exe.
and then use -r (recursive path)
ZCM a -r -m8 -t6 path\*
doesn't work :( & -r is just for compressing files in subdirectories
& another question
zcmx64 a -s -r -v -m8 -t2 shadowofmordor.test I:\Shadow of mordor\Repack\test\
the output archive will be I:\shadow\ :confused::confused::confused:
also I can't extract to a folder which have spaces

~MAK~ 29-09-2016 22:13
Quote:
Originally Posted by aswadd (Post 452246)
doesn't work :( & -r is just for compressing files in subdirectories
& another question
zcmx64 a -s -r -v -m8 -t2 shadowofmordor.test I:\Shadow of mordor\Repack\test\
the output archive will be I:\shadow\ :confused::confused::confused:
also I can't extract to a folder which have spaces
checkout this - http://www.fileforums.com/showthread.php?p=452248

Specially made for you ;)

ChronoCross 01-10-2016 08:42
I need help here in 7z.exe
i want to know what means this code
Quote:
-mb0s0:1 -mb0s1:2 or -mb1s0:3 -mb1s1:4
what is the benefits to use this code in the chain of methods?
Quote:
7z.exe a -t7z -bd -mcrc=0 -m0=bcj2 -m1=lzma:d26:fb64 -m2=lzma:d19:fb64 -mb0s0:1 -mb0s1:2 ^-m3=lzma:d19:fb64 -m4=lzma:d19:fb64 -mb1s0:3 -mb1s1:4 ^ "data" test.7z

aswadd 02-10-2016 09:42
I have a small question ... what is the fastest program for storing files ? 7z or arc or winrar
& can I use srep on Multi files ? or srep is used only on 1 file ??

gozarck 03-10-2016 06:41
1 Attachment(s)
Hello Aswadd.
All those tools can store any folder quickly. the speed depends of your HDD.
About srep maybe you already know the answer :)
Unpack Code:
Quote:
@echo off
for %%f in (*.001,*.7z) do 7z.exe x -bd -y "%%f" -so | srep64.exe -d - - | 7z.exe x -bd -y -si -ttar >nul
pause >NUL

Ichiraku2001 03-10-2016 07:54
Quote:
Originally Posted by RamiroCruzo (Post 451922)
Yeah..Whenever we ask him how he's so good, he tells someone else's name and say he rarely does the thing...Leaving us with the question, "Is he from future or something? -_-"

Anyways, welcome to the forum matey ^_^

Razor bro, had a conflict with Val and blah blah stuff, so am gone....So don't worry.. ;)
starting off with what this thread is xD
how did u compress the crew to 12.3 GB ;_;

aswadd 03-10-2016 13:22
Quote:
Originally Posted by gozarck (Post 452400)
Hello Aswadd.
All those tools can store any folder quickly. the speed depends of your HDD.
About srep maybe you already know the answer :)
Unpack Code:
yeb , I know other methods to make srep work on all files but I thought there was an option to do that :D
I used this :)
Quote:
dir /s /b *.pcf |ppx2 -P 20 -L 1 srep64 -m3f -a2 "{}"
not for speed :D Just for rematching is taking much time
:D:D 20X Srep64 // 20 rematching
or
Quote:
for %%C in (dir /s /b) do (
srep64 -m3f -a2 %%C
)

coveiro 08-10-2016 21:49
What is the best method to compress Mafia III, for a smaller size ?

~MAK~ 08-10-2016 22:13
Quote:
Originally Posted by coveiro (Post 452612)
What is the best method to compress Mafia III, for a smaller size ?
I didn't repacked it, but as far as by seeing game data, I think it will use reflate+srep+lzma in simple way. Reflate will work on .sds files and except other audio and other data will be repacked with srep+lzma ;)

oltjon 09-10-2016 02:32
for some time that I try to force myself to figure out how to make the Russians repacker to compress so much the flie.ttarch2. tell tale games.
there someone who knows something ??

Thank you in advance

coveiro 09-10-2016 06:21
Quote:
Originally Posted by ~MAK~ (Post 452613)
I didn't repacked it, but as far as by seeing game data, I think it will use reflate+srep+lzma in simple way. Reflate will work on .sds files and except other audio and other data will be repacked with srep+lzma ;)
thanks :D

ChronoCross 09-10-2016 09:59
What about freearc next?. I don't undertand very well lua options but what if we can add some lua options and use razor's split tool?.

Bulat 09-10-2016 10:20
you have a good chance to learn Lua and apply your abilities to FA :) if you know programming, learning Lua is just one day. if you don't know it at all, learning Lua may be easiest start

aswadd 09-10-2016 15:01
Quote:
Originally Posted by oltjon (Post 452619)
for some time that I try to force myself to figure out how to make the Russians repacker to compress so much the flie.ttarch2. tell tale games.
there someone who knows something ??

Thank you in advance
rebuilding game data files :) so basically go and find some software to extract ttarch2 files content
then compress the output then when unpacking rebuild the game data again :D

ChronoCross 09-10-2016 20:20
Thanks Bulat! I started to learn LUA. I download zerobrane studio is a free IDE for LUA language.

aswadd 10-10-2016 08:23
Guys why not making a list about best methods to every Game engine like unreal engine / cry engine 3 / unity3d engine :D It will help a lot

Gupta 11-10-2016 00:29
Guys I packed some files using
7zip tar | srep | 7zip_lzma (don't remember exact command)

now the problem is that while i am extracting it via
7za e -txz -y data3.bin -so | srep64 -d -hash- -s -- | 7za x -ttar -y -si -oData3\ *.out *.raw -r

the pipe automatically closes after extracting a file of 4083387777 bytes(3.80Gb)

i try many different commands bt problem still persists

any help please...

Razor12911 11-10-2016 00:44
If pipe closes automatically then one of the programs either found an EOF early or there was an error, you must check process exit code of each program to find out which one failed.

-Either way, you can just normally unpack everything without the use of standard streams, you'll figure out which program is letting you down.

Gupta 11-10-2016 04:02
finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each

this is what 7zip has to say
Quote:
7za x Data3.tar -oData3\

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21

Extracting archive: Data3.tar

ERRORS:
Headers Error


WARNINGS:
There are data after the end of archive

--
Path = Data3.tar
Type = tar
ERRORS:
Headers Error
WARNINGS:
There are data after the end of archive
Physical Size = 4083388416
Tail Size = 14341385728
Headers Size = 512
Code Page = UTF-8


Archives with Errors: 1

Warnings: 1

Open Errors: 1
i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.

Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?

ChronoCross 12-10-2016 19:15
--Freearc NEXT
iam a master of LUA code... Naaaaaaa. LOL

in fa.ini add
Quote:
onArchiveDone(function ()
print("Time Lapsed:",os.clock(),"Sec.")
end)
Quote:
--Razor's split tool
onArchiveDone(function ()
os.execute('split s data.arc 10') --number 10 is 10mb
end)
need to learn how to add external compressors like srep or nanozip

Razor12911 12-10-2016 19:22
Quote:
Originally Posted by PrinceGupta2000 (Post 452684)
finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each

this is what 7zip has to say


i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.

Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?
Well srep on default, if you're using stdio, it has a setting of expecting no more than 25GB/25 600MB from stdin.
"-sBYTES: explicitly specify filesize (for compression from stdin), default 25gb"
All you need to do is guess or have an estimation of your input and set a value, to be safe, set a high value
-s50gb or whatever, just know that this increases memory allocation so also make sure you have enough memory to allocate such, plus for big inputs, you're better off using x64 version of srep to avoid x86 memory allocation barrier.

Quote:
Originally Posted by ChronoCross (Post 452747)
--Freearc NEXT
iam a master of LUA code... Naaaaaaa. LOL

in fa.ini add




need to learn how to add external compressors like srep or nanozip
Is there a need to learn LUA code?
-Never mind.

romulus_ut3 13-10-2016 02:35
How about a thread dedicated to "secrets of the trade" where people will share their methods to achieve the best possible compression/de-compression methods for specific titles that can be understood even by a dummy? I know Razor12911 shared his methods for GTA V and Mad Max IIRC. Lets start with "The Witcher 3 The Wild Hunt Game Of The Year Edition" perhaps? I'm trying to compress this title into archives that'll fit into DVD5. I don't really require a fancy installer or anything, just a simple executable that'll let me extract the game's archives where I want (I believe Razor12911 has an archiver like this which I've used with great success in the past) I know that finding the best compression method for a certain set of files requires trial and error but we're all here to learn, right?

Vapor028 13-10-2016 03:26
Hello everyone.
I'm new to this Forum and have one or two Questions.

Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)

next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma

Razor12911 13-10-2016 03:47
Quote:
Originally Posted by romulus_ut3 (Post 452754)
How about a thread dedicated to "secrets of the trade" where people will share their methods to achieve the best possible compression/de-compression methods for specific titles that can be understood even by a dummy? I know Razor12911 shared his methods for GTA V and Mad Max IIRC. Lets start with "The Witcher 3 The Wild Hunt Game Of The Year Edition" perhaps? I'm trying to compress this title into archives that'll fit into DVD5. I don't really require a fancy installer or anything, just a simple executable that'll let me extract the game's archives where I want (I believe Razor12911 has an archiver like this which I've used with great success in the past) I know that finding the best compression method for a certain set of files requires trial and error but we're all here to learn, right?
Get a deflate precompressor, you can use this one.
http://fileforums.com/showthread.php?t=98317
srep then lzma, I think that's the method that needs to be used to get an output of something like 25-30GB.

Quote:
Originally Posted by Vapor028 (Post 452756)
Hello everyone.
I'm new to this Forum and have one or two Questions.

Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)

next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
-Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
Unfortunately no.

-next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
it sometimes all depends on what you're compressing. 4x4 allows a compressor to run in parallel mode, meaning more speed at a cost of ratio. 4x4 splits data into blocks then runs a chosen compressor, this block splitting loses you ratio, but like I said it all depends on what you're compressing, if its something that can be compressed properly by lzma and is long, you'll see ratio difference.

ChronoCross 13-10-2016 03:48
Quote:
Originally Posted by Razor12911 (Post 452748)
Is there a need to learn LUA code?
-Nevermind-Nirvana-
:D my mistake. No the old way works fine.
i want to make somethig new interesting with LUA code in FA Next.

Razor12911 13-10-2016 03:56
Well I didn't know there was a need, I didn't check fa.ini until this morning, seemed interesting, and definitely something I have no knowledge on so carry on, good luck.

Vapor028 13-10-2016 04:16
Quote:
Originally Posted by Razor12911 (Post 452758)
-Is it possible to use xz compressed files with IC 3.0? if yes how? (btw thx razor for this awesome program)
Unfortunately no.

-next one is how to use 4x4:lzma... i have a bat that runs but its no difference to lzma
it sometimes all depends on what you're compressing. 4x4 allows a compressor to run in parallel mode, meaning more speed at a cost of ratio. 4x4 splits data into blocks then runs a chosen compressor, this block splitting loses you ratio, but like I said it all depends on what you're compressing, if its something that can be compressed properly by lzma and is long, you'll see ratio difference.
Thx for the fast answer.
For the XZ files... is there an IC like program to use?
and for the 4x4 problem i wrote a little wrong... with no difference i mean in terms of speed. its like 4x4 isnt there at all.... it will be subjetive but i think its even slower then lzma alone. maybe something wrong in the bat?

Quote:
-msrep64+delta+4x4:lzma:a1:mfbt4:d1024m:fb128:mc100 0:lc8

Gupta 13-10-2016 04:17
Quote:
Originally Posted by Razor12911
Well srep on default, if you're using stdio, it has a setting of expecting no more than 25GB/25 600MB from stdin.
"-sBYTES: explicitly specify filesize (for compression from stdin), default 25gb"
All you need to do is guess or have an estimation of your input and set a value, to be safe, set a high value
-s50gb or whatever, just know that this increases memory allocation so also make sure you have enough memory to allocate such, plus for big inputs, you're better off using x64 version of srep to avoid x86 memory allocation barrier.
specifying correct size probably more good while decompressing
srep will fail if u specify less and use more ram if u specify more, is little bit slow too

probably the reason why it is failing to restore things data correctly is becoz of the fact that i don't even store checksums while processing with srep(-hash-)

Razor12911 13-10-2016 04:31
Quote:
Originally Posted by Vapor028 (Post 452761)
Thx for the fast answer.
For the XZ files... is there an IC like program to use?
and for the 4x4 problem i wrote a little wrong... with no difference i mean in terms of speed. its like 4x4 isnt there at all.... it will be subjetive but i think its even slower then lzma alone. maybe something wrong in the bat?
On second thought, I think it's actually possible for IC update, I wonder why didn't I think of this before.
Making update for XZ support.

Quote:
Originally Posted by PrinceGupta2000 (Post 452762)
specifying correct size probably more good while decompressing
srep will fail if u specify less and use more ram if u specify more, is little bit slow too

probably the reason why it is failing to restore things data correctly is becoz of the fact that i don't even store checksums while processing with srep(-hash-)
Actually, srep is suppose to fail during compression, removing -hash- will not fix this, to be on the safe side, don't pipe srep when compressing, only do that when decompressing.


All times are GMT -7. The time now is 09:58.
Page 2 of 9 1 2 34 Last »
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com