Helping thread :)

[Gupta]

finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each

this is what 7zip has to say

7za x Data3.tar -oData3\

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21

Extracting archive: Data3.tar

ERRORS:
Headers Error


WARNINGS:
There are data after the end of archive

--
Path = Data3.tar
Type = tar
ERRORS:
Headers Error
WARNINGS:
There are data after the end of archive
Physical Size = 4083388416
Tail Size = 14341385728
Headers Size = 512
Code Page = UTF-8


Archives with Errors: 1

Warnings: 1

Open Errors: 1

i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.

Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?

[Razor12911]

Originally Posted by PrinceGupta2000

finally, i somehow figure it out, actually srep is the culprit who is somehow manages to add random data at the end of archive
and this is happening with all the 4 bins i made, all having around 25gb of reflated files each

this is what 7zip has to say


i don't know why it is happening, it works fine with the 1gb file and a 10gb file i tested earlier
i also try shar instead of tar the results are same, file got broken ... may be a internal bug of srep or might be shar and tar format is not stable yet... no may be in the case of shar bt not in the Tar at all
bt this work fine with freaarc everytime, i even compress 60 gb file with it and that work gracefully bt freearc makes things little bit slower and less control-able never had crc error with it at all
may be should wait for freearc'next, Bulat says he will add srep as internal in a month or may be two
bt as far as my tests arr concerned, the memory uses is too high in its case.

Moral of the story, don't go with 7zip+srep on bigger files
IS anybody has a solution for it?

Well srep on default, if you're using stdio, it has a setting of expecting no more than 25GB/25 600MB from stdin.
"-sBYTES: explicitly specify filesize (for compression from stdin), default 25gb"
All you need to do is guess or have an estimation of your input and set a value, to be safe, set a high value
-s50gb or whatever, just know that this increases memory allocation so also make sure you have enough memory to allocate such, plus for big inputs, you're better off using x64 version of srep to avoid x86 memory allocation barrier.

Originally Posted by ChronoCross

--Freearc NEXT
iam a master of LUA code... Naaaaaaa. LOL

in fa.ini add




need to learn how to add external compressors like srep or nanozip

Is there a need to learn LUA code?
-Never mind.

[ChronoCross]

Originally Posted by Razor12911

Is there a need to learn LUA code?
-Nevermind-Nirvana-

my mistake. No the old way works fine.
i want to make somethig new interesting with LUA code in FA Next.