FileForums - Exdupe with stdout ?!

Page 1 of 2

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - Exdupe with stdout ?! (https://fileforums.com/showthread.php?t=99385)

shazzla

22-07-2017 01:31

Exdupe with stdout ?!

Hi guys !

Few days ago i tried to use Exdupe with stdout in arc.ini ,but i failed.
Anyone succeeded with this ?

Yes ,i know,there is SREP. Its okay,but in some case,it eats insanely hughe amount of RAM,even with -dngb parameter.

Any help appreciated !
Thanks.
(Atm away from PC,im on tablet)

Bulat

22-07-2017 05:07

increase -l parameter in srep, it should be always better than exdupe

shazzla

22-07-2017 08:10

Thank Bulat!

I know about the - l - parameter ,but it doesnt decrease the decompression memory req. Only a little bit!
Real example :
Approx 40 Gb dataset with L512 ,decomp memory 4.8 Gb
l1024-l4096 ,decomp memory around 3.9 gb.
-dngb n=1-4 ,M3F-M5F used in both case !
Those decomp.mem.reqs are too much for me.
Thats why i want to use exdupe without tempfile.
Btw i love SREP...

Any idea to decrease decomp mem reqs?

Bulat

22-07-2017 08:27

-m0 -d1g will definitely use less than 1 GB

78372

22-07-2017 08:34

:confused: m0 is the rep method, so it's memory usage is obviously low.

BTW, in case of x86 srep, how can we prevent srep from creating virtual temp file(I have only 4GB RAM)?

FD_RLT

22-07-2017 08:49

Quote:

Originally Posted by 78372 (Post 460954)

:confused: m0 is the rep method it's memory usage is obviously low.

BTW, in case of x86 srep, how can we prevent srep from creating virtual temp file(I have only 4GB RAM)?

Why you don't want it to create virtual temp file if you only have 4 GB RAM ?

78372

22-07-2017 09:34

Because of the massive read and writes in hdd

shazzla

22-07-2017 09:49

uses razor's CLS-SREP.DLL.
it uses memory for TMP ,but u need to keep decomp memory req within considerable values.

78372

22-07-2017 09:55

That's the problem. I know razor's cls is helpful, but hey it's only useful if you use 64bit srep because it allows you to use that. But the problem is, I use x86 editions and when you use that, you need the virtual memory.

Srep is a big problem for me when repacking games bigger as GTA V, because I use solid compression which requires a lot of ram to compress and decompress

ChronoCross

22-07-2017 16:07

Code:

[External compressor:exdupe]

packcmd   = exdupe.exe  -x0 -m512  $$arcdatafile$$.tmp $$arcpackedfile$$.tmp  

unpackcmd = exdupe.exe  -R  $$arcpackedfile$$.tmp $$arcdatafile$$.tmp   

datafile   = $$arcdatafile$$.tmp

packedfile = $$arcpackedfile$$.tmp

shazzla

23-07-2017 05:26

thanks ,its okay ,but how can i use STDOUT ?

ChronoCross

24-07-2017 16:40

Quote:

Originally Posted by shazzla (Post 460976)

thanks ,its okay ,but how can i use STDOUT ?

why you make so complicated?, go for the most simple way(freearc ->arc.ini)

Quote:

7z.exe -an -ttar a -bd "D:\TEST\*" -so | exdupe.exe -x0 -m512 -stdin - -stdout | zstd.exe -f -19 -v - -o test.zst

shazzla

24-07-2017 22:37

Sorry for the noob question,but how can i use this in arc.ini?
I mean pack and unpack command..

i tried several combination to pack/unpack. sadly without success. :(

Arc.ini PACK command :

packcmd = exdupe.exe -x1 -m512 -stdin -p -stdout <stdin> <stdout>

working on UNPACK......

shazzla

26-08-2019 05:08

2 Attachment(s)

2 years passed and no success. Anyone ? :/

shazzla

20-09-2019 01:21

hmmm..
tried many versions,but what is the unpack command for this ?

7z.exe -an -ttar a -bd "D:\TEST\*" -so | exdupe.exe -x0 -m512 -stdin - -stdout | zstd.exe -f -19 -v - -o test.zst

As the topic describes ,the main goal is use exdupe with stdio with freearc.
It would be nice.
Please ,anybody ?

All times are GMT -7. The time now is 20:57.

Page 1 of 2

Show 40 post(s) from this thread on one page