FileForums - PMT - Parallel Multithreaded Encoder/Decoder

Page 2 of 2

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - PMT - Parallel Multithreaded Encoder/Decoder (https://fileforums.com/showthread.php?t=103230)

hwang4

18-03-2020 01:47

I tried several times , in the end i got always the same error ! i'm using :
"PMT (razor) + srep:m3f " , i have no problem in compression but in decompression , it shows "unsupported compression method srep:m3f+razor"

can someone help me and show me an example for what i should add in :
Ultraarc 2.9.0.0 R6 Final :
config0.txt

ASIS.v7.0.8 :
Compressor.ini
arc.ini
Script.iss

thanks!:)

Kitsune1982

22-03-2020 10:03

Does this work with CIU installer?

Hi guys, havent posted much in awhile so decided i would post a reply.
I saw the info about PMT and it looks great but i have a question.
Will this PMT work with installer scripts like C<ustom>I<nstaller>U<ltimate>
3.0.0.1.b4 by Cesar82 or is CIU's options diskspan and ultraarc only? Would love to be able to use this with CIU. let me know if its possible and if it is, what do I need to do to link the compressed files to my project?

thanks.

-Kitsune

igrice

22-03-2020 10:34

Quote:

Originally Posted by KaktoR (Post 484020)

Code:

Compressed 1 file, 288,981,192 => 112,742,899 bytes. Ratio 39.01%

Compression time: cpu 0.22 sec/real 136.38 sec = 0%. Speed 2.12 mB/s

All OK



Testing 1 file, 288,981,192 bytes. Processed  92%

ERROR: general (de)compression error in razor

So what I doing wrong here?

Post PMT.ini and arc.ini please.

shazzla

22-03-2020 20:51

In some rare case,it happens with me,too.. (for example,Quantum break ,1xx GB version)
Maybe a bug in Razor ?!

KaktoR

23-03-2020 00:48

Use t2 for unpack

78372

26-03-2020 02:47

Quote:

Originally Posted by shazzla (Post 484882)

In some rare case,it happens with me,too.. (for example,Quantum break ,1xx GB version)
Maybe a bug in Razor ?!

With Razor's patch, razor now supports stdio (What a pun :D )
Try using that one to remove temps and probably that will fix some stuff.

shazzla

26-03-2020 03:33

I used the STDIO patched version ,without PMT ! (RZ V1.03.7)

KaktoR

27-03-2020 02:16

I have further tested with full games, it works like a charm and pretty fast :D

Code:

 1:36:00 - Starting Archive Creation. 

 2:54:33 - Archive Creation Complete. 

 2:54:33 - Sorting Storage Directories Started. 

 2:54:34 - Sorting Storage Directories Complete. 

 2:54:34 - Starting Archive Creation. 

 4:11:32 - Archive Creation Complete. 

 4:11:32 - Sorting Storage Directories Started. 

 4:11:32 - Sorting Storage Directories Complete. 

 4:11:32 - Starting Archive Creation. 

 9:56:22 - Archive Creation Complete. 

 9:56:22 - Sorting Storage Directories Started. 

 9:56:22 - Sorting Storage Directories Complete. 



10:28:36 - The Witcher Enhanced Edition Size: 13.67 GB. 

10:28:36 - The Witcher 2 Size: 17.21 GB. 

10:28:37 - The Witcher 3 Size: 42.99 GB. 

10:28:37 - Full Input Size: 73.89 GB. 

10:28:37 - Output Size: 45.63 GB.

Decode only possible with t2 or t1 for me, I don't know why :(

Tested with batch and CIU installer.

I used razor

78372

27-03-2020 07:20

Seems like some internal issues with razor. Did you test with other compressors? How do they behave? Probably the fast speed of razor is causing some I/O bottleneck or something with multiple threads. Single thread is enough for razor to be frank with such lightning decompression speed.

KaktoR

30-03-2020 10:19

Not yet, but it's ok neverthless because rz is lightning fast in decompression :)

What other compressors do you suggest? What about zstd?

78372

30-03-2020 11:13

ZSTD has mt and way too fast to use with PMT.
Try with slow tools.
BCM, precomp, rzm etc. can work better.
We have.alternatives for them though.

Cesar82

30-03-2020 15:52

Has anyone done any tests using PMT + pZLib3?

If yes please share the values for PMT.ini and Arc.ini.
Thanks!

78372

30-03-2020 22:38

Quote:

Originally Posted by Cesar82 (Post 484988)

Has anyone done any tests using PMT + pZLib3?

If yes please share the values for PMT.ini and Arc.ini.
Thanks!

What's the point? pZlib is already MT and fast. Moreover, you have xtool now.

Cesar82

30-03-2020 23:15

@78372
Thanks for the information. I know little about compression methods.
So of the compressors used in CIU only RAZOR has benefits if used with PMT.

CIU supports these compressors in addition to some RAZOR tools for specific games.
SREP, ZTool, XTool(not v2019), pZLib3, UELR, AFR, OodleRec [4,5,6,7,8], MSC, MPZ, OGGRE, BPK, 7z, LOLZ, ZSTD, RAZOR

Does any of this list other than razor benefit if used with PMT?

78372

30-03-2020 23:55

Quote:

Originally Posted by Cesar82 (Post 484991)

mpz only I guess, at least for now.

KaktoR

31-03-2020 02:32

Code:

mpz with PMT

Compressed 64 files, 209,434,083 => 180,728,551 bytes. Ratio 86.29%

Compression time: cpu 0.20 sec/real 38.46 sec = 1%. Speed 5.45 mB/s



Tested 64 files, 180,728,551 => 209,434,083 bytes. Ratio 86.29%

Testing time: cpu 0.19 sec/real 36.54 sec = 1%. Speed 5.73 mB/s



mpz only

Compressed 64 files, 209,434,083 => 180,670,089 bytes. Ratio 86.27%

Compression time: cpu 0.30 sec/real 103.93 sec = 0%. Speed 2.02 mB/s



Tested 64 files, 180,670,089 => 209,434,083 bytes. Ratio 86.27%

Testing time: cpu 0.36 sec/real 95.90 sec = 0%. Speed 2.18 mB/s

Tested with settings -b64m -t100p for encode/decode with a Ryzen 5 2600 (12 threads).

I just wonder why ratio is a little bit better without pmt :D

78372

31-03-2020 20:57

The same reason for everything, chunking.

DiCaPrIo

08-06-2020 02:15

Quote:

Originally Posted by L0v3craft (Post 486217)

Hi. Is possible to add bpk compatibility? Thanks.

see at first page

L0v3craft

02-07-2020 20:15

Is possible to add this?

Code:

https://github.com/encode84/bcm

it is BCM v1.5. If is not possible someone has a cls dll for BCM to make it multithreaded? Thanks.

78372

02-07-2020 22:47

Quote:

Originally Posted by L0v3craft (Post 486638)

Is possible to add this?

Code:

https://github.com/encode84/bcm

it is BCM v1.5. If is not possible someone has a cls dll for BCM to make it multithreaded? Thanks.

BCM 1.4 works with this. I will make a stdio patch for 1.5 when I have time.

CAT8K

27-03-2021 10:28

Thanks very much for this tool "78372"
If possible, I want the latest version from you "PMT_rel3"

78372

23-04-2021 09:44

Quote:

Originally Posted by CAT8K (Post 491130)

Thanks very much for this tool "78372"
If possible, I want the latest version from you "PMT_rel3"

I don't think that the project requires an update, I haven't received such requests and I do not use it anyways.

All times are GMT -7. The time now is 17:54.

Page 2 of 2

Show 40 post(s) from this thread on one page