FileForums

FileForums (https://fileforums.com/index.php)
-   PC Games (https://fileforums.com/forumdisplay.php?f=6)
-   -   Best Compression For Archiving (https://fileforums.com/showthread.php?t=103136)

brispuss 02-11-2019 01:29

Best Compression For Archiving
 
4 Attachment(s)
Not sure if this is the right part of this forum to ask this, but?


Trying to save space on HDD by LOSSLESSLY compressing some games for archiving purposes. And later decompressing these games LOSSLESSLY to match the orginal files exactly (bit perfect).


At this stage, concentrating mostly on compressing .dds, .jpg, .ogg, and .png files from games.

So far, compressing *.dds files seem to be quite good using lolz directly with -d128 option and with all other lolz options at default settings.

Here is a sample dds file in case someone wishes to improve broch_02.dds compression?

Now, having problems compressing *.jpg, *.png, and *.ogg files (LOSSLESSLY). Since these files are already compressed (to a degree), it is difficult compressing these files much smaller.

Here are some sample files -

he-crown-solomon117.jpg

common_game_04.png

amb_in_boat_cabin.ogg


Can anyone suggest suitable tools and their settings to losslessly compress these (sample) files, please?

Thank you!

78372 02-11-2019 22:37

packjpg for jpeg images, png contains zlib streams so xtool or reflate should work, oggrec can compress ogg files losslessly.

brispuss 03-11-2019 01:47

turned command into code
 
Thanks for the post!

I'm experimenting with several compressors now.

But trying to compress the png file common_game_04.png using xtool produces the same sized file output (113 kB)! No compression of this file at all!?

Tried using the command
Code:

xtool e:precomp:c32mb,t4:zlib
on this test png file. Why is the png file not being compressed?

78372 03-11-2019 02:54

Try reflate. XTool doesn't work for all png streams.

brispuss 03-11-2019 18:37

Thanks again.

But where do I get the "reflate", and what (command line) settings should be used to run it?

The *.ogg file recompressor oggre doesn't seem to compress the amb_in_boat_cabin.ogg file much unfortunately. Original ogg file size ~ 1004 kB, and after recompressing, the file size becomes only ~ 971 kB. This is only about 3% reduction in file size.

Is there a better program or group of programs that would LOSSLESSLY compress *.ogg files better? Ideally something around 50% or more reduction in file size would be excellent!

78372 03-11-2019 22:25

oggrec is probably the best(or only) lossless compressor for ogg files. Ogg files aren't much recompressible.

78372 04-11-2019 04:52

Here's reflate CLS-Reflate
As it's a cls release, you don't need to edit arc.ini, just paste the files beside arc.exe

brispuss 04-11-2019 15:51

Thanks again for the comments and link.

EDIT: Not sure how to run reflate now?

1) Which version of reflate should be used "k" or "l"? And why?

2) Pasted files from "k" version into Freearc (v0.67) bin sub-directory where the command line executable version of Freearc (Arc.exe) is.

3) Tried running reflate executable under elevated command prompt and nothing seemed to happen.

4) Ran Arc executable under elevated commend prompt, but I don't see an option for "reflate"; unless I missed it?

5) So how do you run reflate?

6) And how do you run reflate ONLY without any precompression and without any compression applied?

78372 04-11-2019 18:29

reflate is a precompressor. Use l version. Program works like -mreflate+srep+lolz

brispuss 04-11-2019 19:24

Thanks, but which command line executable should be run? Arc.exe? Or?

I tried the command "arc -mreflate" only but only get arc showing commands help!? Also tried "arc -mreflate+srep+lolz" but still get arc showing commands help only!?

Simorq 05-11-2019 00:07

@brispuss
Code:

FreeArc 0.67 (March 15 2014)  http://freearc.org  March 15 2014
High-performance archiver. Free as well for commercial as for non-commercial use
Usage: Arc command [options...] archive [files... @listfiles...]
Commands:
  a        add files to archive
  c        add comment to archive
  ch      modify archive (recompress, encrypt and so on)
  create  create new archive
  cw      write archive comment to file
  d        delete files from archive
  e        extract files from archive ignoring pathnames
  f        freshen archive
  j        join archives
  k        lock archive
  l        list files in archive
  lb      bare list of files in archive
  lt      technical archive listing
  m        move files and dirs to archive
  mf      move files to archive
  modify  modify archive using +/-/* actions
  r        recover archive using recovery record
  rr      add recovery record to archive
  s        convert archive to SFX
  t        test archive integrity
  u        update files in archive
  v        verbosely list files in archive
  x        extract files from archive
Options:
  --                                        stop processing options
  -ac          --ClearArchiveBit          clear Archive bit on files succesfully (de)archived
  -ad          --adddir                    add arcname to extraction path
  -aeALGORITHM  --encryption=ALGORITHM      encryption ALGORITHM (aes, blowfish, serpent, twofish)
  -agFMT        --autogenerate=FMT          autogenerate archive name with FMT
  -ao          --SelectArchiveBit          select only files with Archive bit set
  -apDIR        --arcpath=DIR              base DIR in archive
  -baMODE      --BrokenArchive=MODE        deal with badly broken archive using MODE
  -cfgFILES    --config=FILES              use configuration FILES (default: arc*.ini)
  -d            --delete                    delete files & dirs after successful archiving
  -df          --delfiles                  delete only files after successful archiving
  -diAMOUNT    --display=AMOUNT            control AMOUNT of information displayed: [hoacmnwrfdtske]*
  -dmMETHOD    --dirmethod=METHOD          compression METHOD for archive directory
  -dpDIR        --diskpath=DIR              base DIR on disk
  -dsORDER      --sort=ORDER                sort files in ORDER
  -ed          --nodirs                    don't add empty dirs to archive
  -envVAR                                  read default options from environment VAR (default: FREEARC)
  -epMODE      --ExcludePath=MODE          Exclude/expand path MODE
  -f            --freshen                  freshen files
  -fn          --fullnames                match with full names
  -hpPASSWORD  --HeadersPassword=PASSWORD  encrypt/decrypt archive headers and data using PASSWORD
  -iTYPE        --indicator=TYPE            select progress indicator TYPE (0/1/2)
  -ioff        --shutdown                  shutdown computer when operation completed
  -k            --lock                      lock archive
  -kb          --keepbroken                keep broken extracted files
  -kfKEYFILE    --keyfile=KEYFILE          encrypt/decrypt using KEYFILE
  -lcN          --LimitCompMem=N            limit memory usage for compression to N mbytes
  -ldN          --LimitDecompMem=N          limit memory usage for decompression to N mbytes
  -mMETHOD      --method=METHOD            compression METHOD (-m0..-m9, -m1x..-m9x)
  -maLEVEL                                  set filetype detection LEVEL (+/-/1..9)
  -max                                      maximum compression using external precomp, ecm, ppmonstr
  -mc                                      disable compression algorithms (-mcd-, -mc-rep...)
  -mdN          --dictionary=N              set compression dictionary to N mbytes
  -mmMODE      --multimedia=MODE          set multimedia compression to MODE
  -ms          --StoreCompressed          store already compressed files
  -mtTHREADS    --MultiThreaded=THREADS    number of compression THREADS
  -mx                                      maximum internal compression mode
  -nFILESPECS  --include=FILESPECS        include only files matching FILESPECS
  -oMODE        --overwrite=MODE            existing files overwrite MODE (+/-/p)
  -okfKEYFILE  --OldKeyfile=KEYFILE        old KEYFILE used only for decryption
  -opPASSWORD  --OldPassword=PASSWORD      old PASSWORD used only for decryption
  -pPASSWORD    --password=PASSWORD        encrypt/decrypt compressed data using PASSWORD
  -r            --recursive                recursively collect files
  -rrSIZE      --recovery=SIZE            add recovery information of specified SIZE to archive
  -sGROUPING    --solid=GROUPING            GROUPING for solid compression
  -scCHARSETS  --charset=CHARSETS          CHARSETS used for listfiles and comment files
  -sfxMODULE                                add sfx MODULE ("freearc.sfx" by default)
  -slSIZE      --SizeLess=SIZE            select files smaller than SIZE
  -smSIZE      --SizeMore=SIZE            select files larger than SIZE
  -t            --test                      test archive after operation
  -tTYPE        --type=TYPE                archive TYPE (arc/zip/rar/...)
  -taTIME      --TimeAfter=TIME            select files modified after specified TIME
  -tbTIME      --TimeBefore=TIME          select files modified before specified TIME
  -tk          --keeptime                  keep original archive time
  -tl          --timetolast                set archive time to latest file
  -tnPERIOD    --TimeNewer=PERIOD          select files newer than specified time PERIOD
  -toPERIOD    --TimeOlder=PERIOD          select files older than specified time PERIOD
  -tpMODE      --pretest=MODE              test archive before operation using MODE
  -u            --update                    update files
  -vSIZE        --volume=SIZE              split archive to volumes each of SIZE bytes
  -wDIRECTORY  --workdir=DIRECTORY        DIRECTORY for temporary files
  -xFILESPECS  --exclude=FILESPECS        exclude FILESPECS from operation
  -y            --yes                      answer Yes to all queries
  -zFILE        --arccmt=FILE              read archive comment from FILE or stdin
                --append                    add new files to the end of archive
                --archive-comment=COMMENT  input archive COMMENT in cmdline
                --bypass=URL                setups proxy bypass list for URL access
                --cache=N                  use N mbytes for read-ahead cache
                --crconly                  save/check CRC, but don't store data
                --create-in-workdir        create archive in workdir and then move to final location
                --dirs                      add empty dirs to archive
                --groups=FILE              name of groups FILE
                --language=FILE            load localisation from FILE
                --logfile=FILE              duplicate all information displayed to this FILE
                --noarcext                  don't add default extension to archive name
                --nodata                    don't store data in archive
                --nodates                  don't store filetimes in archive
                --nodir                    don't write archive directories
                --original=URL              redownload broken parts of archive from the URL
                --pause-before-exit=PAUSE  make a PAUSE just before closing program window
                --print-config              display built-in definitions of compression methods
                --proxy=URL                setups proxy(s) for URL access
                --queue                    queue operations across multiple FreeArc copies
                --recompress                recompress archive contents
                --save-bad-ranges=FILE      save list of broken archive parts to the FILE
                --sync                      synchronize archive and disk contents
All OK

Batch Script Example
Code:

:-ioff        --shutdown
SET exclude=-x@Resources\_Exclude.lst
SET DIR1=M:\Journey
SET MC=SR+lolz:dtb1:d32:mtt1:mt6:mc1023

Resources\Arc.exe create -cfgFA.ini -i2 -ds=pns -ep1 -r -ed -w.\Temp -s; -dp"%DIR1%" -m%MC% %exclude% "Journey.Bin"
RD /S /Q TEMP
pause


brispuss 05-11-2019 01:31

Thanks for the detailed reply!

I've basically got cls-reflate working now by using the command-

Code:

arc create -mreflate archive inputfile
Similarly, I've also downloaded and installed cls-srep for Freearc as well. So the command for srep is -

Code:

arc create -msrep archive inputfile
As for the sample batch script, that is a little over my head at the moment!

As of now, by experimenting with different (pre)compressors, the best compression for *.png files seems to be by using -

Precomp V0.4.7 (-cn -brute options), and then using lolz V22c4b (-d128 -mc1023 options).

The test file common_game_04.png at 113 kB is compressed to 90 kB. Not too bad, but I would like better compression if possible.

As for the *.ogg files, they are proving to be extremely difficult to compress much further!


EDIT: I've got the common_game_04.png file compressed down to 77 kB by using Precomp and then using fp8 V6 at maximum compression (-8 option).

And by using paq8px V183fix1 (-9beta options) instead of fp8 V6, common_game_04.png is now compressed to 72 kB, but this compression process is VERY SLOW!

amcic1994 03-01-2020 13:34

Quote:

Originally Posted by brispuss (Post 483320)
Thanks, but which command line executable should be run? Arc.exe? Or?

I tried the command "arc -mreflate" only but only get arc showing commands help!? Also pornjk porn800 redtube tried "arc -mreflate+srep+lolz" but still get arc showing commands help only!?

Trying to save space on HDD by LOSSLESSLY compressing some games for archiving purposes. And later decompressing these games LOSSLESSLY to match the orginal files exactly (bit perfect).


All times are GMT -7. The time now is 15:14.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com