Go Back   FileForums > Off Topic > Software

Reply
 
Thread Tools Display Modes
  #1  
Old 11-02-2017, 00:20
PrinceGupta2000's Avatar
PrinceGupta2000 PrinceGupta2000 is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 195
Thanks: 49
Thanked 222 Times in 89 Posts
PrinceGupta2000 is on a distinguished road
Preprocessing / Optimization / Re compression For Better Compression of Games

[DEFLATE/ZLIB]
1.Precomp: Precomp is a command line pre-compressor. You can use it to achieve better compression on some file-/streamtypes (works on files and streams that are compressed with Zlib or the Deflate compression method, bZip2, GIF, JPG, and MP3). Precomp tries to decompress the streams, and if they can be decompressed and "re-"compressed so that they are bit-to-bit-identical with the original stream, the decompressed stream can be used instead of the compressed one.
Links:
:Gitter Chat
:Github Repository
:Github Releases
:Universal CLS-Precomp By RAZOR12911
:CLS-PRECOMPMT By RAZOR12911
Downloads:
:PrecompInside0.31.7z

2.PZLIB V3: A fast and really very fastest re-compressor for Deflate and Zlib, developed by RAZOR12911 but has some bugs when restoring several files via stdin stdout. Hence advised to be used separately on files
Links:
:Official Thread

3.PZLIB V1: The predecessor of PZLIB V3, fast and good for games containing zlib streams compressed using default and maximum level
which have headers example GTA IV, Dragon Age etc.
Links:
:Official Thread

4.Reflate: universal deflate re-compressor
Links
:CLS-Reflate by RAZOR12911
Download:
:Reflate V1l1 (GCC Compiled)

[Stream/Specific Compressors]
1.Media Stream Compressor (MSC): bmp Packer/preprocessor for multimedia data. It currently supports the following data types: dds (raw, dxtBMP, Wav (PCM), mp3. Because This preprocessor is sharpened by working with freearc, the data processing is carried out with a single input file. The output is just one file of its own format.
Suggestion: Should be used be before rep or basically before any "compressor"
-mmsc+srep+lzma or -mprecomp+msc+srep+lzma -> correct
-mprecomp+srep+msc or -msrep+msc -> incorrect, will ruin compression
Download:
:msc0.064.7z (Contains CLS)

2.MPZ: Best Mp3 compressor Developed by Eugene Shelwein. It usually reduces file size by 15 % but on some files, it can go par 60%
Download:
:mpzapi_v1a.rar

3.PackJPG: A compression program for further compressing JPEG image files Github:

4.PackARC: packARC is an archiver program specially designed for lossless further compression of media files in the JPEG, MP3, BMP and PNM formats. Github:

5.PackMP3: A compression program for further compressing MP3 audio files Github

6.PackPNM: A compression program for further compressing PNM, BMP and Radiance HDR files Github

7.unpackJPG: Unpack only, PackJPG Github

8.Lepton: Lepton is a tool and file format for losslessly compressing JPEGs by an average of 22%.
Links
:Github repository
:Github releases
:Official Blog Post

9.Png optimize suite(Win32) (Ver 2012-06-15): set of console compressor for Pngs
Download
:pngopt-120615.7z
:Blog Post

10.OptimFrog:OptimFROG is a lossless audio compression program. Its main goal is to reduce at maximum the size of audio files while permitting bit identical restoration for all input. It is similar with the ZIP compression, but it is highly specialized to compress audio data.
Downloads:
OptimFROG_Win_x86_5100.zip
OptimFROG_Win_x64_5100.zip

11.TAK: Tom's lossless Audio Kompressor (TAK) is a lossless audio compressor which promises compression performance similar to Monkey's Audio “High” and decompression speed similar to FLAC.
Downloads:
:TAK_2.3.0.zip
Links
:HomePage

[Prerocessors]

Delta: This preprocesses data improving their further compression. It detects tables of binary records and
1) substracts successive values in columns,
2) reorder columns trying to maximize results of further compression.
should be used on raw data i.e. not processed with srep, any deflate compressor, or something other

Usage:
Code:
fazip delta infile outfile
Rep : A LZ77 compressor detects repetition in 1GB Block
Usage:
Code:
fazip rep:<dictionary size>mb infile outfile
Super-rep : Successor of rep, supports dictionaries larger than RAM available.Compression speed is 100 mb/s and decompression runs at 250 mb/s on i3-2100.
:Description of The Algo
:srep393a.7z
CLS:SrepInside0.33.7z

[Loossy Compressors]

OggEnc: A very good lossy Compressor for Waves
Links
Official Site
Downloads:
oggenc2.88-1.3.5-generic.zip
oggenc2.88-1.3.5-x64.zip

Binks Lossy Compressor: Binks Reencoding script written by PANKER1992.
Link:
Official Discussion
Downloads
Bink_Enc.7z
__________________
None knoweth whence creation has arisen; And whether he has or has not produced it;He who surveys it in the highest heaven, He only knows, or haply he may know not.

Last edited by PrinceGupta2000; 03-04-2017 at 21:48.
Reply With Quote
The Following 7 Users Say Thank You to PrinceGupta2000 For This Useful Post:
78372 (01-04-2017), kassane (15-02-2017), oltjon (07-04-2017), Prince4 (13-02-2017), Razor12911 (25-02-2017), romulus_ut3 (31-03-2017), Simorq (06-03-2017)
Sponsored Links
  #2  
Old 13-02-2017, 05:48
Prince4's Avatar
Prince4 Prince4 is offline
Registered User
 
Join Date: Mar 2015
Location: India->Delhi->Rohini
Posts: 91
Thanks: 70
Thanked 15 Times in 12 Posts
Prince4 is on a distinguished road
Smile

Some knowledge collaborative like Razor's!!

Keep it up!
__________________
Who's Got Time To Play Games? Huh??
Reply With Quote
  #3  
Old 14-02-2017, 23:23
PrinceGupta2000's Avatar
PrinceGupta2000 PrinceGupta2000 is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 195
Thanks: 49
Thanked 222 Times in 89 Posts
PrinceGupta2000 is on a distinguished road
ppx2: ppx2 allows parallel processing of arguments
Changes by me:
Quote:
setting priorities of the processes
start process with new console window
verbose mode
and others i don't remembering
Here it is
ppx2.7z

For help:
ppx2 --help

The source can be found here:
https://github.com/guptaprince/ppx2
__________________
None knoweth whence creation has arisen; And whether he has or has not produced it;He who surveys it in the highest heaven, He only knows, or haply he may know not.

Last edited by PrinceGupta2000; 27-02-2017 at 08:01.
Reply With Quote
  #4  
Old 15-02-2017, 00:06
EzzEldin16's Avatar
EzzEldin16 EzzEldin16 is offline
Registered User
 
Join Date: Sep 2016
Location: USA
Posts: 70
Thanks: 35
Thanked 18 Times in 11 Posts
EzzEldin16 is on a distinguished road
i have a question
i compressed a game with zlib streams found alot of them
i compressed with precompressor it packed well but unpacking didn't
is the problem with the .iss script or what i'm using wpi script by razor
__________________
learn it
Reply With Quote
  #5  
Old 15-02-2017, 00:16
PrinceGupta2000's Avatar
PrinceGupta2000 PrinceGupta2000 is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 195
Thanks: 49
Thanked 222 Times in 89 Posts
PrinceGupta2000 is on a distinguished road
Are u talking about pzlib?
Then it is the pre-compressor itself.
there is no way to fix that
__________________
None knoweth whence creation has arisen; And whether he has or has not produced it;He who surveys it in the highest heaven, He only knows, or haply he may know not.
Reply With Quote
  #6  
Old 23-02-2017, 09:28
ZakirAhmad ZakirAhmad is offline
Registered User
 
Join Date: Sep 2015
Location: Kmr
Posts: 75
Thanks: 11
Thanked 3 Times in 2 Posts
ZakirAhmad is on a distinguished road
Xdelta

How xdelta can be used to decompress Compressed files... I have seen it in crysis 3 repack of Fitgirl... The files r md5 after decompression are orginal...
Reply With Quote
  #7  
Old 24-02-2017, 00:13
Joe Forster/STA's Avatar
Joe Forster/STA Joe Forster/STA is offline
Senior forum member
 
Join Date: Nov 2000
Location: Hungary
Posts: 9,356
Thanks: 4
Thanked 146 Times in 89 Posts
Joe Forster/STA is on a distinguished road
Xdelta is used for patches. (You know, game X version 1.0 to game X version 1.1: version 1.0 files are the input, version 1.1 are the output.) It does compress the patch file, though. See .RTPatch for a very similar, but commercial, software.
__________________
Joe Forster/STA
For more information, see the FileForums forum rules and the PC Games forum FAQ!
Don't contact me via E-mail or PM to ask for help with anything other than patches (or software in general) done by me, otherwise your request may be deleted without any reply!
Homepage: http://sta.c64.org, E-mail: [email protected]; for attachments, send compressed (ZIP or RAR) files only, otherwise your E-mail will bounce back!
Reply With Quote
The Following User Says Thank You to Joe Forster/STA For This Useful Post:
Simorq (06-03-2017)
  #8  
Old 24-02-2017, 03:39
EzzEldin16's Avatar
EzzEldin16 EzzEldin16 is offline
Registered User
 
Join Date: Sep 2016
Location: USA
Posts: 70
Thanks: 35
Thanked 18 Times in 11 Posts
EzzEldin16 is on a distinguished road
@joe foster/STA can you explain how it is used precisely ??
__________________
learn it
Reply With Quote
  #9  
Old 24-02-2017, 07:46
Joe Forster/STA's Avatar
Joe Forster/STA Joe Forster/STA is offline
Senior forum member
 
Join Date: Nov 2000
Location: Hungary
Posts: 9,356
Thanks: 4
Thanked 146 Times in 89 Posts
Joe Forster/STA is on a distinguished road
Imagine that you're the developer of game X. You've already released version 1.0 (CD/DVD, Steam etc.) but found some bugs since (wow!). You fixed (some of) them, calling it version 1.1, but you can't re-release the whole game; you rather collect the changes into a patch. This patch can be a package of the full files that have been changed between version 1.0 and 1.1 but that can be really big if it contains many files and/or some of the files are big.

A smarter idea is collecting the actual differences between the files. (Obviously, if those files exist in both versions.) Say, you have a big data file that has an internal structure and it contains many little files. (E.g. as simple as a ZIP archive.) If you add one internal file then only that part of the big data file changes, the rest stays the same. If you fix some code and recompile the program file (executable) then, again, most of the file stays the same, only some parts of it (added code, added data, modified jump offsets, modified data pointers/references) will change.

Xdelta and .RTPatch are programs that can do a binary comparison between a bunch of "old" files (those of version 1.0) and "new" files (version 1.1) and create a patch. If your customer downloads the patch and has the "old" files, xdelta can use ("apply") the patch to convert the "old" files into the "new" ones.

It doesn't matter if the "old" and "new" files are big, if the actual differences are small then the patch will be small, too. You can create a single patch file out of the differences of multiple files, including creating a new file (that didn't exist in version 1.0 yet), deleting a file (that doesn't exist in version 1.1 anymore) or modifying a file. I'm not sure about xdelta but with .RTPatch you can also create multi-version patches (version 1.0 or 1.1 or 1.2 -> 1.3) that can modify many versions of "old" files into the "new" ones.

I've seen xdelta used in the game called "The Thing" (from 2002 or something). Most developers use .RTPatch because it probably has more features but it's extremely expensive (thousands of dollars) and closed source, while xdelta is free and open source.
__________________
Joe Forster/STA
For more information, see the FileForums forum rules and the PC Games forum FAQ!
Don't contact me via E-mail or PM to ask for help with anything other than patches (or software in general) done by me, otherwise your request may be deleted without any reply!
Homepage: http://sta.c64.org, E-mail: [email protected]; for attachments, send compressed (ZIP or RAR) files only, otherwise your E-mail will bounce back!
Reply With Quote
The Following User Says Thank You to Joe Forster/STA For This Useful Post:
EzzEldin16 (01-04-2017)
  #10  
Old 13-03-2017, 23:38
ImmortalArc's Avatar
ImmortalArc ImmortalArc is offline
Registered User
 
Join Date: Nov 2016
Location: Santiago
Posts: 65
Thanks: 15
Thanked 2 Times in 2 Posts
ImmortalArc is on a distinguished road
Quote:
Originally Posted by Joe Forster/STA View Post
.RTPatch it's extremely expensive (thousands of dollars) and closed source, while xdelta is free and open source.
Whoaaa
Attached Images
File Type: jpg IMG_20170314_130618_022.JPG (116.2 KB, 194 views)
__________________
Sharing is caring : immortal words..
Reply With Quote
  #11  
Old 31-03-2017, 21:37
PrinceGupta2000's Avatar
PrinceGupta2000 PrinceGupta2000 is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 195
Thanks: 49
Thanked 222 Times in 89 Posts
PrinceGupta2000 is on a distinguished road
UPDATED

- New Lossy Section
- Add some Other Compressor

Guys, Please please help me further improve this thread
Thanks.
__________________
None knoweth whence creation has arisen; And whether he has or has not produced it;He who surveys it in the highest heaven, He only knows, or haply he may know not.
Reply With Quote
The Following User Says Thank You to PrinceGupta2000 For This Useful Post:
romulus_ut3 (20-04-2017)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Games Are For Kids JoyBoy Chit Chat 81 20-02-2017 07:31
Multiple Games to DVD Grumpy PC Games - CD/DVD Conversions 6 22-12-2005 15:47
Frequently Asked Questions Joe Forster/STA PC Games - Frequently Asked Questions 0 29-11-2005 09:48
i got the email in the mail about copying ps2 games EVOEvo PS2 Games 7 21-11-2003 07:56
WARNING: New games use new compression for movies files! Flavangel DC Games 0 26-06-2001 14:35



All times are GMT -7. The time now is 10:19.


Powered by vBulletin® Version 3.8.10
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Copyright 2000-2017, FileForums @ http://fileforums.com