An Experiment In Data Compression

There are a finite number of compression algorithms. One strategy is to catalog differences in consecutive values and compress a dictionary of the differences.

A simple experiment to perform in Rust or C++20, is as follows.

1. Read the file as an array of unsigned int (16 bits each.) To do this, it is important to understand encoding. I avoid Microsoft's Unicode encoding. An alternative is to read the file in 8-bit bytes, and concatenate every two-byte pair into an array of unsigned int. The numeric value represented is not the purpose. Declare a large enough array space in RAM to accommodate some max_size value.

2. delta_encode the array. There is standardized code to do this. Python3 has a function in module numpy.

3. Golomb-encode the resulting dictionary or array. There is standardized code to do this. This falls into the category of run-length encoding.

Viewing pixels from a raw photo format or an mp3 as unsigned int is the innovation suggested. 

4. Write the result in a file with a ".gcf" compression extension.

5. If the file is already compressed, Golomb-decode first, then delta-decode, dropping any null byte at the end to pad an odd non-pair byte.

6. Save as original file, deleting ".gcf"

This is my attempt to be novel at the compression game.

Comments

Post a Comment

Popular posts from this blog

DRM and Password Authorization

We log in online and are accustomed to being refused service if we supply an incorrect username/password combination.  We persecute "plain-text offenders" and rightly expect to benefit from comparing a hash fingerprint of the credential presented online to a list of hash fingerprints, each paired with a username. Even better if the username is hashed as well. The knowledge gained from online handshakes can benefit us in digital rights management.  It has long been the case that we could obfuscate a password literal in source code by compiling it into object code. However, decompilers provide assembler code that can reconstruct the source code without descriptive variable names and make string values stand out. Now, we can construct a creative solution. We can designate a username/password combination to COMPILE SOURCE UNIQUELY FOR EACH USER. To hide it in the object code, we hash the username and password with a published hash algorithm and use the hash-fingerprint literal in...
Read more

How to best the Bitcoin cryptocurrency lotto;

A  hi-speed box is prerequisite; emphasis on read/write speeds on duel 4T SSDs. 32 GB Hi speed RAM; overclocked Core I5 12600 or Core I7 22700. I estimated 32 or 64 units… An ASIC is a circuit that provides a hash value that (critically) BEGINS with &h0000. Time to calculate/randomly poll for these is the intractable problem. Example: When you play TX lotto, EVERY entry has same probability. Buuut; if you play A SINGLE unique entry repeatedly, it has the same probability the FIRST time. However, thereafter, the winning solution becomes constrained to be x + (n) / total possible. Since the requirement to win BECOMES x2 AND ~x1. Else the entry would have won before.  While this is statistically insignificant, we can capitalize within the cryptomining protocol. After completing a bid, whether successful or unsuccessful instead of discarding the HASH, we can proceed to play the same number again. Starting with &h0000, and hashing once per round, as prescribed. But to ...
Read more

A "Secure" OS made PRACTICAL

I can remember suggesting a theoretical Operating System that requires object code to be compiled in such a way that a decryption key is required for a program object to execute. The program object itself would execute unencrypted in RAM.  I cannot find the entry at the moment, but I have also suggested an OS that uses an RSA key for program installation. Separately, I have contemplated a Linux user, organized such that the encrypted home folder appeared as an UNMOUNTED container over IP, while operating as a MOUNTED container locally, for the profile user. This would defeat some purposes of SSH, but would harden the installation against data theft over IP.  Now, with the development of an offline username/password for a program, I am asking myself if I cannot effectively suggest a practical (as opposed to theoretical) arrangement, whereby the program presents a credential (or two - username/password) to the OS, for permission to execute. Clearly, multiple program objects w...
Read more