An Experiment In Data Compression

There are a finite number of compression algorithms. One strategy is to catalog differences in consecutive values and compress a dictionary of the differences.

A simple experiment to perform in Rust or C++20, is as follows.

1. Read the file as an array of unsigned int (16 bits each.) To do this, it is important to understand encoding. I avoid Microsoft's Unicode encoding. An alternative is to read the file in 8-bit bytes, and concatenate every two-byte pair into an array of unsigned int. The numeric value represented is not the purpose. Declare a large enough array space in RAM to accommodate some max_size value.

2. delta_encode the array. There is standardized code to do this. Python3 has a function in module numpy.

3. Golomb-encode the resulting dictionary or array. There is standardized code to do this. This falls into the category of run-length encoding.

Viewing pixels from a raw photo format or an mp3 as unsigned int is the innovation suggested. 

4. Write the result in a file with a ".gcf" compression extension.

5. If the file is already compressed, Golomb-decode first, then delta-decode, dropping any null byte at the end to pad an odd non-pair byte.

6. Save as original file, deleting ".gcf"

This is my attempt to be novel at the compression game.

Comments

Post a Comment

Popular posts from this blog

A Question About Erasthmus' Sieve

Students of Mathematics are familiar with Erasthmus' Sieve. 2     x 3          x                4       x 5               x 6     x     x 7 8     x 9          x 10       x          x 11 12     x     x 13 14     x 15          x     x 16     x 17 18     x     x 19 20     x          x 21          x 22     x 23 First, we strike all multiples of 2 up to the square root (5.) Then all multiples of 3, then all multiples of 5, etc. I wrote a digital implementation of the Sieve, and used the index cr...
Read more

An Improvement To The Three Second Hold Rule

The rule that a broker has to hold a share for three (3) seconds, or one hundred shares for three hundred (300) seconds, can be improved. Brokers wishing to game the system will simply proliferate buy orders, and hold multiple buys of consolidated numbers of shares for three (3) seconds. To limit this, each broker could be limited to one buy per second. In this way, if he wishes to invest in 60 companies, he will have to use 60 seconds to put the buy orders in. If he wished to buy 1,000,000 shares, but only wishes to hold them for 1000 seconds ( 16.6 minutes,) he would be forced to put in 1000 buys of 1000 shares each. This process would force him to use 16.6 minutes in the buy process, after which each individual buy would be ready to sell in 50 minutes.  Computer programs written to optimize time to hold v time to buy would be forced to keep track of all the buy orders.  An alternative would be to attempt to consolidate all buy orders, but this is game-able by a round robin ...
Read more

Notice of corrupted results: Vigenere may yet be found to be a "group."

Retraction: It is with regret that I do not delete the previous post. I am an individual, not an organization, and I failed to adequately double-check my results, but I am leaving the post to maintain some accountability. I am able to definitively say that the script for 2chars_from_file.py DOES NOT yield an exhaustive list. It develops the ciphertext for all 2 Char combinations of the FIRST 10, three character ciphertexts. This is not due to a logic error, but rather there is a virus that corrupts the indentation of the two write statements.  This may be verified by downloading the relevant code, and examining it, and running it if needed, against the relevant datafile. It is difficult to be exhaustively correct about ALL of the scripts. This calls ALL of my results in question, with regard to the qualification of a mathematical group. If Vigenere IS a group, the resulting substitution is still valid, but there exist 6 Char passwords that will accomplish the same encryption in ONE...
Read more