A consistent maximum size for data compression

In February, I wrote about an experiment in data compression. The resulting algorithm is lossless and has an unanticipated virtue: it yields a compression that is consistent across all data types, as it indiscriminately reduces it to the type integer as a primary step.

This results in a theoretical max value for storing bulk data. While some data types may respond better to specialized compression strategies, ALL data reduces by the SAME ratio when considered by [modified] "Golumb" compression. 

This can apply to storage estimates.

Comments

Post a Comment

Popular posts from this blog

A Simple Open DRM

We love to hate DRM, because we associate it with degraded quality or loss of data control. Sometimes we associate it with loss of privacy when we register.  Here is a formative "Open DRM." The kernel of the system is going to be the idea that a salted hash is a member of a class of some kind. If I start with a MEMBER md5 hash and run 200,000 iterations of the correct salt, I hit a control value predictably.  The potential problem arises not from providing 199,999 different possible keys, but from providing a registered customer with his own valid replacement, on that rare occasion when he proves he has purchased it, and we must re-present his original key. Standing behind the warranty is what makes us reputable. To preserve his privacy AND the security of our system, we do not want to store the enumerated value on the server.  However, we can reasonably suppose that we can manage to keep a secret value secure. I propose that the DBM or LLC choose a passphrase of some kin...
Read more

A Question About Erasthmus' Sieve

Students of Mathematics are familiar with Erasthmus' Sieve. 2     x 3          x                4       x 5               x 6     x     x 7 8     x 9          x 10       x          x 11 12     x     x 13 14     x 15          x     x 16     x 17 18     x     x 19 20     x          x 21          x 22     x 23 First, we strike all multiples of 2 up to the square root (5.) Then all multiples of 3, then all multiples of 5, etc. I wrote a digital implementation of the Sieve, and used the index cr...
Read more

A "Secure" OS made PRACTICAL

I can remember suggesting a theoretical Operating System that requires object code to be compiled in such a way that a decryption key is required for a program object to execute. The program object itself would execute unencrypted in RAM.  I cannot find the entry at the moment, but I have also suggested an OS that uses an RSA key for program installation. Separately, I have contemplated a Linux user, organized such that the encrypted home folder appeared as an UNMOUNTED container over IP, while operating as a MOUNTED container locally, for the profile user. This would defeat some purposes of SSH, but would harden the installation against data theft over IP.  Now, with the development of an offline username/password for a program, I am asking myself if I cannot effectively suggest a practical (as opposed to theoretical) arrangement, whereby the program presents a credential (or two - username/password) to the OS, for permission to execute. Clearly, multiple program objects w...
Read more