A Detailed Topic Summary of Entropy Encoding

All these are interrelated and can hold subsets among themselves. I mean a individual encryption can fall into more than one class.

Based on first categorization we have 2 types of cryptography, one is the fixed length cryptography and the other is variable length cryptography or the information encoding. [ 62 ]

{ Before acquiring into inside informations we ‘ll travel through the rudimentss and a theoretical account that is used as an illustration }

We Will Write a Custom Essay Specifically
For You For Only $13.90/page!


order now

Introduction to Digital Communication system

See the cell phone or in fact any digital communicating system.

Note: This lone gives out the brief lineation of the whole digital communicating procedure, and does non affect the complexnesss of the transition strategies, operating expense or manus off processs.

See the sender of a DC system.

Microphone i? Sampleri? Sampler i? Quantizer i? Encoder i? Modulator

Figure 1 [ 16 ]

The mike is the transducer or convertor which picks up the sound signals through a stop that converts the voice or sound signals into electric signals ( parallel ) , through an affiliated vitreous silica, or electro-mechanical sensitive stuff. [ 18 ]

The following phase is linear to digital convertor. ( Combination of a sampling station and quantizer )

The sampling station samples the linear signal to digital with the usage of a timed switch and obeys the sampling theorem [ 19 ] .

The trying theorem provinces that if any signal is sampled with frequence more than twice the highest frequence, so the original signal can be created utilizing a low base on balls filter [ 22 ] .

Figure 2 [ 23 ]

Now the procedure of quantisation can be described in English as “ rounding off ” . It is rounding off the sampled values to the nearest predefined electromotive force degree which can be treated as a symbol. That means which can be assigned a binary codification word. And ever remain the same degree for that specific symbol [ 17 ] .

Figure 3 [ 26 ]

After the sampling and the quantizing is done, the function comes in for the encoder which maps the electromotive force degrees to binary codification words. Then these codification words are modulated and transmitted [ 20 ] .

Figure 4 [ 25 ]

Example of encoder mapping these electromotive force degrees to binary codification words.

If the electromotive force has been divided in 20 degrees, in stairss of 0.5 V so it is mapped as, 0.0V= ” 00000 ” , 0.5v= ” 00001 ” , 1.0V= ” 00010 ” , 1.5V= ” 00011 ” and so on boulder clay 10.0= ” 1010 ” .

At the receiving system it is simple, it can be decoded back utilizing the expression up tabular array, sent through a low base on balls filter and therefore the original signal can be recreated [ 20 ] .

First categorization of encryption:

Fixed length encryption, and Variable length/Entropy encryption.

Fixed length coding

Fixed length encryption is the process of apportioning a fixed length binary codification word to the symbol that is to be stored or transmitted.

Fixed length coding illustration,

Let us see A=1, B=2, C=3, D=4, E= 5, F=6, G=7, H=8, I=9, J=10, K=11, L=12, M=13, N=14, O=15, P=16, Q=17, R=18, S=19, T=20, U=21, V=22, W=23, Y=25 and Z=26.

At least 5 spots are required to encode all the 26 alphabets, because 2^4 is 8 and 2^5= 32.

Let us get down it as A= ” 00000 ” , B= ” 00001 ” , C= ” 00010 ” , D= ” 00011 ” , E= ” 00100 ” , F= ” 00101 ” , G= ” 00110 ” , H= ” 00111 ” , I= ” 01000 ” , J= ” 01001 ” , K= ” 01010 ” , L= ” 01011 ” , M= ” 01100 ” , N= ” 01101, O= ” 01110 ‘ , P= ” 01111 ” , Q= ” 10000 ” , R= ” 10001 ” , S= ” 10010 ” , T= ” 10011 ” , U= ” 10100 ” , V= ” 10101 ” , W= ” 10110 ” , X= ” 10111 ” , Y= ” 11000 ” , Z= ” 11001 ” .

Now Let us Encode the word “ SAMUEL ” , = ” 10010-00000-01100-10100-00100-01011 ”

Entire figure of bits= 5×6 =30. So in this method 30 spots are required to encode a individual English capital alphabet name. [ 1 ]

Entropy Encoding.

On the contrary in entropy encryption technique is the 1 that involves delegating variable length codifications to symbols, the length of the codification is decided by the chance of happening of that symbol. Again the finding of the chance of a symbol goes to its statistics. [ 1 ]

“ Information encoders really compress the information by replacing symbols represented by equal-length codifications with symbols represented by codifications relative to the negative logarithm of the chance ” [ 1 ] .

“ In this method of information encoding the quantal sub-bands are divided into rectangular codification blocks. The Bit-Plane Coder categorizes each spot in the codification block coefficient in to 3 cryptography base on ballss, each spot belonging to one coding base on balls. It so sends the codification block spot plane from most important spot ( MSB ) to least important spot ( LSB ) and arranges the spots in a random order for each spot plane. ” [ 2 ] .

Entropy encoding illustration:

Let us see the same illustration encoding “ SAMUEL ” in binary, the length of binary codification word is 30 in fixed length.

But in Entropy encoding we assign the codification word based on the frequence,

We Know That

Vowels are the most often repeated letters in English so we assign

A= ” 0 ” , E= ” 1 ” , I= ” 00 ” , O= ” 01 ” , U= ” 10 ” ,

And often repeated letters are S, F, R, V, Z and few others.

So I assign S= ” 000 ” , M= ” 001 ” , L= ” 010 ”

Now when I encode this in binary

SAMUEL = “ 000-0-001-10-1-010 ” . Entire length of binary symbols = 13. ( Entropy Encoding )

Compared to

SAMUEL = ” 10010-00000-01100-10100-00100-01011 ” . Length = 30 binary symbols. ( Fixed length coding )

So in this instance it is more than 200 % efficient than fixed length cryptography.

Another categorization of encoding

There are once more 2 types of encoding based on different categorization.

The lossy encryption and the lossless encryption. ( Other categorizations are non in range of this papers )

Lossy: – Involves actual loss of information, decrease in quantisation degrees and mistakes such as 5.345676 may be shortened and represented as 5.36. Because for a normal human being the difference of.001 does n’t truly count.

Lossless: – There is perfectly no loss of information, like 5.3456 is literally encoded as 5.3456 in double star and non anything else, therefore the name lossless.

Entropy encoding inside informations: –

Lossless compaction is fundamentally some sort of information encryption and it is based on information theory.

All the below are the illustrations of information

* “ The Shannon-Fano Algorithm

* Huffman Coding

* Huffman Coding of Images

* Adaptive Huffman Coding

* Arithmetic Coding

* Lempel-Ziv-Welch ( LZW ) Algorithm ” [ 3 ]

To understand these, foremost we need to understand the rudimentss of information theory, harmonizing to Shannon, the information of an information beginning S is defined as “ The chance that symbol Si in S will happen, indicates the sum of information contained in Si, i.e. , the figure of spots needed to code Si. ” [ 4 ]

For illustration, in the above illustration the chance of happening makes all the difference, the information of vowels and frequent letters is much higher than non frequent letters and consonants and therefore allocated higher information [ 4 ]

I once more like to specify information here in context as “ a step of upset, or more exactly unpredictability ” [ 15 ] .

The Shannon-Fano Algorithm

This is a basic information theoretic algorithm. A simple illustration will be used to exemplify the algorithm:

Symbol S A M U E L

— — — — — — — — — — — — — — — — — — — –

Count 15 7 6 6 5 1

Algorithm Shannon-Fano Algorithm:

“ A top-down attack

1. Kind symbols harmonizing to their repeat, e.g. , SAMUEL.

2. Repetitively divides into two parts, each with approx. same figure of counts. “ [ 5 ]

Symbol Count log ( 1/p ) Code Subtotal ( # of spots )

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

S 15 1.38 00 30

A 7 2.48 01 14

M 6 2.70 10 12

U 6 2.70 110 18

E 5 2.96 111 15

L 1 2.99 0000 4

TOTAL ( # of spots ) : 93 [ 5 ]

Huffman Coding

“ Huffman cryptography is based on the frequence of happening of an information point ( pel in images ) . The rule is to utilize a lower figure of spots to encode the information that occurs more frequently. Codes are stored in a Code Book which may be constructed for each image or a set of images. In all instances the codification book and encoded information must be transmitted to enable decryption. “ [ 6 ] .

The Huffman algorithm is now briefly summarized:

“ A bottom-up attack

1. Low-level formatting: Put all nodes in an Open list, maintain it sorted at all times ( e.g. , ABCDE ) .

2. Repeat until the OPEN list has merely one node left:

( a ) From OPEN choice two nodes holding the least chances and make a parent node of them.

( B ) Assign the amount of the kids ‘s frequencies/probabilities to the parent node and infix it into OPEN.

( degree Celsius ) Assign codification 0, 1 to the two subdivisions of the tree, and cancel the kids from OPEN. ” [ 6 ] .

Symbol Count log ( 1/p ) Code Subtotal ( # of spots )

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — –

S 15 1.38 0 15

A 7 2.48 100 21

M 6 2.70 101 18

U 6 2.70 110 18

E 5 2.96 111 15

L 5 2.96 0000 4

TOTAL ( # of spots ) : 95

Inferences:

Decoding for the above two algorithms is simple every bit long as the cryptography tabular array ( cardinal ) is sent before the information is sent. [ 6 ] .

If statistics are available so Huffman coding is really good. In the above illustration: Number of spots needed for Huffman Coding is: 95 / 39 = 2.43 [ 6 ] .

“ Huffman Coding of Images In algorithm can be given out as: Divide image up into 8×8 blocks

Each block is a symbol to be coded computes Huffman codifications for set of block Encode blocks consequently ” [ 7 ] .

Basically an image is a 3 dimensional matrix, the deepness of the matrix i.e. the 3rd dimension denotes the colour degree or grey degree image ( Based on it is black or white ) .

Adaptive Huffman Coding

The basic Huffman algorithm can be improved based on the conditions.

“ ( a ) The old algorithms require the statistical cognition which is frequently non available ( e.g. , unrecorded sound, picture ) .

( B ) Even when it is available, it could be a heavy operating expense particularly when many tabular arraies had to be sent when a non-order0 theoretical account is used, i.e. taking into history the impact of the old symbol to the chance of the current symbol ” . [ 8 ] .

The solution is to utilize adaptative algorithms. The algorithm does the existent clip analysis of informations before we encode them and based on which bulk of the compaction techniques are based. [ 8 ]

“ {

ENCODER DECODER

— — — – — — — –

Initialize_model ( ) ; Initialize_model ( ) ;

while ( ( c = getc ( input ) ) ! = eof ) while ( ( c = decode ( input ) ) ! = eof )

{ {

encode ( hundred, end product ) ; putc ( hundred, end product ) ;

update_model ( degree Celsius ) ; update_model ( degree Celsius ) ;

} }

} ” [ 8 ]

“ The reply is to hold both encoder and decipherer to hold the same low-level formatting and update_model maps.

Update_model does two things:

( a ) Incrementing the count,

( B ) Update the Huffman tree.

During the updates, the Huffman tree will be maintained its sibling belongings, i.e. the nodes ( internal and foliage ) are arranged in order of increasing weights ( see figure below ) ” . [ 8 ]

Figure 5 [ 8 ]

“ From the figure above when swapping is necessary, the farthest node with weight W is swapped with the node whose weight has merely been increased to W+1. Note: If the node with weight W has a subtree beneath it, so the subtree will travel with it. The Huffman tree could look really different after node swapping, e.g. , in the 3rd tree, node A is once more swapped and becomes the # 5 node. It is now encoded utilizing merely 2 spots. Note: Code for a peculiar symbol alterations during the adaptative cryptography procedure. ” [ 8 ] .

Arithmetical Cryptography

Huffman coding utilizations an whole number figure ( K ) of spots for each symbol. [ 10 ]

Arithmetical cryptography is bit hard to understand if explained abstractly with words so I will utilize a diagram to exemplify. ( I used the pneumonic of “ ZOOM IN ” to retrieve arithmetic cryptography in my undergrad. )

Eg

For all 6 missive words make a series of combinations from “ AAAAAA ” , “ AAAAAB ” , “ AAAAAC ” , and so on boulder clay “ ZZZZZZ ”

And in between one of the beast force dictionary combination would be “ SAMUEL ” .

Figure 6 [ 9 ]

And the chance of “ SAMUEL ” as a whole would be chosen from the figure of 6 missive blocks that are transmitted. It is really, really advantageous for block transmittal of informations where a peculiar block occurs often. Just imagine “ SAMUEL ” the whole block may be mapped to a individual spot ‘1 ‘ or ‘0 ‘ . So all those 30 spots of fixed length cryptography OR 13 spots of Huffman coding can be literally compressed to a individual spot. ( Ignoring overhead compared to big informations, clairvoyance. utile in image compaction ) . [ 24 ]

The image is pretty self explanatory, spliting the word lengths and delegating chances to them.

The bottom line of arithmetic encryption goes like this “ Start with conjecture of symbol frequences. Update frequence with each new symbol Note: This technique can be CPU and memory intensive ” [ 9 ] .

Till this point in the study we have done the rudimentss that are required to understand the deeper facets. The subsequent parts this study chiefly focuses on the deepness of the topic, research oriented affair and proficient challenges. First treatment in this country would be sing different documents that are published in this field.

Recent works Extension of the audio theoretical account explained to images and advanced subjects in this country.

“ Analysis of Fractals, Image Compression, Entropy Encoding, Karhunen-Loeve Transforms ” [ 10 ] .

This is a paper which is published in this field and negotiations about Entopy encoding and many things but they are concerned about Entropy encoding in the Hilberts Space. They are fundamentally seeking to utilize the autocorrelation coefficient of the beginning to accomplish better compaction and turn out it by mathematical techniques. [ 10 ]

They take this thought from another paper which gives the theoretical base for this new mathematical cogent evidence [ 11 ] .

Till now we were speaking about an audio encryption mechanism, in one of the old illustrations I mentioned that, an image is a 3 dimensional matrix with the pel information in the 3rd dimension.

Figure 7 [ 13 ]

This paper extends the same construct of sound that was discussed till now and extends it to image compaction. In this paper ( Comparison of Discrete and Continuous Wavelet Transforms ) , the writers try to convey out the relation between distinct and uninterrupted ripple transforms. The utilizations of subband filters are besides outlined by them. ( Previously I wrote approximately low base on balls filters for signal diversion and high base on balls filter for trying ) . We all know that there is a tool kit in matlab for ripple transmutation, an of import portion of that tool kit was developed based on the mathematics published in this paper. The nexus between the uninterrupted and distinct transform is given through a complex mathematical relation that is developed here [ 12 ] .

Besides a batch of techniques use the mathematical theoretical account of Hilbert ‘s for calculation, really this Hilbert ‘s infinite is a based Euclidian theoretical account where a individual dimensional infinite ( audio ) or 3 dimensional infinite ( image ) or any other theoretical account can be extended any dimensional concretion theoretical account. This is really utile because we can encode multiple redundancies, independently without upseting each other. ( Very helpful because multiple redundancies encoded in same process ) [ 14 ] . With this theoretical account as base a low memory demand line based encoding mechanism has been developed [ 46 ] .

Redundancy encoding purely talking does n’t come under encoding instead it can be called compaction technique. There are fundamentally 2 types of redundancies, spacial and temporal. If we try to take spacial redundancy without the group mapping categorization it becomes lossy compaction. [ 21 ] ( Spatial – redundancy w.r.t infinite, same thing repeated over and once more ; temporal- related to clip same frames exist over and once more ) .

Modern encryption criterions and most widely known 1s are MPEG and JPEG they have been developed and optimized for images and traveling images. They are really widely used around the universe without the regional boundaries. They are called the variable length codifications and utilize the basic rules of Huffman coding. [ 27 ]

The more the compaction ratio the longer it takes to treat the file because a more power algorithm like arithmetic cryptography has to be employed which is really hard on the CPU [ 30 ] .

“ The RVLC decryption decreases the sum of table infinite required, peculiarly for MPEG-4 picture RVLC decryption on a DSP bit. Because of the particular construction of RVLC codewords, the decrypting techniques that are common for regular Variable Length Codes are less efficient comparitively. This method does non utilize the codeword itself to entree a search tabular array, but alternatively forms a kind of hash index. This reordering can significantly diminish the sum of memory required by cut downing spreads or redundancy in the tabular array entries ” [ 33 ] . ( RVLC – Reversible variable length codification )

“ There is one more strategy that allows bidirectional decryption of variable length coded bitstreams from either terminal. Except for a little i¬?xed figure of excess spots appended to a sequence of codification words, the strategy is every bit efi¬?cient as Huffman coding. The excess operations required at programmer and decipherer are code word reversal and one EXOR for each spot. “ [ 34 ] .

Figure 8 [ 34 ]

Research workers have developed one more method to synchronise a variable length codification with the timing. They do by implementing 2 phases of encoding [ 35 ] . There are fundamentally 4 primary documents published that Tell about synchronism added to the Huffman cryptography. Everything is reasonably much the same except for some minor differences. They really have two phase decryption and give them all possible names to do it look fancy [ 36-39 ] .

Fast Decoding technique of Huffman coding is one of the paper published which tries to better the velocity of decrypting by bettering the traditional method utilizing some add-on to it or adding excess search tabular arraies. [ 40 ]

Such sorts of add-ons require some specialised hardware or VLSI circuits or processors with particular purpose architecture ( something like the CUDA ‘s or GPU ‘s ) [ 41 ] . Hence a general intent technique was to be developed which does the reconciliation act, one such thing was proposed by use of extra or external memory [ 42 ] .

The MPEG criterion, Mpeg stands for Traveling Picture Experts Group ; it is developed for both audio picture and synchronism. It uses a proprietary rule called gesture vector. The website claims that “ the file size is smaller for same quality ” [ 28 ] . The JPEG criterion ( Joint Pictures Experts Group ) became an International Standard ( ISO/IEC 15444-1 ) in December 2000

and is presently available for download from ITU-T [ 28 ] . They have the proficient ushers to JPEG and MPEG in their web sites severally [ 44 ] , [ 45 ] .

Role of random noise in image compaction, random noise is really of import to animate an image that is image that is digitally stored ; an image without the quantisation noise in the end product subband filter is really soft and shaded. The acuteness adjust characteristic is based on adding random noise to the end product filters [ 30 ] . Anyhow the noise in any other signifier is extremely disliked [ 31 ] , because it spoils the image quality.

A paper has been written on Fixed-length coding for robust picture compaction. ( still manges to be little ) It uses MPEGs like frame ( outline ) but uses fixed length codifications, it is much faster than variable length cryptography because of its fixed length and easier operability [ 32 ] .

Figure 9 [ 50 ]

The natural image is the pure RGB image. From standard Torahs of natural philosophies we know that any colour can be created utilizing the right mixture of RGB, or their complementary colourss Cyan, Magenta and Yellow. [ 54 ]

The image construct can be really handily assumed as an extension of the digital communicating system audio theoretical account that is developed at the beginning of this papers.

The chief compaction technique of jpeg is the DCT system, in a manner it can be referred to lossy encoding based on the parametric quantity alpha and besides one given to the down sampling station [ 47 ] .

The distinct cosine transform can be given as

Figure 10 [ 51 ]

This is for a individual dimension array the two dimensional DCT ( for images ) , We can specify images as a map degree Fahrenheit ( x, y ) , where the grey graduated table or luminescent value alterations with ( x, y ) [ 58 ] .

So the two dimensional DCT can be given as

Figure 10b [ 52 ]

Here MxN is the size of the image. It ‘s truly fun to imitate DCT on the computing machine utilizing MATLAB plan, as you change the parametric quantity alpha you can see the pixelation growth.

The DCT coefficient matrix consists largely of nothings, ( standards used to cut down file size ) .

The best portion of DCT is that it can be split into blocks and processed. This gives a batch of range to multi nucleuss and tasking to be used expeditiously [ 53 ] .

Parameters inside JPEG, MPEG and other techniques.

Rate control is really of import while a picture file is being played or watched. It should keep standard picture quality throughout the session and should be immune to localised spikes. For this a technique called ( ARCIAM ) is proposed for Motion JPEG2000. [ 43 ] , [ 57 ] .

A really good manner of proving the algorithms, hardware and picture is utilizing the SVT trial set developed and published in paper “ SVT High Dei¬?nition Multi Format Test Set ” [ 55 ] .There is besides one more package ( Kakadu V2.2 Software ) that does this occupation that is available and can be obtained through given mention. [ 56 ]

The rate control systems

There are a batch of documents published on the rate control systems, their all already many discrepancies and each one is optimized for a specific demand.

Low Complexity JPEG2000 Rate Control through Reverse Sub set Scaning Order and Coding Passes Concatenation [ 59 ] . “ This is a rate control method devised to supply quality to JPEG2000 codification watercourses incorporating a less figure of quality beds. It is based on a Reverse subband scanning order and a cryptography passes Concatenation ( ROC ) that does non utilize deformation steps based on the original image. “ [ 59 ] .

Another attack in this way is “ two new methods for efficient rate control and information cryptography in lossy image compaction utilizing JPEG-2000. These two methods allow them to hold a important betterment in calculation complexness and power ingestion compared to the original JPEG-2000 algorithms. First, a avaricious heap-based rate-control algorithm ( GHRaC ) , which achieves efficient station compaction rate control by implementing a avaricious fringy analysis method utilizing the heap kind algorithm. Second, an integrated rate-control and entropy-coding ( IREC ) algorithm that reduces the calculation complexness of information cryptography by selectively entropy coding merely the image informations that is likely to be included in the concluding spot watercourse, as opposed to entropy coding all image informations. “ [ 60 ] .

One of the other attack to rate communicating systems is “ distortion-based model for rate control that enables a JPEG2000 encoder to accomplish a user-specified quality, and hence doing it possible to bring forth changeless quality from frame-to-frame in an image sequence. This method makes direct usage of the same JPEG2000 coding base on balls informations as the traditional attacks, and therefore can easy be adopted at the back terminal of JPEG2000 encoding engines. We compare this method with two other common rate control techniques for JPEG2000. ” [ 48 ] .

Apart from these major 1s there are many more algorithm discrepancies of the same thing, as mentioned, they all are specialised versions of some specific undertaking.

Mistake opposition in coding

It is one more of import factor of the cryptography to be error immune, I mean the cryptography should hold built in steps to observe and if possible right mistakes. So in most of the encoding techniques some sort of mistake opposition capableness is included. One of such codification is the reed Solomon cryptography

Before acquiring into existent inside informations we define few more footings like transition. Modulation is changing of the high frequence bearer signal w.r.t the instantaneous amplitude of the signal to be transmitted [ 49 ] .

Reed Solomon Code

“ Reed Solomon codifications are a subset of BCH codifications and are additive block codifications. A Reed-Solomon codification is specified as RS ( n, K ) with s-bit symbols. ” [ 63 ]

The notation shows aout of n entire spots k spots are for the para.

Figure 11a [ 63 ]

The above illustration shows the reed Solomon codification with K informations spots and 2t para spots

A popular Reed-Solomon codification is RS ( 255,223 ) with 8-bit symbols. Each codeword contains 255 codification word bytes, of which 223 bytes are informations and 32 bytes are para. [ 63 ] .

Reed Solomon Encoder

Fig 11b [ 63 ]

It Works on the mathematical equation described above the image. “ All the 6 registries hold a symbol ( 8 spots ) . The arithmetic operators carry out finite field add-on or generation on a complete symbol. “ [ 63 ]

Reed Solomon Decoder

Figure 11c [ 63 ]

“ The standard codeword R ( x ) is the original ( transmitted ) codeword degree Celsius ( x ) plus mistakes:

R ( x ) = degree Celsius ( x ) + vitamin E ( ten )

A Reed-Solomon decipherer attempts to place the location and magnitude of up to t mistakes ( or 2t erasures ) and to rectify the mistakes or erasures. ” [ 63 ]

Decision

Analogue to digital encryption is required for

Digital transmittal, [ 64 ]

Application of DIP algorithms, [ 58 ]

Digital stored files are easy to copy, ( eg of parallel stored file is tape. ) . [ 65 ]

Digitally encoded signals are more immune to resound, ( Because Noise random, Gaussian, and white noise are analog in nature ) . [ 65 ]

It ‘s easier to threshold binary signals infested with noise instead than parallel signals. [ 65 ]

And eventually the most of import thing is the ability to execute encoding [ 66 ] .

Hardware for parallel processing encoders and encrypters are much more expensive and complicated compared to package 1s. [ 67 ]

Leave a Reply

Your email address will not be published. Required fields are marked *