CRC-32

Article on other languages:

• CRC
• Control_de_redundància_cíclica
• Cyklický_redundantní_součet
• Zyklische_Redundanzprüfung
• Cyclic_redundancy_check
• Control_de_redundancia_cíclica
• CRC
• CRC
• Contrôle_de_redondance_cyclique
• בדיקת_יתירות_מחזורית
• CRC
• Cyclic_redundancy_check
• 巡回冗長検査
• Cyclic_Redundancy_Check
• CRC
• CRC
• Циклический_избыточный_код
• Cyclic_redundancy_check
• Kontrola_cyklickým_kódom
• Cyclic_Redundancy_Check
• CRC
• 循環冗餘校驗

	del.icio.us
	Digg
	Furl
	Reddit
	Rojo
	Add to OnlyWire

This article or section includes a list of references or external links, but its sources remain unclear because it has insufficient inline citations. You can improve this article by introducing more precise citations where appropriate. (January 2008)

A cyclic redundancy check (CRC) is a type of function that takes as input a data stream of any length, and produces as output a value of a certain space, commonly a 32-bit integer. The term CRC denotes either the function or the function's output. A CRC can be used as a checksum to detect accidental alteration of data during transmission or storage. CRCs are popular because they are simple to implement in binary hardware, are easy to analyze mathematically, and are particularly good at detecting common errors caused by noise in transmission channels. The CRC was invented by W. Wesley Peterson, and published in his 1961 paper^[1]. The IEEE-recommended 32-bit CRC used in Ethernet and elsewhere appeared at a telecommunications conference in 1975.^[2]

Introduction

A CRC is an error-detecting code. Its computation resembles a long division operation in which the quotient is discarded and the remainder becomes the result, with the important distinction that the arithmetic used is the carry-less arithmetic of a finite field. The length of the remainder is always less than or equal to the length of the divisor, which therefore determines how long the result can be. The definition of a particular CRC specifies the divisor to be used, among other things.

Although CRCs can be constructed using any finite field, all commonly used CRCs employ the finite field GF(2). This is the field of two elements, usually called 0 and 1, comfortably matching computer architecture. The rest of this article will discuss only these binary CRCs, but the principles are more general.

An important reason for the popularity of CRCs for detecting the accidental alteration of data is their efficiency guarantee. Typically, an n-bit CRC, applied to a data block of arbitrary length, will detect any single error burst not longer than n bits (in other words, any single alteration that spans no more than n bits of the data), and will detect a fraction 1-2^-n of all longer error bursts. Errors in both data transmission channels and magnetic storage media tend to be distributed non-randomly (i.e. are "bursty"), making CRCs' properties more useful than alternative schemes such as multiple parity checks.

The simplest error-detection system, the parity bit, is in fact a trivial CRC: it uses the two-bit-long divisor 11.

CRCs and data integrity vs. correctness

This section does not cite any references or sources. Please help improve this section by adding citations to reliable sources. Unverifiable material may be challenged and removed. (September 2008)

CRCs are not, by themselves, suitable for protecting against intentional alteration of data (for example, in authentication applications for data security), because their convenient mathematical properties make it easy to compute the CRC adjustment required to match any given change to the data.

It is often falsely assumed that when a message and its CRC are received from an open channel and the CRC matches the message's calculated CRC then the message cannot have been altered in transit.

This assumption is false because CRC is not really encryption at all: it is supposed to be used for data integrity checks, but is occasionally assumed to be used for encryption. When a CRC is calculated, the message is left in cleartext and the constant-size CRC is tacked onto the end (i.e. the message can be read just as easily).

Although CRCs share a problem with message digests in that there cannot be a 1:1 relationship between all possible messages and all possible CRCs, the CRC function fares worse because it is not a trapdoor function. That is, it is easy to generate other messages that result in the same CRC, especially messages similar to the original. By design however, a message that is too similar (differing only by a trivial noise pattern) will have a dramatically different CRC and thus be detected.

Alternatively the message could just be intercepted and replaced by a phony message with a new, phony CRC (creating a packet that would be verified by any Data-Link entity). So CRCs can be relied upon to verify integrity but not correctness.

In contrast, an effective way to protect messages against intentional tampering is by the use of a message authentication code such as HMAC.

Computation of CRC

Main article: Computation of CRC

To compute an n-bit binary CRC, line the bits representing the input in a row, and position the (n+1)-bit pattern representing the CRC's divisor (called a "polynomial") underneath the left-hand end of the row. Here is the first calculation for computing a 3-bit CRC:

11010011101100 <--- Input
1011           <--- divisor (4 Bits)
--------------
01100011101100 <--- result

If the input bit above the leftmost divisor bit is 0, do nothing and move the divisor to the right by one bit. If the input bit above the leftmost divisor bit is 1, the divisor is exclusive-ORed into the input (in other words, the input bit above each 1-bit in the divisor is toggled). The divisor is then shifted one bit to the right, and the process is repeated until the divisor reaches the right-hand end of the input row. Here is the last calculation:

00000000001110 <--- result of multiplication calculation
          1011 <--- divisor
--------------
00000000000101 <--- remainder (3 bits)

Since the leftmost divisor bit zeroed every input bit it touched, when this process ends the only bits in the input row that can be nonzero are the n bits at the right-hand end of the row. These n bits are the remainder of the division step, and will also be the value of the CRC function (unless the chosen CRC specification calls for some postprocessing).

Mathematics of CRC

Main article: Mathematics of CRC

Please help improve this section by expanding it. Further information might be found on the talk page or at requests for expansion. (June 2008)

Mathematical analysis of this division-like process reveals how to pick a divisor that guarantees good error-detection properties. In this analysis, the digits of the bit strings are thought of as the coefficients of a polynomial in some variable x -- coefficients that are elements of the finite field GF(2) instead of more familiar numbers. This "polynomial trick" allows bit strings to be viewed as elements of a ring. A ring is, loosely speaking, a set of elements somewhat like numbers, that can be operated on by an operation that somewhat resembles addition and another operation that somewhat resembles multiplication, these operations possessing many of the familiar arithmetic properties of commutativity, associativity, and distributivity. Many analytical tools commonly used with numbers also work on rings, and this is why the "polynomial" view helps the analysis.

Specifying a CRC

The concept of the CRC as an error-detecting code gets complicated when an implementer or standards committee turns it into a practical system. Here are some of the complications:

• Sometimes an implementation prefixes a fixed bit pattern to the bitstream to be checked. This is useful when clocking errors might insert 0-bits in front of a message, an alteration that would otherwise leave the CRC unchanged.

• Sometimes an implementation appends n 0-bits (n being the size of the CRC) to the bitstream to be checked before the polynomial division occurs. This has the convenience that the CRC of the original bitstream with the CRC appended is exactly zero, so the CRC can be checked simply by performing the polynomial division on the expanded bitstream and comparing the remainder with zero.

• Sometimes an implementation exclusive-ORs a fixed bit pattern into the remainder of the polynomial division.

• Bit order: Some schemes view the low-order bit of each byte as "first", which then during polynomial division means "leftmost", which is contrary to our customary understanding of "low-order". This convention makes sense when serial-port transmissions are CRC-checked in hardware, because some widespread serial-port transmission conventions transmit bytes least-significant bit first.

• Byte order: With multi-byte CRCs, there can be confusion over whether the byte transmitted first (or stored in the lowest-addressed byte of memory) is the least-significant byte or the most-significant byte. For example, some 16-bit CRC schemes swap the bytes of the CRC.

• Omission of the high-order bit of the divisor polynomial: Since the high-order bit is always 1, and since an n-bit CRC must be defined by an (n + 1)-bit divisor which overflows an n-bit register, some writers assume that it is unnecessary to mention the divisor's high-order bit.

Commonly used and standardized CRCs

While cyclic redundancy checks form part of several standards, they are not themselves standardized to the point of adopting one algorithm of each degree worldwide: there are three known CRC-12 polynomials^[3], at least eight documented variations of CRC-16, and three of CRC-32^[4]. The polynomials usually seen are not the most efficient ones possible. Between 1993 and 2004 Koopman, Castagnoli and others surveyed the space of polynomials up to 16 bits^[5], and of 24 and 32 bits,^[6][7] finding examples that have much better performance (in terms of Hamming distance for a given message size) than the polynomials of earlier protocols, and publishing the best of these with the aim of improving the error detection capacity of future standards^[7]. In particular, iSCSI has adopted one of the findings of this research.

Far from being arbitrarily chosen the popular CRC-32 polynomial, recommended by the IEEE and used by V.42, Ethernet, FDDI and ZIP and PNG files among others, is the generating polynomial of a Hamming code and was selected for its error detection performance^[2]. Even so it is outperformed by the Castagnoli CRC-32C polynomial used in iSCSI^[7].

The table below lists only the polynomials of the various algorithms in use. Any particular protocol can impose pre-inversion, post-inversion and reversed bit ordering as described above. CRCs in proprietary protocols might use a complicated initial value and final XOR for obfuscation but this does not add cryptographic strength to the algorithm.

Note: in this table the high-order bit is omitted; see Specifying a CRC above.

Known to exist, but technologically defunct -- mainly replaced by cryptographic hash functions

• CRC-128 (IEEE)
• CRC-256 (IEEE)

Designing CRC polynomials

The selection of generator polynomial is the most important part of implementing the CRC algorithm. The polynomial must be chosen to maximize the error detecting capabilities while minimizing overall collision probabilities.

The most important attribute of the polynomial is its length (the number of the highest nonzero coefficient), because of its direct influence of the length of the computed checksum.

The most commonly used polynomial lengths are

• 9 bits (CRC-8)
• 17 bits (CRC-16)
• 33 bits (CRC-32)
• 65 bits (CRC-64)

When creating a new CRC polynomial or improving an existing CRC the general mathematical advice is to use an irreducible polynomial that satisfies all polynomical irreducibility constraints from modular arithmetics.

• Irreducibility in this case means that the polynomial cannot be divided by any polynomial except itself and 1 with zero remainder.
• Reducible polynomials can still be used, but their error correcting and detecting capabilities will be less effective. Some applications may choose to use reducible polynomials under certain conditions.

The properties of the generator polynomial can be derived from the algorithm definition

• CRCs with more than one nonzero coefficients are able to detect all single bit errors in the input message.
• CRCs can be used to detect all double bit errors in the input message shorter than 2^k, where k is the length of the longest irreducible part of the polynomial.
• If the CRC polynomial is divided by x + 1 then no polynomial with odd number of nonzero coefficients can be divided by it. Hence, it can be used to detect odd number of errors in the input message (like single bit parity function).
• CRC polynomials detect (single) burst errors shorter than the number of the position of the highest polynomial coefficient.

General category

• Error correcting code
• redundancy check
• List of checksum algorithms
• Parity

Specific Technological References

• Adler-32
• Fletcher's checksum

References

1. ^ Peterson, W. W. and Brown, D. T. (January 1961). "Cyclic Codes for Error Detection". Proceedings of the IRE 49: 228. doi:10.1109/JRPROC.1961.287814. ISSN 0096-8390. 
2. ^ ^{a b} Brayer, K; Hammond, J L Jr. (December 1975). "Evaluation of error detection polynomial performance on the AUTOVON channel" in National Telecommunications Conference, New Orleans, La. Conference Record 1: p. 8-21 to 8-25, New York: Institute of Electrical and Electronics Engineers. 
3. ^ "(slib) Cyclic Checksum". Retrieved on 2008-04-06.
4. ^ Greg Cook (2008-09-09). "Catalogue of parameterised CRC algorithms". Retrieved on 2008-09-09.
5. ^ ^{a b c} [|Koopman, Philip] & Chakravarty, Tridib (2004), Cyclic Redundancy Code (CRC) Polynomial Selection For Embedded Networks, <http://www.ece.cmu.edu/~koopman/roses/dsn04/koopman04_crc_poly_embedded.pdf> 
6. ^ Castagnoli, G.; S. Braeuer; M. Herrman (June 1993). "Optimization of Cyclic Redundancy-Check Codes with 24 and 32 Parity Bits". IEEE Transactions on Communications 41 (6): 883. doi:10.1109/26.231911. ISSN 0090-6778. . Castagnoli's et al. work on algorithmic selection of CRC polynomials
7. ^ ^{a b c d e f} Koopman, P. (June 2002). "32-Bit Cyclic Redundancy Codes for Internet Applications". The International Conference on Dependable Systems and Networks: 459. doi:10.1109/DSN.2002.1028931. . Verification of Castagnoli's results by exhaustive search and some new good polynomials
8. ^ Perez, A.; Wismer & Becker (1983). "Byte-Wise CRC Calculations". IEEE Micro 3 (3): 40–50. doi:10.1109/MM.1983.291120. ISSN 0272-1732. 
9. ^ Ramabadran, T.V.; Gaitonde, S.S. (1988). "A tutorial on CRC computations". IEEE Micro 8 (4): 62–75. doi:10.1109/40.7773. ISSN 0272-1732. 
10. ^ Thomas Boutell, Glenn Randers-Pehrson, et al. (1998-07-14). "PNG (Portable Network Graphics) Specification, Version 1.2". Retrieved on 2008-04-28.

See also

• Information security
• Simple file verification

External links

• Easy to understand CRC32 C++ Source Code
• Free CRC Source Code from the Boost C++ Libraries
• The CRC Pitstop
• Williams, R. (1993-09) A Painless Guide to CRC Error Detection Algorithms
• Understanding Cyclic Redundancy Check
• Black, R. (1994-02) Fast CRC32 in Software; algorithm 4 is used in Linux and info-zip's zip and unzip.
• Kounavis, M. and Berry, F. (2005). A Systematic Approach to Building High Performance, Software-based, CRC generators, Slicing-by-4 and slicing-by-8 algorithms
• CRC32: Generating a checksum for a file, C++ implementation by Brian Friesen
• CRC16 to CRC64 collision research
• Reversing CRC - Theory and Practice.
• 'CRC-Analysis with Bitfilters'.
• Another Bad CRC Reading Tool, but free
• MathPages - Cyclic Redundancy Checks
• Tool for Bad CRC Reading with advanced options
• A CRC calculation utility and C source code generator written in Python. (MIT licence)
• CRC Encoding - C# Implementation by Marcel de Wijs
• Cyclic Redundancy Check: theory, practice, hardware, and software with emphasis on CRC-32. A sample chapter from Henry S. Warren, Jr. Hacker's Delight.

Online tools

• Free CRC Verilog Circuit generator
• Online CRC32 and CRC32B calculator (uses IEEE 802.3 polynomial)
• Online Tool to compute common CRCs (8/16/32/64) from strings
• Online CRC calculator
• Another online CRC calculator
• A third online CRC calculator
• Online CRC Tool: Generator of synthesizable CRC functions
• Online Char (ASCII), HEX, Binary, Base64, etc... Encoder/Decoder with MD2, MD4, MD5, SHA1+2, CRC, etc. hashing algorithms