Recalculate TCP checksum

Question

I'm writing a userspace application for DDoS mitigation and to implement certain mitigation policies I need to be able to change TCP options and things like sequence and acknowledgement numbers in packet headers on the fly.

The usual way to calculate the checksum would be to recreate a new TCP pseudoheader and iterate over the entire data packet to calculate the checksum but there is another way which involves just calculating the differences between only the words changed and doing a ones-complement subtraction.

My current code seems to be off frequently by 1 or 2. I suspect this is a problem because I'm not handling carries/borrows correctly. I have very little idea on how I would go about fixing this:

unsigned short pseq1, pseq2, pseq3, pseq4
unsigned short sum1, sum2, sum3, prevcheck;
short pdiff1, pdiff2;

u_char *pkt_data;

prevcheck = (pkt_data[50] << 8) | pkt_data[51];

pseq1 = (pkt_data[38] << 8) | pkt_data[39]; 
pseq2 = (pkt_data[40] << 8) | pkt_data[41];

pkt_data[38] = ((seq_num - offsetResult) >> 24) & 0xFF;
pkt_data[39] = ((seq_num - offsetResult) >> 16) & 0xFF; 
pkt_data[40] = ((seq_num - offsetResult) >> 8) & 0xFF;
pkt_data[41] = (seq_num - offsetResult) & 0xFF;

pseq3 = (pkt_data[38] << 8) | pkt_data[39];
pseq4 = (pkt_data[40] << 8) | pkt_data[41];

pdiff1 = pseq1 - pseq3;

pdiff2 = pseq2 - pseq4;

sum1 = ~pdiff1 + ~pdiff2;

sum2 = ~sum1;

sum3 = sum2 + prevcheck; 

pkt_data[50] = (sum3 >> 8) & 0xFF; 
pkt_data[51] = sum3 & 0xFF;

In this instance: 68 05 ca 57 94 05 60 73 5c d0 57 bf 08 00 45 00 00 2c 00 00 40 00 3f 06 bc a5 b9 aa 2a 6a 42 f9 58 19 00 50 ed 48 fc e4 57 e5 6e c0 f6 c8 60 12 72 10 fe f3 00 00 02 04 05 b4 00 00

The produced checksum is fef3 when it should be fef2 .

Any suggestions would be amazing!

Answer 1

According to RFC 1071 , the checksum is calculated using a 16-bit 1's complement sum.

On a 2's complement machine, the 1's complement sum must be computed by means of an "end around carry", ie, any overflows from the most significant bits are added into the least significant bits.

So you should "reverse" the "end around carry" when updating the checksum.

Ie subtract 1 for each negative carry and add 1 for each positive carry.

Something like this:

int32_t sum; // or just int, but make sure it's 32-bit or more
unsigned short pseq1, pseq2, pseq3, pseq4
unsigned short prevcheck;

u_char *pkt_data;

prevcheck = (pkt_data[50] << 8) | pkt_data[51];

pseq1 = (pkt_data[38] << 8) | pkt_data[39]; 
pseq2 = (pkt_data[40] << 8) | pkt_data[41];

pkt_data[38] = ((seq_num - offsetResult) >> 24) & 0xFF;
pkt_data[39] = ((seq_num - offsetResult) >> 16) & 0xFF; 
pkt_data[40] = ((seq_num - offsetResult) >> 8) & 0xFF;
pkt_data[41] = (seq_num - offsetResult) & 0xFF;

pseq3 = (pkt_data[38] << 8) | pkt_data[39];
pseq4 = (pkt_data[40] << 8) | pkt_data[41];

sum = ~prevcheck - pseq1 - pseq2;

while (sum >> 16)
    sum = (sum & 0xFFFF) + (sum >> 16); // "end around carry"

sum += pseq3 + pseq4;

while (sum >> 16)
    sum = (sum & 0xFFFF) + (sum >> 16); // "end around carry"

sum3 = (short)~sum;

pkt_data[50] = (sum3 >> 8) & 0xFF; 
pkt_data[51] = sum3 & 0xFF;

Answer 2

u_int32_t sum;
u_int16_t oldSeq1, oldSeq2, newSeq1, newSeq2;
u_int16_t oldChecksum;

sum = ~oldChecksum - oldSeq1 - oldSeq2;

sum = (sum & 0xFFFF) + (sum >> 16);

sum = sum + newSeq1 + newSeq2;

sum = (sum & 0xFFFF) + (sum >> 16);

sum = (u_int16_t)~sum;

As my requirement was only to edit either sequence or acknowledge sequence numbers the above code was the solution for me using suggestions from all. This will account for all carries as the fold is done after each arithmetic operation instead of at the end where it could skew results.

This will work equally for changes in the IP header or TCP header as long as you pad anything shorter than 16 bits to 16 bits.

Answer 3

TCP/IP checksums use 1's complement arithmetic, which is similar to 2's complement plus carry feedback. Ie if you add two 16bit values, and get a carry, you need to add 1 to the sum.

Your code uses unsigned 16bit integers (BTW, I recommend using fixed width integer types , instead of int and short , when the size matters, as in this case), so when adding them the carry is lost.

A better way is to use 32bit variables for the intermediate result, then feeding back the carry. For example:

uint16_t a, b; ... uint32_t sum = (uint32_t)a + (uint32_t)b; if (sum > 0x10000u) { sum = (sum >> 16) + (sum & 0xffff); }