OSPF - Checksum not working

Question

I'm working on a project on which I need to generate the OSPF packet manually. I am currently having problems getting the OSPF checksum right. I read that I have to keep the Auth data out of the calculation, and even though I'm doing that I can't get it to work. I know that the function being used to generate the checksum is correct because I use the same one to generate the checksum for the IP header, and that works.

*I'm sorry for my bad C programming, it's not my main language.

void generateHello(unsigned char* packet_return,unsigned char* buff,unsigned short *ospf_packet){
ospf_packet = (unsigned short*) malloc(14*sizeof(unsigned short));

//OSPF Version 
packet_return[34] = 0x02;

//Message Type - Hello
packet_return[35] = 0x01;

//Packet Length
packet_return[36] = 0x00;
packet_return[37] = 0x2c;

//Source OSPF Router (IP)
packet_return[38]=local_ip[0];
packet_return[39]=local_ip[1];
packet_return[40]=local_ip[2];
packet_return[41]=local_ip[3];

//Area 

packet_return[42]=0x00;
packet_return[43]=0x00;
packet_return[44]=0x00;
packet_return[45]=0x01;

//ADD CHECKSUM

packet_return[46]=0x00;
packet_return[47]=0x00;

//Auth Type
packet_return[48]=0x00;
packet_return[49]=0x00;

//Auth Data
packet_return[50]=0x00;
packet_return[51]=0x00;
packet_return[52]=0x00;
packet_return[53]=0x00;
packet_return[54]=0x00;
packet_return[55]=0x00;
packet_return[56]=0x00;
packet_return[57]=0x00;

//Network Mask
packet_return[58]=0xff;
packet_return[59]=0xff;
packet_return[60]=0xff;
packet_return[61]=0x00;

//Hello Interval
packet_return[62]=0x00;
packet_return[63]=0x0a;

//Multi-Topology Routing
packet_return[64]=0x12;

//Router Priority
packet_return[65]=0x01;

//Router Dead Interval 
packet_return[66]=0x00;
packet_return[67]=0x00;
packet_return[68]=0x00;
packet_return[69]=0x28;

//Designated Router
packet_return[70]=0x00;
packet_return[71]=0x00;
packet_return[72]=0x00;
packet_return[73]=0x00;

//Backup designated router
packet_return[74]=0x00;
packet_return[75]=0x00;
packet_return[76]=0x00;
packet_return[77]=0x00;

//Checksum
packet_return[78]=0x00;
packet_return[79]=0x00;

//LLS Data Length
packet_return[80]=0x00;
packet_return[81]=0x03;

//Type
packet_return[82]=0x00;
packet_return[83]=0x01;

//Length
packet_return[84]=0x00;
packet_return[85]=0x04;

//Options - LSDB Resynchronization
packet_return[86]=0x00;
packet_return[87]=0x00;
packet_return[88]=0x00;
packet_return[89]=0x01;


int i;
int j;
for(i=0,j=34;i<48;i++,j+=2)
{
    ospf_packet[i]= htons(((packet_return[j] << 8) | packet_return[j+1])); 
}


unsigned short ck_sum = in_cksum(ospf_packet,sizeof(unsigned short)*14);
printf("CHECKSUM OSPF - %.4x \n", ck_sum);

packet_return[46]=ck_sum & 0xff;
packet_return[47]=(ck_sum >> 8) & 0xff;
}

Answer 1

Firstly, please use constants or better a struct with __attribute__((packed)) rather than lots of array offsets.

Secondly, this looks fishy:

for(i=0,j=34;i<48;i++,j+=2)
{
    ospf_packet[i]= htons(((packet_return[j] << 8) | packet_return[j+1])); 
}

ospf_packet is 14 unsigned shorts long, per the malloc . Yet you are writing 48 unsigned shorts into it. That will cause undefined behaviour.

Also, packet_return appears to be a char * so is presumably in wire order. You are then reading it out assuming it is all shorts in wire order (fine as far as it goes I suppose), then converting it from wire order to host order (it seems) - I think that should be ntohs not htons (yes, I know they do the same thing). It is not evident why you are doing this at all.

Lastly, the OSPF checksum is calculated over the entire OSPF packet except the authentication field.

From RFC2328

    Checksum
        The standard IP 16-bit one's complement checksum of the
        entire OSPF packet, excluding the 64-bit authentication
        field.  This checksum is calculated as part of the
        appropriate authentication procedure; for some OSPF
        authentication types, the checksum calculation is omitted.
        See Section D.4 for details.

I can't immediately see why your code sums across the entire packet nor how it omits the authentication field.