C低级读写功能等效于64位吗？

Question

I have some code that uses the low level i/o read and write system calls, as described on page 170 of the C programming language book Kernighan and Ritchie. The function prototypes are this

int n_read = read  ( int fd, char *buf, int n )
int n_read = write ( int fd, char *buf, int n )

now the two .c file that uses these read and write are called by a larger fortran based program to read and write lots of data.

the C code is simply this, with no #include of any kind, having the underscore after the function name and passing by reference:

int read_ ( int *descriptor, char *buffer, int *nbyte )
{
   return ( read( *descriptor, buffer, *nbyte ) );
}

int write_ ( int *descriptor, char *buffer, int *nbyte )
{
   return ( write( *descriptor, buffer, *nbyte ) );
}

and the larger fortran based program will do something like this

INTEGER nbyte
COMPLEX*16 matrix(*)
INTEGER READ, WRITE
EXTERNAL READ, WRITE

status = READ( fd, matrix, nbyte )
if ( status .eq. -1 ) then
   CALL ERROR('C call read failure')
   stop
endif

As you may have already guessed, this works fine for nbyte values less than 2^31. I have a need to read more than 2 GB of data, so i need nbyte to be a long integer and INTEGER*8 in fortran.

Is there an equivalent read64 and write64, like there is an lseek64 provided by unistd.h and features.h ?

what is the best way to recode this? should i use fread and fwrite ? is the int fd from the low level write the same as FILE *stream from fread() ?

my requirement is being able to pass a long integer of 8 bytes to allow for values up to 100 to 500 gigabytes or an integer having 12 digits, which is all for the value of nbyte

Am i gaining anything or losing out by currently using read and write which is identified as a "system call" ? What does this mean?

Answer 1

Edit: You can't, at least not on Linux. read will never transfer more than what a 32-bit integer can hold.

From the manpages of Linux on read :

On Linux, read() (and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)

This is not a contraint of POSIX, it's allowed by POSIX, but in the end it's implementation defined how read behaves. As Andrew Hanle reports, reading a 32GB file works just fine on Solaris. In this case, my old answer is still valid.

Old Answer:

read can work with 64-bit files just fine. It's defined in <unistd.h> as the following:-

ssize_t read(int fd, void *buf, size_t count);

You would have to adjust your routines to work with size_t instead of int , to properly support big files.

You should check SSIZE_MAX (the maximum value supported for count ), before using read with a big file, and abort if it's to small (or split into smaller chunks). SSIZE_MAX is an implementation defined value.

Answer 2

As @Leandros observed, POSIX-conforming implementations of read() and write() accept byte counts of type size_t , and return byte counts of type ssize_t . These are probably the definitions that actually apply to you, as the read() and write() functions are not specified by the C standard. That's a distinction without much difference, however, because size_t is not required to be wider than int -- in fact, it can be narrower.

You anyway have a bigger problem. The Fortran code seems to assume that the C functions it is calling will read / write the full specified number of bytes or else fail, but POSIX read() and write() are not guaranteed to do that when they succeed. Indeed, there was a question around here the other day that hinged on the fact that these functions did not transfer more bytes at a time than can be represented by a signed, 32-bit integer, even on a 64-bit system with 64-bit [s]size_t .

You can kill both of these birds with one stone by implementing the read_() and write_() functions to loop , performing successive calls to the underlying read() or write() function, until the full number of specified bytes is transferred or an error occurs.