[英]reading from multiple lines of a file - scanf in c using Linux cat
[英]Reading a specific number of lines from a file in C (scanf, fseek,fgets)
我有一個流程主控器,它生成N個子進程 ,這些子進程通過未命名的管道與父進程進行通信。 我必須能夠:
我的問題與OS概念無關,僅涉及文件操作:S
也許fseek? 我無法映射日志文件(有些文件大小超過1GB)。
我將不勝感激。 先感謝您
編輯:我試圖讓孩子們在不使用fseek和塊的值的情況下讀取相應的行,所以,有人可以告訴我這是否有效嗎? :
//somewhere in the parent process:
FILE* logFile = fopen(filename, "r");
while (fgets(line, 1024, logFile) != NULL) {
num_lines++;
}
rewind(logFile);
int prev = 0;
for (i = 0; i < maps_nr; i++) {
struct send_to_Map request;
request.fp = logFile;
request.lower = lowLimit;
request.upper = highLimit;
if (i == 0)
request.minLine = 0;
else
request.minLine = 1 + prev;
if(i!=maps_nr-1)
request.maxLine = (request.minLine + num_lines / maps_nr) - 1;
else
request.maxLine = (request.minLine + num_lines / maps_nr)+(num_lines%maps_nr);
prev = request.maxLine;
}
//write this structure to respective pipe
//child process:
while(1) {
...
//reads the structure to pipe (and knows which lines to read)
int n=0, counter=0;
while (fgets(line, 1024, logFile) != NULL){
if (n>=minLine and n<=maxLine)
counter+= process(Line);//returns 1 if IP was found, in that line, between the low and high limit
n++;
}
//(...)
}
我不知道它是否會起作用,我只是想使其起作用! 即使這樣,是否有可能勝過讀取整個文件並打印日志文件中找到的ip總數的單個過程?
如果您不希望完全均勻地分割文件,並且行長在整個文件中的分布有些均勻,則可以避免一次在父級中讀取整個文件。
start
並讀取chunk_size
字節。 這是該策略的概圖。
編輯簡化一些事情。
編輯 :這是下面第3步和第4步的一些未經測試的代碼。 這都是未經測試的,並且我還沒有仔細研究過一次的錯誤,但是它使您了解fseek
和ftell
的用法,聽起來像您要找的東西。
// Assume FILE* f is open to the file, chunk_size is the average expected size,
// child_num is the id of the current child, spawn_child() is a function that
// handles the logic of spawning a child and telling it where to start reading,
// and how much to read. child_chunks[] is an array of structs to keep track of
// where the chunks start and how big they are.
if(fseek(f, child_num * chunk_size, SEEK_SET) < 0) { handle_error(); }
int ch;
while((ch = fgetc(f)) != FEOF && ch != '\n')
{/*empty*/}
// FIXME: needs to handle EOF properly.
child_chunks[child_num].end = ftell(f); // FIXME: needs error check.
child_chunks[child_num+1].start = child_chunks[child_num].end + 1;
spawn_child(child_num);
然后在您的孩子中(第4步),假設該孩子可以訪問child_chunks[]
並知道其child_num
:
void this_is_the_child(int child_num)
{
/* ... */
fseek(f, child_chunks[child_num].start, SEEK_SET); // FIXME: handle error
while(fgets(...) && ftell(f) < child_chunks[child_num].end)
{
}
}
/* get an array with line-startpositions (file-offsets) */
fpos_t readLineBegins(FILE *f,fpos_t **begins)
{
fpos_t ch=0, mark=0, num=0;
*begins = 0;
do {
if( ch=='\n' )
{
*begins = realloc( *begins, ++num * sizeof(fpos_t) );
(*begins)[num-1] = mark;
mark = ftell(f);
}
} while( (ch=fgetc(f))!=EOF );
if( mark<ftell(f) )
{
*begins = realloc( *begins, ++num * sizeof(fpos_t) );
(*begins)[num-1]=mark;
}
return num;
}
/* output linenumber beg...end */
void workLineBlocks(FILE *f,fpos_t *begins,fpos_t beg,fpos_t end)
{
while( beg<=end )
{
int ch;
fsetpos( f, &begins[beg] ); /* set linestart-position */
printf("%ld:", ++beg );
while( (ch=fgetc(f))!=EOF && ch!='\n' && ch!='\r' )
putchar(ch);
puts("");
}
}
main()
{
FILE *f=fopen("file.txt","rb");
fpos_t *lineBegins, /* Array with line-startpositions */
lb = readLineBegins(f,&lineBegins); /* get number of lines */
workLineBlocks(f,lineBegins,lb-2,lb-1); /* out last two lines */
workLineBlocks(f,lineBegins,0,1); /* out first two lines */
fclose(f);
free(lineBegins);
}
我認為它可以幫助您: 從文本文件中讀取特定范圍的行
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.