[英]Shared memory "Too many open files" but ipcs doesn't show many allocations
I'm writing unit tests for code which creates shared memory.我正在为创建共享 memory 的代码编写单元测试。
I only have a couple of tests.我只有几个测试。 I make 4 allocations of shared memory and then it fails on the fifth.我进行了 4 次共享 memory 分配,然后在第五次分配失败。
After calling shmat()
perror()
says Too many open files
:调用shmat()
perror()
后说Too many open files
:
template <typename T>
bool Attach(T** ptr, const key_type& key)
{
// shmemId was 262151
int32_t shmemId = shmget( key.key( ), ( size_t )0, 0644 );
if (shmemId < 0)
{
perror("Error: ");
return false;
}
*ptr = ( T * ) shmat(shmemId, 0, 0 );
if ( ( int64_t ) * ptr < 0 )
{
// Problem is here. perror() says 'Too many open files'
perror( "Error: ");
return false;
}
return true;
}
However, when I check ipcs -m -p
I only have a couple of shared memory allocations.但是,当我检查ipcs -m -p
时,我只有几个共享的 memory 分配。
T ID KEY MODE OWNER GROUP CPID LPID
Shared Memory:
m 262151 0x0000a028 --rw-r--r-- 3229 0
m 262152 0x0000a029 --rw-r--r-- 3229 0
In addition, when I check my OS shared memory limits sysctl -A | grep shm
此外,当我检查我的操作系统共享 memory 限制sysctl -A | grep shm
sysctl -A | grep shm
I get: sysctl -A | grep shm
我得到:
kern.sysv.shmall: 1024
kern.sysv.shmmax: 4194304
kern.sysv.shmmin: 1
kern.sysv.shmmni: 32
kern.sysv.shmseg: 8
security.mac.posixshm_enforce: 1
security.mac.sysvshm_enforce: 1
Are these variables large enough/are they the cause/what values should I have?这些变量是否足够大/它们是原因/我应该拥有什么值?
I'm sure I edited the file to increase them and restarted machine but perhaps it hasn't accepted (this is on Mac/OSX).我确定我编辑了文件以增加它们并重新启动机器,但它可能没有被接受(这是在 Mac/OSX 上)。
Your problem may be elsewhere.您的问题可能在其他地方。
Edit: This may be a shmmni
limit of macOS.编辑:这可能是 macOS 的shmmni
限制。 See below.见下文。
When I run your [simplified] code on my system (linux), the shmget
fails.当我在我的系统(linux)上运行您的 [simplified] 代码时, shmget
失败。
You didn't specify IPC_CREAT
to the third argument.您没有将IPC_CREAT
指定给第三个参数。 If another process has created the segment, this may be okay.如果另一个进程创建了该段,这可能没问题。
But, it doesn't/shouldn't like a size of 0. The [linux] man page states that it returns an error ( errno
set to EINVAL) if the size is less than SHMMIN
(which is 1).但是,它不/不应该喜欢 0 的大小。[linux] 手册页指出,如果大小小于SHMMIN
(即 1),它将返回错误( errno
设置为 EINVAL)。
That is what happened on my system.这就是我的系统上发生的事情。 So, I adjusted the code to use a size of 1.因此,我调整了代码以使用 1 的大小。
This was done [as I mentioned] on linux.这是在 linux 上完成的[正如我提到的]。
macOS may allow a size of 0, even if that doesn't make practical sense. macOS可能允许大小为 0,即使这没有实际意义。 (eg) It may round it up to a page size. (例如)它可能会将其四舍五入到一个页面大小。
For shmat
, it returns (void *) -1
.对于shmat
,它返回(void *) -1
。
But, some systems can have valid addresses that have the high bit set.但是,某些系统可以具有设置了高位的有效地址。 (eg) 0xFFE0000000000000 is a valid address, but would fail your if
test because casting that to int64_t
will test negative. (例如) 0xFFE0000000000000 是一个有效的地址,但会失败您的if
测试,因为将其转换为int64_t
将测试为否定。
Better to do:更好的做法:
if ((int64_t) *ptr == (int64_t) -1)
Or [possibly better]:或者[可能更好]:
if ((void *) *ptr == (void *) -1)
Note that errno
is not set/changed if the call succeeds.请注意,如果调用成功,则不会设置/更改errno
。
To verify this, do: errno = 0
before the shmat
call.要验证这一点,请在shmat
调用之前执行: errno = 0
。 If perror
says "Success", then the shmat
is okay.如果perror
说“成功”,那么shmat
就可以了。 And, your current test needs to be adjusted as above--I'd do that change regardless.而且,您当前的测试需要如上所述进行调整-无论如何我都会进行更改。
You could also do (eg):你也可以这样做(例如):
printf("ptr=%p\n",*ptr);
Normally, errno
starts as 0.通常, errno
从 0 开始。
Note that there are some differences between macOS and linux.请注意,macOS 和 linux 之间存在一些差异。
So, if errno
is ever set to "too many open files", this can be because the process has too many open files ( EMFILE
).因此,如果errno
曾经设置为“打开的文件过多”,这可能是因为该进程有太多打开的文件 ( EMFILE
)。
It might be because the system-wide limit is reached ( ENFILE
) but that is "file table overflow".这可能是因为达到了系统范围的限制( ENFILE
),但那是“文件表溢出”。
Note that under linux shmat
can not generate EMFILE
.注意在linux下shmat
不能生成EMFILE
。 However, it appears that under macOS it can .但是,似乎在 macOS 下它可以.
However, if the number of calls to shmat
is limited [as you mention], the shmat
should succeed.但是,如果对shmat
的调用次数是有限的 [正如您提到的那样],则shmat
应该会成功。
The macOS man page is a little vague as to what the limit is based on. macOS 手册页对于限制的依据有点模糊。 However, I checked the FreeBSD
man page for shmat
and that says it is limited by the sysctl
parameter: kern.ipc.shmseg
.但是,我检查了FreeBSD
手册页中的shmat
并且说它受sysctl
参数的限制: kern.ipc.shmseg
。 Your grep
should have caught that [if applicable].您的grep
应该已经注意到了 [如果适用]。
It is possible some other syscall elsewhere in the code is opening too many files.代码中其他地方的一些其他系统调用可能会打开太多文件。 And, that syscall is not checking the error return.而且,该系统调用没有检查错误返回。
Again, I realize you're running macOS
.同样,我意识到您正在运行macOS
。
But, if available, you may want to try your program under linux.但是,如果可用,您可能想在 linux 下尝试您的程序。 For example, it has much larger limits from the sysctl
:例如,它对sysctl
有更大的限制:
kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
vm.hugetlb_shm_group = 0
Note that shmmni
is the system-wide maximum number of shared memory segments.请注意, shmmni
是系统范围内共享 memory 段的最大数量。
Note that for macOS
, shmmni
is 32 (vs. 4096 for linux)??!?请注意,对于macOS
, shmmni
是 32(对于 linux 是 4096)??!?
That means that the entire system can only have 32 open shared memory segments for any/all processes???这意味着整个系统对于任何/所有进程只能有 32 个开放共享 memory 段???
That seems very low.这似乎很低。 You can probably set this to a larger number and see if that helps.您可以将其设置为更大的数字,看看是否有帮助。
Linux has the strace
program and you could use it to monitor the syscalls. Linux 有strace
程序,您可以使用它来监控系统调用。
But, macOS has dtruss
: How to trace system calls of a program in Mac OS X?但是,macOS 有dtruss
: 如何在 Mac OS X 中跟踪程序的系统调用?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.