简体   繁体   English

POD类型的二进制I / O如何不破坏别名规则?

[英]How does binary I/O of POD types not break the aliasing rules?

Twenty plus years ago, I would have (and didn't) think anything of doing binary I/O with POD structs: 二十多年前,我会(而且没有)想到用POD结构进行二进制I / O的任何事情:

struct S { std::uint32_t x; std::uint16_t y; };
S s;
read(fd, &s, sizeof(s)); // assume this succeeds and reads sizeof(s) bytes
std::cout << s.x + s.y;

(I'm ignoring padding and byte order issues, because they're not part of what I am asking about.) (我忽略了填充和字节顺序问题,因为它们不是我要问的部分。)

"Obviously", we can read into s and the compiler is required to assume that the contents of sx and sy are aliases by read() . “显然”,我们可以读入s并且编译器需要假设sxsy的内容是read()别名。 So, sx after the read() isn't undefined behaviour (because s was uninitialized). 所以, read()之后的sx不是未定义的行为(因为s未初始化)。

Likewise in the case of 同样在。的情况下

S s = { 1, 2 };
read(fd, &s, sizeof(s)); // assume this succeeds and reads sizeof(s) bytes
std::cout << s.x + s.y;

the compiler can't presume that sx is still 1 after the read() . 编译器不能假定read()之后sx仍为1

Fast forward to the modern world, where we actually have to follow the aliasing rules and avoid undefined behaviour, and so on, and I have been unable to prove to myself that this is allowed . 快进到现代世界,我们实际上必须遵循别名规则并避免未定义的行为,等等,我无法向自己证明这是允许的

In C++14, for example, [basic.types] ¶2 says: 例如,在C ++ 14中,[basic.types]¶2说:

For any object (other than a base-class subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes (1.7) making up the object can be copied into an array of char or unsigned char. 对于普通可复制类型T的任何对象(基类子对象除外),无论对象是否保持类型T的有效值,构成对象的基础字节(1.7)都可以复制到char或者数组中。无符号字符。

42 If the content of the array of char or unsigned char is copied back into the object, the object shall subsequently hold its original value. 42如果将char或unsigned char数组的内容复制回对象,则该对象应随后保持其原始值。

¶4 says: ¶4说:

The object representation of an object of type T is the sequence of N unsigned char objects taken up by the object of type T, where N equals sizeof(T). 类型T的对象的对象表示是由类型T的对象占据的N个无符号字符对象的序列,其中N等于sizeof(T)。

[basic.lval] ¶10 says: [basic.lval]¶10说:

If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined: 54 如果程序试图通过以下类型之一以外的glvalue访问对象的存储值,则行为未定义: 54
... ...
— a char or unsigned char type. - char或unsigned char类型。

54 The intent of this list is to specify those circumstances in which an object may or may not be aliased. 54此列表的目的是指定对象可能存在或不存在别名的情况。

Taken together, I think that this is the standard saying that "you can form an unsigned char or char pointer to any trivially copyable (and thus POD) type and read or write its bytes". 总而言之,我认为这是标准的说法“你可以形成一个unsigned charchar指针,指向任何可复制的(因而是POD)类型并读取或写入其字节”。 In fact, in N2342 , which gave us the modern wording, the introductory table says: 事实上,在给我们现代措辞的N2342中 ,介绍表说:

Programs can safely apply coding optimizations, particularly std::memcpy. 程序可以安全地应用编码优化,尤其是std :: memcpy。

and later : 后来

Yet the only data member in the class is an array of char, so programmers intuitively expect the class to be memcpyable and binary I/O-able. 然而,该类中唯一的数据成员是一个char数组,因此程序员直观地期望该类具有memcpyable和二进制I / O能力。

With the proposed resolution, the class can be made into a POD by making the default constructor trivial (with N2210 the syntax would be endian()=default), resolving all the issues. 使用提议的解决方案,可以通过使默认构造函数变得简单(使用N2210语法将是endian()= default)将类设置为POD,从而解决所有问题。

It really sounds like N2342 is trying to say "we need to update the wording to make it so you can do I/O like read() and write() for these types", and it really seems like the updated wording was made standard. 这听起来像N2342试图说“我们需要更新措辞,以便你可以为这些类型执行像read()write()这样的I / O”,看起来更新的措辞似乎是标准的。

Also, I often hear reference to "the std::memcpy() hole" or similar where you can use std::memcpy() to basically "allow aliasing". 另外,我经常听到引用“ std::memcpy() hole”或者类似的地方你可以使用std::memcpy()基本上“允许别名”。 But the standard doesn't seem to call out std::memcpy() specifically (and in fact in one footnote mentions it along with std::memmove() and calls it an "example" of a way to do this). 但是标准似乎没有特别调用std::memcpy() (实际上在一个脚注中提到它和std::memmove()并且称之为“示例”的方法)。

Plus, there's the fact that I/O functions like read() tend to be OS-specific from POSIX and thus aren't discussed in the standard. 此外,事实上像read()这样的I / O函数往往是POSIX特定于操作系统的,因此标准中没有讨论。


So, with all this in mind, my questions are: 所以,考虑到所有这些,我的问题是:

  • What actually guarantees that we can do real-world I/O of POD structs (as shown above)? 实际上,我们可以保证我们可以对POD结构进行真实的I / O操作(如上所示)?

  • Do we actually need to need to std::memcpy() the content into and out of unsigned char buffers (surely not) or can we directly read into the POD types? 我们是否真的需要std::memcpy()将内容输入和输出unsigned char缓冲区(当然不是)或者我们可以直接读入POD类型吗?

  • Do the OS I/O functions "promise" that they manipulate the underlying memory "as if by reading or writing unsigned char values" or "as if by std::memcpy() "? 操作系统I / O函数是否“承诺”他们操纵底层内存“好像通过读取或写入unsigned char值”或“好像通过std::memcpy() ”?

  • What concerns should I have when there are layers (such as Asio ) between me and the raw I/O functions? 当我和原始I / O功能之间存在层(例如Asio )时,我应该有什么顾虑?

Strict aliasing is about accessing an object through a pointer/reference to a type other than that object's actual type. 严格别名是通过指向对象的实际类型以外的类型的指针/引用来访问对象。 However, the rules of strict aliasing permit accessing any object of any type through a pointer to an array of bytes . 但是,严格别名的规则允许通过指向字节数组的指针访问任何类型的任何对象 And this rule has been around for at least since C++14. 至少从C ++开始,这个规则已经出现了14。

Now, that doesn't mean much, since something has to define what such an access means. 现在,这并不意味着什么,因为必须要定义这样的访问意味着什么。 For that (in terms of writing), we only really have two rules: [basic.types]/2 and /3 , which cover copying the bytes of Trivially Copyable types. 为此(在写作方面),我们实际上只有两个规则: [basic.types] / 2和/ 3 ,它们包括复制Trivially Copyable类型的字节。 The question ultimately boils down to this: 问题最终归结为:

Are you reading the "the underlying bytes making up [an] object" from the file? 您是否正在从文件中读取“构成[一个]对象的基础字节”?

If the data you're reading into your s was in fact copied from the bytes of a live instance of S , then you're 100% fine. 如果你正在阅读到您的数据s实际上从现场实例的字节复制S ,那么你100%的罚款。 It's clear from the standard that performing fwrite writes the given bytes to a file, and performing fread reads those bytes from the file. 从标准中可以清楚地看出,执行fwrite会将给定的字节写入文件,执行fread会从文件中读取这些字节。 Therefore, if you write the bytes of an existing S instance to a file, and read those written bytes to an existing S , you have perform the equivalent of copying those bytes. 因此,如果将现有S实例的字节写入文件,并将这些写入的字节读取到现有S ,则执行相当于复制这些字节的操作。

Where you run into technical issues is when you start getting into the weeds of interpretation. 遇到技术问题的地方就是你开始深入解读杂草的时候。 It is reasonable to interpret the standard as defining the behavior of such a program even when the writing and the reading happen in different invocations of the same program. 将标准解释为定义此类程序的行为是合理的,即使在同一程序的不同调用中发生写入和读取时也是如此。

Concerns arise in one of two cases: 在两种情况之一中出现了担忧:

1: When the program which wrote the data is actually a different program than the one who read it. 1:编写数据的程序实际上是与读取数据的程序不同的程序。

2: When the program which wrote the data did not actually write an object of type S , but instead wrote bytes that just so happen to be legitimately interpret-able as an S . 2:当编写数据的程序实际上并没有写出类型为S的对象时,而是编写了恰好可以解释为S字节。

The standard doesn't govern interoperability between two programs. 该标准不管理两个程序之间的互操作性。 However, C++20 does provide a tool that effectively says "if the bytes in this memory contain a legitimate object representation of a T , then I'll return a copy of what that object would look like." 但是,C ++ 20确实提供了一个有效地说“如果这个内存中的字节包含T的合法对象表示,那么我将返回该对象看起来像什么的副本”的工具。 It's called std::bit_cast ; 被称为std::bit_cast ; you can pass it an array of bytes of sizeof(T) , and it'll return a copy of that T . 你可以传递一个sizeof(T)的字节数组,它将返回该T的副本。

And you get undefined behavior if you're a liar. 如果你是个骗子,你会得到不确定的行为。 And bit_cast doesn't even compile if T is not trivially copyable. 如果T不是可复制的,那么bit_cast甚至不会编译。

However, to do a byte copy directly into a live S from a source that wasn't technically an S but totally could be an S , is a different matter. 但是,要从一个技术上不是S但完全可能是S的源直接进行实时S的字节复制,则是另一回事。 There isn't wording in the standard to make that work. 标准中没有措辞来完成这项工作。

Our friend P0593 proposes a mechanism for explicitly declaring such an assumption, but it didn't quite make it into C++20. 我们的朋友P0593提出了一种明确声明这种假设的机制,但它并没有完全融入C ++ 20。

The type-access rules in every version of the C and C++ Standard to date are based upon the C89 rules, which were written with the presumption that implementations intended for various tasks would uphold the Spirit of C principle described in the published Rationale as "Don't prevent [or otherwise interfere with] the programmer from doing what needs to be done [to accomplish those tasks]." 到目前为止,C和C ++标准的每个版本中的类型访问规则都基于C89规则,这些规则的编写假定用于各种任务的实现将支持已发布的原理中描述为“Don”的C语言原则不要[或以其他方式干扰]程序员做[需要完成的任务]。“ The authors of C89 would have seen no reason to worry about whether or not the rules as written actually required that compilers support constructs that everyone would agree that they should (eg allocating storage via malloc , passing it to fread , and then using it as a standard layout structure type) since they would expect such constructs to be supported on any compiler whose customers would need them, without regard for whether or not the rules as written actually required such support. C89的作者没有理由担心编写的规则是否实际上要求编译器支持每个人都同意他们应该的结构(例如通过malloc分配存储,将其传递给fread ,然后将其用作标准布局结构类型)因为他们希望在客户需要它们的任何编译器上支持这样的构造,而不考虑所写的规则是否实际上需要这样的支持。

There are many situations where constructs which should "obviously" work, actually invoke UB, because eg the authors of the Standard saw no need to worry about whether the rules would eg forbid a compiler given the code: 在许多情况下,构造应该“显然”工作,实际上调用UB,因为例如标准的作者不需要担心规则是否会例如禁止给定代码的编译器:

struct S {int dat[10]; } x,y;
void test(int i)
{
  y = x;
  y.dat[i] = 1; /// Equivalent to *(y.dat+i) = 1;
  x = y;
}

from assuming that object y of type struct S could not possibly be accessed by the dereferenced int* on the marked line(*), and thus need not be copied back to object x . 假设struct S的类型的对象y不可能被标记的行(*)上的dereferenced int*访问,因此不需要复制回对象x For a compiler to make such an assumption when it can see that the pointer is derived from a struct S would have been universally recognized as obtuse regardless of whether or not the Standard would forbid it, but the question of exactly when a compiler should be expected "see" how a pointer was produced was a Quality of Implementation issue outside the Standard's jurisdiction. 对于编译器做出这样的假设,当它可以看到指针是从struct S派生时,无论标准是否会禁止它,它都会被普遍认为是钝的,但是关于编译器何时应该被预期的问题“看”指针的生成方式是标准管辖范围之外的实施质量问题。

(*) In fact, the rules as written would allow a compiler to make such an assumption, since the only types of lvalue that may be used to access a struct S would be that structure type, qualified versions of it, types derived from it, or character types. (*)事实上,编写的规则将允许编译器做出这样的假设,因为可用于访问struct S的唯一类型的左值是结构类型,它的限定版本,从它派生的类型或字符类型。

It's sufficiently obvious that functions like fread() should be usable on standard-layout structures that quality compilers will generally support such usage without regard for whether the Standard would actually require them to do so. 很明显,像fread()这样的函数应该可以在标准布局结构上使用,质量编译器通常会支持这种使用,而不考虑标准是否真的要求它们这样做。 Moving such questions from Quality of Implementation issues to actual conformance issues would require adopting new terminology to describe what a statement like int *p = x.dat+3; 将这些问题从实施质量问题转移到实际的一致性问题,需要采用新的术语来描述像int *p = x.dat+3;这样的语句int *p = x.dat+3; does with the stored value of x [it should cause it to be accessible via p under at least some circumstances], and more importantly would require that the Standard itself affirm a point which is currently relegated to the published Rationale--that it is not intended to say anything bad about code which will only run on implementations that are suitable for its purpose, nor to say anything good about implementations which, although conforming, aren't suitable for their claimed purposes. 与x的存储值有关[至少在某些情况下应该使它可以通过p访问],更重要的是要求标准本身确认一个目前已降级为已公布的理由的点 - 它不是意在说任何关于代码的坏事,这些代码只能在适合其目的的实现上运行,也不能说任何有关实现的东西,虽然符合要求,但不适合他们声称的目的。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM