[英]Passing textures with UInt8 component type to Metal compute shader
I have an image that I generate programmatically and I want to send this image as a texture to a compute shader. 我有一个以编程方式生成的图像,我想将此图像作为纹理发送到计算着色器。 The way I generate this image is that I calculate each of the RGBA components as UInt8
values, and combine them into a UInt32
and store it in the buffer of the image. 我生成这个图像的方法是我将每个RGBA组件计算为UInt8
值,并将它们组合成UInt32
并将其存储在图像的缓冲区中。 I do this with the following piece of code: 我使用以下代码执行此操作:
guard let cgContext = CGContext(data: nil,
width: width,
height: height,
bitsPerComponent: 8,
bytesPerRow: 0,
space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: RGBA32.bitmapInfo) else {
print("Unable to create CGContext")
return
}
guard let buffer = cgContext.data else {
print("Unable to create textures")
return
}
let pixelBuffer = buffer.bindMemory(to: RGBA32.self, capacity: width * height)
let heightFloat = Float(height)
let widthFloat = Float(width)
for i in 0 ..< height {
let latitude = Float(i + 1) / heightFloat
for j in 0 ..< width {
let longitude = Float(j + 1) / widthFloat
let x = UInt8(((sin(longitude * Float.pi * 2) * cos(latitude * Float.pi) + 1) / 2) * 255)
let y = UInt8(((sin(longitude * Float.pi * 2) * sin(latitude * Float.pi) + 1) / 2) * 255)
let z = UInt8(((cos(latitude * Float.pi) + 1) / 2) * 255)
let offset = width * i + j
pixelBuffer[offset] = RGBA32(red: x, green: y, blue: z, alpha: 255)
}
}
let coordinateConversionImage = cgContext.makeImage()
where RGBA32
is a little struct that does the shifting and creating the UInt32
value. 其中RGBA32
是一个小结构,可以进行移位并创建UInt32
值。 This image turns out fine as I can convert it to UIImage
and save it to my photos library. 这个图像很好,因为我可以将它转换为UIImage
并将其保存到我的照片库。
The problem arises when I try to send this image as a texture to a compute shader. 当我尝试将此图像作为纹理发送到计算着色器时,会出现问题。 Below is my shader code: 以下是我的着色器代码:
kernel void updateEnvironmentMap(texture2d<uint, access::read> currentFrameTexture [[texture(0)]],
texture2d<uint, access::read> coordinateConversionTexture [[texture(1)]],
texture2d<uint, access::write> environmentMap [[texture(2)]]
uint2 gid [[thread_position_in_grid]])
{
const uint4 pixel = {255, 127, 63, 255};
environmentMap.write(pixel, gid);
}
The problem with this code is that the type of my textures is uint
, which is 32-bits, and I want to generate 32-bit pixels the same way I do on the CPU, by appending 4 8-bit values. 这段代码的问题是我的纹理类型是uint
,它是32位,我想通过附加4个8位值来生成32位像素,就像我在CPU上一样。 However, I can't seem to do that on Metal as there is no byte
type that I can just append together and make up a uint32
. 但是,我似乎无法在Metal上这样做,因为没有byte
类型我可以一起追加并构成一个uint32
。 So, my question is, what is the correct way to handle 2D textures and set 32-bit pixels on a Metal compute shader? 所以,我的问题是,在Metal计算着色器上处理2D纹理和设置32位像素的正确方法是什么?
Bonus question: Also, I've seen example shader codes with texture2d<float, access::read>
as the input texture type. 额外问题:另外,我看过示例着色器代码,其中texture2d<float, access::read>
作为输入纹理类型。 I'm assuming it represents a value between 0.0 and 1.0 but what advantage that does that have over an unsigned int with values between 0 and 255? 我假设它代表一个介于0.0和1.0之间的值,但是这对于一个值为0到255之间的unsigned int有什么好处呢?
Edit: To clarify, the output texture of the shader, environmentMap
, has the exact same properties (width, height, pixelFormat, etc.) as the input textures. 编辑:为了澄清,着色器的输出纹理environmentMap
具有与输入纹理完全相同的属性(width,height,pixelFormat等)。 Why I think this is counter intuitive is that we are setting a uint4
as a pixel, which means it's composed of 4 32-bit values, whereas each pixel should be 32-bits. 为什么我认为这是反直觉的是我们将uint4
设置为像素,这意味着它由4个32位值组成,而每个像素应该是32位。 With this current code, {255, 127, 63, 255}
has the exact same result as {2550, 127, 63, 255}
, meaning the values somehow get clamped between 0-255 before being written to the output texture. 使用此当前代码, {255, 127, 63, 255}
具有与{2550, 127, 63, 255}
{255, 127, 63, 255}
完全相同的结果,这意味着在写入输出纹理之前,值以某种方式被钳位在0-255之间。 But this is extremely counter-intuitive. 但这非常违反直觉。
There's a bit more magic at play than you seem to be familiar with, so I'll try to elucidate. 玩起来比你似乎更熟悉,所以我会试着澄清一下。
First of all, by design, there is a loose connection between the storage format of textures in Metal and the type you get when you read/sample. 首先,通过设计,Metal中纹理的存储格式与读取/采样时获得的类型之间存在松散的联系。 You can have a texture in .bgra8Unorm
format that, when sampled through a texture bound as texture2d<float, access::sample>
will give you a float4
with its components in RGBA order. 您可以使用.bgra8Unorm
格式的纹理,当通过绑定为texture2d<float, access::sample>
的纹理进行采样时texture2d<float, access::sample>
将为您提供一个float4
,其组件处于RGBA顺序。 The conversion from those packed bytes to the float vector with swizzled components follows well-documented conversion rules as specified in the Metal Shading Language Specification. 从那些压缩字节到具有混合组件的浮点向量的转换遵循金属着色语言规范中规定的记录良好的转换规则。
It is also the case that, when writing to a texture whose storage is (for example) 8 bits per component, values will be clamped to fit in the underlying storage format. 还有一种情况是,当写入存储(例如)每个组件8位的纹理时,将钳制值以适合底层存储格式。 This is further affected by whether or not the texture is a norm
type: if the format contains norm
, the values are interpreted as if they specified a value between 0 and 1. Otherwise, the values you read are not normalized. 这还受纹理是否为norm
类型的影响:如果格式包含norm
,则将值解释为它们指定的值介于0和1之间。否则,您读取的值不会标准化。
An example: if a texture is .bgra8Unorm
and a given pixel contains the byte values [0, 64, 128, 255]
, then when read in a shader that requests float
components, you will get [0.5, 0.25, 0, 1.0]
when you sample it. 例如:如果纹理是.bgra8Unorm
并且给定的像素包含字节值[0, 64, 128, 255]
.bgra8Unorm
[0, 64, 128, 255]
,那么当在请求float
组件的着色器中读取时,您将得到[0.5, 0.25, 0, 1.0]
当你采样它。 By contrast, if the format is .rgba8Uint
, you will get [0, 64, 128, 255]
. 相反,如果格式为.rgba8Uint
,则会得到[0, 64, 128, 255]
.rgba8Uint
[0, 64, 128, 255]
。 The storage format of the texture has a prevailing effect on how its contents get interpreted upon sampling. 纹理的存储格式对其内容在采样时如何解释具有主要影响。
I assume that the pixel format of your texture is something like .rgba8Unorm
. 我假设纹理的像素格式类似于.rgba8Unorm
。 If that's the case, you can achieve what you want by writing your kernel like this: 如果是这种情况,您可以通过编写内核来实现您想要的目标:
kernel void updateEnvironmentMap(texture2d<float, access::read> currentFrameTexture [[texture(0)]],
texture2d<float, access::read> coordinateConversionTexture [[texture(1)]],
texture2d<float, access::write> environmentMap [[texture(2)]]
uint2 gid [[thread_position_in_grid]])
{
const float4 pixel(255, 127, 63, 255);
environmentMap.write(pixel * (1 / 255.0), gid);
}
By contrast, if your texture has a format of .rgba8Uint
, you'll get the same effect by writing it like this: 相比之下,如果您的纹理格式为.rgba8Uint
,您可以通过这样编写它来获得相同的效果:
kernel void updateEnvironmentMap(texture2d<float, access::read> currentFrameTexture [[texture(0)]],
texture2d<float, access::read> coordinateConversionTexture [[texture(1)]],
texture2d<float, access::write> environmentMap [[texture(2)]]
uint2 gid [[thread_position_in_grid]])
{
const float4 pixel(255, 127, 63, 255);
environmentMap.write(pixel, gid);
}
I understand that this is a toy example, but I hope that with the foregoing information, you can figure out how to correctly store and sample values to achieve what you want. 我知道这是一个玩具示例,但我希望通过上述信息,您可以弄清楚如何正确存储和取样值以实现您想要的效果。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.