D3D11 Multiple Vertex Buffer 2D Depth sorting with depth buffer

Question

I am trying to create some 2D UI drawing in D3D11. I am trying to get the draw calls down as much as possible to improve performance. Previously, I batched up as many textures as possible to send to one draw call, they were accessed with an else if. Well it turned out that sending many textures, especially large ones, to a single shader destroyed performance in some scenarios which is exactly the opposite of what I was trying to achieve.

The goal is to render things back to front from 2 VBOs. So now, I am trying split some things up into a few vertex buffers. I have one vertex buffer that contains non textured vertices, and one that contains different textures. This is because I send one texture to the shader at a time, and want to keep the amount of texture swapping between vertices being drawn at a minimum. The problem I am encountering is that the depth sorting does not work as I want it to. I render the textured vertex buffer first, then the non textured one. When I try to render textures over something rendered in the non textured vertex buffer, it simply doesn't work. There may be some other strange effects that I cant quite explain.

Maybe this is something with my alpha blending? It seems to me that the Z value I write has no effect on the output. Here is the relevant code:

desc->AlphaToCoverageEnable = false;
desc->RenderTarget[0].BlendEnable = true;
desc->RenderTarget[0].SrcBlend = D3D11_BLEND_SRC_ALPHA;
desc->RenderTarget[0].DestBlend = D3D11_BLEND_INV_SRC_ALPHA;
desc->RenderTarget[0].BlendOp = D3D11_BLEND_OP_ADD;
desc->RenderTarget[0].SrcBlendAlpha = D3D11_BLEND_INV_DEST_ALPHA;
desc->RenderTarget[0].DestBlendAlpha = D3D11_BLEND_ONE;
desc->RenderTarget[0].BlendOpAlpha = D3D11_BLEND_OP_ADD;
desc->RenderTarget[0].RenderTargetWriteMask = D3D11_COLOR_WRITE_ENABLE_ALL;
d3d->device->CreateBlendState(desc.get(), &d3d->blend_state);

I also tried Depth -= 1.0f and keeping the depth between 0.0 and 1.0 here to no effect

void put(float x, float y, float u, float v) {
    CurrentVBO->put({ {x,y, Depth }, current_color, {u,v}, Scale });
    Depth += 1.0f;
}

vertex shader

cbuffer vertex_buffer : register(b0) {
    float4x4 projection_matrix;
    float render_time;
};

struct VS_INPUT {
    float3 pos : POSITION;
    float4 color : COLOR;
    float2 uv : TEXCOORD;
    float scale : SCALE;
};

struct PS_INPUT {
    float4 pos : SV_Position;
    float4 color : COLOR;
    float2 uv : TEXCOORD;
    uint tex_id : TEXID;
};

PS_INPUT main(VS_INPUT input) {
    PS_INPUT output;
    output.pos = mul(projection_matrix, float4(input.pos.xy * input.scale, input.pos.z, 1.0f));
    output.color = input.color;
    output.uv = input.uv;
    return output;
}

desc->DepthEnable = true;
desc->DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL;
desc->DepthFunc = D3D11_COMPARISON_LESS;

desc->StencilEnable = false;
desc->FrontFace.StencilFailOp = desc->FrontFace.StencilDepthFailOp = desc->FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP;
desc->FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;

desc->BackFace = desc->FrontFace;

d3d->device->CreateDepthStencilState(desc.get(), &d3d->depth_stencil_2d);

After some more reading, I may need to do some more work to make this work properly.

Answer 1

Since the problems of blending and ordering transparent objects with depth. I went with my original plan, but only allow a few textures to be passed to one shader. I also removed the high resolution textures as I think they were causing bandwidth problems in general, as the GPU is already being heavily strained in my case.