Three.js: 3D texture MIP depth face clipping when using a perspective camera

Question

I am working on a web GUI using three.js for one of my light simulators. The GUI requires rendering a 3D volume (integers or float), so I adapted the 3D texture demo script and made a test page.

After some tweaking, I was able to render the 3D texture in MIP style, but the display appears to have some issue in correctly rendering the depth - some of the bounding box faces may disappear at certain camera angles, and make the render inaccurate.

The below screen capture shows the problem. The texture is a 40x20x30 volume with two z-layers: value 1.0 for 0<z<20 and value 2.0 for 20<z<30.

As you can see from the above screenshot, the MIP rendering is ok at the left view angle, but starts showing jagged boundaries (or missing bbx faces) at most other angles, shown on the right.

You can see it yourself by clicking on the below link

http://mcx.space/test/0235423747321423142346464758/

to input this volume, please click on the "JSON" tab, delete the existing text, and copy/paste the below JSON data (as you can see, the "Shapes" object defines a 3D volume using the JData ND array format , and is decoded to a numjs NdArray object). Once you replaced the JSON text, you can then click on the "Preview" tab to see the rendering.

{
  "Session": {
    "ID": "mcx",
    "Photons": 100000,
    "DoMismatch": true,
    "DoAutoThread": true,
    "DoSaveVolume": true,
    "DoPartialPath": true,
    "DoNormalize": true,
    "DoSaveRef": false,
    "DoSaveExit": false,
    "DoSaveSeed": false,
    "DoDCS": false,
    "DoSpecular": true,
    "DebugFlag": "",
    "SaveDataMask": "DP",
    "OutputFormat": "nii",
    "OutputType": "x",
    "RNGSeed": 1648335518
  },
  "Forward": {
    "T0": 0,
    "T1": 5e-9,
    "Dt": 5e-9
  },
  "Optode": {
    "Source": {
      "Type": "pencil",
      "Pos": [30,30,0],
      "Dir": [0,0,1],
      "Param1": [0,0,0,0],
      "Param2": [0,0,0,0]
    },
    "Detector": []
  },
  "Shapes":{
    "_ArrayType_":"uint8",
    "_ArraySize_":[40,20,30],
    "_ArrayZipType_":"zlib",
    "_ArrayZipSize_":24000,
    "_ArrayZipData_":"eJztybENAAAIA6DU/492dOgHBlaSMqfTWmuttdZaa6211lprrbXWWmuttdZaa6211lprrbXWWmuttdZaa6211lprrbXWWmuttdZaa6211lprv+4CkyqAIQ=="
  },
  "Domain": {
    "OriginType": true,
    "Dim": [60,60,60],
    "VolumeFile": "",
    "Media": [
      {
        "mua": 0,
        "mus": 0,
        "g": 1,
        "n": 1
      },
      {
        "mua": 0,
        "mus": 0,
        "g": 1,
        "n": 1
      }
    ]
  }
}

My script was largely adapted from the 3d texture demo. I tried to add transparent: true, opacity: 0.6, alphaTest: 0.5, depthWrite: false to the material setting, but it did not make any difference.

Because my data is the same float32 array as the demo script, but the demo script runs perfectly fine in the MIP mode. I am wondering if I overlooked something.

Your comment and pointers are highly appreciated! thanks

function render(){
  renderer.render( scene, camera );
}
        
function drawvolume(volume){
  const dtype={
    uint8:THREE.UnsignedByteType,
    uint16:THREE.UnsignedShortType,
    uint32:THREE.UnsignedIntType,
    int8:THREE.ByteType,
    int16:THREE.ShortType,
    int32:THREE.IntType,
    float32:THREE.FloatType
  };
  const dim=volume.shape;
  const buf=nj.array(volume.flatten().selection.data, 'float32');//had to cast to float32, otherwise, uint8 won't render

  const texture = new THREE.DataTexture3D( buf.selection.data, dim[2], dim[1], dim[0]);
  texture.format = THREE.RedFormat;
  texture.type = THREE.FloatType;
  texture.minFilter = texture.magFilter = THREE.LinearFilter;
  texture.unpackAlignment = 1;

  // Colormap textures
  const cmtextures = {
      viridis: new THREE.TextureLoader().load( 'https://threejs.org/examples/textures/cm_viridis.png', render ),
      gray: new THREE.TextureLoader().load( 'https://threejs.org/examples/textures/cm_gray.png', render )
  };

  // Material
  const shader = VolumeRenderShader1;

  const uniforms = THREE.UniformsUtils.clone( shader.uniforms );

  uniforms[ "u_data" ].value = texture;
  uniforms[ "u_size" ].value.set( dim[2], dim[1], dim[0] );
  uniforms[ "u_clim" ].value.set( volume.min(), volume.max() );
  uniforms[ "u_renderstyle" ].value = 0;
  uniforms[ "u_renderthreshold" ].value =  0.2;
  uniforms[ "u_cmdata" ].value = cmtextures[ "viridis" ];

  const material = new THREE.ShaderMaterial( {
      uniforms: uniforms,
      vertexShader: shader.vertexShader,
      fragmentShader: shader.fragmentShader,
      side: THREE.BackSide // The volume shader uses the backface as its "reference point"
  } );

  // THREE.Mesh
  const geometry = new THREE.BoxGeometry(  dim[2], dim[1], dim[0] );
  geometry.translate(dim[2]*0.5 - 0.5, dim[1]*0.5 - 0.5, dim[0]*0.5 - 0.5 );

  const mesh = new THREE.Mesh( geometry, material );
  return mesh;
}

...

     if(isWebGL2Available()){
       let jd=new jdata(cfg.Shapes,{});
       let vol=jd.decode().data;
       boundingbox.add( drawvolume(vol.transpose()) );
     }

An addition question: I am also having trouble to render a 3D integer array instead of a float array. I tried updating the texture setting to the following, but it shows a uniform block. I suspect that I also need to write a new shader?

  const texture = new THREE.DataTexture3D( volume.selection.data, dim[2], dim[1], dim[0]);
  texture.format = THREE.RedIntegerFormat;
  texture.type = dtype[volume.dtype];
  texture.minFilter = texture.magFilter = THREE.LinearFilter;
  texture.unpackAlignment = 1;

Answer 1

I'm not sure if I got your problem right. For me this looks like an issue with the side that the material is applied to. Maybe you should try THREE.DoubleSide or THREE.FrontSide in the ShaderMaterial .