writing huge number of points to vtk file format

Question

I'm writing script in python which is parsing about 500 images (slices of 3d object) of width 1810px and height 1808px. I want to write points (with original color) from all these images and save them to vtk file format. For test purposes i've modified example from vtk:

import vtk
from vtk import *

#setup points and vertices
Points = vtk.vtkPoints()
Vertices = vtk.vtkCellArray()

for x in xrange(0, 1808):
    for y in xrange(0, 1810):
        for z in xrange(0, 544):
            id = Points.InsertNextPoint(x, y, z)
            Vertices.InsertNextCell(1)
            Vertices.InsertCellPoint(id)


polydata = vtk.vtkPolyData()
polydata.SetPoints(Points)
polydata.SetVerts(Vertices)
polydata.Modified()
if vtk.VTK_MAJOR_VERSION <= 5:
    polydata.Update()

writer = vtk.vtkXMLPolyDataWriter()
writer.SetFileName("TriangleColoredPoints.vtp")
if vtk.VTK_MAJOR_VERSION <= 5:
    writer.SetInput(polydata)
else:
    writer.SetInputData(polydata)
writer.Write()

For now it's not even reading an image, but the problem is that when i add all these points, i'm going out of memory. Is there any way to save these points in chunks?

Answer 1

Looks like you're attempting to write a polydata with 1800x1800x500 points approx (about 1.6 billion points). Millions may be possible, but not billions, not yet anyway. Looks like you will need to reduce your problem size. If you do reduce the problem size, then use the vtkXMLPolyDataWriter. It has

writer->SetDataModeToBinary();
writer->SetCompressorTypeToZLib();

Which are useful in writing large models to a filesystem compactly.

Alternatively, you can store the images as a singe volume (1800x1800x500, vtkImageData) as the above comment mentions, there will be very large memory use still though. Then you can do volume rendering or marching cubes to get a surface etc. But without more detail is to what you're trying to achieve, its difficult to answer any further.