Apache Beam 如何对文件使用 TestStream

Question

I have a simple pipeline that just copies files from source to destination. Im trying to write tests for the windowing I had set up.

Is there a way to use the TestStream class for files?

For example:

    @Test
    public void elementsAreInCorrectWindows() {
     
        TestStream<FileIO.ReadableFile> testStream = TestStream.create(ReadableFileCoder.of())
            .advanceWatermarkTo(start)
            .addElements(readableFile1)
            .advanceWatermarkTo(end)
            .addElements(readableFile2)
            .advanceWatermarkToInfinity();        
    }

However the constructor for ReadableFile is packaged protected so I wouldn't be able to create those objects.

Answer 1

I think it would be a reasonable feature/pull request to make this Coder public. In the meantime, you could have a TestStream that produces elements of another type that you then transform with a DoFn into ReadableFiles.