How to Create JavaFX 16 bit Greyscale Images

Question

I have an application that generates 16 bit grey scale images. Currently these images are generated as an awt BufferedImage with data type DataBuffer.TYPE_USHORT.

In my app, I extract the 16 bit data from the BufferedImage, normalise to 8 bit, then render it into an FX Canvas with canvas.getGraphicsContext2D().getPixelWriter().setPixels(...)

It works but looks messy I am wondering if it is possible to implement a JavaFX Image or WritableImage, lets call it a UShortImage that is backed by my 16 bit data. The UShortImage could have FX properties for the normalisation levels, but otherwise could be used exactly as any JavaFX Image.

Any help or pointers on how to achieve this with good run time efficiency would be appreciated!

Answer 1

I don't know if this is exactly what you are looking for as perhaps I don't understand your question. But you can convert a color image to a gray image by desaturating the color using a ColorAdjust effect.

ColorAdjust monochrome = new ColorAdjust();
monochrome.setSaturation(-1);
ImageView gray = new ImageView(new Image(IMAGE_LOC));
gray.setEffect(monochrome);

I am not sure why poor grandma and grandpa must be faded to black and white, but it must be so.

import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.effect.ColorAdjust;
import javafx.scene.image.*;
import javafx.scene.layout.HBox;
import javafx.scene.paint.Color;
import javafx.stage.Stage;

public class GrayScale extends Application {

    @Override
    public void start(Stage stage) {
        ColorAdjust monochrome = new ColorAdjust();
        monochrome.setSaturation(-1);
        Image image = new Image(IMAGE_LOC);
        ImageView color = new ImageView(image);
        ImageView gray = new ImageView(image);
        gray.setEffect(monochrome);

        HBox layout = new HBox(10, color, gray);
        layout.setPadding(new Insets(10));

        Scene scene = new Scene(layout);
        scene.setFill(Color.BLACK);
        stage.setScene(scene);

        stage.show();
    }

    public static void main(String[] args) {
        launch(args);
    }

    // source: http://www.photoshopessentials.com/photo-editing/black-and-white-tutorials/desaturate/
    private static final String IMAGE_LOC =
            "http://pe-images.s3.amazonaws.com/photo-effects/black-and-white/grandparents.jpg";
}

If ColorAdjust doesn't work for you, then you could just do manipulation on a pixel-by-pixel bases using PixelReaders and PixelWriters on a WritableImage. (I don't see a reason for a canvas to be involved as you mention in your question). For an related implementation using a WritableImage see:

• Reduce number of colors and get color of a single pixel

---

Perhaps you are also looking to extend PixelFormat with a new type and extend PixelReader to read your data buffer, which may be possible, though I guess that wouldn't be necessary to accomplish what you want:

PixelReader reader = new GrayScalePixelReader(); 
reader.getPixels(0, 0, W, H, 
    GrayScalePixelFormat.instance(), grayScaleByteBuffer, 0, scanlineStride); 
Image img = new WritableImage(reader, W, H); 

where GrayScalePixelReader and GrayScalePixelFormat are new classes you create via extension.

FX supports pixel types define by PixelFormat such as BYTE_BGRA and BYTE_BGRA_PRE; I am looking to add support for a new pixel type which has a single sample per pixel which is a 16 unsigned data value.

 pixelFormat = PixelFormat.createByteIndexedInstance(LUT); getPixelWriter().setPixels(0, 0, w, h, pixelFormat, normalisedImageData, 0, w); 

Yeah, you can almost do it, but it is a little tricky. I agree that existing PixelFormat in Java 8 is not really very readily user extensible, for reasons such as you mention:

Some methods eg getPixelWriter() of the WritabelImage class are final so you can't even override them. The Image infrastructure looks like it is not designed to be extended in this way.

Usually, when it comes to JavaFX, this lack of extensibility is built in on purpose. Some reasons are:

1. Security, to make it more difficult for people to subvert the platform into doing something malicious by overriding core functionality to also perform malicious side effects.
2. Present a simplified interface to the application developer to make it easier to understand and use the API.
3. Deliberately make some parts of the platform opaque, so that it is easier for the library developers to maintain the core library implementation and change it at will without impacting user programs.

Of course the tradeoff (as in this case) can be a lack of flexibility in trying to accomplish what you wish.

If you look into the Image implementation in JavaFX, it kind of has four different components:

1. The javafx.scene.image APIs .
2. The platform implementation ( prism ) backing the image APIs (see source ).
3. The image io libraries for reading common image formats (png, jpg, etc).
4. Interfaces to graphics hardware accelerator APIs or software rendering APIs, for example es2 , which ultimately, usually treats the image as a texture input to a hardware graphics API such as OpenGL or Direct3D.

The interesting thing about grayscale pixel formats is that they are not directly exposed in (1) the javafx.scene.image APIs. However, the prism platform implementation, the image io libraries and hardware accelerators (2, 3 and 4), do have some level of inbuilt grayscale support. For instance, a PNG file could have a grayscale format and the imageIO will recognize the format and pass the grayscale buffer directly to the prism Image class to be interpereted.

Grayscale buffers are also supported by the hardware acceleration layer, as can be seen in the ES2Texture implementation. So this gives an ideal situation of the grayscale pixel format encoded buffer for the image being able to be directly rendered to a hardware accelerated texture. Note however, that the current pixel format for the grayscale byte buffer supported by the Prism/ES2 rendering system, is for an 8 bit encoding, not a 16 bit encoding you ideally ask for (though comments on the prism pixel format do not that a 16-bit type might be needed in the future ;-):

// L8, A8 types:
// NOTE : we might need L8A8 16-bit type
BYTE_GRAY    (DataType.BYTE,  1, true,  true), 

So this means that much of the infrastructure that you are looking for that allows direct interpretation of the grayscale encoded image buffer by the JavaFX runtime is already there, however it is placed inside com.sun classes and private API and not directly exposed to the user in the public javafx.scene.image APIs.

A feature request could be created to enable a public API for grayscale pixel formats or you could ask about it on the openjfx-dev mailing list . Shortest timeframe to get such features implemented would be Java 10. It looks like such a request may already exist, so you could comment on it or bring it up on the openjfx-dev mailing list to help move it forward:

• High bit-depth Grayscale WritableImage support .
• Support Writable Gray Image .

Does the above info help you in anyway with your current problem -> probably not :-) But hopefully it helps understanding of the platform and its capabilities, limitations and potential mechanisms for future extension to support your requirements for fully.