A Java Based Toolkit

Shrinath Shanbhag, KRESIT, IIT Mumbai

 Dr. Sharat Chandran, CSE IIT Mumbai

 

[1]          Introduction

 

This toolkit is a Java based class library, which simplifies routine image handling operations. It is targeted primarily at students and researchers working in the field of image processing and computer vision. It is an abstraction of the underlying complexities of the java-imaging framework and provides its user with a simplified logical image model. It includes a readymade extensible GUI shell (or template) for use in an application.

 

For students entering the field of computer vision, working with standard image formats and visualizing results is a challenging task. There is also the additional burden of creating a modern GUI for the application. The complexity can be attributed to a variety of reasons, some of which are mentioned below.

 

A general toolkit cannot avoid complexity as it has to deal with a diverse range of underlying hardware and image representation formats.

Fortunately, we have a solution around this problem now – Java. The write once run anywhere philosophy of java goes a long way in helping one out. But other problems still exist.

 

A student programmer thus has to expend considerable effort into activities not directly related to solving the problem at hand. Groping with nuances of toolkits can often take away disproportionate amounts of time. Therefore there exists a need for an image toolkit, bridging the gap between students and complexity.  Such a toolkit should possess the following characteristics

 

The image toolkit has to provide a logical image model consistent with simplified image model assumed/presented in the course.

This facilitates easy access to resources provided by the underlying framework.

·        Should provide a starter application with a modern GUI, for appeal and ease of use.

 

This toolkit is such a toolkit written in java.

 

Section 2 lists related work. Section 3 describes a general image model. Section 4 provides a description of image toolkit and its components. Section 5 is a short tutorial section illustrating usage of the toolkit classes then follows. Section 6 discusses interoperability with java’s imaging API. Section 7 compares this toolkit with other toolkits.

 

[2]          Related Work

 

Commercial and shareware/freeware libraries are available to developers of imaging and vision applications. A detailed discussion of many of these can be found in [1]. The primary goal of these libraries is to support application programming. They aim to provide working solutions for common imaging requirements. Most of the libraries support the whole gamut of imaging operations. Some like Khoros even support a visual programming environment.

The Java Vision Toolkit [1] is a notable exception. This java-based toolkit is designed with goals similar to those discussed here. It supports reading, writing and display of greyscale, colour and range based images. It also supports a host of image operations like histogram operations, convolution, image arithmetic, morphology, etc. It uses Java Advanced Imaging API (JAI) extensively. Most of the supported operations are provided by JAI. The JVT provides templates that simplify application programming. These templates form the basis of its extensible architecture. It also provides support for integrating user extension in to its GUI. Similarities and differences are further discussed in Section 7.

 

[3]          Image Model

 

An image is a two dimensional array of pixels. Each pixel can be represented as a single number (which represents intensity, as in the case of binary or greyscale images) or as a collection of numbers (that represent red, green, blue components as in the case of an RGB colour image).

This is a very simple logical model. In reality however, there are a lot of other considerations one has to contend with. For e.g. the image data may be compressed or encoded, the image may have an associated colour lookup table with the pixel values storing indices into it and so on.  A more general image model is the one used by the java imaging framework and is as shown below.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


The raster in this model, corresponds to the two dimensional array of pixels in our simple logical model. This raster comprises of two parts a data buffer and a sample model.

The data buffer here is the actual pixel store. The sample model indicates the data type of the pixel store and certain other attributes. This other attributes are image format specific. Different image formats exist for optimal representation of images, most often to reduce image size and sometimes to account for different colour reproduction techniques.

Image format dictates the maximum number of colours in an image, the colour model, number of components in a pixel, the organisation of these pixels in memory and whether the pixel data contains colour value or an index into a colour component palette.

The sample model defines the number of colour components and how they are packed together in memory. For e.g. an (r, g, b) pixel can be represented as three separate bytes or packed together into a single short.

The colour model defines what the individual components of the pixel mean (i.e. the semantics). This meaning is dependent on the colour space being used by the colour model.

Colour spaces map directly to colour reproduction techniques. Two colour spaces, which are very popular, are the CMY and the RGB colour spaces. CMY space is a subtractive colour space used by hardcopy devices like printers. RGB is an additive colour space used by display devices like monitors.

From this description, it is apparent that, handling image data involves a lot of detail. Most algorithms assume the simple logical image model. The image toolkit provides this model and is described in the following section.

 

[4]          Components of the Image Toolkit

 

The image toolkit provides the following classes:

ImageToolkit, BinaryImage, GrayScaleImage, ColorImage, Plugin, UIShell

 

Each of these classes and member functions are described below

 

class ImageToolkit

            This class provides various utility functions for loading and creating image types supported by the toolkit and conversion between image types supported by the toolkit.

 

            Methods:

            public static BinaryImage createBinaryImage(int width, int height);

public static GreyScaleImage createGreyScaleImage(int width, int height);

public static ColourImage createColourImage(int width, int height);

These three methods are used to create an unintialized image.

 

            public static BinaryImage createBinaryImage(BufferedImage src);

public static GreyScaleImage createGreyScaleImage(BufferedImage src);

public static ColourImage createColourImage(BufferedImage src);

These three methods are used to clone an existing image. These can also be used to convert from one image type to another.

 

            public static Image loadImage(String name);

            public static BufferedImage loadBufferedImage(String name);

            public static BinaryImage loadBinaryImage(String name);

public static GreyScaleImage loadGreyScaleImage(String name);

public static ColourImage loadColourImage(String name);

These five methods facilitate loading image files from local disk.

 

class BinaryImage

            An instance of this class represents a binary image. Each binary image pixel represents either foreground or background information in a byte.

            Methods:

            public int [] getPixel(int x, int y, int [] colour);

            public void setPixel(int x, int y, int [] colour);

            These methods provide access to image data

 

class GreyScaleImage

            An instance of this class represents a greyscale image. Each greyscale image pixel represents intensity information as a byte.

            Methods:

            public int [] getPixel(int x, int y, int [] colour);

            public void setPixel(int x, int y, int [] colour);

            These methods provide access to image data

 

class ColorImage

An instance of this class represents a colour image. The colour image is an RGB image and uses bytes to store individual RGB components of each pixel.

            Methods:

            public int [] getPixel(int x, int y, int [] colour);

            public void setPixel(int x, int y, int [] colour);

            These methods provide acess to image data

 

class Plugin

            This is the base class for deriving user defined application plugin classes implementing specific algorithms.

            Member Variable:

            public UIShell shell

            This member references the current instance of UIShell. This can be used to access shell functions from within the plugin (for e.g. to display the result of the computation within the shell).

 

            Methods:

            public void compute(BufferedImage src);

            public void compute(ColourImage src);

            public void compute(GreyScaleImage src);

            public void compute(BinaryImage src);

            These methods are overridden in the user defined plugin class to implement the desired functionality.  The first function is implemented in the base “Plugin” class to check the image type and call appropriate methods from the remaining methods. Such functionality may be desired if the plugin works with only certain types of images. If the plugin is general enough then the functionality can be implemented in the override of the first method itself.

 

class UIShell

            An instance of this represents and manages a GUI front end for an application. In addition this class provides useful functions for loading images, managing open images as a multi-document application, handling user input and invoking a plugin algorithms in response to user input.

            Methods:

            public void addImage(String name, BufferedImage img);

            This method is used  to add the result of an image computation to the UIShell.

 

            public void initApplication(String name);

            This method is used to intialize UIShell

 

            public BufferedImage getSelectedImage();

            This method returns the currently selected image

 

 

            public void registerPlugin (String name, Plugin plug);

            This method is used to register user defined plugins with the UIShell.

 

These six simple classes together provide the bridge between the logical image model and the underlying API.

           

[5]          Using the image toolkit

 

The image toolkit is simple to use. Its simplicity and functionality is best illustrated by a code sample. The following is the source code for a simple starter application.

 

import imageToolkit.*;

 

public class StarterApp {

     public static void main(String [] args){

          UIShell shell = new UIShell(“Starter App”);

     }

}

Listing 1 : A basic application

 

Figure 1 : StarterApp with an open image

 

This is simplest of applications that can be created using this toolkit. It provides a lot of functionality even for a program this small. It creates a complete multi-document user interface. The application can open and display images. All this is part of the built-in functionality.

 

The next sample shows creation of a simple plugin class and making it accessible from the shell’s “Plugins” menu.

 

import imageToolkit.*;

 

class MyFirstPlugin extends Plugin {

}

 

public class StarterApp {

     public static void main (String [] args) {

          UIShell shell = new UIShell(“Starter App”);

          MyFirstPlugin plug = new MyFirstPlugin();

          shell.registerPlugin(“MyFirstPlugin”, plug);

}

}

Listing 2 : Creating a plugin class

 

Figure 2 : MyFirstPlugin integrated with the UIShell menu

 

The next sample illustrates encoding an algorithm in the plugin by overriding plugin’s compute method. This sample acts on the input image and halves the input image intensity . The input image to compute is an RGB image and each pixel has three components.

 

import imageToolkit.*;

 

class MyFirstPlugin extends Plugin {

     public void compute(ColourImage src){

          int width = src.getWidth();

          int height = src.getHeight();

          int []pix=null;

 

          for(int y=0; y<height; y++)

              for(int x=0; x<width; x++)

          {

              pix = src.getPixel(x, y, pix);

              pix[0] /= 2;

              pix[1] /= 2;

              pix[2] /= 2;

              src.setPixel(x, y, pix);

          }

}

}

 

public class StarterApp {

     public static void main (String [] args) {

          UIShell shell = new UIShell(“Starter App”);

          MyFirstPlugin plug = new MyFirstPlugin();

          shell.registerPlugin(“MyFirstPlugin”, plug);

}

}

Listing 3 : Writing your own algorithm

 

The next sample extends the previous one to illustrate creation of a new image as a result of algorithmic computation and displaying it within UIShell. Unmodified parts of previous app have been dimmed out.

 

import imageToolkit.*;

 

class MyFirstPlugin extends Plugin {

     public void compute(ColourImage src){

          int width = src.getWidth();

          int height = src.getHeight();

int []pix = null;

 

ColourImage result;

result = ImageToolkit.createColourImage(width,       height);

 

         

          for(int y=0; y<height; y++)

              for(int x=0; x<width; x++)

          {

              pix = src.getPixel(x, y, pix);

              pix[0] /= 2;

              pix[1] /= 2;

              pix[2] /= 2;

              result.setPixel(x, y, pix);

          }

 

          shell.addImage(“Result”, result);

}

}

 

public class StarterApp {

     public static void main (String [] args) {

          UIShell shell = new UIShell(“Starter App”);

          MyFirstPlugin plug = new MyFirstPlugin();

          shell.registerPlugin(“MyFirstPlugin”, plug);

}

}

Listing 4 : Computing a new image and adding it to UIShell

 

Figure 3 : MyFirstPlugin output within the UIShell

 

These samples demonstrate the process of implementing algorithms using classes provided by the Image Toolkit. Further extensions can be programmed by extending the toolkit and customizing them as desired.

[6]          Interoperating with java imaging api

The image classes provided by image toolkit work seamlessly with the java api. Essentially, all image classes provided by the toolkit are derived from Java 2D’s BufferedImage class. Hence these classes can we used instead of java’s Image or  BufferedImage class. All the methods and operations defined for BufferedImage’s are applicable to these classes as well.

Java imaging API provides a large assortment of image processing operators. All of these can be used on this toolkits image classes.

The operations supported include:

·         Affine transformation

·         Amplitude scaling

·         Lookup-table modification

·         Linear combination of bands

·         Color conversion

·         Convolution

 

More information about these classes can be found in any Java 2D Imaging reference.

 

The following sample implements edge detection filter using Java2D’s ConvolveOp

 

import imageToolkit.*;

import java.awt.image.*;

 

class MyFirstPlugin extends Plugin {

     public void compute(ColourImage src){

          int width = src.getWidth();

          int height = src.getHeight();

 

ColourImage result;

result = ImageToolkit.createColourImage(width,       height);

 

          float []krnl = new float[9];

          krnl[0] = -1.0f; krnl[1] = -1.0f; krnl[2] = -1.0f;

 

          krnl[3] = -1.0f; krnl[4] = 8.0f;  krnl[5] = -1.0f;

 

          krnl[6] = -1.0f; krnl[7] = -1.0f; krnl[8] = -1.0f;

 

          ConvolveOp cp = new ConvolveOp(new Kernel(3, 3,  

                                                     krnl));

         

          cp.filter(src, result);

 

          shell.addImage(“Result”, result);

 

}

}

 

public class StarterApp {

     public static void main (String [] args) {

          UIShell shell = new UIShell(“Starter App”);

          MyFirstPlugin plug = new MyFirstPlugin();

          shell.registerPlugin(“MyFirstPlugin”, plug);

}

}

Listing 5 : Using Java 2D’s Convolve operator

Figure 4 : Using Java2D’s ConvolveOp

[7]          Comparison

 

JVT discussed in section 2 is similar to this toolkit in the following respects

·        It is java based.

·        It is extensible.

·        It provides a built in GUI.

·        It supports integration of user-defined operations with its built in GUI.

There is one important difference between the two

 

This toolkit is built on Java 2 platform alone, where as JVT need JAI in addition to the Java 2 platform. This can be a significant disadvantage for JVT for the following reasons

·        JAI is not a part of the widely available Java 2 platform and needs to be separately downloaded.

·        Most of the JAI routines are implemented as native language libraries. Hence code-using JAI is not 100% pure java code.

·        JAI is not implemented on all platforms hosting the java VM.

This toolkit does not suffer from these disadvantages. Moreover this toolkits image classes or those derived from it can easily be used in conjunction with JAI, should the need arise.

 

[8]          Conclusion

 

We’ve discussed a simple toolkit written to ease the programming effort required in creating imaging applications. The toolkit though simple is fully functional and easily extensible. We hope you find these classes useful.

[9]          References

 

[1]   Mark W. Powell, Vision Software for Teaching and Research                    

http://figment.csee.usf.edu/teach_res/powell.pdf, 1996

 

[2]   Tom Olson, “CVPR panel discussion on teaching image computation,”

http://figment.csee.usf.edu/teach_res/olson.html, 1996

     

[3]   R. Jordan and R. Lotufo, “Interactive digital image processing course on the world wide web”, IEEE International Conference on Image Processing, vol. 2, pp. 433-436, September 1996.

 

[4]   R. Jordan and R. Lotufo, “Hands-on digital image processing”, IEEE 26th National Conference on Frontiers in Education, vol. 3, pp. 1199-1202, November 1996.

 

[5]   KHOROS,

http://www.khoral.com

 

[6]   Image/J,

http://rsb.info.nih.gov/ij

 

[7]   Java Tutorial

http://web2.java.sun.com/docs/books/tutorial

 

[8]   Java API Specs

Sun Micro Systems

http://java.sun.com/j2se/1.3/docs/index.html

 

[9]   Java Advanced Imaging API

Sun Micro Systems

http://java.sun.com/products/java-media/jai/index.html

 

[10]    Jonathan Knudsen, Java 2D Graphics,

O’Reilly Associates