|
|
|
Paul Cockshott |
|
Faraday partnership |
|
|
|
|
The AWT and how to put a simple image onto the
screen |
|
Layout managers |
|
The Jimage Class, |
|
|
|
|
At the end of this lecture you should have an
idea of how to display a JPEG image on the screen, and how to load it into
the Jimage class to carry out further image processing. |
|
|
|
|
AWT Images |
|
Image Producers and Consumers |
|
Jimage class |
|
Pixel Representations |
|
JPEG files |
|
|
|
|
AWT
abstract windows toolkit, supported by JavaSoft |
|
Operating system independent layer for windowing
in Java |
|
Fiendishly obscure |
|
Designed around requirements of images being
streamed off the web |
|
|
|
|
Simple image display program to show how to
display a JPEG file |
|
Pipeline
model of image production |
|
Jimages act as image consumers |
|
Jimages allow arithmetic on image |
|
Jimages provide output to AWT images and JPEG |
|
|
|
|
import java.awt.*; |
|
import java.awt.image.*; |
|
import java.util.*; |
|
class JPEGshow extends Frame { |
|
... |
|
static public void main(String[] args) { |
|
if (args.length == 1) new JPEGshow(args[0]); |
|
else System.err.println("usage: java JPEGshow <image
file>"); |
|
} |
|
} |
|
This is a standard Java Program class with a public
static void main method |
|
|
|
|
JPEGshow(String
filename) { |
|
super("JPEG show Example"); |
|
add( |
|
new ImageCanvas(getToolkit().getImage(filename) ), |
|
BorderLayout.CENTER); |
|
setSize(700, 540); |
|
show(); |
|
} |
|
|
|
|
Each frame has associated with it a toolkit
object the provides an interface to OS specific operations. |
|
CreateImage |
|
CreateMenu |
|
CreateLabel |
|
CreateMenuBar …. etc |
|
|
|
|
class ImageCanvas extends Component { |
|
Image
image; |
|
ImageCanvas(Image image) |
|
{this.image = image;} |
|
public void paint(Graphics g) |
|
{
g.drawImage(image, 0, 0, this);} |
|
} |
|
|
|
|
Pipeline flow model of image processing |
|
Images are just tokens linking producers and
consumers |
|
|
|
|
addConsumer(ImageConsumer ic) This method
is used to register an ImageConsumer with the ImageProducer for access to
the image data during a later reconstruction of the Image. |
|
removeConsumer(ImageConsumer ic) This
method removes the given ImageConsumer object from the list of consumers
currently registered to receive image data. |
|
startProduction(ImageConsumer ic) This
method starts an immediate
reconstruction of the image data |
|
|
|
|
void setDimensions(int width,
int height) |
|
The dimensions of the source image are
reported using the setDimensions method call. |
|
Void setPixels(int x, int y,
int w, int h, ColorModel model, byte[] pixels,
int off, int scansize) |
|
The pixels of the image are delivered using one
or more calls to the setPixels method. |
|
|
|
|
|
|
|
AWT is
operating system independent |
|
Streaming image model |
|
Images
as tokens |
|
Producer
- consumer pipeline |
|
|
|
|
Library of image processing classes developed at
Faraday |
|
Available for student practicals |
|
Algebraic rather than stream oriented |
|
Interfaces to MMX hardware under windows |
|
|
|
|
By this we mean the it is structured around
algebraic expressions whose values are images |
|
Thus if A and B are images and ‹ is some
operator then |
|
A ‹ B is also an image |
|
|
|
|
Arithmetic |
|
I+J Universal plus(Universal) |
|
I-J Universal minus(Universal) |
|
I‚J Universal times(Universal) |
|
I„J Universal divide(Universal) |
|
˝I˝Universal abs() |
|
Filtering |
|
Jimage convolve(double[] k) convolve with
symmetrical separable kernel. |
|
public
abstract Jimage convolve(double[][] kernel)with non separable kernel |
|
|
|
|
|
|
Jimage getScaledInstance(int nwidth, int
nheight)
This scales with bicubic interpolation. |
|
Jimage getScaledInstance(int nwidth, int
nheight, int ndepth)
This method allows the depth as well as area
of an image to be altered if it is reduced the planes are aggregated if
increased they are interpolated. |
|
|
|
|
|
|
|
Data access |
|
int rgbpixel(int x,int y) |
|
Converts the plane information into a pixel in
the direct color model of java. |
|
public abstract int upixel(int x, |
|
int y, |
|
int plane) |
|
- returns unsigned integer pixel |
|
public abstract float fpixel(int x, |
|
int y, |
|
int plane) |
|
Returns the pixel in the range -1 to +1. |
|
|
|
|
|
public abstract void setPixel(int x, |
|
int y, |
|
int plane, |
|
double pix) |
|
Pixel information in range -1 to +1 |
|
public void setSubImage(int x, |
|
int y, |
|
int z, |
|
Jimage im) |
|
Update an area of an image with another one. The
other one must not run off the edge of the one being written to. The source
of the copying is the 0th plane of the source jimage. |
|
|
|
|
|
public void putJPEGImage( |
|
java.lang.String fileName, |
|
int quality) |
|
throws java.io.IOException |
|
Outputs the image to a jpeg file |
|
public boolean getImage(java.lang.String
fileName) |
|
Initialise the Jimage from the specified file.
The file must be jpeg or gif. |
|
|
|
|
|
|
public java.awt.Image getAWTImage() |
|
|
|
public java.awt.image.ImageProducer getProducer() |
|
|
|
|
|
|
|
class Jimageshow extends Frame { |
|
Jimageshow(String filename) { |
|
super("Jimage show Example"); |
|
Jimage raw=new ByteImage(100,200,3); |
|
if (raw.getImage(filename)){ |
|
Jimage cooked = (Jimage)raw.times(0.3); |
|
add(new ImageCanvas(cooked.getAWTImage()),
BorderLayout.CENTER); |
|
setSize(700, 540); |
|
show(); |
|
} |
|
} |
|
|
|
|
When dealing with displays it is conventional to
assume that pixels are bytes holding numbers in the range 0 to 255. |
|
0 Is assumed to be black |
|
1 Is assumed to be white or maximum brightness
of any given colour. |
|
For multicolour displays with 3 colour
components, the convention is to have 3 fields of range 0..255 to hold the
colour information. |
|
|
|
|
|
For
multicolour displays with 3 colour components, the convention is to have 3
fields of range 0..255 to hold the colour information. The AWT does this
with the class Color. |
|
public Color(int rgb) |
|
Creates a color with the specified RGB value,
where the red component is in bits 16-23 of the argument, the green
component is in bits 8-15 of the argument, and the blue component is in
bits 0-7. The value zero indicates no contribution from the primary color
component. |
|
A Jimage returns this format with int
rgbpixel(). |
|
|
|
|
|
|
|
|
|
The byte data type in Java does not take on the
values 0..255. Instead it takes on the values -128 to 127. |
|
There are no unsigned bytes in Java. |
|
This creates a problem for the representation of
pixels in Jimages. |
|
The solution adopted is to adopt the following
representation |
|
-128 = black |
|
0 = mid grey |
|
127 = white |
|
|
|
|
|
|
|
|
|
If byte
pixels are signed then so must other representations be. |
|
The solution adopted is to adopt the following
representation for floats |
|
-1 = black |
|
0 = mid grey |
|
1 = white |
|
|
|
|
|
|
|
|
unsigned bytes
shorts float |
|
min value 0 -128 -2048 -1 |
|
maxval 255 127 2047 1 |
|
medianval m 127.5 -0.5 -0.5
0 |
|
range r 255 255 4095 2 |
|
As shown in table a pixel prin
representation r is converted to a pixel ps in representation s
by the operation: |
|
ps = ms+(rs(pr-mr))¸rr |
|
|
|
|
|
|
Signed
pixels seem at first to be counter-intuitive but they have numerous
advantages. |
|
A value of 0 or mid grey can be viewed as the
‘most likely’ value that a pixel takes on in the absence of other
information. |
|
If you do arithmetic on images, in particular
subtract one image from another, then negative values of pixels naturally
arise. |
|
Signed pixels allow straightforward
implementation of contrast adjustments. For instance multiplying an image
by 0.5 halves the contrast in the image. |
|
|
|
|
|
|
Signed
pixels allow straightforward implementation of contrast adjustments. For
instance multiplying an image by 0.5 halves the contrast in the image. |
|
|
|
|
|
|
|
|
|
|
|
|
Image may contain a million pixels, |
|
Arithmetic may be required on each one |
|
Important to optimise operations or they are
very time consuming |
|
May need to use assembler kernels |
|
May need to use special purpose instructions |
|
|
|
|
Intel and other CPU manufacturers have been
adding to the instruction sets of their computers new extensions that
handle multi-media data. |
|
The aim is to allow operations to proceed on
multiple pixels each clock cycle |
|
|
|
|
Standard Intel register set |
|
|
|
|
Standard Intel register set operating in MMX
mode |
|
|
|
|
Existing operating systems must still work
unchanged |
|
Applications not using MMX run unchanged |
|
No new state added to the CPU |
|
Hence, shared use of the FP registers, since
these are already supported by exising OS’s |
|
|
|
|
|
A problem with limited precision arithmetic is
that overflows frequently occur. This can give rise to meaningless results:
consider |
|
200+175 = 375 but in 8 bit binary |
|
11001000 |
|
+10101111 |
|
=101110111 |
|
|
|
|
You can fix this by using conditionals |
|
unsigned char p1,p2,p3; |
|
int I3= (int)p1 + (int)p2; |
|
p3=(I3>255?255:(unsigned char)I3); |
|
|
|
|
|
|
12: j=(int)(*p1++)+(int)(*p2++); |
|
00401043
mov ecx,dword ptr
[ebp-4] |
|
00401046
xor edx,edx |
|
00401048
mov dl,byte ptr [ecx] |
|
0040104A
mov eax,dword ptr
[ebp-8] |
|
0040104D
xor ecx,ecx |
|
0040104F
mov cl,byte ptr [eax] |
|
00401051
add edx,ecx |
|
00401053
mov dword ptr
[ebp-14h],edx |
|
00401056
mov edx,dword ptr
[ebp-8] |
|
00401059
add edx,1 |
|
0040105C
mov dword ptr
[ebp-8],edx |
|
0040105F
mov eax,dword ptr
[ebp-4] |
|
00401062
add eax,1 |
|
00401065
mov dword ptr
[ebp-4],eax |
|
|
|
|
13: |
|
14:
*p3 = (unsignedchar)(j>255?255:j); |
|
00401068
cmp dword ptr
[ebp-14h],0FFh |
|
0040106F
jle main+6Ah
(0040107a) |
|
00401071
mov dword ptr
[ebp-18h],0FFh |
|
00401078
jmp main+70h
(00401080) |
|
0040107A
mov ecx,dword ptr
[ebp-14h] |
|
0040107D
mov dword ptr
[ebp-18h],ecx |
|
00401080
mov edx,dword ptr
[ebp-0Ch] |
|
00401083
mov al,byte ptr
[ebp-18h] |
|
00401086
mov byte ptr [edx],al |
|
15: p3++; |
|
00401088
mov ecx,dword ptr
[ebp-0Ch] |
|
0040108B
add ecx,1 |
|
0040108E
mov dword ptr
[ebp-0Ch],ecx |
|
|
|
|
Iu8vec8
*v1,*v2,*v3; |
|
int i,j,k; |
|
for(i=0;i<31;i++){ |
|
*v3=(*v1++)+(*v2++); |
|
v3++; |
|
} |
|
_mm_empty(); |
|
|
|
|
mov
ecx ,32 ; load counter with
32 |
|
l1: movq mm0,[esi] ; load 8 bytes |
|
add
esi,8 ; inc src pntr |
|
paddusb
mm0,[edx] ; packed unsigned add bytes |
|
add
edx,8 ; inc src pntr |
|
movq
[edi],mm0 ; store 8 byte result |
|
add
edi,8 ; inc dest pntr |
|
loop
nz,l1 ; dec counter, |
|
; repeat non zero |
|
|
|
|
On image of 256x256 pixels |
|
Old C code executes 26*256*256 instructions =
1,703,936 instructions |
|
Optimised mmx code executes 6*256*32
instructions = 49,152 |
|
Note that no compiler currently will give the
optimised code. It has to be hand assembled. |
|
|
|
|
Intel Provide an image porcessing library that
can be downloaded from their web site. |
|
It provides efficient access to the MMX
hardware. |
|
It provides frequently used Image Processing
Operations. |
|
It requires a set of DLLs in your path to run |
|
|
|
|
At the core of IPL is the ability to write to a
single API and get the best possible results for any Intel processor. The
libraries have as many as six processor-specific branches for each function
and six sets of carefully written assembly code, but only one entry point
to each function. |
|
|
|
|
|
|
|
Use of Intel IPL complex and requires C |
|
I have provided 2 java classes that call the
IPL. |
|
IntelBImage and IntelFImage. These are
documented in the Jimage web pages. They inherit from ByteImage and
FloatImage |
|
To use them the Intel IPL must have been
installed on your machine and be on the path. |
|
If you are forced to use Unix machines the
libraries will not be available to you. |
|
|
|
|
|
|
http://www.javasoft.com/products/jdk/1.2/docs/api/java/awt/package-summary.html |
|
http://developer.intel.com/vtune/perflibst/ipl/index.htm |
|
http://developer.intel.com/vtune/perflibst/ipl/ipapi.htm |
|
|
|
|
|
|
|
|
What did you not understand here? |
|
What would you like more information on? |
|