Graphics

When video graphics cards became able to per-pixel addressing, graphics started to become very important to computers. The first graphics that a computer could do involved sprites. These sprites were limited to different character blocks that were installed in the extended ASCII code. This ASCII code changed between computers, and that is one of the reasons it was never permanently part of the ASCII code. How computers would make graphics out of this is by placing these characters in different parts of the scene, and by making them different colors. I'm sure that all of you Windows users are familiar with the blue scandisk scene which is show after a bad shut down. That makes use of ASCII characters to make a primitive GUI.

Sprites

Sprites brought the use of graphics into the computer desktop. Sprites are basic 2D dimensional images, and by the use of layering, complex graphics can be made. All of the windows in a GUI based computer OS make use of sprite layers. When a window is minimized, the window's sprite layer is moved behind another layer, usually the desktop image. This makes it invisible, because it is no longer being drawn onscreen.

3D graphics are progressing at amazing rates. Through the use of dedicated video hardware acceleration, seemingly 3D worlds can be drawn onscreen in the form of games, 3D rendering, and 3D applications / 3D modeling. 3D graphics are much more complicated then 2D graphics, since they no longer rely on layered based sprites. Now each frame has to be drawn from scratch. The scene is saved as a series of vertexes to different polygons. Each of these polygons are usually mapped, or coated, in a texture. As you can see, 3D graphics are far more complex and computer intensive then 2D graphics. This is one of the major reasons why our OS desktop screens and GUI are still based in 2D graphics.

Banding

Banding occurs when too few different colors are used to make smooth transitions. The ability to notice color banding is effected by how large the transition is, how many colors are used, and the color transition to be covered.

In general, most color transitions that are done using less then 16-bit color will have noticeable signs of banding or dithering. Banding is most noticeable when light toned colors are used.

WARNING : SCREEN DEPTH MUST BE AT LEAST 24-BIT TO BE ABLE TO DIFFERENCIATE 16-BIT AND 24-BIT PICTURES

24-bit Depth	16-bit Depth	8-bit Depth
16777216 colors	65536 colors	256 colors
4-bit Depth	2-bit Depth	1-bit Depth
16 colors	4 colors	2 colors

Something to note about the use of these colors. Most people use 16-bit color, so their should be no noticeable difference between 24-bit and 16-bit color. This is actually based on the available colors that people will see when they are using the respective color depth display.

Dithering is used to produce gradual changes in colors with a limited number of available transitional colors. Without dithering, banding will occur. Dithering is more favorable then banding because it will not create lines or blocks, which can lead to image misinterpretation.

WARNING : SCREEN DEPTH MUST BE AT LEAST 24-BIT TO BE ABLE TO DIFFERENCIATE 16-BIT AND 24-BIT PICTURES

Banding occurs when too few different colors are used to make smooth transitions. The ability to notice color banding is effected by how large the transition is, how many colors are used, and the color transition to be covered.

In general, most color transitions that are done using less then 16-bit color will have noticeable signs of banding or dithering. Banding is most noticeable when light toned colors are used.

WARNING : SCREEN DEPTH MUST BE AT LEAST 24-BIT TO BE ABLE TO DIFFERENCIATE 16-BIT AND 24-BIT PICTURES

24-bit Depth	16-bit Depth	8-bit Depth
16777216 colors	65536 colors	256 colors
4-bit Depth	2-bit Depth	1-bit Depth
16 colors	4 colors	2 colors

Something to note about the use of these colors. Most people use 16-bit color, so their should be no noticeable difference between 24-bit and 16-bit color. This is actually based on the available colors that people will see when they are using the respective color depth display.

Dithering is used to produce gradual changes in colors with a limited number of available transitional colors. Without dithering, banding will occur. Dithering is more favorable then banding because it will not create lines or blocks, which can lead to image misinterpretation.

WARNING : SCREEN DEPTH MUST BE AT LEAST 24-BIT TO BE ABLE TO DIFFERENCIATE 16-BIT AND 24-BIT PICTURES

Monitors have surprising no moving parts. The only mechanical objects in a monitor are the controls and power switch. The internal workings of a monitor is all electrical.

The Cathode Ray Tube
Called CRT for short, it is a fancy name for a vacuum tube. Cathode ray tubes are used mostly in televisions and computer monitors. Every television and monitor that has a lot of depth is made from a CRT, although the newer flat screen monitors and televisions are made of a different technology called LCD, or Liquid Crystal Display. This vacuum tube is special in that it has an electron gun of one end, and a phosphor screen at the other end. The electron gun uses a high voltage of around 10kV to 50kV. This high amount of energy is used to project a steam of electrons, the very small particles with are electricity. This is know as a cathode ray. These electrons are projected at the phosphor screen which absorbs their energy, creating a spark of light. The phosphors glow when struck by electrons, and the electron stream can vary in intensity. If the electrons have too much energy, the phosphor's energy which it absorbed may leak over into the adjacent phosphors, creating a blurred image. If the electrons are too weak, the phosphors do not glow enough, so a dull image is created. Phosphors also come in a different levels of "persistence". Persistence is the amount of time the phosphor can hold its energy. If the phosphor is being hit by electrons frequently, for example with a high refresh rate monitor, short persistence phosphors are used because the phosphors need to switch from bright to off very quickly for a good picture. In a low refresh rate CRT, like a television, long persistence phosphors are used to prevent the phosphors to create a flickering effect because they are losing energy before they are recharged again. The phosphors will eventually wear out, and not be able to absorb enough energy to create a bright enough picture. This is why some older monitors and televisions seem to have a dull, dark picture.

Making an Image
The cathode ray of electrons can be controlled and directed by a magnetic field. Because electrons are charged particles, they can be deflected by a magnetic field. CRT's have a powerful magnet between the electron gun and phosphor screen called a deflection coil. A defection coil can also be referred to as deflection yoke, or just plain yoke. The magnetic strength of the deflection coil can be varied so that the electron beam can be swept horizontally from side to side on the phosphor screen. This sweeping motion is called horizontal sync. CRT's horizontal sync can take place anywhere from 15.75kHz in televisions and between 85-90kHz in monitors. The deflection coil is not only able to sweep the ray horizontally, but also vertically as well. This vertical sweep is called vertical sweep or refresh rate. The beam starts at the top right corner is drawn horizontally on the phosphor screen, and after each line is finished, the beam is moved down a little and another line is drawn.

When the beam gets to the bottom left corner of the screen, it starts again from the top right to draw another frame on the screen.

Interlaced Monitors
Early monitors and all televisions are interlaced. This means that instead of drawing every horizontal line in sequence, the picture is divided into 2 frames. The first frame draws only the odd lines, know as the odd field. After that frame is finished all of the even lines are drawn to complete the picture, known as the even field. This was done because the early CRT's couldn't draw the complete screen fast enough, so the top half of the phosphor screen would be already dark by the time the bottom was being drawn. Interlacing fixed this, because the CRT could draw half lines of the screen quick enough. Newer monitors do not interlace because they are plenty quick enough to get the whole image on screen.

color
So how can one beam of electrons create color? It can't. Instead, 3 separate beams are used. Each beam is responsible for one of the primary colors. Light is a little different than the primary colors that are taught in school. When dealing with light; red, green and blue can be used to make any color. This is different than the three other material or pigment primary colors; magenta, yellow, and cyan. Light is additive, to make white, all of the colors have to be used. The more light, the brighter it is. Pigment color is subtractive, to make white, none of the colors have to be used. The more pigment used, the darker it gets.

	Light			Pigment
	RED	GREEN	BLUE	MAGENTA	YELLOW	CYAN
Red	X			X	X
Yellow	X	X			X
Green		X			X	X
Cyan		X	X			X
Blue			X	X		X
Magenta	X		X	X
White	X	X	X
Black				X	X	X

Light (Red/Green/Blue)

Pigment (Magenta/Yellow/Cyan)

There are three electron guns in the back of the CRT, each with their own deflection coil. Instead of one phosphor on screen for each pixel, 3 are used. Each one of the phosphors is manufactured out of different materials so that it will glow a different color.

Using three electrons can have problems. One is that the electron gun responsible for one color will at one time or another light up the phosphor of another color. Is would distort the color displayed on screen, so manufactures have designed 3 different ways to prevent this.

INVAR Shadow Mask
This technology was designed first, and is used in televisions and most computer monitors. What it is, is basically a very thin sheet of metal which has tiny holes punched into it which are aligned and positioned to only let the correct electron beam hit the colored phosphor it is responsible for. One downside of this technology is that it will absorb a lot of the electron's that will pass through the mask, because the holes need to be very small to do their job properly. This will make the use of a lot of electrons to create a bright image, increasing power consumption. While these electrons are being absorbed by the shadow mask, they will cause the metal in the shadow mask to heat up. The current material used is a nickel/iron mixture called INVAR that has nearly zero expansion due to heat. Older materials that expanded more due to heat would distort the image on screen. The use of a shadow mask necessitates that the screen is to be parabolic in shape in not only the horizontal direction, but also in the vertical direction so that the shadow mask can correctly filter the electrons.

Aperture Grill
This technology was originally designed by Sony, and is more expensive to manufacture then the shadow mask technology. Aperture grills are composed of thin wires drawn vertically up and down the screen. This allows for brighter images because less electrons are impeded, with a more efficient filtering. A major downside of this technology is that as the wires heat up, they will expand and begin to bur the image. To prevent a blurred image, multiple thin horizontal "damper wires" are used to tighten the wires. These "damper wires" can become visible with bright colors because they themselves can block the electron beam.

Slot Mask
This technology was recently designed by NEC and Panasonic as a hybrid of the shadow mask and aperture grill technology. It is still a shadow mask, but differs in hole arrangement. The holes are oval shape instead of round. This compliments the oval shape that the electron beam forms, more correctly filtering the beam resulting in a better picture.

A TFT screen works on a completely different principle then CRT's. In TFT display, there is a fluorescent back light which shines light through a layer of liquid crystals. TFT screens work on the same principles of light, there are still 3 separate sub pixels which each display one primary color. An image is formed by the liquids ability to block light when it is charged with a current. Every TFT flat panel screen is made through a process called photolithography. This process will engrave small transistors on the sheet of glass, and each one can be digitally controlled. Behind this transistor sheet is a powerful fluorescent light. In front of this sheet of transistors is a thin film of liquid that is less than 1/100th of a millimeter thick. In front of that is a protective sheet of glass which is the screen. How it works is the liquid's crystal molecules will align differently under electrical current. When no electrical current is applied, all of the back light is allowed to pass through the liquid layer. When the transistor has a charge, the liquid molecules begin to align in such a way that they will block all light from passing through. Depending on the charge of the transistor, different light intensities can be made.

The benefits of a TFT flat panel are that they consume less energy, have lower radiation, and have a brighter picture than CRT's. Also TFT's are lighter, thin, and do not suffer any geometric distortion because they are perfectly flat. The downside to flat panels are that they have a fixed native resolution. This means that all other resolutions have to be scaled up to be displayed, which can result in some jazzed distortion due to interpolation. Another downside to them is that they have a limited color range. Some really bright whites, and really dark darks are almost impossible to display because of the liquid's sensitivity. This means that anything over 16-bit color will not be displayed. They can also suffer from broken transistors, meaning that some pixels may not work. This will only be noticeable in some situations, but finding a perfect display may take a little effort. Because TFT's are inherently digitally operated, it is unnecessary to convert to an analog picture like for a CRT. The better TFT displays have a digital video connection, which can be plugged into some video cards, called a DVI. This is to prevent image quality loss through the analog to digital to analog conversion that the signal goes through.

Current TFT displays have poorer quality than CRT's, and so they have only being used in situations where size is more important than quality. Laptops and hand held devices use TFT screens rather than CRT, because they screens can be manufactured to be almost completely flat, while CRT will always require a large foot print. As manufacturing techniques improve, LCD image quality is starting to improve and is getting very competitive to CRT's. The advantages of flat panels over CRT's are that they can be smaller, lighter in weight, have a brighter picture, and consume less energy. The current down sides are poorer image quality, fixed resolution, smaller screen size, slower refresh rate, limited color range and cost. The materials to produce TFT's are less expensive than CRT's, but manufactures are still refining the manufacturing process, and are still having difficulty in production, meaning higher prices.

When 3D acceleration is done, each pixel has have a color value. The color value is the result of the rendering pipeline. But to get that color, a lot of other information had to be know about that pixel.

Color Depth
color depth is the amount of colors that the pixel can be. The most common are 16-bit and 32-bit. But why only 3D accelerated only played in 16-bit and 32-bit? What happened to being able to play in 8-bit 256 color mode, or 24-bit color? And why can games run in 32-bit but note 24-bit, when both have the same number of colors?

16-bit
16-bit color has been the most used depth for a while now, but newer video cards are making the progression to 32-bits because of increased RAM bandwidth and frequencies. 16-bit color is capable of over 65 thousand different colors. There are 3 primary colors, but only 16-bits, so there is an improper bit allocation. When running in 2D, the extra bit is given to the green value, and can be expressed as 5-6-5. This means that there are 32 different shades of both red and blue, but green is capable of 64 different shades. This is a sloppy way to do things.

Games use the extra bit as an alpha value. This is represented as 5-5-5-1. This is good for sprites or simple transparency, but it is severely limited. A pixel can be opaque, or it can be translucent. There is no in between. So some games choose to do things a bit differently. They re- divide the bits more fairly. They give 4 bits to each value; red, green, blue and alpha. This is written as 4-4-4-4, and is much better because it allows for proper transparencies. This limits 3D accelerated games to only 4096 colors, which 16 levels of alpha. You can now appreciate why game programmers want to use 32-bit color depths to make their games look better.

32-bit VS 24-bit
The reasoning behind color depths is in the bit arrangement. Since computer RAM stores information in 64-bit chunks, 24-bit color can not be used without wasted bandwidth. If 2 pixels were to be stored in memory, there would be 16-bits that were left unused. If 3 24-bit pixels were to be stored in memory, two trips would have to be made. 24-bit just isn't suited for 3D. That is where 32-bit color comes in. 24-bit and 32-bit color depths use the same 24-bits for color information, 8-bits for each of the 3 primary colors. This yields a maximum color range of over 16 million colors. 32-bit color "pads" its 24-bits with 8 extra bits to maximize RAM efficiency. These bits can't go unused, so they are assigned to the alpha value. In short, the alpha value determines how transparent the pixel is. This is very useful when creating smoke or water effects.

More Bits
Pixel position in the X and Y plans is based on the pixel's order in an image. But how is the pixel's depth determined? That is the job of the Z-buffer. The z-buffer is usually 16-bits in 16-bit color mode, and 24-bits in 32-bit color mode. The more Z bits, the more accurately the pixel can be presented. For alpha transparencies, Z depth is very important, because a the renderer has to know if the pixel goes behind a transparent pixel to blend with it, or if it goes in front totally blocking that transparent pixel out.

Now, the same problem with 32-bit color Z-buffers. 24-bits is an awkward number of bits, and it just won't do. This is where the stencil buffer comes in. The stencil buffer is a 8-bit scratch pad which can be used any way the programmer chooses. Most commonly, it is used as a marker for pixels that will be reused, such as in a reflection in another object.

32-bit color Rendering (64-bits per pixel)
8-bit Red	8-bit Green	8-bit Blue	8-bit Alpha Transparency
24-bit Z Buffer			Stencil Buffer

Scene Generation
This is the first stage, and what happens here is the that the CPU has to figure out what is going on in the scene, and how to draw it. All of the scene information and settings are recorded, so that geometry transformation can begin.

Geometry Transformation, And Culling
This step is done by the CPU, with the exception of some of the newer video cards which feature a dedicated Transform & Lighting processor. This step is responsible for the creation of the coordinate matrix for every object, and the scene. These data matrixes are used in the later pipeline for calculations. All geometry information has to be recalculated every time an object moves. All geometry which is out of view or behind the viewpoint is removed, called culling.

Lighting, And Clipping
This step is done by the CPU, with the exception of some of the newer video cards which feature a dedicated Transform & Lighting processor. This step is responsible for the basic appearance of all of the polygons which were created by the previous stage. But matrix multiplication and comparisons, base colors are assigned to each polygon based on their orientation to the light sources, opacity and reflectivity. Any geometry which is hidden behind another objects is removed now, called clipping.

Triangle Setup
Triangle setup is done by all video accelerators, and this involves the breakdown of each polygon into horizontal strips called spans based on what textures they use. This stage on is also referred to as rasterization. Each one of these spans is then sent individually to the next pipeline stage.

Texturing
This pipeline takes each span and pixel by pixel begins to texture it. Based on what type of texture filtering is needed, and how many textures are to be used, a pixel may have to spend up to three clock cycles before it is complete. Most current video cards are capable of applying 2 or even 3 textures to each pixel in a single clock. This is called multi-texturing, and it is one of the major advancements in computer gaming, creating life-like realism out of simple polygons. Once each pixel is complete, it is sent to the assembly stage.

Rendering Assembly
This stage takes each pixel and determines its position in the buffer. If a pixel based on the pixels x and y coordinates already exists, the z-buffer is compared. If the current pixel has a higher z buffer value than the current pixel, transparency is compared. If the pixel already in the frame buffer has less than 100% opacity, than the current pixel is blended with that pixel, called alpha blending. If the current pixel has a lower z-index value and 100% opacity, the buffer pixel is replaced, if not, the two are blended.

Screen Output
This stage involves transferring that image in the back buffer to the front buffer so that it can be displayed.

Monochrome Display Adapter
The first video cards were made by IBM, and were called Monochrome Display Adapters. These video cards were only capable of a single color, at one shade, and at only one resolution. These cards were not capable of per pixel definement, meaning that they could not change the value of a specific pixel. Rather they had to change the screen in 9 by 14 pixel blocks, and they could only change the block to draw predefined alphanumeric characters. This meant only one font, and only one character size. The video card was capable of only 720X250 resolution. These cards would fit into ISA slots. What the card was just a connector for the monitor cable. The card itself had no memory or processing capabilities of it's own. It did reserve a small area of main memory which it used as a frame buffer. The processor and main system BIOS were responsible for the image processing, and because of this the MDA was the cause of poor system performance. All output to the monitor was in digital form, as opposed to analogue form, and all digital to analog conversion was done in the monitor.

Hercules Graphics Card
Hercules also had their own primitive video card. Like the MDA, it had no memory or processor of its own, it relied on the main CPU for all of its work. It was also fixed resolution and monochrome. This card did have a limited graphics capability, but not as good as a CGA video card.

Computer Graphics Adapter
CGA video cards were introduced around the same time as the MDA and HGC video cards. What made the CGA cards unique was the ability of being able to control specific pixels, a feature called "All Point Addressable". Along with graphics capability, CGA added the the ability to use up to 16 colors, and different character sizes. Although CGA still had no processing or memory onboard, it took advantage of faster CPU's, and it eventually progressed into being able to run two different resolutions in text mode, 320 by 200 and 640 by 400. It also had the ability to scale two different text sizes, although not at the same time. It could either display character blocks of 40 column by 25 row, or 80 column by 25 row.

Enhanced Graphics Adapter
EGA video cards was released by IBM in 1984 and was considered the second generation of PC graphics. It brought about advancements such as onboard memory, and a separate onboard video BIOS. Before EGA, all BIOS information had been incorporated into the system BIOS. EGA memory was also modular, meaning some EGA cards were sold with no memory, but had the ability to be upgraded with a memory expansion board and memory modules. EGA supported up to 256k of onboard memory and were significantly faster than other video cards, but EGA still was doomed with the same graphics limitations as the older CGA standard.

Professional Graphics Adapter
PGA cards were introduced in 1984, the same time as EGA, but they sold for over 4 times as much. The reason was because PGA had it's own separate processor that would operate all video related tasks. PGA cards were designated for the high end business market. The PGA would take up a whole 3 expansion slots, and was capable of a full 60fps of 3D clipping, tasks that were impossible for the other cards. PGA never really caught on because of their steep price, and VGA alternative.

MultiColor Graphics Array
MCGA was released by IBM in 1987 as a low cost replacement to VGA. This technology never caught on because IBM was legally trying to charge royalties for anyone who used it. MCGA had a higher color depth and resolution capability then VGA, but it was compatible only with IBM's PS/2 model MCA (Micro Channel Architecture) bus systems.

8514/A
This was released in 1987 by IBM as a high end replacement to VGA.

Video Graphics Adapter
VGA was introduced the same time in 1987 as both 8514 and MCGA by IBM. VGA became an instant success because unlike the MCGA,8514 and the later XGA, VGA was capable of operating on the standard ISA bus, and so no royalties would have to be paid to IBM. This was the first technology that would send information to the monitor in analog form. This removed the restrictions of color depth limitations, because analogue allows for a much wider frequency of colors. This created a need for the use of a RAMDAC on the video card. VGA also implemented the use of a video accelerator. This wasn't as fast as a dedicated co-processor, but it still maintained a low manufacturing cost while keeping the price down. A video accelerator would handle most of the video processing, but still needed the processor for direction and high level tasks.

Extended Graphics Array
IBM soon dropped support for the failing MCGA and 8514/A in 1990 it introduced XGA. XGA cards were build with up to 1MB of frame buffer, and were equipped with their own co-processor like the older PGA cards. This gave them better performance than VGA, but they were never very big in systems other than IBM's PS/2 computers. XGA was capable of 16 bit color at 640x480 and up to 256 colors at 1024X768 resolution. IBM later came out with an XGA2 which were a little faster, and supported higher resolutions and color depths.

Super Video Graphics Display
This was created as an improvement over VGA, adding higher resolutions and color depths. There was a major problem, because IBM wasn't responsible for its release, other smaller companies were. This created a lack of a standard, so although most SVGA cards were compatible with VGA monitors, and visa versa, software and programming differences started to create difficulties.

Video Electronics Standards Association SVGA
This standard was created by an organization of manufacturers as a solution to the SVGA problems. The organization soon released a VESA BIOS Extension that all video cards had to be compatible with, solving the incompatibilities programmers were having. This standard has been extended to cover every resolution and color depth of today.