LCD Multitouch Table Mark 1

Multitouch is cool, but its also becoming cheaper and more accessible thanks to the work of folks like Jeff Han, and communities like the NUI Group. Of course, devices like the iPhone have made many of these ideas mainstream, but not everyone wants to do multitouch on a tiny display. Multitouch surfaces have great potential as intuitive, multi-user, social computing environments.

I wanted to build a small, portable touch table that can be used by one or two people. The display surface is the horizontal table top, and people "play" at opposite sides. Its a bit like those old-style sit-down arcade games, though for the life of me I can't remember any games that were simultaneous two-player. Both users can touch the surface, but also maintain face-to-face interactions across the table. The device is designed particularly with kids in mind, for play between two children, or between a child and parent.

One of the main motivations for building this was to come up with an aid for educating kids as young as 2 years old. Some children struggle to learn stuff that most of us pick up effortlessly, like how to talk and interact socially. Very early intervention can make a huge difference for kids with conditions like autism and other language disabilities. While therapies like ABA can be very effective, they can also be gruelling and intense for both the child and the therapist (usually, the parent). If we can simplify this process using engaging interactive games, and by sharing and building resources collaboratively on-line, this has to be a good thing. More on this later...

Design

There are many ways to do multitouch sensing, but the basic idea used here is to measure pressure optically with a camera. A great place to get some background is this thread at NUI group. Since I wanted to do fiducial tracking, I chose an approach based on diffuse illumination. This means that the display does not rely on a very tight coupling with the surface to illuminate objects.

I wanted my table to be small, cheap and robust, so I chose to use an LCD rather than a projector. LCDs don't transmit diffuse IR very well, so the illumination is provided by the surface itself (Diffuse Surface Illumination, or "DSI"). A special type of acrylic (called "Endlighten" or "DLux", depending on the manufacturer) is used as the top surface of the display. This has embedded particles which reflect light out the surface when lit from the side. This material is often used in display signs.

Parts List

  • LG W2252TQ-PFT 22" LCD Monitor
  • Unibrain Fire-i B/W Camera
  • 2.5mm focal length lens
  • Mac Mini
  • Firewire 9-pin to 6-pin cable
  • IR pass filter 850nm
  • Acrylic panel, 10mm "Endlighten"
  • IR LED strip illuminator
  • Power supply, 12V for illuminator
  • Misc hardware: MDF, wood screws, nut/bolts, square dowell, 1" aluminium angle, steel strip, T-nuts, hinge, construction adhesive, silicone sealant, magnetic catches, gaffa tape

I found a lot of useful information on-line, but much of it was fragmented. So here is an illustrated guide to how I built my system. Click on the images for higher resolution versions.

Step 1: Disassemble Monitor

I selected a 22" LCD display from (LG W2252TQ-PFT), as these seem to give a good balance between size and cost. I wanted something cheap so that it didn't matter too much if I trashed it.

First problem is to get the thing apart without breaking it. The monitor is held together with pesky snap fittings, so you have to carefully prise it apart with a screwdriver. Hint: if you remove the grey front bezel, try levering from the bottom left and right corners.

Once you've removed the plastic there are basically just two parts left, the panel and the power supply unit, which also has the video connector. Be careful not to scratch the display surface. The black back-light cables are only just long enough in the original configuration. Eventually, you'll need to carefully lengthen these.

The control buttons are supported by the original casing. I just cut the whole thing out with a dremel tool.

We need to separate the LCD driver board from the panel, so that we can remove the backing. Its held on by three black screws - I couldn't shift these with a screw driver and had to drill them out.

This close-up of the driver board shows the connector for the ribbon cable. Be very careful not to stress the yellow flexi cables that connect the driver to the panel. These are held on by glue, and if you tug on them they will pull off the panel and you will lose entire rows or columns of pixels. If you do damage the cables, they may be repairable, but I haven't ever tried this, so play it safe.

The metal backing is held on with clips around the edges. Carefully prise these open and you'll be able to lift the backing off. The display is made of several layers: the panel (front) with a polarising film on the front and back, a diffuser, a fresnel lens, the backlight (a perspex panel), and a glossy white plastic reflector. Most of this stuff is just to get smooth lighting of the display. I discarded the fresnel lens and the reflector, though you'll probably want to keep them until you're finished.

At this point you'll probably want to test the panel to make sure it still works. I set mine up on a perspex sheet. The display will still run without the backlight.

For experiments, I set up the camera under the panel in a little frame made of Duplo. You can light it from the back with a desk lamp.

Here you can see the panel running with the original backlight.

Here you can see how much brightness is lost by removing the back reflector. It makes a big difference, though the screen is usable without it. You can probably get some brightness back by lining the interior of the box with a similar glossy white coating.

Step 2: Modify the backlight

We can use the existing backlight with two modifications: cut the center out of the back, and lengthen the backlight cables.

The backlight has two long fluorescent tubes, mounted in the sides of the panel. I decided to keep as much of the frame as possible, removing the center portion.

The backlight cables connect at one end of the tubes, which are supported by plastic insulators.

I cut the center out of the panel using a combination of hack-saw and tin snips. If you have some sort of cutting wheel that might work better. Do it carefully so you don't damage the fluoro tubes.

Try to leave some space around the corners. As you can see here, its quite thin in places. When its done, the whole LCD assembley should fit back into the frame.

The perspex panel has an interesting pattern etched in the bottom surface. I guess this is to help reflect the light forward. Otherwise, the perspex looks quite clear, so I don't think its a diffusing edge-lit type.

The density of the cross-hatch pattern is higher in the middle, presumably to normlise the intensity of reflected light across the surface.


Now it time to lengthen the backlight cables. Each fluoro has two plugs (four wires). The blue and pink wires are high voltage (3kV), so you need to use a thick wire with good insulation. The power supply includes some sort of sensing that will shut down the supply if there is too much voltage drop. This means that you can't add more than about 20cm to the cable length. I cut the cables near the connector and joined the extra lengths in, insulating the joins with heat-shrink tubing.

Step 3 : Assemble the touch surface

I wanted a design that would enable sensing of objects on the surface, so I chose to use an IR diffusing surface. The top panel is 10mm edge-lit acrylic. This has a couple of brand names, including "Endlighten" and "DLux", depending on the manufacturer.

The acrylic panel is lit from the side with 850nm IR LEDs. To save time, I used 850nm IR LED Flex ribbon, from <http://www.environmentallights.com/categories/1303_2399/infrared-led-ribbon-flex">Environmental Lights. This can easily be cut to length. Its not cheap, but a 5m reel gave me enough to make two surfaces.

Because its flexible, the flex ribbon is very easy to attach to the panel. I just taped it on with masking tape. Here's the completed panel sitting on the beginnings of my box.

Step 4 : Putting it all together


1.9mm lens

2.5mm lens

3.6mm lens

4.3mm lens

Depending on the size of the panel, and desired height of the table you need to choose a lens of the appropriate focal length. Short focal lengths allow you to minimise the distance between the surface and the camera, but they also introduce spherical distortion. With too much distortion, the finger-touch blobs will be smaller at the edge of the surface, requiring more pressure to get the same response. I initially tried a 1.9mm lens, which allowed me to get the camera about 20cm from the surface but the response was not uniform enough to make drag gestures work well. I bought a few different size lenses, and tested them for distortion and distance. I settled on a 2.5mm lens, which could see the entire panel from about 30cm distance.

Once I had the distance between the camera and panel, I made the box to fit. It takes a bit of juggling to get everything in the right place. Other critical distances are the distance between the power supply and the corners of the backlight, and the distance between the power supply and the driver board (for the flat ribbon cable). For safety, I chose to keep the power supply in its original shield, but the shield actually contains two small boards with a short cable between them. These could be separated if things get tight with the spacing.

Here you can see an early side view with the camera and power supply in place.

This shows a top view of the completed system, including all electronics and cabling.

Here is the finished system up and running.

Lessons Learned

Tracking

I found the DSI surface to be surprisingly good. The contrast is low because in addition to the IR reflected from finger touches, it also receives IR diffused through the bottom surface of the acrylic, IR from the backlight, as well as ambient IR from the room. Despite the low contrast, it does seem very consistent and with appropriate settings the blob detection is reliable. If anything, it is a little too sensitive at times, responding to very light touches, brushes from sleeves, etc. It works well in normal room lighting. It does not have to be dark, though if there is bright sunlight on the surface or nearby it has some problems. Direct sunlight on the surface produces spurious blobs, and a very high ambient brightness can result in shadows and reduced sensitivity.

I used Community Core Vision to do the tracking. I tried the Reactivision tracker, but could get nothing but noise out of it (perhaps it doesn't work with low contrast?).

Fiducials Fail

I tried using Reactivision fiducial markers, but found the images very blurry. This blurring comes partly from the LCD panel, and partly from the diffuser.

<

In this image, the diffuser covers only the right hand side of the display, so you can see the difference between the panel by itself (left) and with the diffuser (right). In both cases, the tracker fails to segment the fiducial from the background, though the images could possibly be recovered with some specialised signal processing, at least without the diffuser.

For comparison, here is the marker in front of the panel (right) and behind the panel (left).

The purpose of the diffuser is to produce a constant background illumination for the LCD panel. It does this by blurring, so this effect should not really be surprising. Without the diffuser, you can see objects behind the panel which can be confusing.

This image shows the effect on image clarity of removing the diffuser. Without the diffuser (right) we can see the cross-hatch pattern on the lower surface of the backlight. This could be improved using a different design of backlight, but then we may end up seeing through to objects behind the panel (eg. the camera and wiring). Ideally, we need a material that diffuses visible light, but transmits IR.

Anyway, although its not a showstopper, its still is an issue that I would like to solve. If anyone has any ideas or suggesions, please let me know.

Future Work

Some things still to do:

  • Add cooling. With the box completely closed, it can overheat. I plan to add some inlet vents at the top, which will also let out the sound from the speakers. A fan will exhaust air out the bottom of the unit - there's a small gap between the bottom of the cabinet and the ground.
  • Switch the illuminators. Currently these are always on while the unit is plugged in, but there's an activity LED from the monitor which I could easily use to switch the power to the illuminators.
  • Switch the CPU. Currently need to open the table to press the power button on the Mac Mini. I don't know if the Mac can be remotely switched. I'm reluctant to hack it just yet.
  • Get fiducials working.
  • Write some more code!

And one observation. If you leave your touch table open, it may become home to unwanted guests. I walked in today to find things wildly moving about the screen. On closer investigation I found this:

I don't blame her - its midwinter here, and for a short-haired cat, a heated box is pretty cozy place to be.

21 Comments

  1. Multitouch computer tables for libraries (and cold cats) | Librarians Matter:

    [...] how he incorporated a Mac mini and keyboard in the casing – here in this blog post,  LCD Multitouch Table Mark 1 . There is even a photo of our cat, warming herself in the  computer table during [...]

  2. guillaume:

    bonjour je suis francais et parle pas anglais je voulais savoir si on pouvais utilisé une tele lcd pour avoir un plus grand ecran merci de votre reponse et beau travaille

  3. Stewart:

    Hi Guilluame,

    An LCD TV should work fine. With any monitor, the most important thing is that you can move electronic components (connectors, power supply) away from the back of the display. Unfortunately, you never know this until you've actually disassembled the display. If you can, try to find a model which someone has tried, so that you can avoid "FFC issues".

    Apart from that, the design of the backlight may be different for a TV, especially if the display is much larger. With my 22" monitor I was able to get good illumination by retaining the side-lit design. I doubt that a very large backlight could work like this, so you may need to implement your own backlighting. You could do this using fluorescent lamps, or LEDs behind the panel.

    Hope this helps.

  4. Filip:

    Hi, it is great setup. a am building MT-lcd too.
    your setup is:
    ------------ lcd matrix
    ------------ lcd ambilight
    ------------ Endlighten

    ? camera

    am i right???

  5. Stewart:

    Hi Filip,

    Actually, it looks like this:

    ------------ Endlighten
    ------------ LCD matrix
    ------------ diffuser
    ------------ LCD backlight

    v camera

    The endlighten sits on the top. Its edge-lit with IR LEDs. When you touch the endlighten it lights up in the IR spectrum. The camera can see this through the LCD matrix. On all the monitors I tried, the LCD backlight is just a transparent piece of perspex, edge lit using white fluorescent tubes. The visible light does not interfere with the IR signal. The diffuser is part of the original backlight. I left it in because it makes the illumination of the panel more even and seems not to affect the IR signal.

    Hope this helps.

    Cheers,
    Stewart

  6. Filip:

    Thank you on reply. You help me a lot. Now i am waiting Endlighten. On my setup i puted backlight on bottom and i dont have good brightness. This is much better. Are you polished edges of perspex? Try Windows 7 TouchPack if you didn't.

  7. Microsoft Surface Table at Libraries « Library Hat:

    [...] geek in me wanted to know how this table is built and found this excellent blog post by Stewart Greenhill about how to build a home-made surface table with a relatively cheap LCD monitor with the total [...]

  8. Christian:

    Great article!

    How much did you spend on the project?

  9. Stewart:

    Hi Christian,

    Sorry for the lengthy delay - I'm finally getting back to my sadly neglected blog, and sorting comments from the spam.

    The main cost is the Mac Mini. I don't have exact costings, but here are some rough figures. Some things I bought locally (in Australian dollars), and others are in US dollars.

    Monitor: LG W2252TQ-PFT ($AU 270)
    Camera: Unibrain Fire-i (B/W, $US 150)
    Lens: 2.5mm focal length ($AU 30)
    Brain: Mac Mini ($AU 1000)
    Cable: Firewire 9-pin to 6-pin ($AU 50)
    IR Filter: 850nm IR pass ($AU 20)
    Acrylic Panel: Endlighten 10mm ($AU 30)
    IR illuminator: 850nm IR LED strip light (~2meters, $US 100)
    Power Supply: 12V, for illuminator ($AU 30)

    Assorted hardware: MDF, wood screws, nut/bolts, square dowell, 1" aluminium angle, steel strip, T-nuts, hinge, construction adhesive, silicone sealant, magnetic catches, gaffa tape (~$AU 100)

    Hope this helps.

  10. vossler:

    Hi Stewart,

    Impressed work!

    I am curious if you put some IR LEDs in the table's bottom, is that gonna make fiducial marker more clear?

  11. Stewart:

    That's what I thought would happen, but it doesn't seem to work. If you put IR illuminators behind the LCD, you don't get any signal back from the front surface. I'm not sure of the reason, but I haven't found anyone who has got it to work that way.

  12. John:

    I am looking on Environmental Light's website and see 658 mW/meter and 1,752 mW/meter brightness. Which one did you use?
    Do you still have the part number?

    Thanks
    John

  13. John:

    Sorry, I thought of another question.
    How much larger should the Acrylic be than the actual LCD?

    Thanks again,
    John

  14. Stewart:

    Hi John,

    Here's the part description from the invoice:

    1 x Narrow Dimmable LED Ribbon Flex by the 5-meter reel
    (InfraRed 850 nm) (irrf850-reel)

    At the time, the cost was $177 plus freight.

    The size of the acrylic depends on how you're going to mount it. I ended up mounting the acrylic as the top surface of my box, so it rested on top of the four sides. I made it big enough for a 25mm border on all sides. Initially I was going to sandwich the panel between the endlighten and a separate smaller sheet of perspex, but in the end I supported the panel on a small ledge inside the box. An advantage of the "sandwich" idea is that the whole assembley is more-or-less self supporting, but if you include the driver board in the sandwich its quite a bit larger on the top side. Make sure you support the weight of the acrylic - don't just rest it on top of the LCD or you'll end up distorting or cracking it.

    Hope this helps.

  15. John:

    Hi again,

    You site is a big help and thanks for answering our questions.

    Did you polish the edges of the acrylic? If so, what method did you use?

    I polished mine with 320 - 400 grit sand paper and then a buffing wheel on a drill.
    It doesn't have the same finish you would expect from something like flame polishing and I am afraid that it the way I will have to go.
    I am just not getting the IR lights into the acrylic good enough.

    Thanks again,
    John

  16. Stewart:

    Hi John,

    I asked the suppliers to polish the edges after cutting. I'm not sure what method they used. The surface looked pretty smooth, though it was not completely "flat", probably cut with a bandsaw. If it is laser-cut then you'd probably get a better finish.

    A flat, highly polished surface is more important for FTIR designs, where you need as much internal reflection as possible. If you are using a DSI design (with Endlighten), its probably not that critical.

    Are you sure you've got enough power to your LEDs? If you're using the flex ribbon you'll need at least a 2A at 12V. Check the data sheet for the power consumption per meter.

  17. Industrial LCD:

    I’ve always loved the idea of tangible computer interfaces. I’ve built some plain paper and cardboard box multitouch displays, and some advanced IR projections ones. I found that both were lacking in giving you a nice clear display to work with

  18. LCD Display guy:

    That is probably the coolest mod I've ever seen, home made touch screen table.

    Would any other touch sensors be viable to work with on a home project for example resistive, or is that likely to put oo much strain on the LCD?

  19. LCD Displays:

    I love these home made touch screen projects, so mean seem to be popping up all over the web, it makes you wonder why companies like apple charge so much for these things?!

  20. Corey:

    Where did you get your acrylic from?

  21. Stewart:

    Hi Corey,

    I'm not sure if they still do it but I got my Endlighten from Snapper Frames, a Perth company which makes acrylic display boards.
    http://www.snapperdisplay.com.au/

    The guy I spoke to (Tom?) was very helpful. The distributer in Australia is Plastral, but unless you're buying huge quantities you've got no chance of getting it through them.

Leave a comment