Experience Extension, what is it?
Experience Extension strives to let immobile people experience the outdoor world by letting mobile people act as their eyes, ears and legs. The mobile person walks trough an outdoor location picked by the immobile person, while wearing a camera, microphone and a small laptop. The laptop is connected to the internet, using a high speed mobile card. The mobile person is thus not bound to a certain location and is able to go to any place that the immobile person would like to experience.
The mobile and the immobile person then will communicate using the internet, whereby the immobile person receives audio and video, the mobile person only audio. The mobile person wears the camera on the side of the head, thus creating a first person view for the immobile person watching. This way the immobile person can 'use' the mobile person to go wherever he or she desires, coming close to controlling a real life avatar.
We, a team of students from the HKU (Hogeschool voor de Kunsten Utrecht), have tested the Experience Extension in the field. This document reflects on the process and the implications.
Technical specifications
Hardware and software
The Experience Extension needs two systems to work. One for the immobile and one for the mobile side.
We used the following hardware for the mobile side:
- Acer Aspire one UMPC.
- ERS headset (http://www.dogcamsport.co.uk/ERS-headcam-headset.htm)
- EZ grabber converter (http://www.tvstickshop.com/products/productinfo.asp?catid=2&prodid=22&gclid=CLm10_ikn5cCFQoh3godaz-sDA)
- HSDPA expresscard (Merlin XU870)
- Expresscard to USB converter (http://www.virtuavia.eu/shop/index.php?target=products&product_id=29836&sl=EN)
- Skype (http://www.skype.com)
- TrackerCam (http://www.trackercam.com/TCamWeb/download.htm)
We used the following hardware for the immobile side;
- Standard laptop, but any computer with a decent video card capable of running Skype will do.
- Extern monitor or television screen.- USB telephone, old model.
And the following software needs to be installed on the UMPC;
- Skype (http://www.skype.com)
The technology
We have chosen to use Skype for our communication software, for it's ease of use, the fact that it is free to download and its capability to run on multiple platforms. We chose the Acer Aspire One because of it's size, weight and power. Since there is no UMPC currently available with an expresscardslot, we had to resort to an USB-to-expresscard-converter to connect the expresscard to the Acer.
Since most elderly people are not familiar with the technology we are using, we tried to show as little of the system as possible. For this purpose we chose to give them a 'normal' telephone to communicate with the mobile person. Elderly people like to use familiar objects that they know from the past rather than new technology that is completely new to them. The added advantage of this is that elderly people seem to be used to speak extra clear when using a telephone, thus making the communication easier. We remove the mouse and the keyboard, so the only objects the immobile person sees are a telephone handset and a monitor. The people thus have the experience of being in a normal call with the mobile person, and see the images on the screen as an extra. Even if they don't really understand the connection between the two media, the system still works. Likewise, it is better to substitute the regular computer screen with a normal TV set, to create an even better recognisable situation for most elderly people.
The Voltaic generator solar bag provides the battery for the camera, which is being charged by sunlight when the bag is in use and when stored. When there is sufficient sunlight, the UMPC can also drain extra power from the bag, thus enabling the Experience Extension to go on for a much longer time, depending on the amount of sunlight.
How to start
Since we are using different pieces of software, it is important to boot the system in a specific order. On the immobile side it is sufficient to start up skype, disable any screen savers, automatic energy saving or sleep mode and set skype to automatically answer incoming calls (tools --> options --> calls --> call settings --> show advanced options --> tick 'automatically answer incoming calls').
On the mobile side starting up is a bit more complicated.
Hook up the camera to the laptop via the EZ grabber, and plug in the microphone and the headphones. In the Generator back, find the battery and hold the power button for approximately 5 seconds to turn it on. Then double click multiple times until the light at the 6-9 Volt option turns green. Now plug the power-input of the camera into the battery output plug. Make sure the mini laptop remains indifferent whether the lid is closed or not (you can set this up in the control panel, under the 'Power options' tab), and disable sleep mode and hibernation. Then start up the connection to the internet with the mobile expresscard, and wait till you have a connection. After this, start up 'Trackercam' and wait till you receive and image from the camera (no image? Set Trackercam to the right camera input or check the camera battery). When Trackercam is running, enlarge the image quality (click the button at the lower left corner and click on the 'increase video size' button in the 'display' tab) and subsequently start up skype. Set skype to automatically answer incoming calls and to start the video automatically when calling (tools --> options --> general --> video settings --> tick 'start my video automatically when I am in a call'). Then you're ready to go. Call the other side, and put the laptop in the bag of the mobile person.
Skype automatically adjusts the audio in the standard settings, and since the system is made for contact in a environment with low noise levels, the audio does not work to well in the outdoors. Specially when there are passing cars or there is lots of background noise it's hard to get the microphone volume right. The best option is to use a directional microphone, attached close to the mouth of the mobile person and configure skype to NOT automatically manage the microphone audio. Put the audio setting in skype to a very low input level, and turn up the volume on the immobile side, thus getting the least amount of background noise and the best audible quality.
It could happen that the sound from the immobile person doesn't get through to the mobile person very well. Configuring Skype to manually adjusting the sound input could help here too.
Possible fields of work for Experience Extension
We used the Experience Extension only with immobile elderly people, for we thought that this would be the best place to implement the principle and we regrettably did not have the time to try it with different target audiences. Nevertheless, we chose to include a list with what we consider to be possible fields of work where Experience Extension can be used. These are hypothetical implementations, to be tested in the future.
Immobile people
Elderly
Elderly people can use Experience Extension in combination with family, friends or volunteers who enable the older person to relive old memories about a place they can not get to anymore, or visit places they have never been to and always wanted to see. The cooperation with family or friends works best, since the experience works the most intense both ways; the dialogue between them will be much more lively, for they already share memories and feelings.
Early stages of dementia
For people in early stages of dementia it can be helpful and healthy to use experience Extension to relive old memories. Seeing places where the person spend her or his life can enhance the memories and thus create a comforting feeling. This visual aid is enhanced further by the accompaniment of the voice of a familiar person, preferably a close member of the family.
Long term illnesses
In the case of long term illnesses it is paramount that the patient is ready to be exposed to experience Extension. The treating physician will need to judge whether or not the patient is ready to be offered the experience. People who have a serious illness often don't like to be exposed to the outside world, for it emphasises their limitations and incapability to experience the world themselves. For people who are more stable and ready to cope with the outside world, experience Extension can be an interesting product, but less so then for elderly people.
Disabled
Disabled people, for example people with a permanent illnesses or disease, are often not prepared to experience the outside world. This will be mostly the case for people who have become disabled, rather than being born that way. For the latter group experience Extension can be a good way to help this group.
Prisoners
For people who are long time imprisoned it can also be difficult to connect to the outside world. If this is wanted non the less, experience Extension can be an easy way to do so.
People suffering from Phobia's
When people suffer from phobia's they are exposed to increasingly 'real' substitutions of the object of their phobia. Experience Extension can add an extra step between the exposure to static images and the real object.
Distant relations
People who are living abroad, or are often separated because of work or otherwise are able to show each other 'their world' like it is now through the use of Experience Extension. Here the Experience Extension can be initiated by the persons themselves, and the technology in their own possession to not let it be dependant upon external facilities.
Entertainment
The technology in Experience Extension can be also used for diverse applications in the field of mobile gaming, or different forms of entertainment.
Case study one - St. Carolus; Mr. Apell
In this first field test we worked together with the St. Carolus nursing home in Hilversum. We tried experience Extension for the first time outside, with 92 years old mister Apell. In his younger years, Mr. Apell sold fur coats in the center of Hilversum. He had not seen his old store in a few years, even though it was 'just' a ten minute walk from his current home. He wanted to see how much the center of the village was changed and to visit his old store again. Since there were no family or friends available to be the 'legs, eyes and ears' of Mr. Apell, Michiel from the Experience Extension team chose to walk with the camera.
Michiel was 'geared up' while Mr. Apell sat down in front of the monitor. Michiel started right there at the table where Mr. was sitting. He left the St. Carolus building and walked outside while wearing the camera gear. Mr. Apell behind the monitor immediately started guiding Michiel to the bus stop. In the first couple of minutes Mr. Apell did not really look at the monitor, just using the telephone to communicate with Michiel, and seemingly not completely aware of the fact that the images on the screen where live. After a while mister Apell heard a dog barking through the telephone and pointed at the screen; 'there is that barking dog!'. From that moment on mister Apell his eyes lay fixed on the screen and he started guiding Michiel more actively through the city. He started noticing little things that we wouldn't have noticed, the many empty stores in the city center for example. We ended up in Mr. Apell's old store, where we met an old colleague of his. At a certain point he addressed Michiel as if he was Mr. Apell himself, asking how he was doing, and how his wife was.
It turned out that the option to use an USB phone, instead of a microphone and earphones or external speakers was a good choice. It gave mister Apell the option to concentrate on the screen and slowly figure out how the images related to the audio, without being spooked by a headset or little microphone.
The day was a very nice experience for both mister Apell but also for the team. We learned that the images were best and most understandable for mister Apell when the camera was held still and fixed at one point, and Michiel did not move his head too quickly when walking around.
Case study two - Sarphatihuis; Floor and Anton.
The second field test took place at the Sarphatihuis in Amsterdam, where we met Anton and Floor. Anton is a volunteer working at the Sarphatiehuis, Floor is an older lady who is unable to go outside because she is bound to her wheelchair. Floor told us she wanted to see 'Artis', the zoo in the neighbourhood. So we decided to take Anton to the Zoo and convert him into Floor's eyes and legs by hooking up the camera and microphone. In Artis it took slightly longer to get the complete set-up right, since we had two people working with the system who where not familiar with the options and capabilities of the different parts. It's important to not 'just' hook somebody up to the laptop and send them away, but take some time to adjust the audio levels on both sides, immobile and mobile, and adjust the camera to a convenient position. To do the latter, make the person with the camera walk around a bit, like he or she would normally do. Adjust the panning and the tilt of the camera so that it is adjusted right for the way the person holds his or her head.
When the system was running as smoothly as possible, Floor got an extensive tour of the Zoo. She told Anton what she wanted to see, and he walked through the place according to what she said. Anton was quite talkative, for he did not hear Floor very well in the beginning and tried to fill up the space, but later on we heard from Floor that she tried to talk to him but the audio simply did not get through. Once again, make sure to check the audio connection both ways before you go. Floor understood that the audio belonged to the images very quickly, and seemed to be able to enjoy the experience without being bothered by the technology. Again, using the usb telephone made the connection between Floor and Anton more understandable and easy.
Some things in this Experience Extension did not really work out the way we wanted to. We got the feeling that Anton positioned himself in a slightly superior position from Floor, and too much tried too 'be kind' to her, not being interested in the two way communication that should have appeared, but talking just because there was a connection. Furthermore he did not really understand the system, for he often told Floor what he was seeing, and what was happening, while at the same time she was able to see the image with her own eyes. Experience Extension works best if it is done with two people who know each other, or at least have a genuine interest in each others stories.
Even though not everything went smoothly, a lot of animals passed by the camera. Floor told us in retrospect that, especially when we were close to the animals, it really "felt like she was there herself". She was very happy with the experience and in return, we learned a bit more.
Conclusion
The testing of Experience Extension was an interesting and fun period for the team and we learned that although the principle is quite easy it takes some work to get it right.
When working with older people, it is important to not let them get distracted by the new technologies that are being used, so we chose to use a USB telephone handset instead of a headset and microphone. Next to that it is better to use an old television instead of a computer or laptop screen. By replacing the new technology with electronics that look and feel 'old' but are in fact new products, it is easier for elders to do the Experience Extension.
Like the importance of using the right technology, it is also important to combine the right people in Experience Extension. Both of them must be interested in each other's stories and the mobile person who walks with the camera should be instructed about the technology (preferably by experiencing it themselves first) and should receive some instructions about how to behave.
When walking with the camera, it is important to remember that the person on the other side is seeing the same as you are, so you don't have to tell what you see, and to keep your head still for a while if you want the images to be most clear at the other side.
The person with the camera should act like the immobile person is next to him or her, sharing an experience in a way much more direct than through a telephone conversation, and treating the conversation in a similar way, relaxed and open, sharing an experience rather than telling about it.
Experience Extension turned out to be what we hoped, and more. The people we worked with where really happy with the experience and they participated enthusiastically and actively, giving us a interesting and fun experience as well.
We hope that in the future, much more people will have the chance to experience this!
Copyright: HKU & Telematica
Creators: Michiel Ten Kleij, Jean-michel Molenaar, Ruben Rekker
Written by: Jean-michel Molenaar
