Moving experiences / Home

« Our Little Legoland is sprawling with life. | Main | Mobile VOIP and the way forward.... »

Wednesday, May 02, 2007

My speech at MEX, The SW Transformer A Vision for a mobile OS

I was asked by Marek Pawlowski from PMM to give a keynote at the MEX conference, a small mobile user experience conference. It is great small conference with key people in mobile user experience.

My brief for the talk was this manifesto:

"Manifesto point #4:

Handsets, applications and services should be more aware of the user's physical environment and adapt to provide the most appropriate interface for sound and visual conditions. We think multi-modal interfaces should be supported on many more mobile devices and can dramatically improve the user experience."


We are at a stage where a new mobile operating system should be created. One that enables the creation of a cheap monoblock enabling best of breed convergence of mobility and computing. I envision an OS that morphs to accommodate both moving use and mobile use. Focused use and multi-tasking. I call it a Transformer OS.

One way to think of this is to think of RSS in terms of comand. Each command or feature in the user experience is wrapped into a meta language of context. This language of context will drive the use cases and the rendering. We do not only separate funtion and presentation we make function and context interdependant.

With such an operating system we would tear down classic application boundaries: like calling, camera, idle, and calendar into a fluid dynamic environment. The operating system is broader than the footprint of the silicon. It extends into the environment and the network.

I think this kind of device and system could be a massive hit in emerging markets where they choose a mobile before they buy a computer. 

Designing passion – A vision towards convergence

Passionate users is the ultimate recognition. This picture is in conflict with common user behavior, users buy form. Not function. The blackberry is function turned form. When you are true to ergonomics, environment and context you can get passionate users. This picture is inspiring (photo of user licking his Blackberry)

I have written and been very exited about convergence for a longtime,  most recently I have seen a future through transformers. A devices, which the user can morph into three or more optimal shapes. A examples are the Nokia N90 or N93.

These solve convergence with mechanics, we get highest common denominator. There are however three key drawbacks. Bulk, Cost and reliability.

These drawbacks are in stark contrast where there is growth in the mobil market, the low end and emerging markets, where PC is not the first or dominant door to Internet.

This talk is a vision for a contextually aware mobile.  A SW transformer.

Nokia claims in their recent advertising that their N95 is what Computers have become. I think this slogan is inspired by this Economist cover from November 23rd 2002 where the 7650 is heralded at the future of computing.  It was such a great cover, I decided to frame it.

---The phone appears as the monolith, or the invisible hand sending its power to alter the future.

The phone being the future of the computer. - These statements are bold statements. They are very inspiring.

I have had the N95 for some time now, I like it, but when I cannot find it, I shout to my family – Anyone seen my phone.  The N95 is what phones have become.

Phones and computer are inherently different.

1. A phone is used with one hand
2. A phone must fit into the pocket
3. It should be possible to use the phone while moving.

It is probably the last point that is the most interesting. Used while moving.

If we do not apply any budget constraint we will always have at least two complementing devices: a phone and a laptop for our computing needs. If we apply budget restrictions the cheaper will prevail, that being the phone, it is cheaper because it does less and is smaller.

The question in my mind is how much most the phone evolve before it becomes a decent substitute for both. 

If we make compromises on the second point we are in trouble, like I think Steve Jobs realized with the iPod. If we make compromises on the third point we are in trouble.

Where we can make compromises is on the first point.  We need to be clever how to make them.

The Blackberry is a phone, but a lot of Blackberry users use a second phone. This is typically for two reasons: 1. Battery does not last for both tasks 2. Form factor is too wide for telephony.

If we want to reach mass market, we need to be cheap. A Mono block is always cheapest.

My opinion is that when we do a device wider than 60mm we are starting to compromise one hand use. The iPhone is 61mm and the Nokia Communicator is 57mm. Length of the device also matters the E90 is 132mm and the iPhone is 115.

The Nokia N800 is 144mm long and 74.9mm wide, hardly pocketably and hardly one-hand operatable. I really struggle with it.

With a thin device and good one-hand use grip. I think one could stretch to about 65mm x 120mm.  One should be no more than 15mm thick.

Styling is crucial. Users buy form, not function.

A computer substitute at this form factor will need some pretty awesome SW to substitute for a full size screen and keyboard of the laptop.

This is why I think we need a SW transformer. A totally new operating system and user experience.

A SW Transformer is a contextually aware multi-modal operating system, with seamless applications and services. An OS larger than what runs on the silicon.

It does not exist. Its closest ancestors are the phone, the Blackberry and the Nokia E90 Communicator and the N800 Internet tablet.  They do not really transform. From the iPhone demos, it is the closest thing, but it is not one hand operatable. Potentially a fatal mistake.

What would this SW Transformer do for its user.

In short it will leverage the context, of the user, the environment, the task to transform and offer the most optimal user experience at any moment.

1. It needs to adapt to moving use.

Browsing while moving is not the same as browsing when sitting

The Web is 800 or 1024 wide and is best perceived in its whole width. However if the user is moving carrying a device in landscape is difficult. By moving and turning the device portrait the ergonomics of moving use improve. This should be sensed from the environment by the OS and signals should be sent to the services redirected to a moving friendly version of the site. Rendered in a single or double columns. Use cases optimized for multi-tasking. I still believe that we will get moving optimized websites. Keyhole browsing, or panning across a big web-page is not a solution for moving data.

E-Mail on the move is different than e-mail when sitting

Blackberry has taught us alot about mobile e-mail use. Again when moving the E-mail tasks would be skewed towards moving use. tasks like read, deleting, archiving, tagging are favored over replying, composing etc. Applications in general have to be designed to support a contextual meta framework which tells the app renderer how the UE should be rendered on the screen in any given situation.

I am reasonably comfortable that one can have the user experience adapt to users habits and different context.  Just look at one of the most popular pages on the Internet, My Yahoo, it adapts to its users usage, without any user input.

Larger fonts, less information when moving
The layout and texts on the screen should adapt to users environment. When moving one needs a bigger font than when the device is stable. The layout should be lighter and the information density should be decreased. The cognitive load is so much greater when moving than when sitting down.

2. Adapt to location

The user experience needs to adapt to the location, the information from all the sensors need to be analyzed and be used to adapt the applications and the user experience. If places could with low cost radio become beacons, like marked rocks are at sea, the mobile users might have a much safer environment to navigate in.

Triggers should have contextual input

User Experience triggers: alarms and notifications should adapt to the environment and location. For example an alarm about replying to a specific e-mail is useless to display if the user is in the car, or walking rapidly. Same applies to an advert. I will not be receptive to book ads if I am rushing to catch a train, when on the train I might want the ads. Understanding context is king in the experience business particularly if you plan to monetize with advertising.

3. The operation system should adapt to multimedia multi-modal communication

The environment should drive ambient communication

If the user is moving and the location is not where calendar assumes the user to be;  a very typical sms: ”I am 15 min late could be automated.”  We know that typing while moving is hard, and if the system can automate this we have an improved user experience. I am a tiptronic man. automatic with manual override. I know nothing of gearboxes, but I am sure Tiptronic was pretty darn hard to engineer, so will the SW transformer be.

The user should be able to seamlessly glide into and glide out of different types of communication.

A good way to describe this is to think of Skype, it is a fusion of calling, presence, IM and video driven out of a dynamic live phonebook. I have noticed that the older generation, the so called Internet Immigrants, or the 50 to 60 year olds, refer to this behavior as Skyping. I find it fascinating. Skype is becoming a verb. Anytime we get cross generational communication we have a killer app at hand.

Adapt to the audio environment.

Many phones have different profiles for different environments, but why could the device not switch it for you automatically. Going from normal to outdoor based on ambient noise should be a nobrainer.

3. Let your community help you make the most of now.

Let the community guide you.

The web, the natural place for communities, organic, without borders aggregated based on common needs and interest. If the communities could extend into the functional priority we could simplify all kinds of tasks. You trust your friends on most things, let them prioritize order of command in your UI.

Mobile moments, or small chunks of empty time, could become significantly more useful if users could have their community propose things that are hot now. Vodafones slogan Make most of Now, is in the right direction, they just need community generated task recommendations fed to the users.

A SW Transformer is more efficient than current experiences.

So to summarize.

One way to think of this kind of Transformer operating system and its application is to think RSS in terms of command. Each command or feature in the user experience is wrapped into a meta language of context. This language of context will drive the use cases and the rendering. We do not only separate function and presentation we make function and context interdependent.

With such an operating system we would tear down classic application boundaries: like calling, camera, idle, and calendar into a fluid dynamic environment. For example if the user is on a call and the lens cap is opened, one could immediately create an video link between the user and the remote party and enable the important ”see what I see” use case. The user experience needs to encourage the switching of modes in conversations. The devices need to support: See, Look, here, where, and touch types of tasks, typical in any real life conversation. It would become vitality rich.

If the phones leverage context, become multi-modal and physically grow a bit,  I think they stand a much better chance of becoming the future of computing as envisioned by the Economist and desired by Nokia and the rest of the mobile industry.

I am one of the ones who believe the emerging markets will leapfrog the western world in mobility, they will drive mobile data. What they need, and what they probably will build are solutions that transform the monoblock into a simple device when moving and a powerful device when still, yet mobile, yet pocketable pocket, nothing revolutionary, just a better phone.

All of these things are rather subtle changes and should not be presented to the users as a revolution. I do believe that the aggregated smartness will however create a new communications revolution.

07:33 PM in Technology | Permalink


Did you see this video? I think you and Intel share the same vision:

What steps do we have to take to start making this a reality?

Posted by: Stefan Constantinescu at May 3, 2007 9:39:44 AM

Niceness. A comment:

Application boundaries are black boxes, in all meanings of the word. People don't have to care about them unless they run into them. Well-defined applications and their boundaries hide an immense amount of complexity in the data structures and their manipulation in the GUI. I understand and agree with your points regarding the boundaries, however there is a pandora's box waiting beyond that point..

There is some stuff by D.Norman (activity-based computing philosophy driven at Apple in 80s, which points out the direction) and "Pliant Research" (research by A. Henderson et al into more 'pliant' computing). They in my opinion define very similar design challenges and in the latter case try to tackle the complexities emerging from it. But, to be honest, I think the problem is that they are not designers but computer and cognitive scientists..

In the direction you set in your proposal, I guess the challenge will be to define the proper mechanisms through which the "computer", its user and their environment can in symbiosis infer the proper context for the situation at hand. Or something like that.

BTW: Thanks for sharing the measurements and the grip issues. Revealing. I've for years wished for someone to make a cylinder-shaped "communication device" which would afford a power-grip (like the one afforded by public transport vehicles' interiors). Good size would be the 35mm film cylindric box, with telescopic transforming capability. And why are there no transformers which become smaller when put into use? Now they all grow larger. ;)

Posted by: ville at May 8, 2007 1:11:27 AM

Very interesting thoughts - must have been a good presentation.

Your next speech as a video on YouTube please.

Just a Joke ;-) or maybe not ?

Posted by: Wolfram Herzog at May 30, 2007 1:01:57 PM


Posted by: Tomota at Apr 10, 2008 10:22:51 AM


Posted by: maomao at Apr 28, 2008 11:38:58 AM

Nice posting, I like the way how you express yourself. Keep up the good work.

Posted by: Authentic Jordan Shoes at Jul 6, 2012 11:34:18 AM

I should have learned this for a long time. Thanks,

Posted by: android casino at Apr 29, 2013 2:57:49 PM