The New York Times has an article titled Nobody Likes Smart Machines.

Dr. Norman, who the story follows, highlights the problem of predictability that comes as a result of delegation through automating tasks to smart machines that still occasionally require human intervention. They give the example of window blinds that change to suit the weather, the side-effects and resultant feeling of loss of control when your computer screen in the office randomly glares annoyingly back at you. It’s an interesting article that really only covers half the picture.

The conclusion the professor comes to is one many people in the design space probably would. That of making the user experience better by making it more predictable, either by removing functionality or improving control. This is where intelligent design really comes into the fore, but not just intelligent design needs to be considered. Intelligent systems that behave and self-adapt can really shine in this area. The big problem is how to do that effectively so as to account for not one person’s vision of how the world works, but that of many peoples. This is where personalization comes in and what I call object profiles, personalised profiles for objects just like that of the seat in an up-market car that adapts to an individual upon entry.

The key here is context. Maintaining the context of the environment surrounding the intelligent system in order to teach it personal traits and habits.

You can’t explain to your car’s navigation system why you dislike its short, efficient route because the scenery is ugly. Your refrigerator may soon know exactly what food it contains, what you’ve already eaten today and what your calorie limit is, but it won’t be capable of an intelligent dialogue about your need for that piece of cheesecake.

What this fails to account for is Machine Learning, narrow AI and that of the power of context in deriving best outcomes.

By context I mean monitoring everything you do and building a current and future picture of the world based upon past history.

Say you own one of those new fangled fridges that automatically orders the food to be home delivered if you run out of anything. What happens if you decide the next week that you want to go on holiday? How can the fridge know you won’t be needing it’s services until say the day you get home?

By keeping a record of whatever context you find yourself in. For example the online travel site your entering you holiday bookings into… simply give your intelligent appliances access to that knowledge - that’s how.

Big Brother 2020 style.

There has been a lot of focus lately on attention aggregation. Twitter/Facebook activity feeds are a good example of unstructured partial-attention aggregation, yet that only gets you so far. Real augmentation will come when the systems we interact with can change and adapt to our whim. And that my friends, is the power of context tracking in the digital age. Maintaining a dynamic real-time context profile our surrounding can then use - all based on a list of likely probabilities - probabilities then used to predict what augmentation intelligent systems can assist with next.

Information overload is already here. In a world of never ending pop-up boxes, call marketers, Facebook requests, mail you just throw in the bin and systems asking if you want to allow this or disallow that - systems that learn what we want will be needed to manage the never ending pile of relationship management with others and with our surroundings. When your in giving a presentation somewhere you really don’t want your phone ringing or IM window popping up.

Another advantage of these kinds of systems is the automated feedback loop AFL they make possible. This means device interfaces can be improved through historical collection of context, and actual usage patterns based upon factors that include demographics. New market devices may even emerge as a result of how people used these devices rather than how a designer thinks it should be used. Imagine devices that build an interface reputation for ease of use or productivity in different areas. How much easier would shopping for a new device online become? Increasingly we’re purchasing items without ever seeing or using them and more often than not we’re less satisfied with the outcome. An AFL can help minimize that problem.

What’s needed is the ability to delegate intelligently those monotonous tasks and the user experience that was once a burden becomes a boon.

Now you may think this is years away, but it’s happening now. Google augments personalised search based upon search history and your location aka your context. Individual data silos are already using these techniques to serve advertisements. I read recently that Marissa Mayer stated they use the keywords 411GOOG users enter for machine learning and better translation and search results. Games have long personalised themselves to users, even now to the point of detecting when the gamer is getting bored in order to maximize the user experience (and revenue).

It’s cross-silo and ubiquitous networking of dynamic devices that will bring the next wave of personalisation. I look forward to what my future digital self has in store for me there.

On that note, I’m thinking this needs a name… so building on the great work of VRM, I bring you ARM.

Automated Relationship Management. CRM was already taken and everyone can always use another ARM or a leg.

I’ve been reading Backus among other things, attempting to grasp the differences between first-order predicate logic, function-level, and variable-level combinatory logic. One area that has stood out for me in reading Backus was the notion that function-level may be more efficient to reason about than the more user-friendly variable-level.

[rant] In some circles I understand function-level is also known as pointfree (or pointless) while the most common variable-level usages appears to be lambda calculus based. Talk about my brain having a lexical meltdown. Reading Backus is bad enough… but trying to understand just what pure functional programming is ain’t easy either when there appears to be no consensus on exactly what that means. [/rant]

Backus goes on after his efficiency claim to discuss the notion of using primitive functions to compose functional forms and type hierarchies from composing those functional forms in succession. This immediately piqued my interest because in the back of my mind it had been nagging me why someone could possibly continue along the function-level line of thought when the mainstream had moved on to the more popular get-things-done lambda calculus way of doing things.

Then something struck me. Something I’ve thought about with regard to intelligence and that of the realm of referential transparency and the way in which we all operate in life. I began to draw parallels between the notion of type hierarchies, atoms, elements, molecules, proteins, enzymes, cells and organisms. The composition of primitive functions into these functional forms. The hierarchies these then build based upon the physical properties (quantum mechanics?) of those primitive functions and the unique ways in which functions may apply to others creating yet more uniquely propertied functional forms. The complexity of properties of these functions meaning some will be more inherently stable than others (potentially the types). That these unique properties could themselves be derived from evolutionary encapsulation. In other words; turtles all the way down.

Months back I set out to model my own theory of encapsulation. Initially known as my Principal of Rational Dynamic Complexity. Later I prefixed Robust. Then after finding that difficult to remember it became the Law of Encapsulation with each of those elements becimong principles. My The Law of Encapsulation states that; for any complex system to evolve in a constrained environment(eg. the earth) it must be dynamic yet rational in order to be robust.

So while others may prefer the variable world of the lambda calculus to get things done today, tomorrow is a new day, one I believe may yet see the rise of function-level. Only your likely to never know it because it will come naturally to you. In other words… an interface abstraction. Just as our bodies are for our brains.

Jon Udell discusses the idea that technical mastery requires social innovation. While I agree in general I don’t necessarily agree with his final statement that major innovation would require more social than technical. I just don’t believe you can separate the two cleanly yet. It’s still all about how you compose the data with the view(s) and technically I still see a lot of problems in information architecture today that needs solving.

Recent talk by Dave Winer and Tim Bray on what happens to your data and records when you die are the classic example. Data is spattered everywhere, yet there’s a technical solution to that, one that requires a social element as well. I say this as I’m attempting to create my own environment for historical data and innovation. So while the social hurdles are first and foremost in my mind - my user-facing API is defined by that - actually architecting what I consider important elements to achieve my goals, is trying technically. Sure if you have a lot of man hours of a big company it may well be simple to implement but conceptualizing how exactly to implement an entire environment with innovative paradigms, requires a lot of forethought. Usually by one BDFL slaving over that while guiding others to help implement that. eg. Tim Berners-Lee, Guido van Rossum. There’s an excellent talk Fred Brooks gave at OOSPLA07 that talks about innovation I’d highly recommend listening to.

Where Jon asks whether operations are beyond the capabilities of mainstream users I begin to envision a much simpler and more direct approach to how people interact with objects in a system. I always recall a mainstream user testing a web site I’d designed. The number one interaction she responded to - one that guided her into learning new capabilities - happen to be the direct object interactions. Popup’s on hovering menu’s, self-explanatory animated transitions on acting etc, all of these were at the point of object contact. It taught me how important it is to see and learn how to interact with each component and how those can effect others. More importantly how the system integrates and functions to get tasks done. Jon’s screencasts, the CoScripter example, they exemplify this.

I remember my first steps playing with the Self programming language desktop GUI. The ability to inspect objects was fantastic even though the overall user experience was awkward. I think we need more of this interaction if users are to learn new capabilities. Interfaces simply must be object accessible.

Applications today just aren’t built with this in mind and I think the problem is one of default functionality. By default 99% of systems don’t guide people as they use them. There is so little by way of best practices and the advantages they afford because more often than not it’s educated developers doing the implementation.

I think the only way to solve this is to give users the tools do the implementation and customization themselves by lowering the barrier to entry through an object accessible interface. Community driven applications modifiable at the object level comes to mind. Wikipedia is a good example at making information more accessible to the masses, Twine a level up. Why not do the same for application development? The Linden Labs guys also talked at OOPSLA07 that is worth listening to. A large number of scripts were developed by people with little previous programming skill because they could grasp basic constraints to the interface and architecture. If we can create an interface to do this and get tasks done while the hard technical aspects are hidden; great! But technically I don’t believe we’re there yet on a distributed level nor social. The two problems are just too similar.

It’s really one distributed problem over two different mediums.

I couldn’t leave without linking to an interactive application example The Neuron.

A quote by Sep Kamvar repeated by Greg Linden sparked some thoughts on personalization that I’ve managed to boil down to a simple old age adage:

It’s what people do, not what they say that matters when it comes to personalization.

This is why I believe contextual and personalized attention data is the most important personal information one can aggregate. I see this happening with services like Google’s Search History and Twitter/Facebook’s activity feeds at the very edges of this personalization movement.

It’s an area I believe the semantic web needs to work on in order to fill out the middle and become successfully adopted. Standardizing actions on the side of the developer and openly(with privacy in mind) recording these actions with dynamic ontologies. Associating the data with these actions. I say dynamic because meaning often changes over time and as Sep highlighted; checking every box and labeling someone as having a general interest doesn’t necessarily return useful results. It’s the history that matters (why dynamic ontologies) and personal information has to be personal to be effective.

Twine and other sites like it are interesting to me in capturing the data they do for a number reasons, however they all seem to be one shop shows, only capturing some attention from their own narrow usage patterns. This is where I see the real power to the people; aggregating attention cross-application and intelligently customizing usage for user experience. VRM is one area that comes to mind to manage all of this.

The way I see it, right now the only actions we have on users visiting sites are; entry, internal actions and exit. I think this is why only search history and location are returning results. I’m more likely to visit local business and services more if the ads are relevant to my location and search contexts. The closer something is, the more personal it often feels and closer to instant gratification you become.

Contexts include the who what where when how and why of a persons action. Who they are and their history of performing actions, what they’re looking for in relation to, where they are while seeking that. When they are seeking the what aka the time/day, how they’re attempting to obtain their goal and why being their motivation for doing so.

Who is the history - The records of a persons actions.
What is they seek - The goal aka resource aka search term
Where they are - Their location (IP address / wifi mesh / GPS location)
When they’re seeking - Is it after work? Middle of the day? Just after school?
How they’re seeking - What service or applications are they utilizing? Voice, Phone or Web?
Why they’re motivated - Is it research or are they in the mood to purchase something? History is telling.

If I’ve missed any, please leave a comment.

I believe this is why results always come back to context as Sep highlights with his statement that “recent searches are much more important.” It’s where they are and what people are doing right now.

Having said that, another area I see yet to be fully exploited is using time-varying attention information to predict what users may be motivated to do next or soon and return results(not necessarily all of them) that reflect that before the user even knows they want them. I call this the consumer cycle as most people are just patterns stuck in a cycle that evolves over time. I hear Microsoft Research are doing this with Visual Studio by tracking actions and customizing the interface. I know Werner Vogel’s of Amazon in a recent InfoQ video displayed a diagram Jeff Bezos drew that highlights their business growth cycle and the feedback loop. People are growth cycles too and if you can figure out those patterns then and you can pander to their every whim even before they know they have them. Targeted suggestion can really be a powerful form of persuasion. Friends do it to other friends all the time. This is why it’s important to also know a person’s friends and in what contexts they pay attention to them. A friend might buy a new phone and the person ask them about that purchase. Great time to personalize an experience and display the information from other sources(friends/known sites) the user can use.

What I find interesting is that all of this is related to a more fine-grain approach to handling users personal data in applications. The decision is then one of how dynamic to make an application such that collaboration and convenience are enhanced and not sacrificed in the overall user experience. One person might require a completely different experience to the next. So how best then to share those disparate world views? Links with history and session data that form historical views on data comes to mind. I’ve not seen much work on this regard but I haven’t really look that hard. Still, I believe this will be vital for the platforms of the future. Avi’s Seaside has been one of the few attempts I do know about, if you know any others I’d like to hear about them.

Now all of this dynamic data collection requires architecture to support it. I’m of the belief that a deterministic approach with history and session data stored for context recall may be the best approach to personalization across future contexts. Most of my research is focused in this area, so it better be.

With that, I’m off to revise my Combinatory Logic skills.

I believe we’re entering the Functional Reactive Programming paradigm shift. At least I hope we are.

Behaviors that update views on data only when that data changes.

Named resources ala URIs that expose those views, thereby exposing that data through APIs ala REST, ala CouchDB.

Views that can then be dynamically compiled into pages using transclusion (the embedding of externally identified, content) and functional reactive programming client behaviors ala Flapjax or more conventionally Redux. Views that can be programmed live using FRP techniques ala SuperGlue.

Flat tables ala CouchDB, Googles BigTable, Hadoop? and Vertica, storing large datasets viewed with client-driven(pay?) behavioral views that form customised APIs. Behaviour Orientated Architecture. BOA.

Forget about maintaining the APIs and worry about maintaining the data and the ability to view that data. ie. computational resources to generate and distribute those views, as well as uptime and security permissions for that data. HTTP Digest, view access permissions? [Insert Identity Layer Here] Better yet, automated content-centric networking, encrypted packet creation of those views as named data that can be pushed into the cloud.

Next we take it to the architecture level and synchronous reactive programming. Then we get functional reactive AI systems.

 New fMRI research from Vanderbilt neuroscientists gives insight into the limits of multi-tasking in the brain.

The results revealed that the central bottleneck was caused by the inability of the lateral frontal and prefrontal cortex, and also the superior frontal cortex, to process the two tasks at once. Both areas have been shown in previous experiments to play a critical role in cognitive control.

"We determined these brain regions responded to tasks irrespective of the senses involved, they were engaged in selecting the appropriate response, and, most importantly, they showed 'queing' of neural activity–the neural response to the second task was postponed until the response to the first was completed," Dux said.

"Neural activity seemed to be delayed for the second task when the two tasks were presented nearly simultaneously – within 300 milliseconds of each other," Marois said. "If individuals have a second or more between tasks, we did not see this delay.

Based on memory work I've read I'd postulate that neurons required for multiple tasks, block synapse unrelated to the current task based on temporal information it receives and later predicts. That the "queueing" shown is a simple battery of requested actions on different synapse from other circuits in the brain. And that priority is given next to the most active synapse.

I need to think about this further when my head is clearer.