Intelligent User Interfaces

I’m researching options for incorporating natural language inquiries into business applications. For example, we should be able to ask questions like “how much did we make from XCorp last year”, “how many clients do we have in Ohio”, “is Bob Smith still with XCorp”, “how many orders came in through Atlanta yesterday”, “is truck 192 on schedule”, or “where is Tom Jones today?” (looking for an employee).

Modern language analysis technology facilitates converting those queries into atomic elements which can be used programmatically. We see this in Google Now (“OK Google”), Cortana, Siri, and other wannabee speech/text processors. Microsoft announced the availability of Cognitive Services which make the processing of requests much easier than it used to be. This goes far beyond speech recognition and on to what we do with the speech that has been recognized.

Speech can come in the form of text or voice. Since speech recognition initially just converts spoken words into text, that’s just the front-end part. In the bigger picture, input and output can go through Slack, Skype, SMS, web pages, voice-enabled mobile apps, and other media. Everyone is familiar with phone services these days where we’re prompted with “please tell me in a few words how we can help you today”. Even good old Pick BASIC green screen apps can now accept a request with an INPUT statement, pass the request to an external handler for conversion into a structured query, return the query to a BASIC program for processing, and then return the response back to the character UI user. All of this is done transparently behind the scenes.

With a broad variety of I/O media, consumers/Clients can pass in requests like “let me know when YourCompany has widgets on sale for less than $50” or “what’s my account balance”. Management can store requests like “let me know when Connie is in Boston”, “send a message to Dana if the open receivable ever gets over a million dollars”, “how many contracts does Ken need to process now”, or even “shutdown line 3 and move everyone there to line 2”.

Once the request data is available it can be passed to business modules that handle the requests, in exactly the same way that requests are processed from a web form or a thick client app. In terms of MV BASIC this means a program like LocatePerson(name,location) can be used to pass in a person’s name and return their location, or ChangeProductionLine(fromLine,toLine) can be called. The response can be crafted in a way that makes sense based on the request, like “Tom is in Denver today” or “Ken has 9 contracts to process this morning”. You provide the logic for getting the information and making changes – this is all about exchanging a focused request and data with people using the media they prefer.

There’s a whole-nuther angle to this new world of data processing, and that’s visual processing. Optical Character Recognition is much more sophisticated. A mobile phone can now be used to take a picture of a sign in any language and translate it into another – this can even be done live just by holding the phone up to text. Aside from the translation application, basic scanning can be used to extract data from documents live, eliminating redundant data entry.

Even better, recognition of human faces is now quite good. Cognitive processes can now pull multiple faces from an image, estimate the gender, age, and mood (!) of each individual, and even help to identify an individual from a database. Consider applying this technology to identify customers or as a part of internal security.

We’ve moved from punched cards and greenbar to dumb terminals, then to GUI, to web, to mobile, and now we have no idea where users are. In addition to the old pattern of designing input forms and reports, to get data from users the way We want and presenting data back to them the way We want, we now need to cater to accepting requests and responding to users however and wherever They want.

The average developer is probably going to fall back on history and make assertions about how all of this is unnecessary, that everything is just fine the way it has been for decades, that this is just a fad that will go away. If you’re still trying to sell a character-based application, are you really still thinking like that? A lot of development companies have gone out of business with that attitude about GUI, web, and mobile. Successful companies provide whatever the consumer wants, not just what fits into their comfort zone.

We can make use of all of the functionality discussed here. What’s required of course is a good business case for it and funding for the initial development investment. The concept of investment requires a vision of how the offering will be used, perhaps monetized, how it will be offered to consumers (internal employees or anyone else in the world), and how it will help to generate revenue or save expense. Now that you know what’s possible, try to apply this knowledge to your own business, to your prospect audiences, and current client base. Does this present new opportunities for communication or for using existing data in better ways? Will it help to improve customer retention or to improve brand awareness? I guarantee you’re going to be seeing more of this from other companies that do see the benefits. We’ve already been seeing it for years and only now is it becoming more accessible to the average business. Consider this an opportunity to let you mind wander, to consider new ways of communicating and new ways to make use of data that’s already available, and perhaps new ways to generate revenue by providing what people want in a different package.

Please feel free to contact me about options for enhancing your business software with cognitive abilities.