Platypus Header

Platypus Innovation Blog

1 March 2019

Why I'm Giving This Talk (And not a Bot)

This is the talk notes and slides from a talk I gave at a Scotland Internet of Things workshop. My apologies for where the notes are incomplete.

Thank You

Let's start with me.

I'm Daniel Winterstein. I came to Edinburgh in 1999 to study Artificial Intelligence. It's a good city. It's a good subject.

I'm the Founder and CTO at Winterwell, we're a machine-learning consultancy. We make a product called SoDash, which is a social media tool, used by Harrods, Selfridges, Network Rail, and others.

We're pivoting to become Good-Loop, which is an ethical advertising and data-management platform.

Conversational UI - or "bots"


What if we're successful?

Someday, you're going to be sacked by a computer.

Which is convenient, as you'll presumably be able to get your P45 at the same time. The joined up process will be so smooth, it will be a bureaucrats wet dream. With cross-channel conversational follow-through and automated data-entry - It will make grown men weep.

Solution: Citizen's Wage / Basic Income

It's understandable to find this scary.

However, it's a sad reflection on the human condition that a life without hard or menial work scares us Imagine a life of pleasant contented happiness what a scourge on the face of the earth it would truly be... Douglas Adams' writing on the dolphins springs to mind.

Bots should deliver freedom from drudge work

Let's talk a bit about how today's bots go wrong, or make things worse.

Insincerity, Poor Etiquette, and Being Useless

These sins are not inherent to bots. 
Pushy sales-people and useless customer-service is not a new invention. 
But bots allow companies to be insincere, annoying, and useless at scale. 

I tried getting a bot to do the talk.   
Me: Hey Cortana, Could you help with my talk?
Me: Thank you Cortana

So that wasn't a success.

Let's look at another example. There's an anti-pattern emerging here: Bots shouldn't pretend to be human. - brilliant idea: you want to schedule a meeting, you cc their bot, and it arranges the meeting.

Simple focused - Where does it go wrong?

It turns out even this really focused problem is surprisingly hard. They've been going 3 years, and they haven't cracked it yet. Right now, is only part AI; they also have teams of people processing messages. So in order for the bot to pretend to be human, they have people pretending to be bots.
This is not living the dream.

And the kicker: Doodle is a better service, inspite of being much simpler.
Because Doodle isn't confined by pretending to be human. So it can offer a user-interface that fits the problem.

Example emails

“Daniel, open this email for 12 people you should meet :)”

“Re: Making Great Customer Experiences”

If it's a sales message - don't pretend to be friends. If it's a cold email,
don't pretend we're having a conversation.

A simple test for if you want to deploy a chat bot: How would you feel as the recipient?

If the person you're talking to knew the full picture -- what's automated and what the goals are -- what would they think?
Would they be happy to receive fast service? Or annoyed at a pretence at caring?

We need a New Etiquette for Bots

Clippy - was intrusive. Though the modern web has bots that are worse.

Etiqutte and Sincerity are about how we as companies use bots. The solution is not technical - it's caring for our public.

Being Useless -- that is a technical problem.

Fear: That the bot will do more harm than good.

Quality: The bots can't deliver (yet).

Time/Cost: To learn a system, work out the common conversations,
and code them up.

Instead of programming the Bot. What if the bot learns from you?

We want bots to do repetitive tasks. If it's repetitive - there will be lots of examples for
them to learn form.

In general -- machine learning bots is hard, because communication is hard and needs a full
human understanding, and there is never enough data. But that's in general.

If you frame the task -- something specific, structured, and where failing to understand is OK
(bots should know when to stop and hand over gracefully) -- then it becomes possible.

15 February 2019

GPT-2: A new AI with amazing creative writing skills

This language AI is really amazing. Let's cut to the unicorns:

The AI was given a creative writing task: Given a prompt, and asked to continue writing the story. The example reported has been cherry-picked (i.e. the researchers filtered out the duds and are showing off the best output), but is still very very impressive.

In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
Model completion (machine written!)
The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science. 
Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved. 
Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.
Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez. 
Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns. 
While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.” 
Dr. Pérez believes that the unicorns may have originated in Argentina, where the animals were believed to be descendants of a lost race of people who lived there before the arrival of humans in those parts of South America. 
While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.” 
However, Pérez also pointed out that it is likely that the only way of knowing for sure if unicorns are indeed the descendants of a lost alien race is through DNA. “But they seem to be able to communicate in English quite well, which I believe is a sign of evolution, or at least a change in social organization,” said the scientist.

And: the AI was entirely self-taught. It has a structure designed for sequence learning, which is then trained on the text from millions of web pages. It learns to predict the next word when reading. The impressive knowledge of language patterns and story structures shown above is all learned from the data.

As the researchers note, this level of AI has a lot of applications - good and bad. So they are not releasing the full model yet, asking the AI and wider society to consider how we manage this technology.

I read this yesterday. Still processing it with my jaw on the floor.

Naming things is an important part of humanising them, so the researchers have called this system GPT-2. See for a summary of GPT-2 and a link to the technical paper. The neural net architecture is not given, but the paper and partial code suggest it may be surprisingly simple and generic, though large and expensive to train. Spoiler alert: it's not an LSTM - long short-term memory, the neural net architecture which has ruled NLP work for the last few years. It uses an attention-based short term memory in an encode-decode setup called a Transformer. Though attention functions do have some common ground with the memory-gates of an LSTM. So it's evolution not revolution. Except there's a point where evolution becomes revolutionary.

By Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever from

Good-Loop Unit