tag:blogger.com,1999:blog-32899725395478132622024-03-14T06:24:47.860+00:00Platypus InnovationDaniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.comBlogger48125tag:blogger.com,1999:blog-3289972539547813262.post-34650256587679962062020-01-06T16:51:00.001+00:002021-11-08T14:21:22.901+00:00My Ideas - Please take them!<br />
<blockquote class="tr_bq">
Two roads diverged in a yellow wood,<br />
And sorry I could not travel both<br />
And be one traveller, long I stood<br />
And looked down one as far as I could<br />
To where it bent in the undergrowth. </blockquote>
<blockquote class="tr_bq">
<i>The Road Not Taken, by Robert Frost</i></blockquote>
<div>
<br /></div>
<div>
Here are some company / project ideas I've had. Since I cannot develop these ideas properly, let me share them in the hope that they will inspire someone else. Free for use!<br />
<span style="font-size: xx-small;">No guarantee is offered as to the quality of ideas, or their right to exist in an enlightened society.</span></div>
<div>
<br /></div>
<div>
I have a long backlog of these (I have a habit of emailing stray thoughts to myself). </div><div><br /></div><span><a name='more'></a></span><div><br /></div><div>Let's have some green ideas for COP26:</div><div><br /></div><div><b>Random idea #127: The Eurovision solution to business conferences</b></div><div>Remote conferences allow people to take part without travel, reducing carbon emissions. But let's face it: they're not great. The in-person networking element is nowhere near as good. This is where Eurovision provides a blueprint for success. </div><div>If you can visit Eurovision in person - that's obviously best. Most people can't. We don't watch at home on their own though. We get together and have Eurovision parties - it's remote, but locally together.</div><div>This can work for business conferences too! E.g. suppose there's an event in London which a few people from Edinburgh are interested in. The event could organise local hubs where they can meet up to watch the talks and participate.</div><div><br /></div><span></span><span><!--more--></span><div><br /></div><div>Kicking off with a random 3: <i>electric cars</i>, a <i>group movie app</i>, and a <i>robot politician</i>.</div>
<h4>
Random idea #81: Shared batteries for electric cars</h4>
<div>
Climate change needs to be tackled; you know this. OK: Car batteries. They take a while to charge. Even with a fast charger, the charging time makes for a frustrating stop -- and limits the throughput of the charging station.</div>
<div>
<br /></div>
<div>
Car batteries are <i>heavy</i>, e.g. one ton. You could not swap them in and out by hand! But a charging station could have a lift and trolley for that.</div>
<div>
<br /></div>
<div>
Checking Wikipedia, this is not a new idea: The concept of an exchangeable battery service was first proposed as early as 1896!</div>
<div>
<br /></div>
<div>
<div>
Benefits: Instant charge up. More scaleable infrastructure.<br />
<br /></div>
</div>
<h4>
Random Idea #82: A poll app for group picking of what movie or TV show to watch</h4>
<div>
If you and a friend want to pick a movie -- it's a pleasure. It may take a while to find one you both want to see, but the conversation is a pleasant one, touching on movies you loved and hated. However, as the number of people in the group grows, the process quickly becomes unwieldy, especially if done by email or chat. Part of the group gets all excited about one film, only to find that someone has already seen it.</div>
<div>
<br /></div>
<div>
This could be fixed with an app.</div>
<div>
<br /></div>
<div>
Basic features:</div>
<div>
- You can make a poll as easily as with Doodle, and people put in movie/TV ideas, and say yes/ok/no.</div>
<div>
<br /></div>
<div>
Excellent features would be:</div>
<div>
- Easy input and memory of your past watch history and want-to-watch list.</div>
<div>
- Links to trailers & reviews</div>
<div>
<br /></div>
<div>
<h4>
Random Idea #83: Stand an open-source (& open data) robot candidate for election as MP</h4>
<div>
On the grounds of transparency, predictability, and better than so many of the humans who do stand.</div>
<div>
<br /></div>
<div>
Also, for interesting debates.</div>
<div>
<br /></div>
<div>
Legally fine -- it'd just need a human spokesperson to be the mouthpiece for the robot party. That person could dress Daft Punk style, or less weird -- just with a conspicuous earpiece connecting them to the AI.</div>
<div>
<br /></div>
<div>
Architecture?</div>
<div>
A deep neural network (e.g. GPT2) for speech writing. You wouldn't get coherent consistent policies from it, but then, you don't get that from Boris Johnson either.</div>
<div>
Decision making?</div>
<div>
Could be GPT2 style question-answering -- i.e. its a language model being used to complete the answer to a question. This is flexible, but it lacks explicit semantics (i.e. it wouldn't know what its saying).<br />
Train it on a set of good politicians (Mandela, Asquith, Gandhi, Churchill, etc) + neutral news (Wikipedia, BBC), and the outputs should be -- well bad, but comparable to many modern politicians.</div>
<div>
<br /></div>
<div>
To stand initially against the Prime Minister and Lord Buckethead.</div>
<div>
<br /></div>
</div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-28135872684881855522019-11-18T12:39:00.000+00:002019-11-18T13:17:19.632+00:00How Nick Bostrom's "Fable of the Dragon-Tyrant" is an elitist folly<a href="https://1.bp.blogspot.com/-H5yFGjEKMH0/XdKN1iXnabI/AAAAAAAAooI/goguqIrExLcUjXpH6i5V6rWmad3nF6_uQCNcBGAsYHQ/s1600/snapshot_2019-11-18_30392.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="600" data-original-width="1029" height="186" src="https://1.bp.blogspot.com/-H5yFGjEKMH0/XdKN1iXnabI/AAAAAAAAooI/goguqIrExLcUjXpH6i5V6rWmad3nF6_uQCNcBGAsYHQ/s320/snapshot_2019-11-18_30392.png" width="320" /></a>I recently met Nick Bostrom's <a href="https://nickbostrom.com/fable/dragon.html">Fable of the Dragon-Tyrant</a> via a rather charming <a href="https://www.youtube.com/watch?v=cZYNADOHhVY">animation</a> by the enigmatically named CGP Grey. The moral of the fable -- and in case anyone misses it, Bostrom spells it out in his notes -- is that death is a terrible thing, and we should devote much more of society's output into researching an end to death i.e. longevity medicine.<br />
<br />
This is initially appealing. I don't want to die; you don't want to die; nobody wants to die, and nobody wants their loved ones to die.<br />
<br />
However if you look at Bostrom's fable as a proposal for change -- the folly becomes clear.<br />
<br />
When a rich person dies before the age of 100, we can see that as a failure of medicine, and perhaps more R&D could help, and yes that would be good. But when a poor person dies before the age of 50 - that is more often a failure of Economics. So the greater problem by far is the economic and social challenges we face. Bostrom's Fable is a call for more R&D money and more focus on the needs of the rich, which is implicitly at the cost of the needs of the poor. The idea that R&D is not doing enough to look after the needs of the rich is, to put it simply, horseshit. What we need is more efforts directed toward the needs of <i>everyone,</i> and particularly economic and political change.<br />
<br />
That is not to knock the huge value of R&D, and I speak as someone who is in the R&D field. But actually greater change could be delivered <i>today</i> and with greater certainty, through simply changing our emphasis to <b>care more, for more people</b>.<br />
<br />
I am also concerned that longevity, especially longevity for the elite prioritised above a just society, might be a very bad thing. It could well pose an existential risk to freedom and justice.<br />
<br />
The limit on the greedy and power-hungry has always been that, eventually, they too shall pass. To quote Death from Bill & Ted: "Whether you're a King or a street-sweeper, sooner or later, you dance with the Grim Reaper." (<a href="https://www.youtube.com/watch?v=N9YpnklYhxY">clip</a>).<br />
<br />
The transience of our lives is often cited as a reason for not being materialistic - epitomised in the phrase "You can't take it with you when you go". But what happens if you <i>don't </i>go? If you could live much longer -- or even think you could, given enough money? This might curb generous impulses, in favour of hoarding wealth for your own much longer and more costly needs.<br />
<blockquote class="tr_bq">
<b>Our <i>mortality</i> may be the source of our <i>morality</i>.</b></blockquote>
I don't want to die. But I'm not devoting my energy and surplus money into R&D towards an eternal Zuckerbergian Elite. In the end, the Fable of the Dragon-Tyrant is peddling visions of eternal life, in return for your money and obedience. That is and always has been a lousy deal.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-49696128285105328432019-07-22T10:47:00.001+01:002019-07-22T10:49:57.645+01:00Fast Company innovation event says: Slow down!<blockquote class="tr_bq">"The real danger is not that computers will begin to think like men, but that men will begin to think like computers."<br />
- Sydney J Harris</blockquote><br />
Last week was Fast Company's inaugural European Innovation Festival -- launching an old world version of it's annual New York event. The theme was super-human technology: Innovation from technologies like AI and the increasingly blurry line between the real and digital worlds.<br />
<br />
<blockquote style="float:right; border:solid 1px #770f00; background:#770f00; color:white; width:165px; height:125px !important; border-radius:50%; font-size:200% margin:auto; padding-top:40px;"><b><center><span style='font-size:200%'>40,000</span> people in the US already have computer chips in their head.</center></b></blockquote><br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-SW9OKKkoH_g/XTWBLhwJekI/AAAAAAAAllw/TfivODJp1fc2-yjxqXJNZe2oHi1JQX4VQCK4BGAYYCw/s1600/yuval0.jpeg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="264" src="https://3.bp.blogspot.com/-SW9OKKkoH_g/XTWBLhwJekI/AAAAAAAAllw/TfivODJp1fc2-yjxqXJNZe2oHi1JQX4VQCK4BGAYYCw/s320/yuval0.jpeg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Yuval Harari predicts some changes. Photo by Fast Company</td></tr>
</tbody></table>If there was one consistent message from the dozen talks and panel sessions, it was a call to slow down the pace of disruption. Not because these technologies are bad, but they are so powerful they will reshape society -- and we need to consider our end goals.<br />
<br />
Keynote speaker Yuval Harari gave a great talk, where he confidently predicts the end of Homo Sapiens as we know it, as AI, gene editing, and neural implant technologies come of age.[1] He is perhaps the only person who could describe bionic re-engineering of human bodies as a "conservative approach"! On the grounds that it would be a smaller change than a shift to computer-based life.<br />
<div><br />
</div>He was not making Terminator-style doomsday predictions: Harari sees the advances of technology as morally open. Evolution is a process, and current-day Homo Sapiens is not the end of it. Nor would the end of Homo Sapiens mean the end of humanity.<br />
<br />
The worry is that AI and bio-hacking driven by raw competition has the potential to "downgrade humanity". E.g. by strengthening discipline and manipulation, at the cost of caring and creativity. We need global cooperation and wisdom to use technology for the benefit of humanity - now more than ever.<br />
<br />
The recent development of a de facto arms race in AI between the US and China particularly concerned Harari, as the harsh us-or-them logic of an arms race could pull us towards the worst outcomes.<br />
<br />
Putting some real-world flesh on Harari's vision of upgraded humans, neuroscientists Moran Cerf and Riccardo Sabatini gave an excellent session on "hacking the human mind" -- the potential for computers to link directly with the brain.<br />
<br />
For many people, their online life is as real as their offline one, and their phone is already part of them. Cerf and Sabatini predict that this integration will start to be physical. The line is increasingly blurred, with for example, augmented reality, the extra digital senses of a fit-bit, and most sci-fi of all: direct brain-computer interaction.<br />
<br />
40,000 people in the US already have computer chips in their head, as part of how their brains work.<br />
<br />
These are mostly simple chips which release electric pulses to help alleviate epilepsy and other brain diseases. But more complex brain-computer integrations are already in use: Cochlea implants provide hearing to those who have lost it. A microphone sends electrical signals into the hearing region of the brain. Wonderfully, the brain learns to hear the new electrical impulses, growing new neural connections to interpret them.<br />
<br />
What about a chip to allow the brain to make a Wikipedia or Google search, and hear the results internally. How far off is that?<br />
<br />
Professor Cerf talked of his work in recording dreams, using electrodes implanted during brain surgery. This is real technology (if still crude in its outputs -- don't expect a video), being used in post-surgery therapy. The conference also demoed an easy-to-use baseball-cap device which can measure simple moods from brain signals.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-KAFNNJAgQEk/XTWDY0Wy70I/AAAAAAAAlmI/NYv0AvBnGHQ-EJNN0p1FbpEoICrPp57cwCK4BGAYYCw/s1600/66436386_2386874498039053_4965719378067718144_n.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://1.bp.blogspot.com/-KAFNNJAgQEk/XTWDY0Wy70I/AAAAAAAAlmI/NYv0AvBnGHQ-EJNN0p1FbpEoICrPp57cwCK4BGAYYCw/s200/66436386_2386874498039053_4965719378067718144_n.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Gucci CEO Marco Bizzarri and poet Shanelle Gabriel.</td></tr>
</tbody></table>It was a glamourous and diverse conference. The event was hosted by Gucci at their HQ in Milan. Gucci's CEO Marco Bizzarri attended, a dapper giant. Film-star and rock-star Jared Leto added to the glitz.[3] Alongside the rock-star persona, there is also an impressive tech investor -- his portfolio includes Slack, Uber, Snapchat, Spotify, and Airbnb. The attendees were wonderfully diverse. Partly marking how AI has become mainstream, and partly due to Fast Company's design to create a stimulating cross-sector event, there were entrepreneurs, investors, artists, and HR people (amongst others). The event opened with a short music set[4], and a performance poet created live synopsis poems to end each session.<br />
<br />
The recurrent theme was looking at how technology affects us. Ben Schwerin from Snapchat spoke well on how tech companies should take more responsibility for the mental effect of systems, and the importance of designing for user benefit, not just good user stats. He also called for more government involvement:<br />
<blockquote class="tr_bq">"[Social media] platforms have gotten so big and so powerful that it probably is not the healthiest thing to put that power in the hands of a few people who are motivated by running a for-profit business."</blockquote>Although the technology discussed at the Fast Company event was new, the questions it raises are old ones. What do we value? What is important about being Human? and, How do we organise for the general good?<br />
<br />
The questions may be as old as humanity -- but the emphasis has changed. Super-human technology has moved these questions from a matter of individual philosophy, to one for public policy.<br />
<br />
[1] More on <a href="https://www.fastcompany.com/90373620/yuval-noah-harari-humans-are-on-the-verge-of-merging-with-machines?utm_source=winterstein&utm_medium=good-loop">Yuval Noah Harari's talk</a> <br />
[2] More on neuroscientists <a href="https://www.fastcompany.com/90374460/the-ethics-of-hacking-humanity-from-bionics-to-brain-chips?utm_source=winterstein&utm_medium=good-loop">Cerf and Sabatini</a><br />
[3] A toke of <a href="https://www.youtube.com/watch?v=8TtY8jEtCD4">Jared Leto</a><br />
<div><br />
</div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-DErBV1VoSMU/XTWCUwJrHmI/AAAAAAAAll8/LCM2qQxZrDMSYVGoDxa7JYAhjH82USauQCK4BGAYYCw/s1600/DSC_0298.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="81" src="https://2.bp.blogspot.com/-DErBV1VoSMU/XTWCUwJrHmI/AAAAAAAAll8/LCM2qQxZrDMSYVGoDxa7JYAhjH82USauQCK4BGAYYCw/s640/DSC_0298.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A panorama of nice people, and me.</td></tr>
</tbody></table><div><br />
</div>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-12732260248449586512019-03-01T13:25:00.003+00:002019-03-01T13:25:28.042+00:00Why I'm Giving This Talk (And not a Bot)<i>This is the talk notes and slides from a talk I gave at a <a href="https://www.meetup.com/iotScotland/">Scotland Internet of Things workshop</a>. My apologies for where the notes are incomplete.</i><br />
<br />
Hello<br />
Thank You<br />
<br />
Let's start with me.<br />
<br />
I'm Daniel Winterstein. I came to Edinburgh in 1999 to study Artificial Intelligence. It's a good city. It's a good subject.<br />
<br />
I'm the Founder and CTO at Winterwell, we're a machine-learning consultancy. We make a product called SoDash, which is a social media tool, used by Harrods, Selfridges, Network Rail, and others.<br />
<br />
We're pivoting to become Good-Loop, which is an ethical advertising and data-management platform.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-O8t3b-qBxQY/XHkujP8UY3I/AAAAAAAAhuI/ple1oLHmi0sdDJeAMtAI-_45c7KH-_4-wCLcBGAs/s1600/snapshot_2019-03-01_13127.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="683" height="246" src="https://1.bp.blogspot.com/-O8t3b-qBxQY/XHkujP8UY3I/AAAAAAAAhuI/ple1oLHmi0sdDJeAMtAI-_45c7KH-_4-wCLcBGAs/s320/snapshot_2019-03-01_13127.png" width="320" /></a></div>
<br />
<br />
<h3>
Conversational UI - or "bots"</h3>
<br />
Why?<br />
<br />
<br />
What if we're successful?<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-fVYLS-kYyyk/XHkujyccuPI/AAAAAAAAhu8/RFYmBvKUxME6qm5CE5t5YtZKYb_rNR7WACEwYBhgL/s1600/snapshot_2019-03-01_17351.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="683" height="246" src="https://3.bp.blogspot.com/-fVYLS-kYyyk/XHkujyccuPI/AAAAAAAAhu8/RFYmBvKUxME6qm5CE5t5YtZKYb_rNR7WACEwYBhgL/s320/snapshot_2019-03-01_17351.png" width="320" /></a></div>
<br />
Someday, you're going to be sacked by a computer.<br />
<br />
Which is convenient, as you'll presumably be able to get your P45 at the same time. The joined up process will be so smooth, it will be a bureaucrats wet dream. With cross-channel conversational follow-through and automated data-entry - It will make grown men weep.<br />
<br />
<h3>
Solution: Citizen's Wage / Basic Income</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-qV3CPCzucyY/XHkujeKLB9I/AAAAAAAAhvA/uEUMyvGcmHEFhjeeyxobr8Qra8hPIesYACEwYBhgL/s1600/snapshot_2019-03-01_14132.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://4.bp.blogspot.com/-qV3CPCzucyY/XHkujeKLB9I/AAAAAAAAhvA/uEUMyvGcmHEFhjeeyxobr8Qra8hPIesYACEwYBhgL/s320/snapshot_2019-03-01_14132.png" width="320" /></a></div>
<br />
It's understandable to find this scary.<br />
<br />
However, it's a sad reflection on the human condition that a life without hard or menial work scares us Imagine a life of pleasant contented happiness what a scourge on the face of the earth it would truly be... Douglas Adams' writing on the dolphins springs to mind.<br />
<br />
Bots should deliver freedom from drudge work<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-xv4h5SMa8Fg/XHkuiQrZF-I/AAAAAAAAhuw/Kg74kdMB4toYThnv2704BeA_koxGIjEiACEwYBhgL/s1600/snapshot_2019-03-01_10753.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://3.bp.blogspot.com/-xv4h5SMa8Fg/XHkuiQrZF-I/AAAAAAAAhuw/Kg74kdMB4toYThnv2704BeA_koxGIjEiACEwYBhgL/s320/snapshot_2019-03-01_10753.png" width="320" /></a></div>
<br />
<br />
Let's talk a bit about how today's bots go wrong, or make things worse.<br />
<h3>
Insincerity, Poor Etiquette, and Being Useless</h3>
These sins are not inherent to bots. <br />
Pushy sales-people and useless customer-service is not a new invention. <br />
But bots allow companies to be insincere, annoying, and useless at scale. <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-ZwmMbHl-Vxo/XHkujqNCLNI/AAAAAAAAhuw/1nW0YO54Zx46nbxZZazyuUXUvJJr7RjQgCEwYBhgL/s1600/snapshot_2019-03-01_15886.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="177" src="https://3.bp.blogspot.com/-ZwmMbHl-Vxo/XHkujqNCLNI/AAAAAAAAhuw/1nW0YO54Zx46nbxZZazyuUXUvJJr7RjQgCEwYBhgL/s200/snapshot_2019-03-01_15886.png" width="200" /></a><a href="https://3.bp.blogspot.com/-wpQp5_0aM6k/XHkulW_pRdI/AAAAAAAAhvA/tgnG-qeejbQ71y_05NgBpYIaa16Jpr3MQCEwYBhgL/s1600/snapshot_2019-03-01_8762.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="683" height="244" src="https://3.bp.blogspot.com/-wpQp5_0aM6k/XHkulW_pRdI/AAAAAAAAhvA/tgnG-qeejbQ71y_05NgBpYIaa16Jpr3MQCEwYBhgL/s320/snapshot_2019-03-01_8762.png" width="320" /></a></div>
<br />
<br />
I tried getting a bot to do the talk. <br />
Me: Hey Cortana, Could you help with my talk?<br />
Cortana: <opens a="" browser="" for="" help="" my="" search="" talk=""></opens><br />
Me: Thank you Cortana<br />
Cortana: <opens a="" browser="" for="" search="" thank="" you=""></opens><br />
<br />
So that wasn't a success.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-ZllMV3xr4rY/XHkukG5wg0I/AAAAAAAAhvA/a3N2vCmDv64h3YZNLre8jCktm7RS6L0tgCEwYBhgL/s1600/snapshot_2019-03-01_19276.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://4.bp.blogspot.com/-ZllMV3xr4rY/XHkukG5wg0I/AAAAAAAAhvA/a3N2vCmDv64h3YZNLre8jCktm7RS6L0tgCEwYBhgL/s320/snapshot_2019-03-01_19276.png" width="320" /></a></div>
<br />
Let's look at another example. There's an anti-pattern emerging here: Bots shouldn't pretend to be human.<br />
<br />
x.ai - brilliant idea: you want to schedule a meeting, you cc their bot, and it arranges the meeting.<br />
<br />
Simple focused - Where does it go wrong?<br />
<br />
It turns out even this really focused problem is surprisingly hard. They've been going 3 years, and they haven't cracked it yet. Right now, x.ai is only <i>part</i> AI; they also have teams of people processing messages. So in order for the bot to pretend to be human, they have people pretending to be bots.<br />
This is not living the dream.<br />
<br />
And the kicker: <i>Doodle is a better service, inspite of being much simpler.</i><br />
Because Doodle isn't confined by pretending to be human. So it can offer a user-interface that fits the problem.<br />
<br />
<br />
Example emails<br />
<br />
“Daniel, open this email for 12 people you should meet :)”<br />
spam<br />
<br />
“Re: Making Great Customer Experiences”<br />
spam<br />
<br />
If it's a sales message - don't pretend to be friends. If it's a cold email,<br />
don't pretend we're having a conversation.<br />
<br />
A simple test for if you want to deploy a chat bot: How would you feel as the recipient?<br />
<br />
If the person you're talking to knew the full picture -- what's automated and what the goals are -- what would they think?<br />
Would they be happy to receive fast service? Or annoyed at a pretence at caring?<br />
<br />
<h3>
We need a New Etiquette for Bots</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-w5z1vSB6CfM/XHkukQPJiTI/AAAAAAAAhu8/-BKB4KoIHegIs_pJdAKVaxL7Rxbhx9RngCEwYBhgL/s1600/snapshot_2019-03-01_21538.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://4.bp.blogspot.com/-w5z1vSB6CfM/XHkukQPJiTI/AAAAAAAAhu8/-BKB4KoIHegIs_pJdAKVaxL7Rxbhx9RngCEwYBhgL/s320/snapshot_2019-03-01_21538.png" width="320" /></a></div>
<br />
<br />
Clippy - was intrusive. Though the modern web has bots that are worse.<br />
<br />
<br />
Etiqutte and Sincerity are about how we as companies use bots. The solution is not technical - it's caring for our public.<br />
<br />
Being Useless -- that is a technical problem.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-JydVIco5FtY/XHkuk5GymjI/AAAAAAAAhu4/ukLefZq0amcbPdVbvsV4NMmcRzKtwX0NQCEwYBhgL/s1600/snapshot_2019-03-01_4074.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://2.bp.blogspot.com/-JydVIco5FtY/XHkuk5GymjI/AAAAAAAAhu4/ukLefZq0amcbPdVbvsV4NMmcRzKtwX0NQCEwYBhgL/s320/snapshot_2019-03-01_4074.png" width="320" /></a></div>
<br />
<br />
Fear: That the bot will do more harm than good.<br />
<br />
Quality: The bots can't deliver (yet).<br />
<br />
Time/Cost: To learn a system, work out the common conversations,<br />
and code them up.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-5pAShZ90yfg/XHkulLJVq-I/AAAAAAAAhu8/fefhk8F7n1UHLXHS-S0FjdsQv2GAKyq6ACEwYBhgL/s1600/snapshot_2019-03-01_7690.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://4.bp.blogspot.com/-5pAShZ90yfg/XHkulLJVq-I/AAAAAAAAhu8/fefhk8F7n1UHLXHS-S0FjdsQv2GAKyq6ACEwYBhgL/s320/snapshot_2019-03-01_7690.png" width="320" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-6R4Z3KAyLss/XHkuidumMjI/AAAAAAAAhu4/huWSITyXRsAyI6xlVHqCXNRi_dUNdGqNACEwYBhgL/s1600/snapshot_2019-03-01_13096.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="526" data-original-width="589" height="285" src="https://4.bp.blogspot.com/-6R4Z3KAyLss/XHkuidumMjI/AAAAAAAAhu4/huWSITyXRsAyI6xlVHqCXNRi_dUNdGqNACEwYBhgL/s320/snapshot_2019-03-01_13096.png" width="320" /></a></div>
<br />
<br />
Instead of programming the Bot. What if the bot learns from you?<br />
<br />
We want bots to do repetitive tasks. If it's repetitive - there will be lots of examples for<br />
them to learn form.<br />
<br />
In general -- machine learning bots is hard, because communication is hard and needs a full<br />
human understanding, and there is never enough data. But that's in general.<br />
<br />
If you frame the task -- something specific, structured, and where failing to understand is OK<br />
(bots should know when to stop and hand over gracefully) -- then it becomes possible.<br />
<br />Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com1tag:blogger.com,1999:blog-3289972539547813262.post-10868729741368259032019-02-15T08:14:00.001+00:002019-02-22T10:43:51.134+00:00GPT-2: A new AI with amazing creative writing skillsThis language AI is really amazing. Let's cut to the unicorns:<br />
<br />
The AI was given a creative writing task: Given a prompt, and asked to continue writing the story. The example reported has been cherry-picked (i.e. the researchers filtered out the duds and are showing off the best output), but is still very very impressive.<br />
<br />
Prompt:<br />
<blockquote class="tr_bq">
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.</blockquote>
Model completion (machine written!)<br />
<blockquote class="tr_bq">
The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science. </blockquote>
<blockquote class="tr_bq">
Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved. </blockquote>
<blockquote class="tr_bq">
Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.<br />
Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez. </blockquote>
<blockquote class="tr_bq">
Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns. </blockquote>
<blockquote class="tr_bq">
While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.” </blockquote>
<blockquote class="tr_bq">
Dr. Pérez believes that the unicorns may have originated in Argentina, where the animals were believed to be descendants of a lost race of people who lived there before the arrival of humans in those parts of South America. </blockquote>
<blockquote class="tr_bq">
While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.” </blockquote>
<blockquote class="tr_bq">
However, Pérez also pointed out that it is likely that the only way of knowing for sure if unicorns are indeed the descendants of a lost alien race is through DNA. “But they seem to be able to communicate in English quite well, which I believe is a sign of evolution, or at least a change in social organization,” said the scientist.</blockquote>
<a href="https://3.bp.blogspot.com/-od7V6iGzmjo/XGZ0OUx1ZUI/AAAAAAAAhcc/6bFy6g7UmigLX-lhPO4TjobmmbZ-kU0bACLcBGAs/s1600/Atmosphere-Fairy-Tales-Unicorn-Landscape-Mystical-1737897.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="720" data-original-width="510" height="320" src="https://3.bp.blogspot.com/-od7V6iGzmjo/XGZ0OUx1ZUI/AAAAAAAAhcc/6bFy6g7UmigLX-lhPO4TjobmmbZ-kU0bACLcBGAs/s320/Atmosphere-Fairy-Tales-Unicorn-Landscape-Mystical-1737897.jpg" width="226" /></a>Wow.<br />
<br />
And: the AI was entirely self-taught. It has a structure designed for sequence learning, which is then trained on the text from millions of web pages. It learns to predict the next word when reading. The impressive knowledge of language patterns and story structures shown above is all learned from the data.<br />
<br />
As the researchers note, this level of AI has a lot of applications - good and bad. So they are not releasing the full model yet, asking the AI and wider society to consider how we manage this technology.<br />
<br />
I read this yesterday. Still processing it with my jaw on the floor.<br />
<br />
Naming things is an important part of humanising them, so the researchers have called this system GPT-2. See <a href="https://blog.openai.com/better-language-models/">https://blog.openai.com/better-language-models/</a> for a summary of GPT-2 and a link to the technical paper. The neural net architecture is not given, but the paper and partial code suggest it may be surprisingly simple and generic, though large and expensive to train. Spoiler alert: it's <i>not</i> an LSTM - long short-term memory, the neural net architecture which has ruled NLP work for the last few years. It uses an attention-based short term memory in an encode-decode setup called a Transformer. Though attention functions do have some common ground with the memory-gates of an LSTM. So it's evolution not revolution. Except there's a point where evolution becomes revolutionary.<br />
<br />
By Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever from OpenAI.comDaniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-40758504446432588732018-11-21T16:15:00.005+00:002018-11-21T16:15:47.015+00:00Redundant Onion or Vital Old Pipe?I was talking the other day with a fellow tech entrepreneur (Matthew Davis from Wittin[1]) about the complexities of some public sector processes -- the red tape which creates extra work, and worse, which acts as an impedance that stops people using the services. It is tempting to blame the bureaucrats - to think government process is complex because it is made so by bureaucrats who like complexity. But this is not the case (well, with a couple of exceptions - immigration policy perhaps). Most government process is made by civil servants who are doing their best, who have the right intentions, and who want to serve the public.<br />
<br />
That is not to say that things cannot be improved.<br />
Undoubtedly there are places where it can be made better, where the people making the process do not have all the answers - how could they? - or where the process is out of date.<br />
<br />
The first challenge is understanding: before we can improve we must understand.<br />
<br />
Business processes are rarely well documented.<br />
Sometimes the steps are documented -- often there is documentation for the public, the internal steps less so but sometimes. However <i>the rational behind the process</i>, the reasons that drove the process to be the way it is -- that crucial information is almost never documented.<br />
<br />
It is all too easy for a process to become cumbersome, where every step was done for a good reason at the time. The road to over-complexity is paved with good intentions.<br />
<br />
I am reminded of a story told by Primo Levi in <i>The Periodic Table</i>, the story of the redundant onion in the oil.<br />
<br />
Levi was an industrial chemist by trade, and worked at one stage in varnish production. In a textbook on the topic he had found the strange advice, when making varnish, to introduce two slices of onion into the principal ingredient of linseed oil. No comment was given on the purpose of this curious additive. Levi spoke about it with Signor Giacomasso Olindo, his predecessor and teacher:<br />
<blockquote class="tr_bq">
Smiling benevolently behind his thick white moustache, he explained to me that in actual fact, when he was young and boiled the oil personally, thermometers had not yet come into use: one judged the temperature of the batch by observing the smoke, or spitting into it, or, more efficiently, immersing a slice of onion in the oil on the point of a skewer; when the onion began to fry, the boiling was finished. Evidently, with the passing of the years, what had been a crude measuring operation had lost its significance and was transformed into a mysterious and magical practice. [2]</blockquote>
Processes accumulate, like sediment into stone.<br />
<br />
<i>But</i> the counterpoint to the Redundant Onion, is the Vital Old Pipe.<br />
<br />
This story is my own from a few years ago, when I was living on the top floor of a block of tenement flats in Edinburgh. At some unknown time earlier, the flat below had redone their kitchen. Whilst clearing out the old kitchen, the builders found an antiquated old pipe for which they could see no purpose. It was in the way of what they wanted to do. The block of flats was 200 years old, and they summized that this pipe had no purpose, that it was some obsolete piece of junk, and so they simply cut it out. Nothing went wrong, validating their decision.<br />
<br />
Or rather, nothing went wrong <i>until later</i>, when our boiler broke. We arranged for a plumber to install a new boiler. First, the plumber flushed out the water from the radiators - an easy task as there is a drainage pipe in old Edinburgh tenement flat buildings to do exactly this. This specialised drain pipe is rarely used. This was the drainage pipe which had been cut sometime before in the flat below.<br />
<br />
We and the other residents in the block learned all about these drainage pipes when water started coming through the walls of the flat below us. And through the ceiling of the flat below them. Finally it <i>took out the ceiling</i> of the shop on the ground floor.<br />
<br />
What had seemed to be a useless old pipe, was in fact an important part of the system, that was just poorly known and not properly understood.<br />
<br />
Therein lies the conundrum. In reforming government process, we start out being unable to tell which bits are Redundant Onions, and can be safely be reformed, and which bits are Vital Old Pipes that still serve a purpose, even though it is not instantly obvious. What's lacking is <i>a manual</i>, or more precisely, the equivalent of good code documentation.[3] If our block of flats had come with a manual, the builders could have checked that, and would have known what the pipe was, and why it was there. Business processes need to come with such a manual, something that says this is how we do it, and this is why we do it -- because that understanding is the key to being able to change it. So that understanding is empowering.<br />
<br />
That manual lets you reorganize and evolve your processes. It's also really useful for normal operations, e.g. when a new staff member comes in. They need to know how to do things. They usually learn that from their colleagues - but this is a slow and patchy way to manage knowledge. The learning is gradual and piecemeal, and they must hold it all in their head.<br />
<br />
But if there's a manual, they can look it up as and when they need it. <br />
And if the process changes, they know where to look for up-to-date answers.<br />
<br />
More than that, the manual is key to improving process.<br />
<br />
Here at Good-Loop we use an internal wiki which anyone in the company can edit, and it documents our processes: how to book a holiday, how to onboard a new colleague (and also, since we are an adtech company, technical material like how to spin up a new server). This has been invaluable in both avoiding mistakes and efficient working.<br />
<br />
We're a small company. Does a wiki approach (by which I mean, having a shared central knowledge-base, maintained by the team) extend to large organisations? I believe it can. Wikipedia is a truly inspiring example here - a high-quality knowledge-base built by the many, for the many. Let's do something similar for government processes.<br />
<br />
<br />
[1] Wittin - <a href="https://www.wittin.co.uk/">https://www.wittin.co.uk</a><br />
[3] <a href="https://platypusinnovation.blogspot.com/2014/08/documentation-for-educated-stranger.html">Documentation for the Educated Stranger </a><br />
[2] Primo Levi - <a href="https://granta.com/chromium/">Opening extract of <i>Chromium</i></a> from <a href="https://www.goodreads.com/book/show/427282.The_Periodic_Table">The Periodic Table</a>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-34739340986117502452018-11-21T15:55:00.002+00:002018-11-21T15:55:24.497+00:00How could it be otherwise? Talking with Matthew from WittinI was talking the other day with Matthew Davis from <a href="https://www.wittin.co.uk/">Wittin</a>, a Dundee-based start-up. He's doing some work with a council, and we were talking about government processes -- how government works, and doesn't always work, and often government isn't really sure itself how it works. By government here I mean the range of public sector institutions that make up the modern UK state. It starts with Westminster and its departments, but includes the regional parliaments of Holyrood / Cardiff / London, the councils, the big public sector services like the NHS, and the many smaller organisations who together carry out the sprawling complex business of running a modern country.<br />
<br />
Whilst the modern UK government (in the broad sense described above; I don't mean the Prime Minister) sincerely wants to be open -- Matt made a good point that it is, and always has been, the preserve of a very select group of people. To take the case of a council he's working with, the council has been in existence in its current form for over 200 years, and during that time the people who ran the council -- sitting in it's chambers as elected officials, or in its offices as civil servants -- have been largely from a limited demographic. If we look at the people who have been making policy, and extend this to include the newspaper journalists and others who get involved in policy making -- Even extending to this wider group, it's still quite a narrow set of people. Let's characterise that as educated middle-class busy-bodies (amongst which I proudly count myself). Even amongst this group, the vast majority are not involved in making policy or carrying out policy. Running the country is left to the few who are willing to do it. How could it be otherwise? <br />
<br />
<i>How</i> could it be otherwise? A small change in emphasis, but a big change in direction. <br />
<br />
The answer that Matt is exploring is around transparency, and using software to make policy more accessible to the many. Ideally it should be easy for people to contribute, and in a way that those contributions are genuinely useful to policy makers.<br />
<br />
It's a big challenge, Matt has two prongs to his approach. One is around more open data, and this is something that the British government is overall very good at. The public sector has been implementing more open data for some years. It's not easy, because government IT is not easy, and there are also privacy issues that limit how data can be opened up. Year by year, progress is being made.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-79552794339416121932018-10-24T17:06:00.001+01:002018-10-24T17:06:51.133+01:00EgBot Maths Q&A Dataset<div class="c5">
<span class="c4">(If you want to skip ahead to our dataset visit our</span><span class="c4"><a class="c1" href="https://www.google.com/url?q=https://zenodo.org/communities/egbot&sa=D&ust=1540400534589000"> </a></span><span class="c2 c4"><a class="c1" href="https://www.google.com/url?q=https://zenodo.org/communities/egbot&sa=D&ust=1540400534589000">Zenodo community page</a></span><span class="c4"> and for our code there is our</span><span class="c4"><a class="c1" href="https://www.google.com/url?q=https://github.com/good-loop/egbot&sa=D&ust=1540400534590000"> </a></span><span class="c2 c4"><a class="c1" href="https://www.google.com/url?q=https://github.com/good-loop/egbot&sa=D&ust=1540400534590000">Github repository</a></span><span class="c0 c4">)</span></div>
<div class="c5">
<span class="c0 c4"><br /></span></div>
<div class="c5">
<span class="c0 c4"><i>Guest post by Irina Preda from the EgBot project, a collaboration between the University of Dundee and Good-Loop</i></span></div>
<div class="c5">
<span class="c0 c4"><i><br /></i></span></div>
<div class="c3">
<span class="c0"></span></div>
<div class="c5">
<span class="c0">Our research focuses on the role of examples in mathematical discourse. One of the ways in which we examine this is through the construction of an autonomous example generator. The generator would be able to contribute to mathematical discussions by proposing appropriate examples, in a socially-appropriate way. This example generator would be a first step towards a machine-enhanced mathematics, where humans and machines collaborate to produce novel mathematics research.</span></div>
<div class="c3">
<span class="c0"></span></div>
<div class="c5">
To be able to build a model of how humans use examples, we would need a large dataset of examples and the context in which they are provided. Unfortunately such a dataset does not exist, but there is a lot of potential for generating one. First we must find a source that would allow us to collect all of this data. Online collaborative mathematics platforms (such as<a class="c1" href="https://www.google.com/url?q=https://polymathprojects.org/&sa=D&ust=1540400534591000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://polymathprojects.org/&sa=D&ust=1540400534591000">Polymath</a></span> and<a class="c1" href="https://www.google.com/url?q=https://mathoverflow.net/&sa=D&ust=1540400534591000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://mathoverflow.net/&sa=D&ust=1540400534592000">MathOverflow</a></span><span class="c0">) provide a remarkable opportunity to explore mathematical discourse and elements of the mathematical process. They are also high quality data-rich sources that provide the perfect resources to analyse discourse as well as train models.</span></div>
<div class="c3">
<span class="c0"></span></div>
<div class="c5">
<br /></div>
<div class="c5">
As StackExchange is a platform with an abundant amount of data (MathStackExchange has approximately<a class="c1" href="https://www.google.com/url?q=https://math.stackexchange.com/questions&sa=D&ust=1540400534592000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://math.stackexchange.com/questions&sa=D&ust=1540400534593000">1 million questions</a></span>) and a well-documented API, we decided to use the<a class="c1" href="https://www.google.com/url?q=https://api.stackexchange.com/&sa=D&ust=1540400534593000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://api.stackexchange.com/&sa=D&ust=1540400534593000">StackExchange API</a></span> to extract their Math Q&A data and thus generate our dataset. From the start of the project we focused on making our work accessible, which is why we decided to publish our code openly on Github, as well as to publish our dataset online. We considered this to be very important as a good dataset is an extremely valuable resource for the data science and machine learning community and can provide a significant boost to further research efforts. Making a dataset truly accessible requires for it to be well-constructed and documented such as to be easily understood, but also needs to exist on a platform that allows for it to be easily found. So we turned to<a class="c1" href="https://www.google.com/url?q=https://zenodo.org/&sa=D&ust=1540400534593000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://zenodo.org/&sa=D&ust=1540400534594000">Zenodo</a></span>, which is an open access research publication framework. It assigns all publicly available uploads a DOI (Digital Object Identifier) to make the data uniquely citeable and also supports harvesting of all content via well-known open web standards (<span class="c2"><a class="c1" href="https://www.google.com/url?q=https://en.wikipedia.org/wiki/Open_Archives_Initiative_Protocol_for_Metadata_Harvesting&sa=D&ust=1540400534594000">OAI-PMH protocol</a></span>) which makes the meta-data easily searchable. <span class="c0">The only limitation we found with using Zenodo is that it doesn’t allow the uploading of json files using the online upload tool, however this was easily fixed by archiving the files (which conveniently reduces the size of the download as well).</span></div>
<div class="c3">
<span class="c0"></span></div>
<div class="c5">
<br /></div>
<div class="c5">
Data collection was only the first stage in our project, there is also data analysis, building a conversational model and an interactive web application. Our intention to use deep learning to build the conversational model meant that this first stage was very important, as deep learning neural networks require an immense amount of data to train. Thankfully the approach we sketched above was successful and we were able to harvest 6GB worth of mathematical question-answering and discourse data. If you would like to take a look at our dataset visit our<a class="c1" href="https://www.google.com/url?q=https://zenodo.org/communities/egbot&sa=D&ust=1540400534595000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://zenodo.org/communities/egbot&sa=D&ust=1540400534595000">Zenodo community page</a></span> and for our code there is our<a class="c1" href="https://www.google.com/url?q=https://github.com/good-loop/egbot&sa=D&ust=1540400534595000"> </a><span class="c2"><a class="c1" href="https://www.google.com/url?q=https://github.com/good-loop/egbot&sa=D&ust=1540400534596000">Github repository</a></span><span class="c0">.</span></div>
<div class="c3">
<span class="c0"></span></div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-52343618850775233322018-08-17T11:27:00.001+01:002018-08-17T11:27:16.443+01:00Kurt Vonnegut on Being ObsoleteThinking on AI and the effects it will have on unemployment -- Kurt Vonnegut, the great American sci-fi writer, wrote back in 1965 on how we should adjust our attitude to unemployment:<div>
<br /></div>
<blockquote class="tr_bq">
"Americans have long been taught to hate all people who will not or cannot work, to hate even themselves for that. We can thank the vanquished frontier for that piece of common-sense cruelty. </blockquote>
<blockquote class="tr_bq">
The time is coming, if it isn’t here now, when it will no longer be common sense. It will simply be cruel."</blockquote>
<br />From the excellent, funny, angry, and hopeful book: <i>God Bless You, Mr. Rosewater</i>.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-52472852964121733502018-07-31T10:57:00.002+01:002018-07-31T13:05:02.767+01:00My company is over 50% female - Why I'm not happy about that<div style="color: #222222; font-family: arial, sans-serif;">
<br class="Apple-interchange-newline" />
There are far more men in the tech sector than women. So it is surely an achievement to be celebrated that my company, Good-Loop, is over half (55%) female. Draw up humble speeches about our greatness, put us on posters and give us an award!</div>
<div style="color: #222222; font-family: arial, sans-serif;">
<br /></div>
<div style="color: #222222; font-family: arial, sans-serif;">
Well yes and no.</div>
<div style="color: #222222; font-family: arial, sans-serif;">
<br /></div>
<div style="color: #222222; font-family: arial, sans-serif;">
Firstly, that 55% depends on how you count it. If we adjust for full-time vs part-time, the balance would be 1/3 female to 2/3 male -- still a lot more balanced than the sector average, but less worthy of a poster. Stats can often be chosen to suit the message ("cherry picked" to use the technical term). Statistics should be compiled by someone independent, with no axe in the race or horse to grind. Or produced to a fixed formula, which will no doubt be inappropriate in many cases, but has the benefit of being consistent and comprehensible.</div>
<div style="color: #222222; font-family: arial, sans-serif;">
<br /></div>
<div style="color: #222222; font-family: arial, sans-serif;">
<div>
Moreover, 50/50 is not the goal of diversity. The gender % is a symptom, and should be treated as such -- it is not the disease itself. If you visit a doctor with a fever, you expect to be tested for the underlying cause, not put in a fridge. We should treat gender % statistics, and indeed, all statistics, in the same way. </div>
<div>
<br /></div>
<div>
All things being equal we expect 50/50 male/female, 2% black, 6% red-heads etc.[1] If there's an imbalance, then prejudice at work might be a problem - but there are other possible causes. For example, when I collect my child from nursery today, most of the staff are female -- but I doubt the nursery is sexist; the cause is at the society level, where more men aren't encouraged towards care roles. Also, nice stats could merely mask problems, and treating the stats would certainly mask problems. For example, if a workplace is biased against mothers who hit a glass-ceiling, then promoting non-mothers won't fix that. </div>
<div>
<br /></div>
<div>
<span style="background-color: white;">Statistics (and more generally: data) is a powerful lens for examining the world. But like any lens, it distorts. The useful thing about a statistic is it simplifies the world and allows for easy comparisons. This is also the dangerous thing -- the world is rarely simple. The data scientist must be alert to the wider picture, the complex causes behind the circumstantial summary. And we all use statistics, so we are all data scientists now.</span></div>
<br class="m_3754762887426080196gmail-Apple-interchange-newline" />
<span style="background-color: white;">The goal of diversity is a workplace free from prejudice, where all kinds of people can achieve their potential and contribute. I believe that <i>is</i> our company, and <i>that</i> is worth being proud of.</span></div>
<div style="color: #222222; font-family: arial, sans-serif;">
<br /></div>
<div style="color: #222222; font-family: arial, sans-serif;">
<hr style="font-size: small;" />
<span style="background-color: white; font-size: x-small;">[1] Approximate UK statistics. <a data-saferedirecturl="https://www.google.com/url?hl=en-GB&q=https://en.wikipedia.org/wiki/Black_British&source=gmail&ust=1533117080921000&usg=AFQjCNF68mA92LHqqeI4XSD-XO74D5qRoA" href="https://en.wikipedia.org/wiki/Black_British" style="color: #1155cc;" target="_blank">https://en.<wbr></wbr>wikipedia.org/wiki/Black_<wbr></wbr>British</a>, <a data-saferedirecturl="https://www.google.com/url?hl=en-GB&q=https://www.telegraph.co.uk/news/celebritynews/9788039/Bullied-and-worse-for-being-ginger.html&source=gmail&ust=1533117080921000&usg=AFQjCNFK5HtshhcQ8hzKVqG18zmuXiPUBQ" href="https://www.telegraph.co.uk/news/celebritynews/9788039/Bullied-and-worse-for-being-ginger.html" style="color: #1155cc;" target="_blank">https://www.telegraph.co.uk/<wbr></wbr>news/celebritynews/9788039/<wbr></wbr>Bullied-and-worse-for-being-<wbr></wbr>ginger.html</a></span></div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-66124696821386895372018-05-28T11:52:00.001+01:002018-05-28T11:52:17.340+01:00Splitting out a React js project for code reuse - The easy way<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
We like React, and we run a few projects on it: <a href="http://portal.good-loop.com/">Good-Loop's portal</a>, <a href="http://app.sogive.org/">SoGive</a>, and some internal tools. So we wanted to reuse code between these projects.</div>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
One way would be to make separate npm packages. But this is a painful solution: The compilation setup is painful, and you lose source-code maps (so you end up debugging from babelled code).</div>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
Instead, we found a simple solution using symlinks:</div>
<ol style="background-color: white; border: 0px; box-sizing: inherit; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; list-style-image: initial; list-style-position: initial; margin: 0px 0px 1em 30px; padding: 0px; vertical-align: baseline;">
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px 0px 0.5em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">Just sym-link the "packages" in, using <code style="background-color: #eff0f1; border: 0px; box-sizing: inherit; font-family: Consolas, Menlo, Monaco, "Lucida Console", "Liberation Mono", "DejaVu Sans Mono", "Bitstream Vera Sans Mono", "Courier New", monospace, sans-serif; font-size: 13px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 1px 5px; vertical-align: baseline; white-space: pre-wrap;">ln -s</code> (for Linux or Mac; I don't know what the equivalent is for Windows).</li>
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; word-wrap: break-word;">Set webpack's resolve.symlinks property to false. In our setup that meant editing webpack.config.js to have <code style="background-color: #eff0f1; border: 0px; box-sizing: inherit; font-family: Consolas, Menlo, Monaco, "Lucida Console", "Liberation Mono", "DejaVu Sans Mono", "Bitstream Vera Sans Mono", "Courier New", monospace, sans-serif; font-size: 13px; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 1px 5px; vertical-align: baseline; white-space: pre-wrap;">resolve: {symlinks: false }}</code>.</li>
</ol>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
Then your code is split out -- but as far as the build process is concerned, or your code editor, nothing has changed.</div>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
You can also unit test with tests in each "package".</div>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
E.g. your project folders might end up looking something like this:</div>
<ul style="background-color: white; border: 0px; box-sizing: inherit; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; list-style-image: initial; list-style-position: initial; margin: 0px 0px 1em 30px; padding: 0px; vertical-align: baseline;">
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px 0px 0.5em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">myapp<ul style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; list-style: disc; margin: 0px 0px 0px 30px; padding: 0.5em 0px 0px; vertical-align: baseline;">
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px 0px 0.5em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">src<ul style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; list-style: disc; margin: 0px 0px 0px 30px; padding: 0.5em 0px 0px; vertical-align: baseline;">
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; word-wrap: break-word;">subpackage: a symlink to subpackage/src</li>
</ul>
</li>
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; word-wrap: break-word;">webpack.config.js, package.json, etc.</li>
</ul>
</li>
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; word-wrap: break-word;">subpackage<ul style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; list-style: disc; margin: 0px 0px 0px 30px; padding: 0.5em 0px 0px; vertical-align: baseline;">
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px 0px 0.5em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">src</li>
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px 0px 0.5em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">test: unit tests for src files</li>
<li style="border: 0px; box-sizing: inherit; font-family: inherit; font-stretch: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline; word-wrap: break-word;">webpack.config.js for the unit tests</li>
</ul>
</li>
</ul>
<div style="background-color: white; border: 0px; box-sizing: inherit; clear: both; color: #242729; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; font-size: 15px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; padding: 0px; vertical-align: baseline;">
This is easy and it works.</div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-31504433786732509702018-04-12T15:24:00.000+01:002018-04-12T15:24:18.287+01:00Invest in Good-LoopThis is an unusual post for me. After 10 years as an entrepreneur, I am <a href="https://seedrs.com/good-loop">asking for money</a>.<br />
<br />
Amy and I set out in 2016 with a mission: to make something positive and good in the often sordid world of online advertising. Good-Loop is an ad network, but different from any other:<br />
<br />
<ul>
<li>50% of the money goes to charity. To be clear: that's not 50% of profits, or 50% of our commission. That's 50% of the total revenue. The rest is split, approximately 30-40% to the publisher and 10-20% to us.</li>
<li>The user is in control: of whether we show them an ad, and what we do with their data. This is the "ad-choice" alternative to adblockers, which avoids annoying ads whilst still supporting publishers to create.</li>
</ul>
<br />
We believe that if you treat people with respect and aim to do good, then you can create both a positive impact (money raised for charity), and a better space for advertising (happy viewers are more likely to listen than annoyed ones).<br />
<br />
The results so far back this up: because we engage users in a positive way, Good-Loop adverts are watched rather than skipped (benchmarked against YouTube and Facebook video ads), and generate more actual customer activity (measured with one of our clients, Lifecake by Canon). Unilever's chief marketing officer, Keith Weed, describes our approach as a "win-win!" (namedropping, because, wow, not only are we working Unilever, who have one of the world's largest ad budgets and an ethical ethos, but their CMO tweeted about it).<br />
<br />
We are not a charity: we plan to make profits and deliver a return. Our aim is to be both profitable and good for the community. I think this investment is a good bet: we have a reasonable chance of delivering an unreasonable return.<br />
<br />
Warning: Any investment in a start-up is risky - do not invest money unless you're OK losing it. I cannot provide financial advice here.<br />
<div>
<br /></div>
If you want to be part of what we're doing - now you can! Over April 2018, we are running an open investment round: <a href="https://seedrs.com/good-loop">https://seedrs.com/good-loop</a><br />
<br />
This is open to anyone. The bulk of the round will likely be established investors who we've pitched to. However if you know and like what we're doing, then you can invest, regardless of cheque size -- the investments so far range from £15 to £30,000.<br />
<br />
I hope you'll join us.<br />
If you have any questions, feel free to contact me: daniel@sodash.comDaniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-51344076376308248412018-01-10T14:12:00.000+00:002018-02-08T11:09:30.366+00:00Upgrading from Jetty 8 to 9I'm upgrading from Jetty 8.1 to Jetty 9.4. Here are the changes I've encountered so far:<br />
<br />
Several classes renamed / replaced:<br />
<br />
* SelectChannelConnector has become ServerConnector<br />
* ServletContextHandler has become ServletHandler<br />
<br />
Server.setThreadPool() has gone.<br />
<br />
Lots of jar changes.<br />
<br />
The JSON class (org.eclipse.jetty.util.ajax.JSON) has moved out into a separate Maven project: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-util-ajax, which gives you jetty-util-ajax.jar<br />
<br />
We also needed the Maven packages: jetty-server, jetty-servlet<br />
<br />
<br />
You must upgrade to servlet-api 3.1 (so change anything which pulls in the servlet-api-3.0.jar or earlier).<br />
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-28785549736551701492017-11-24T06:55:00.001+00:002017-11-24T06:55:27.605+00:00UK Chancellor says: Driverless cars by 2021 The AI revolution is getting real: Chancellor Philip Hammond aims for the UK to be using driverless cars in 4 years time.[1] He warns people to: "retrain and re-skill [in what?] so they can take up the many, many new jobs that this economy will be throwing up" [doing what?]<br />
<br />
The AI revolution will reduce jobs, especially the ones that employ lots of people. Blue collar, but also white collar, and also management. Computer industry jobs like, er, mine, will go too. This is potentially but not necessarily terrible: if society can be run for the common good, then the extra time & wealth will be a good thing. <br />
<br />
Back to Hammond's many many jobs. Here's one new job automation creates: being human for legal reasons.<br />
<br />
<iframe style='width:100%; height:320px;' src="https://www.youtube.com/embed/UbwxGgR-EAM" frameborder="0" allowfullscreen></iframe><br />
<br />
[1] Philip Hammond pledges driverless cars by 2021 and warns people to retrain. <a href='https://www.theguardian.com/world/2017/nov/23/philip-hammond-pledges-driverless-cars-by-2021-and-warns-people-to-retrain'>The Guardian</a>.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-69356716369201670872017-10-06T10:36:00.001+01:002017-10-06T10:36:02.521+01:00GDPR Jargon: What is a Data Controllers vs a Data Processor?The incoming GDPR legislation introduces roles and responsibilities that will affect a great many companies.<br />
<br />
A <b>Data Controller</b> is any organisation that collects and uses data about individuals.<br />
<br />
E.g. does your company have a mailing list? Then it's a data-controller.<br />
<br />
Data-controllers are responsible for using the data correctly (i.e. only as permitted), for not retaining data beyond the agreed purpose, <br />
and for interacting with the individual about their data.<br />
<br />
A <b>Data Processor</b> is an organisation that provides data processing facilities -- e.g. databases and computer systems.<br />
<br />
They are responsible for how the data is stored, and for providing proper security (and notification of any security breaches).<br />
E.g. MailChimp is a data-processor.<br />
<br />
Of course, many companies are both data controllers and data processors.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-92126900463798272022017-07-28T12:26:00.002+01:002017-07-28T12:26:19.662+01:00Visiting the Future in ScandinaviaTo travel Scandinavia using AirBnB is to visit a version of the future. A clean, well-designed, if somewhat aseptic, future. You need hardly meet another human being. After booking (which is entirely online), I received instructions by email. There is no reception of course; check-in is by a code to unlock a key-box (a miniature safe), and collect a key, which then unlocks the front door. It's like an escape-the-room game (only in reverse: get-into-the-room). Payment is seamless, handled digitally in the background.<br />
<br />
There's a definite AirBnB chic, and it's nice enough but generic. Posters from warehouse outlet stores, especially those with universal quotes. Interior decor by the numbers; it could have been designed by a machine.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-5j8fFjggV3M/WXsbb6QYp2I/AAAAAAAASMs/-FpePvbaUkQl9BPvhE1UDSF2TjBk0zLDQCLcBGAs/s1600/DSC_0160.JPG" imageanchor="1" style="float:left;margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-5j8fFjggV3M/WXsbb6QYp2I/AAAAAAAASMs/-FpePvbaUkQl9BPvhE1UDSF2TjBk0zLDQCLcBGAs/s320/DSC_0160.JPG" width="120" data-original-width="900" data-original-height="1600" /></a><a href="https://3.bp.blogspot.com/-6d274MclU5w/WXsbb5_mU7I/AAAAAAAASMo/JUIeSHWwFqMc9EMhOLdtgGNhor9S8vUogCLcBGAs/s1600/DSC_0169.JPG" imageanchor="1" style="float:left; margin-left: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-6d274MclU5w/WXsbb5_mU7I/AAAAAAAASMo/JUIeSHWwFqMc9EMhOLdtgGNhor9S8vUogCLcBGAs/s320/DSC_0169.JPG" width="220" data-original-width="1600" data-original-height="900" /></a><a href="https://4.bp.blogspot.com/-4NqzPV2sNjg/WXsbb4ZkPNI/AAAAAAAASMk/ymde5wjYqdwW-o4BSnf5jq34RSccp8oswCLcBGAs/s1600/DSC_0206.JPG" imageanchor="1" style="float:left; margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-4NqzPV2sNjg/WXsbb4ZkPNI/AAAAAAAASMk/ymde5wjYqdwW-o4BSnf5jq34RSccp8oswCLcBGAs/s320/DSC_0206.JPG" width="220" data-original-width="1600" data-original-height="900" /></a><br />
</div><div class="separator" style="clear: both; text-align: center;"></div><br />
At Vielje Fjord, we are met by a robot lawnmower - a roomba-style contraption that trundles autonomously around the garden, before returning by itself to its charging bay. This meeting with our robot overloads made a great impression on the younger generation. Fascinated and a little nervous, their garden games centred on it: Escape from the Robot Lawnmower, and Robot Lawnmover Will Eat Your Pizza. The object of this attention calmly continued on with its tasks, unperturbed.<br />
<br />
I have the sensation of my family as a small bubble of humanity, travelling effortlessly through the landscape, our path mediated and managed by bots.<br />
<br />
As we leave the cabin at Vielje Fjord, we notice the robot lawnmower has failed. It navigated itself into a narrow patch between a fence and some garden furniture, then stuck, it's sensors reporting no safe way out, until it's batteries drained. The future is impressive but it's still a work in progress; expect glitches.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-7jZEY0KqfD4/WXsexV-auSI/AAAAAAAASNI/wGEU3BqedhUbhg6nZOLtzHxUz9SjZU_MgCLcBGAs/s1600/DSC_0125.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-7jZEY0KqfD4/WXsexV-auSI/AAAAAAAASNI/wGEU3BqedhUbhg6nZOLtzHxUz9SjZU_MgCLcBGAs/s320/DSC_0125.JPG" width="252" height="320" data-original-width="1258" data-original-height="1600" /></a><a href="https://2.bp.blogspot.com/-x-S-CLvDlp0/WXsexWNlr9I/AAAAAAAASNM/Vi2dDBXXcBMSWvNk7tLAW_OvRDxi_8i4gCLcBGAs/s1600/DSC_0163.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-x-S-CLvDlp0/WXsexWNlr9I/AAAAAAAASNM/Vi2dDBXXcBMSWvNk7tLAW_OvRDxi_8i4gCLcBGAs/s320/DSC_0163.JPG" width="320" height="180" data-original-width="1600" data-original-height="900" /></a></div>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com1tag:blogger.com,1999:blog-3289972539547813262.post-67636620614542119872017-06-26T15:07:00.001+01:002017-07-12T09:37:35.671+01:00BitCoin: It won't make you look 5lbs thinner<img border="0" src="https://4.bp.blogspot.com/-oyHALeiXfFA/WVES-cYvQbI/AAAAAAAARGo/2ePRWxpnh6MT98IqR_c_wulcEMjYV_3jQCK4BGAYYCw/s400/4880265002_5bf1a62db3_m.jpg" style='width:95%' /><br />
<br />
Recently I've been reading up on BitCoin, Ethereum, blockchain, and the assorted crypto-currencies than now bloom across the techscape.<br />
<br />
These are exciting times, but not clear times. Consensus ledgers and public cryptographically signed audit trails are an interesting thing, and they'll find some powerful applications. But it's notable that we've all been perfectly happy to use money systems without that -- witness banks, Visa/Mastercard, PayPal, or the numerous companies who use direct-debits to tap your wealth according to their own notion of credit/debt.<br />
<br />
Money systems are essentially about trust (that the the token you hold can be exchanged for goods & services). Money has to be backed by something we trust, such as the nation state & it's economy, or the bank's solvency and integrity at the book-keeping level, or for IOUs, in the individual. BitCoin is the first system where trust is grounded in an <i>algorithm</i>.<br />
<br />
That alone doesn't change how money is used. Perhaps BitCoin's biggest achievement is to get enough momentum to puncture the Visa/Mastercard monopoly (alongside PayPal, and now Android and Apple payment systems). These are (welcome) variations on an existing theme. We're still waiting to see a digital currency that really shakes up how commerce works.<br />
<br />
People talk of low transaction costs -- in fact BitCoin transactions are computationally very expensive! This is by design: BitCoin uses computational cost to defend against attack by a swarm of bots. As I write this, a BitCoin transaction costs about $0.50 and takes about 20 minutes, but these numbers vary.* That BitCoin is cheaper than debit card or bank transfers is not due to technological breakthroughs. It is because the existing system is massively over-priced.<br />
<br />
Reading the many earnest thought-pieces on how blockchain will change everything -- it seems people sometimes project their thoughts about the wider potential of digital currency onto the specific technology of blockchain and BitCoin. These thoughts then get repeated by others (and ratified by repetition), until it can become hard for everyone to see what's going on.<br />
<br />
I've seen a similar concept-creep happen in AI. Specific techniques get described by analogy, and those analogies are extended, until people see The Terminator in every AI project. Your future toaster might have senses (to see when the toast is cooked), and voice recognition and machine-learning to learn your preferences. And probably some spare intelligence, because it's cheaper to put in generic over-powered components than to custom make simpler ones. But no amount of feature-creep will turn it into The Terminator. Although it will get hacked by some internet d-bag, inscribe rude messages into your toast, then burn your house down. If you want a picture of AI over the next 5 years, imagine everything as smart toasters.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://s-media-cache-ak0.pinimg.com/600x315/cf/6a/6a/cf6a6ab8a41c8a2c3c37808dc49ea8c0.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://s-media-cache-ak0.pinimg.com/600x315/cf/6a/6a/cf6a6ab8a41c8a2c3c37808dc49ea8c0.jpg" width="400" data-original-width="600" data-original-height="315" /></a></div><br />
<br />
<hr /><br />
*BitCoin transactions offer a fee to encourage BitCoin miners to process them, and to do so ahead of other transactions (there is a queue). There are plans to alter BitCoin to allow for cheaper transactions, though this requires a reduction in security, and changing a currency on the fly is not easy.<br />
<br />
Images: (cc) FamZoo Staff, famzoo on Flickr, and the Talkie Toaster from Red Dwarf.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-40762174665640094342017-05-15T14:24:00.001+01:002017-05-15T14:28:54.843+01:00How to make a lightbox in DoubleClick for Publishers (DfP), or Wrestling with an AdTech MonsterIf you're working in adtech, you'll probably encounter DoubleClick for Publishers (DfP) - it is the most common publisher-side adserver (or SSP). We've recently produced a version of <a href="http://good-loop.com/">Good-Loop</a> for DfP. This article lists the various booby traps we found along the way. For further details, see our scars.<br />
<br />
First up: If you use an adblocker, turn it off. Understandably, that can break DoubleClick, given that it is an adserver.<br />
<div>
<br />
So you want to write an expanding lightbox -- well DoubleClick supports that, via DoubleClick Studio Enabler (Enabler.js), and how lovely, they even provide a template to get you started.<br />
<div>
Except it is broken. We modified the template and it didn't work. We used the original exactly as provided, and it did not work. As far as I can tell, this is obsolete and broken code, they just forgot to update the documentation. Hidden in a disused basement of the DoubleClick website, inside a cabinet labelled "beware of the leopard", there is a note to say that SafeFrame is the preferred method. <b>Do not use DoubleClick Studio Enabler -- use SafeFrame instead</b>.<br />
<div>
<br /></div>
<div>
SafeFrame requires you to know the size of the ad slot. And it provides a query for finding out page size info. However this is not as useful as you'd hope, because you have to specify the slot size first, before you can query the page size. So you need to pass that info into your creative -- which is done via a couple of macros: %%WIDTH%% and %%HEIGHT%%.</div>
<br />
You'll need to use the SafeFrame <span style="font-family: "courier new" , "courier" , monospace;">ext.geom()</span> method to provide the right expansion size. Beware that this can be affected by timing issues. Don't call it until you're about to do the expansion.</div>
<div>
<br /></div>
<div>
At the time of writing, DfP's implementation of SafeFrame was slightly flaky on iOS. To get reliable results, we put in code to retry the lightbox expansion (and the close) a few times, stopping if your handler receives a success message.</div>
<div>
<br />
<div>
<div>
You'll probably want to know where your advert is going -- but DfP doesn't tell you by default, because: I don't know. You can get this by adding the %%SITE%% macro to your script url. As an example, Good-Loop's final DfP creative looks like this:<br />
<br /></div>
<div>
<div style="text-align: left;">
<span style="color: #24292e; font-family: "courier new" , "courier" , monospace; font-size: 13.6px;"><script src="//as.good-loop.com/unit.js?site=%%SITE%%&width=%%WIDTH%%&height=%%HEIGHT%%&adunit=%%ADUNIT%%&cb=%%CACHEBUSTER%%"></script></span></div>
</div>
<div>
</div>
</div>
<div>
<div style="text-align: left;">
<br /></div>
The next gotcha is the DfP Preview feature. This handy feature provides a link to see your creative in situ. Except DfP preview does not work for rich media ads. This is due to a bug in DfP: the preview simply doesn't properly support the lightbox features -- although the real DfP iframe does. This makes preview confusingly useless for lightbox ads like Good-Loop. So <b>avoid Creative -> Preview</b>.<br />
<br />
So how can you preview an advert? The best approach we found was to setup a very targeted line-item. In DfP, create a line-item that targets a specific (and rare) device. Then in your Chrome browser, open the developer console, toggle device mode, and set Chrome to emulate that device type.<br />
<br />
An annoying feature of DfP is that it will sometimes fill slots with random adverts, rather than the line item you want. Swearing and mashing the settings is one solution.<br />
<div>
<br /></div>
<div>
<div>
Another source of frustration is the delays. Edits in DoubleClick can take 10 minutes to percolate through the slow-as-treacle systems and actually take effect. This multiplies the pain of the trial-and-error stress test that is working with DfP. To minimise this, we moved our html code to being dynamically generated, with the DoubleClick creative reduced to a single .js script tag.</div>
<div>
<br /></div>
Now we're serving adverts...<br />
<br />
Bugs from the host-page CSS: It's easy to break a SafeFrame's ability to expand to full-page. A common css rule goes something like:<br />
<div class="mw-geshi mw-code mw-content-ltr" dir="ltr" style="background-color: #f9f9f9; border: 1px solid rgb(221, 221, 221); box-sizing: inherit; direction: ltr; font-family: monospace, Courier; font-size: 14px; padding: 1em;">
<div class="css source-css" style="box-sizing: inherit; font-family: monospace; line-height: normal;">
<pre class="de1" style="background: none; border-radius: 4px; border: 0px none white; box-sizing: inherit; font-family: monospace, monospace; font-size: 1em; font-stretch: normal; line-height: 1.2em; overflow: auto; padding: 1px; vertical-align: top; word-break: break-all; word-wrap: break-word;">iframe<span class="sy0" style="box-sizing: inherit; color: #00aa00;">,</span> some other element types <span class="br0" style="box-sizing: inherit; color: #00aa00;">{</span>
<span class="kw1" style="box-sizing: inherit; font-weight: bold;">max-width</span><span class="sy0" style="box-sizing: inherit; color: #00aa00;">:</span> <span class="re3" style="box-sizing: inherit; color: #993333;">100%</span><span class="sy0" style="box-sizing: inherit; color: #00aa00;">;</span>
<span class="br0" style="box-sizing: inherit; color: #00aa00;">}</span></pre>
</div>
</div>
<br />
Looks reasonable, but it restricts the iframe's width to 100% of the parent element's - which is almost always very small. Your SafeFrame does an expand... to the same size it was before.<br />
<br />
If the publisher is cooperative, they can fix this by adding a very specific CSS rule such as:</div>
<div>
<br class="Apple-interchange-newline" />
<span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;">iframe</span><span class="br0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">[</span><span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;">id</span><span class="sy0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">^=</span><span class="st0" style="background-color: #f9f9f9; box-sizing: inherit; color: red; font-family: monospace , monospace; font-size: 14px; white-space: pre;">'google_ads_iframe_'</span><span class="br0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">]</span><span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;"> </span><span class="br0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">{</span><span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;"> </span><span class="kw1" style="background-color: #f9f9f9; box-sizing: inherit; font-family: monospace , monospace; font-size: 14px; font-weight: bold; white-space: pre;">max-width</span><span class="sy0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">:</span><span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;"> </span><span class="kw2" style="background-color: #f9f9f9; box-sizing: inherit; color: #993333; font-family: monospace , monospace; font-size: 14px; white-space: pre;">none</span><span class="sy0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">;</span><span style="background-color: #f9f9f9; font-family: monospace , monospace; font-size: 14px; white-space: pre;"> </span><span class="br0" style="background-color: #f9f9f9; box-sizing: inherit; color: #00aa00; font-family: monospace , monospace; font-size: 14px; white-space: pre;">}</span></div>
<div>
<br /></div>
</div>
</div>
</div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com23tag:blogger.com,1999:blog-3289972539547813262.post-43330420121427359272017-04-25T19:26:00.004+01:002017-06-29T07:58:00.769+01:00The Council of the Gods, by Kit WrightLay no blame. Have pity.<br />
Put your fingers in the wounds of the Committee.<br />
<br />
They never reached your item.<br />
Disputing Item One ad infinitum.<br />
<br />
Lay no blame. Be tender.<br />
The retrospective start of the agenda.<br />
<br />
Was all they managed treating.<br />
Consider, pray, the feeling of the meeting.<br />
<br />
(They felt awful). Not surprising<br />
They never came to matters not arising.<br />
<br />
From Matters Arising:<br />
<blockquote class="tr_bq">Who took the chair when the standing committee last sat?<br />
Who kept the minutes for hours and hours and hours?<br />
Who tabled the motion,<br />
Who motioned the table<br />
Whereat<br />
The standing committee<br />
Sat?</blockquote>Have pity.<br />
Put your fingers in the wounds of the committee.<br />
<br />
The gods have not been sleeping.<br />
All night they sat, in grief and boredom, weeping.<br />
<br />
By Kit Wright, <a href="http://amzn.to/2q3w8YB">Amazon book link</a>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-82954258626068541982016-12-29T10:29:00.002+00:002016-12-29T10:29:36.091+00:00O Have You Caught the Tiger?, a poem by A.E. Housman<span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">O have you caught the tiger?</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://upload.wikimedia.org/wikipedia/commons/4/49/Panthera_tigris_tigris.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="213" src="https://upload.wikimedia.org/wikipedia/commons/4/49/Panthera_tigris_tigris.jpg" width="320" /></a></div>
<span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And can you hold him tight?</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And what immortal hand or eye</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Could frame his fearful symmetry?</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And does he try to bite?</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Yes, I have caught the tiger, </span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And he was hard to catch.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">O tiger, tiger, do not try</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">To put your tail into my eye,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And do not bite and scratch.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Yes, I have caught the tiger.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">O tiger, do not bray!</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And what immortal hand or eye</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Could frame his fearful symmetry</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">I should not like to say.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And may I see the tiger?</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">I should indeed delight</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">To see so large an animal</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Without a voyage to Bengal</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And mind you hold him tight.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Yes, you may see the tiger;</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">It will amuse you much.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">The tiger is, as you will find,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">A creature of the feline kind.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And mind you do not touch.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And do you feed the tiger,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And do you keep him clean?</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">He has a less contented look</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Than in the Natural History book,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And seems a trifle lean.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Oh yes, I feed the tiger,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And soon he will be plump;</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">I give him groundsel fresh and sweet,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And much canary-seed to eat,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And wash him at the pump.</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">It seems to me the tiger</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Has not been lately fed,</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Not for a day or two at least;</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">And that is why the noble beast</span><br style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;" /><span style="background-color: white; color: #012944; font-family: tahoma; font-size: 16px;">Has bitten off your head.</span>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com2tag:blogger.com,1999:blog-3289972539547813262.post-657109007851390042016-08-16T11:38:00.002+01:002016-08-16T11:42:59.409+01:00Computer Generated Haiku - a project by Aji Alham Fikri & Daniel WintersteinHere are a couple of write-ups of the computational creativity in poetry research that Aji and I did last year:<br />
<br />
<ul>
<li><a href="https://dl.dropboxusercontent.com/u/5692145/ai-haiku/ai-haiku-slides.html">AI Haiku talk slides</a></li>
<li><a href="http://www.sodash.com/the-poetry-of-technology/">Ryan's blog-post</a></li>
<li><a href="http://project-archive.inf.ed.ac.uk/msc/20150275/msc_proj.pdf">Aji's MSc thesis</a></li>
</ul>
I'm planning to extend this into a general purpose poetry generator / evaluation. You can see the work-in-progress notes for that here: <a href="https://github.com/winterstein/HaikuGen/blob/master/code/haikugen/src/com/sodash/poetry/poem-format.js">a JSON specification format for poetry</a>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-55128999002143061452016-06-22T11:19:00.003+01:002016-06-22T11:19:51.192+01:00Agile Procurement?<div style="border-radius: 5px; border: 1px solid #666; float: right; margin: 5px; padding: 5px; width: 30%;">
<img src="https://1001movienights.files.wordpress.com/2014/01/brazil_movie.jpg" style="width: 100%;" /><br />
<small>Image from Brazil, a gloriously dark comedy about bureaucracy and power by Terry Gilliam. Actual relevance to this post: low, but I like the movie.</small></div>
When it comes to software, public bodies spend a lot of money yet often have second-rate web-sites and systems. Why?<br />
<br />
Partly, because high-profile software projects are difficult, and public-service software often has to handle lots of corner-cases that make off-the-shelf solutions harder to use. But partly it is their own fault: Public procurement sets up systems that almost ensure they will pay too much for second-rate software. Why?<br />
<br />
One such model is the framework agreement. Companies first tender to be on the list to tender for actual work. Bureaucratic framework agreements create an overhead that eliminates most small software companies (who are of mixed quality but contain many of the best developers) in favour of large contractors (who tend to charge more, often a lot more, and deliver older and less flexible software).<br />
<br />
I expect the bureaucracy is trying to manage risk & overhead: Do some heavy vetting once, then re-use it. But this vetting is of little value. A large contractor will submit their successful past projects, possibly carried out by teams who have no connection to the teams that will then work on the tender. A small contractor has less track record to draw on, so is at a disadvantage. Inspite of the care taken by procurement, government software projects often over-run or fail to deliver. Inspite of... or because of?<br />
<br />
There are better ways to manage risk in software projects! <b>We need Agile Procurement. Not procurement of agile software, but agile ideas in procurement itself.</b> That is, procurement teams who work iteratively with the supplier and consumer teams, taking small short-term risks as the best way to manage costs and avoid large risks.<br />
<br />
I'd also like to see <b>multiple redundancy in procurement</b>. Instead of betting everything on one big contract with one supplier... have several suppliers produce prototypes, at least for the early stages. If one supplier fails to deliver -- it's not a problem. This would allow for lighter touch procurement -- opening the door to SME software development companies. Given the difference in costs, I believe this would actually lower the overall price. It also allows more ideas to be explored, and it introduces some post-tender competition -- and hence better software at the end.Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-17150820999798641102016-05-10T15:02:00.002+01:002016-06-03T11:03:09.779+01:00A Simple Intro to Gaussian Processes (a great data modelling tool)<div style="border: solid 1px #999; float: right; width: 330px; padding:2px; margin:4px; border-radius:5px;"><img border="0" height="261" src="https://2.bp.blogspot.com/-zIrp2Iwlrm8/VzHJ4ZNu6uI/AAAAAAAALjk/9MCUUXoUuwMB_QsJfD2wbkEi4ZMOSL2iACLcB/s320/gp-via-matlab.gif" width="320" /><div><small><br />
A Gaussian Process, fitted using MatLab, showing most-likely-value & confidence interval. Note how the shape hugs the data, and how the uncertainty varies depending on the data - sometimes the model is confident, sometimes it isn't, and you know which is which.</small></div></div><br />
Gaussian Processes (GPs) are a powerful technique for modelling and predicting numerical data. Being both relatively new and mathematically quite complex, they're not as well known as other techniques. They have some strong advantages:<br />
<ol><li>Flexible: Can be used to model many different patterns.</li>
<li>You make fairly few assumptions about the model.</li>
<li>Based on probability theory, so they have a solid mathematical grounding.</li>
</ol>This article is an easy-to-read introduction. Instead of diving into the maths behind a Gaussian Process, let's start with a simpler algorithm.<br />
<h2>K Nearest Neighbours (KNN)</h2>K Nearest Neighbours is a classic AI algorithm, and very easy to understand. It also provides a surprisingly good basis for explaining Gaussian Processes (without any maths involved yet). Here's how KNN works:<br />
<br />
The task: Given an item <i>x</i>, predict the category of <i>x</i>. E.g. you might ask, Is this email spam or not-spam?<br />
The method:<br />
<ol><li>Let's pick k=5.</li>
<li>Given an input item <i>x</i>... E.g. a fresh email has arrived, is it spam?</li>
<li>Look through your training data, and find the 5 items most similar to the input item <i>x</i>. These are the nearest neighbours.</li>
<li>Look at the categories those 5 items have, and predict the most common category from them.</li>
<li>Done :)</li>
</ol>A few things to note about the KNN algorithm:<br />
<ul><li>It is a lazy algorithm. The model isn't trained in advance, as with say Linear Regression or Naive Bayes -- instead the work is done when an input item is presented.</li>
<li>The data <i>is</i> the model. If you have enough data, KNN can model a really wide range of patterns. Rather than being forced into a particular shape, the data can speak for itself. There is a cost to this though - it requires more training data.</li>
<li>The key part is judging when items are similar. How you do that will depend on the problem you're looking at.</li>
</ul>These are also key properties of Gaussian Processes.<br />
<br />
The classic KNN algorithm is for predicting categories (e.g. spam / not-spam), but we can modify it as follows to make numerical predictions (e.g. the price of fish):<br />
Step 4': Having found the k nearest neighbours, take the average value.<br />
<h2>From KNN to a Gaussian Process</h2>So what is a Gaussian Process?<br />
It deals in numerical data, e.g. the price of fish -- and for the rest of this article, let's say it <i>is</i> the price of fish as a function of date that we're modelling. We keep all the training data, and say that when predicting new data points from old ones, the uncertainty/noise will follow a multivariate Gaussian distribution. The relationship given by that distribution lets us make predictions from the training examples.<br />
<br />
For non-Mathematicians, let me briefly cover the terminology from that paragraph. A distribution tells you how likely different values are. The Gaussian Distribution, aka the Normal Distribution, has a bell-shaped curve: the mean is the most likely point, and the probability drops off rapidly as you move away from the mean. The multivariate Gaussian (which is what we want) can specify correlations between multiple variables. The Gaussian Distribution naturally arises in lots of places, and is the default noise model in a lot of machine learning.<br />
<br />
Similar to KNN, the Gaussian Process is a lazy algorithm: we keep the training data, and fit a model for a specific input. Also like KNN, the shape of the model will come from the data. And as with KNN, the key part is the relationship between examples, which we haven't defined yet...<br />
<h2>Introducing the Kernel Function</h2>A Gaussian Process assumes that the covariance between any set of points is a multivariate Gaussian.<br />
The "variables" here are the different values for the input <i>x</i>. In our price-of-fish example, the input is the date, and so every date gives a dimension! Yes, in principle, there are an infinite number of variables! However we only have a limited set of training data -- which gives us a finite set of "variables" in the multivariate Gaussian -- one for each training example, plus one for the input we're trying to predict.<br />
<br />
A multivariate Gaussian is defined by it's mean and covariance matrix, so these are the key things for us to calculate.<br />
<br />
The kernel function of a GP says how similar two items are, and it is the core of a specific Gaussian Process. There are lots of possible kernel functions -- the data analyst (e.g. you) picks an appropriate one.<br />
<br />
The kernel function takes in any two points, and outputs the covariance between them. That determines how strongly linked (correlated) these two points are.<br />
<br />
A common choice is to have the covariance decrease as the distance between the points grows, so that the prediction is largely based on the near neighbours. E.g. the price of fish today is more closely linked with the price yesterday than the price last month. Alternatively, the kernel function could include a periodic part, such as a sine wave, to model e.g. seasonal ups and downs. The Wikipedia article lists some example kernel-function formulas.[1]<br />
<div><br />
</div><div>The kernel function will often have some parameters -- for example, a length parameter that determines <i>how</i> quickly the covariance decreases with distance. These parameters are found using optimisation software -- we want parameters that optimise the likelihood of the observed data. We can write down the probability of the observed data (the likelihood) as a function of the kernel function parameters, and then pick kernel function parameter values to maximise the likelihood.</div><div><br />
</div>Given an input <i>x </i>(e.g. "next Thursday") with the training examples <i>x</i>1, <i>x</i>2, ... <i>xn</i> (e.g. the price of fish each day for the last few months), then the GP model is that (x, x1, x2, ...<i>xn</i>) has a distribution with mean 0 and a covariance matrix defined by the kernel function.<br />
<br />
From the covariance matrix, you can then calculate the prediction for <i>x</i>. The prediction for <i>x</i> (or in probability theory terminology, the marginal distribution for <i>x) </i>will be a simple one-dimensional Gaussian. It has a mean value (which is the most likely value for <i>x</i>) and a standard-deviation for the uncertainty<i>.</i><br />
<h2>Building a GP</h2>This article has skipped over the technical details of how you carry out certain steps. I've blithely written about "optimising the likelihood" without saying how you do that. That's partly because there are multiple ways, and partly to keep this article simple. The short answer is: you'll be using software of course, and most likely software that someone has kindly already written for you.<br />
<br />
I'm not going to recommend a particular software tool here, as the choice really depends on what you're familiar with and where you're using it. There are GP calculators available for many environments, e.g. Weka has one for Java [2], or you can talk to your local Winterwell office[3] :)<br />
<h2>Going Deeper</h2>[1] Commonly used kernel functions, in Wikipedia: <a href="https://en.wikipedia.org/wiki/Gaussian_process#Usual_covariance_functions">https://en.wikipedia.org/wiki/Gaussian_process#Usual_covariance_functions</a><br />
[2] <a href="http://www.cs.waikato.ac.nz/ml/weka/">Weka</a>, a Java toolkit for machine learning.<br />
[3] <a href="http://www.winterwell.com/">Winterwell, data science consultancy</a><br />
<br />
So you want to know the mathematical details? Good for you!<br />
Try reading these resources:<br />
<br />
[4] <a href="http://arxiv.org/find/math,stat/1/au:+Ebden_M/0/1/0/all/0/1">"Gaussian Processes: A Quick Introduction"</a> by Mark Ebden.<br />
[5] <a href="http://www.gaussianprocess.org/gpml/chapters/">"Gaussian Processes for Machine Learning"</a> by Carl Rasmussen and (my boss once-upon-a-time) Chris Williams.<br />
<br />
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0tag:blogger.com,1999:blog-3289972539547813262.post-9895990911027118812016-04-01T23:21:00.001+01:002016-04-01T23:23:35.299+01:00“Have a Nice Day!” smiled the robot<div class="et_post_meta_wrapper" style="background: rgb(255, 255, 255); border: 0px; box-sizing: border-box; color: #314046; font-family: 'Open Sans', Helvetica, Arial, Lucida, sans-serif; font-size: 14px; line-height: 23.8px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">
<h1 style="background: transparent; border: 0px; box-sizing: border-box; color: #444444; font-size: 28px; font-weight: 500; line-height: 1em; margin: 0px; outline: 0px; padding: 0px 0px 10px; vertical-align: baseline;">
<span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;">Automation makes it easy to spam people. Makes it easy for normal business-people — people who are quite genuine in their day-to-day dealings — to spam people.</span></h1>
<h1 style="background: transparent; border: 0px; box-sizing: border-box; color: #444444; font-size: 28px; font-weight: 500; line-height: 1em; margin: 0px; outline: 0px; padding: 0px 0px 10px; vertical-align: baseline;">
<span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;">Part of the problem is the curse of subjectivity — </span><em style="background: transparent; border: 0px; box-sizing: border-box; color: #314046; font-size: 14px; line-height: 23.8px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">their</em><span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;"> email is spam, </span><em style="background: transparent; border: 0px; box-sizing: border-box; color: #314046; font-size: 14px; line-height: 23.8px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">my</em><span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;"> message is an valuable communication. Caught up in our own projects, it’s easy to forget that people receiving the email/sms/tweet may not care.</span></h1>
<h1 style="background: transparent; border: 0px; box-sizing: border-box; color: #444444; font-size: 28px; font-weight: 500; line-height: 1em; margin: 0px; outline: 0px; padding: 0px 0px 10px; vertical-align: baseline;">
<span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;">But that kind of unwanted message is relatively forgiveable. There is a worse kind — the message that pretends to be something other, to worm it’s way past your defences. The insincere message. And it’s surprisingly easy to write one,</span></h1>
<h1 style="background: transparent; border: 0px; box-sizing: border-box; color: #444444; font-size: 28px; font-weight: 500; line-height: 1em; margin: 0px; outline: 0px; padding: 0px 0px 10px; vertical-align: baseline;">
<span style="background-color: transparent; color: #314046; font-size: 14px; line-height: 23.8px;">The main culprit is the advert that doesn’t acknowledge that it’s a sales message, but pretends to be something else — helpful or part of a conversation. My inbox is full of these.</span></h1>
</div>
<div class="entry-content" style="background: rgb(255, 255, 255); border: 0px; box-sizing: border-box; color: #314046; font-family: 'Open Sans', Helvetica, Arial, Lucida, sans-serif; font-size: 14px; line-height: 23.8px; margin: 0px; outline: 0px; padding: 30px 0px 0px; vertical-align: baseline;">
<ul style="background: transparent; border: 0px; box-sizing: border-box; line-height: 26px; margin: 0px; outline: 0px; padding: 0px 0px 23px 16px; vertical-align: baseline;">
<li><em style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">“Are you validating performance for your mobile users?”</em></li>
<li style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">“Re: Making Great Customer Experiences”</em><br style="box-sizing: border-box;" />Double spam points there! Pretending to be a reply to me, plus a subject line that doesn’t acknowledge it’s an ad.</li>
<li style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">“Daniel, open this email for 12 people you should meet :)”</em><br style="box-sizing: border-box;" />Sure, the cheap trick of using my name did help them get noticed – but to what end?</li>
<li style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">“I <span class="il" style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">want</span> you <span class="il" style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">back</span> for </em><span class="il" style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="background: transparent; border: 0px; box-sizing: border-box; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">good”</em><br style="box-sizing: border-box;" />Suggestive for a B2B message! But pretending to be on friendly terms with a stranger is insincere. It doesn’t make the message cute, it makes it annoying.</span></li>
</ul>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
Straight to spam! And when you do read one of these? Disappointment lies ahead: the headline makes click-bait promises to lure you in, but then post does not deliver. You’re likely annoyed at having been fooled into a click — hardly the best start to a customer relationship. If a company starts the conversation with an insincere email — do you trust them?</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
As a contrast, here’s a sincere sales message:</div>
<blockquote style="background: transparent; border-color: rgb(21, 127, 185); border-left-style: solid; border-width: 0px 0px 0px 5px; box-sizing: border-box; margin: 20px 0px 30px; outline: 0px; padding: 0px 0px 0px 20px; quotes: none; vertical-align: baseline;">
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px; vertical-align: baseline;">
Network @ BDX Glasgow Next Week (And Win Your Own Office!)</div>
</blockquote>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
Good honest work BDX: It’s clear from the subject that this is an ad for a Glasgow networking event. If I’m interested, I’ll read it. It is an advert, but that's OK.</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
In real life, if someone behaves insincerely, soon enough they notice people avoiding them. With digital we may not notice the bad impression insincerity leaves. And with automation… Automation can amplify the problem. With automation, one can be insincere at scale. Pushy insincere tactics can get more clicks. But click-counting doesn’t measure that it also annoys. The pushy marketer doesn’t see the people who don’t click. This can lead to companies blindly optimising for spam, and trashing their own brand.</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
The problem is not automation, nor is it measurement and optimisation — these are tools that can amplify an underlying problem. The problem is a lack of sincerity.</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
Here’s a simple test of sincerity: If the recipient knew the full story behind the message — how the recipient was chosen, how the message was crafted, and what the sender hopes to achieve — Would that affect how they read it? If so — some insincerity may have crept in.</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
So: At <a href="http://sogrow.co.uk/">SoGrow</a> we’ll be trying to keep it sincere. When one of our bots talks to you, they will hopefully act in a sincere manner — e.g. being open when doing sales.</div>
<div style="background: transparent; border: 0px; box-sizing: border-box; outline: 0px; padding: 0px 0px 1em; vertical-align: baseline;">
Yours sincerely,<br />
<span style="background-color: transparent; line-height: 23.8px;"> – Daniel</span></div>
</div>
Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com1Edinburgh, Edinburgh, UK55.953252 -3.188266999999996255.8109675 -3.5109904999999961 56.0955365 -2.8655434999999962tag:blogger.com,1999:blog-3289972539547813262.post-40652701754771119222016-04-01T23:19:00.001+01:002016-04-01T23:19:04.295+01:00Should charities share profile data, after the tragic death of Olive Cooke?<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-mpFIuLYtMkg/Vv7zahsNukI/AAAAAAAAK8Q/GS6NZTw43P4NmEzuGRTLMkhgoUoV3y6Cg/s1600/PAY-Olive-Cooke.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://4.bp.blogspot.com/-mpFIuLYtMkg/Vv7zahsNukI/AAAAAAAAK8Q/GS6NZTw43P4NmEzuGRTLMkhgoUoV3y6Cg/s320/PAY-Olive-Cooke.jpg" width="320" /></a></div>
<br />
The Olive Cooke case is a striking example where an individual who cared enough to take an interest in charities can end up being bombarded with mail. The case became more dramatic with claims that Olive Cooke was actually killed by this, although the family vociferously denies this.<br />
<br />
There has been much soul searching since, and Third Sector magazine has pointed out that 99 charities had Olive Cooke's contact details, of which 16 failed to provide any opportunity to opt out, 56 required her to proactively contact them if she wished to opt out, and only 14 provided an opt-out tick-box.<br />
<br />
It is natural to want to curb the data sharing that led to Olive Cooke receiving so much mail, and I don't want to discourage the valuable measures that help people better understand what happens with their data.<br />
<br />
However are calls for charities to avoid using profile data the right reaction?<br />
<br />
The average person is bombarded by marketing, with charities only being a fraction of the problem. Other mailings are often more dangerous and certainly less defensible. It would be perverse to rule that charities must limit their marketing, whilst giving free rein to all manner of profit-focused companies. That includes more widely dangerous material, such as seductive loan offers, from the credit card offers banks bombard us with through to the more obviously unscrupulous pay-day lenders.<br />
<br />
In principle, there are already tools to protect you from unwanted contact. The Mail Preference Service, Telephone Preference Service, and Email Preference Service provide a way to opt out of cold-calls and junk mail. But for the charity marketing industry to point to these as solutions is not good enough. Most people do not know of these. Also many organisations do not check against these lists (it is not straightforward, and there is no free service to do so). Finally, these opt-outs do not restrict contact from organisations who can claim a connection (e.g. from a previous contact, which might be as little as an online petition), or where you have, often accidentally, consented to your data being shared.<br />
<br />
We might ask charities to employ more care. But how can a charity really screen large mailing lists, and cross-check with other charities? The admin challenge there is large and complex. If charities were to spend on this, they would find themselves even more criticised for high admin costs.<br />
<br />
We might urge self-restraint -- but it is unrealistic to expect self-restraint from marketing teams to be sufficient. Even where marketers have the best of intentions, their position is too subjective -- “other organisations send junk mail; our messages have valuable information of hopefully mutual interest.” Nor would we want charities to act in a half-hearted one-arm-behind-their-back manner -- we want charities to be as effective as possible, and that includes efficient marketing and effective fund-raising.<br />
<br />
What is the way forward for the Third Sector? <br />
<br />
It begins by acknowledging there is room to improve practices in charity marketing, and following the death of Olive Cooke, a real need to improve. That does not mean avoiding modern marketing tools. Turning back the clock is rarely the answer. Instead of asking charities to be less-effective, we should establish best-practice guidelines, educate the sector about them (which includes educating more charities on how to effectively use these tools), and ensuring best practice is adopted and adhered to.<br />
<br />
Let us suggest some concrete measures charities should adopt in their use of profile data:<br />
<br />
<ul>
<li>It must be easy to opt-out of communications.</li>
<li>If consent is withdrawn, that should be passed on to any partners with whom the data has been shared.</li>
<li>Rather than having to opt-out from every individual charity, it should be possible to opt-out of communications from whole sectors.</li>
<li>Consent to sharing profile data must be genuine -- the current practice of sneaking in a tick-box is not acceptable.</li>
</ul>
<br />
Longer term, could data be managed better to give a more holistic view of communications with the person, with trusted bodies mediating marketing? Such bodies could better understand whether someone was being bombarded with messages from other charities, and understand whether someone has a specific cause they are interested in, in which case they wouldn't want to hear from other charities in other areas. Or indeed whether someone is just not interested in giving to charity at all. This would be good for the charity, because it can reduce their mailing costs and allow for more personalised messages, and good for people like Olive Cooke too.<br />
<br />
<i>Written with Sanjay Joshi for sogive.org</i>Daniel Wintersteinhttp://www.blogger.com/profile/01100168408709296559noreply@blogger.com0