Questions of Bias, Trust and Security in Emerging Technology: Speaking Notes

© Steven David Brown

These speaking notes were presented at a round table discussion with students of the International Anti-Corruption Academy (IACA) Summer Academy, Laxenburg, Austria on 3 July 2019.

Emerging technology, that’s our theme, but what do we mean by ‘technology’? A hammer and a nail is technology. A ball point pen is technology. The Oxford Dictionary defines ‘technology’ as:

‘The application of scientific knowledge for practical purposes.’

We’re here at IACA, so perhaps we can agree that technology in this context is:

‘The application of scientific knowledge to understand, detect, deter and disrupt corruption’.

But actually we are supposed to be talking specifically about ‘emerging’ technology. The implication being that we are discussing new types of digital tools or software that have become recently available and which have received a certain level of visibility and prominence. My first word of caution is that, just because something is new and is popular, does not mean it is effective or, even desirable.

Technology can be used for good and for evil, … and it can be highly manipulative. We have seen claims that Internet troll factories are being used to influence political and economic interests in foreign countries.[1] But feeding carefully customised information to the masses is nothing new. It’s called propaganda and all countries and organisations practice it in one form or another. In the cigar factories of Cuba, pro-government news was read out to the workers as a form of indoctrination and I heard only last week how a major cigarette manufacturer employs 10,000 people in Asia to hand roll cigarettes and that while they work they listen to specially commissioned soap operas broadcast over the workplace loudspeakers with stories that reflect positive role models and encourage their workers to adopt ethical behaviour. Even in my country, in the UK, it was revealed recently that, in the 50s, a highly popular radio programme called the Archers – a drama series about a farming community in middle England – was used by the government to represent the virtues of modern farming techniques.[2] It’s still running on the BBC and is the longest running soap in the world and, although correlation is not the same as causation, UK farming techniques are among the most advanced.

We have now got to the point where technology registers everything we write, it logs everything we search and read. It generates a detailed chronicle of our virtual selves which is stored by and under the control of a limited number of unaccountable international corporations.

For the past 15 years or so, we have had social media which is even more intense and powerful as an opinion former. Barrack Obama was dubbed the ‘first social media president’[3] because of the way his campaign harnessed the power of social media to get elected. President Trump has another reputation, but he used the technology in much the same way. On one view we could be said to be voluntarily submitting ourselves to this brainwashing.

There’s a phenomenon called ‘confirmation bias’ where we seek out information that supports and reinforces our existing opinions and views. When there is no authoritative body curating what we read and unconsciously digest, when we just follow the headlines that appeal to us, we subject ourselves to an addictive diet of data that is the intellectual equivalent of fast food.

The free and unfettered access to information afforded by the Internet has been called the ‘democratisation of information’.[4] But just how much free will do we have? How can we be expected to distinguish between the wheat and the chaff when the chaff comes seductively bundled with fancy fonts and video clips and a high review rating?

How many of you have ever hit the ‘like’ button on Facebook? Google ‘Tristan Harris’ and ‘TED Talks’ and see how social media companies design their websites to keep you clicking and to turn your clicks into profit.[5]

What has this all got to do with understanding, detecting, deterring and disrupting corruption? Well, when someone’s data is being recorded all the time it can be used to reconstruct events, it can show states of mind, identify suspects and their associates and even prove elements of a modus operandi.

Suppose you are looking into a suspect whom you think has been on the take. You might want to perform a lifestyle analysis – what I mean by that is, see if they’re living beyond their means with exotic holidays, expensive hobbies and unrestrained luxury. You might well find evidence of that on social media. Even If the target is careful about what he or she posts, there are always the kids.  You see, it is not enough for you to be a data hermit, you have friends and family who will post about you too.

But the integration of data with physical lives and events goes even deeper than that.

I know of at least two cases where the authorities in the USA have asked Amazon for information about what may or may not have been recorded at a crime scene by an Amazon Echo device.

And how about location-based-services? You may not have heard the term before, but you certainly know what they are. It is where you get services offered to you through your smartphone based on your current location. You know the sort of thing: taxis or local restaurants or how to get from A to B. These services are becoming more sophisticated all the time. But for your smartphone to offer you location based services, the telecoms company must first know where you are.

A lot has been said about the scandal involving Cambridge Analytica and Facebook, but have you heard about Google’s Sensorvault?[6]  If you have Google’s location history enabled on your device, your movements are recorded in a database along with millions of others. If law enforcement wants to see who might have been in a certain location at a certain time, they can get a geo-fence warrant – this is a warrant that sets geographical boundaries and a timeframe. They send the warrant to Google. Google then searches the database for anyone registered in that area at that time. Now, it’s true to say that the data in Sensorvault is anonymised, but Google is able to decrypt and collate it and supply the details to the requesting agency.

Of course, you can disable the location feature on your device, but how many people do that? And if it is disabled, the software will ask you periodically if you want to enable it.

There are also commercial services offered by companies like Echosec[7] and Geofeedia[8] that scoop up information from Twitter and Facebook and Instagram posts and collate it against GPS coordinates so that a subscriber can find out who was tweeting or posting from a certain location at a certain time. You don’t need to go to court to get a warrant, you don’t even need to be law enforcement, just pay the fees.

This is the sort of technology that led Sir Geoffrey Voss, a senior English Judge and chancellor of the High Court of England and Wales to suggest that, in the future, fewer trials will be contested because of this pervasive level of surveillance.[9]  But is this degree of surveillance desirable? Last year, in the USA, the Supreme Court in a case called Carpenter upped the legal threshold for law enforcement when applying for cell phone location records because, they said,

‘… the time-stamped data provides an intimate window into a person’s life, revealing not only his particular movements, but through them his “familial, political, professional, religious and sexual associations”.’ [10]

This is not ‘emerging’ technology, ladies and gentlemen, it is existing technology.

But there is so much data. We are drowning in it. A 64GB smartphone can potentially hold the data equivalent of 11,600 times the complete works of Shakespeare.[11] Phones accumulate data at a frightening rate.

How many investigators have the time to sit down and review 50,000 text messages?  And that is not unheard of. What if there is just one text message amongst all those thousands that indicates guilt or exonerates the innocent. Key word searches are not good enough. Sometimes they are not technically possible. Sometimes people substitute code words for illegal or confidential things. Sometimes you simply just don’t know which words to search for.

If a key word search doesn’t return any results, we automatically assume that there is nothing there to be found, not that we haven’t looked for the right thing.

We have evolved to believe whatever technology tells us without question or closer inspection.  We think that machines are more reliable than humans and we are predisposed to trust their outputs. But I want to suggest that our trust is misplaced.  We are allowing technology to do the thinking for us when we should only be using it as a tool to help us think.

Lawyers in particular should be ready to challenge any output produced by a machine.

When, in Oregon in 2004, a defence attorney was told by the prosecutor that cellsite phone location data was ‘almost as accurate as DNA’, he believed him and advised his client to accept a plea bargain on a charge of manslaughter. In 2014, 9 ½  years later the client walked free on appeal.[12]

Talking of DNA, here’s another cautionary tale of emerging technology. In 2008 a British man was arrested and charged with a stranger rape from 2001. The man’s DNA had been found on a hair on the victim’s ring during a cold case review. The victim had always maintained her attacker was a tall ‘large’ man of colour. However, the man they charged was white, slim and 5’ 6” (that’s about 165cm). Now he was no angel. He had served time in prison for robbery (which was how they had his DNA), but he did not commit this offence and it was only at the court door that the prosecution offered no evidence.[13]

How did this happen? Well, perhaps, because the police and the prosecutors had blind faith in the science and looked no further for an explanation. It turned out that the hair had been discovered on the victim’s ring after the attack while she was in the same hospital where the suspect was working as a hospital porter.

Now, don’t get me wrong. There was nothing wrong with the DNA profile here. DNA has been a Godsend for criminal justice. It has helped solve many crimes by generating evidence where before none existed, but unless DNA and other forms of technology are viewed with a sceptical professional eye, there is also huge scope for miscarriages of justice.

Those of you who are trial lawyers and investigators will know what I mean if I say witnesses are unreliable. They forget, they’re biased, they tell you what they think you want to hear, they lie! But did you know there is also such a thing as ‘machine bias’? There’s an interesting monograph by Andrea Roth called ‘Machine Testimony’ published in the Yale Law Journal which goes into the topic far better than I can.[14]

In short, Roth looks at how poor analogue practice can get transferred to the operation and design of technology: for instance through the automation of procedures and processes that weren’t sufficiently robust in the first place, the use of reference databases that contain skewed samples, assumptions made by the people who write the code as well as good old fashioned defective components and weaknesses in the programming – not to mention operator or key-in error.

There is one potential risk in particular I would like to highlight here: The deliberate tampering with electronic evidence. Digital data is all just zeros and ones and there is no such thing as a 100% secure system. Digital data is volatile, easily altered and destroyed by poor handling, by automatic processes, but also on purpose by malicious actors.

At the beginning of June this year, Eurofins Forensic Services, the biggest private company dealing with forensics for the UK police  (including digital forensics) was hacked. No one knows what the hackers stole, changed or deleted.

Even more worrying, in 2017, the American National Security Agency, probably the best resourced signal’s intelligence agency in the world, was hacked by a group called the Shadow Brokers who exfiltrated a number of hacking tools.  If the NSA can be hacked, what hope is there for the rest of us?

And I am sure you’ve all seen or read about the Ibiza-Gate scandal where the Austrian Vice-Chancellor was filmed in a compromising conversation with someone pretending to be a Russian oligarch’s daughter?  It brought down the government. But how many of you have also seen the Instagram video in which Mark Zuckerberg claims all his power and success is down to a mysterious criminal organisation called SPECTRE?[15]

‘Imagine this for a second: One man, with total control of billions of people’s stolen data, all their secrets, their lives, their futures. I owe it all to Spectre. Spectre showed me that whoever controls the data, controls the future.’

It’s not real. It’s a Deep Fake and it’s not quite perfect, but given a few years, it will be and then who will be able to believe the evidence of their own eyes?

So how can you be 100% sure that the data and evidence you are gathering is complete, has the necessary integrity and can be relied upon? The short answer is you can’t, but you can reduce the risks by applying strict procedures when gathering electronic data; by adopting a common-sense approach to evaluating the data; and, by not taking emerging technology at face value.

[1] Internet Research Agency, St Petersburg aka  Glavset

[2] https://www.telegraph.co.uk/news/2019/06/25/archers-tricked-1950s-farmers-modern-methods-portraying-small/

[3] https://www.theatlantic.com/technology/archive/2017/01/did-america-need-a-social-media-president/512405/

[4] https://er.educause.edu/articles/2017/7/globalization-open-access-and-the-democratization-of-knowledge

https://en.wikipedia.org/wiki/Democratization_of_technology

[5] https://www.ted.com/talks/tristan_harris_the_manipulative_tricks_tech_companies_use_to_capture_your_attention?language=en

[6] https://www.eff.org/deeplinks/2019/04/googles-sensorvault-can-tell-police-where-youve-been

[7] https://www.echosec.net/

[8] https://geofeedia.com

[9] https://www.judiciary.uk/…/chc-speech-future-of-law-lecture-may-2018-1. pdf

[10] Opinion p12 https://www.supremecourt.gov/opinions/17pdf/16-402_h315.pdf

[11] https://encroaching.wordpress.com/2018/01/30/disclosure-and-digital-evidence/

[12] https://www.oregonlive.com/portland/2014/04/se_portland_woman_may_not_have.html; http://www.wcjn.org/Lisa_Roberts.html

[13] https://www.dailymail.co.uk/news/article-512980/DNA-farce-My-nightmare-white-man-charged-hunt-black-rapist.html

 

[14] https://www.yalelawjournal.org/article/machine-testimony

[15] https://www.theverge.com/2019/6/11/18662027/instagram-facebook-deepfake-nancy-pelosi-mark-zuckerberg  The creators of this Deep Fake are no doubt fans of Ian Fleming novels.