Category Archives: Algorithmic bias

Thursday 26 May, 2022

Posted on by

Meadow walk

Seen yesterday evening.

Quote of the Day

”Apparently Anna Wintour wants to be seen as human, and Amy Odell’s biography goes some way to helping her achieve that aim. Nearly all the photographs show her smiling, looking friendly, even girlish. And the text quite often mentions her crying. On 9 November 2016 she cried in front of her entire staff because Hillary Clinton lost the election. But then she immediately set about trying to persuade Melania Trump to do a Vogue shoot. Melania, another tough cookie, refused unless she was guaranteed the cover.”

  • Lynn Barber, reviewing Amy Odell’s biography of the heroine of The Devil Wears Pravda.

(It’s a characteristically sharp review, btw, and may even be outside the paywall.)

Musical alternative to the morning’s radio news

Haydn String Quartet No. 62, Op. 76 No. 3 “Emperor” (second movement) | Veridis Quartet

Link

Long Read of the Day

Heather Cox Richardson on the ‘right to bear arms’

In the aftermath of the Texas child massacre, this has to be today’s Long Read.

For years now, after one massacre or another, I have written some version of the same article, explaining that the nation’s current gun free-for-all is not traditional but, rather, is a symptom of the takeover of our nation by a radical extremist minority. The idea that massacres are “the price of freedom,” as right-wing personality Bill O’Reilly said in 2017 after the Mandalay Bay massacre in Las Vegas, in which a gunman killed 60 people and wounded 411 others, is new, and it is about politics, not our history.

The Second Amendment to the Constitution, on which modern-day arguments for widespread gun ownership rest, is one simple sentence: “A well regulated militia, being necessary for the security of a free state, the right of the people to keep and bear arms, shall not be infringed.” There’s not a lot to go on about what the Framers meant, although in their day, to “bear arms” meant to be part of an organized militia.

As the Tennessee Supreme Court wrote in 1840, “A man in the pursuit of deer, elk, and buffaloes might carry his rifle every day for forty years, and yet it would never be said of him that he had borne arms; much less could it be said that a private citizen bears arms because he has a dirk or pistol concealed under his clothes, or a spear in a cane.”

Today’s insistence that the Second Amendment gives individuals a broad right to own guns comes from two places…

Read on. It’s worth it. Especially the bit about Steve Kerr, the basketball coach.

And then read James Fallows’s advice below.

The rituals we are about to see.

An excerpt from James Fallows’s blog post  on the Texas shooting…

Four years ago, after the Parkland gun massacre, I wrote about the deflection steps that were likely to keep any mass killing from affecting gun policy. The sequence was this, slightly updated from what I had written after the gun massacre in Aurora, Colorado six years before.

Please use this as your guide in the days to come:

  • As news of the killing comes in, cable channels give it wall-to-wall coverage.
  • The NRA ducks its head down and goes dark for hours or days, in its Twitter and other social-media outlets.
  • Politicians who have done everything possible to oppose changes in gun laws, and who often are major recipients of NRA contributions, offer “thoughts and prayers” to the victims, say they are “deeply saddened,” praise the heroes of law enforcement and of medical treatment who have tried to limit the damage, and lament the mental-health or cultural problems that have expressed themselves via an AR-15.
  • “Thoughts and prayers” are of course admirable. But after an airline crash, politicians don’t stop with “thoughts and prayers” for the victims; they want to get to the bottom of the cause. After a fatal fire, after a botched response to a hurricane, after a food-poisoning or product-safety failure or a nursing-home abuse scandal, “thoughts and prayers” are the beginning of the public response but not the end. After a shooting they are both.
  • These same politicians say that the aftermath of a shooting is “not the right time” to “politicize” the tragedy by talking about gun laws or asking why only in America do massacres happen week after week after week. The right time to discuss these policies is “never.”
  • The news moves on; everyone forgets except the families and communities that are forever changed.
  • The next shooting comes, “thoughts and prayers” are offered, and the cycle resumes.

Welcome to democracy, American-style.

Ukraine is using electric bikes to move Anti-Tank weapons around

As regular readers may know, I’ve been a fan of e-bikes for years. But I never anticipated how they might be useful in wartime.

Fascinating Motherboard report.

On Telegram last week, pictures surfaced of the Delfast branded bikes that had been modified to carry massive anti-tank weapons. The two photos showed the e-bike modified with a crate on the back and a huge missile launcher poking from the back.

The e-bikes are used for transporting the launchers; the anti-tank weapons aren’t fired from the back of the bikes. The quiet design and fast speed—a Delfast can reach speeds up to 50 mph—allow the bikes to move NLAWS into position and quickly flee once fired…

Thanks to Charles Arthur for spotting it.

My commonplace booklet

The annual Farne Islands puffin count returns (in pictures)

Link

Magical!

This Blog is also available as a daily email. If you think that might suit you better, why not subscribe? One email a day, Monday through Friday, delivered to your inbox. It’s free, and you can always unsubscribe if you conclude your inbox is full enough already!

Thursday 10 September, 2020

Posted on by

Quote of the Day

“The battle for the mind of Ronald Reagan was like the trench warfare of World War I. Never have so many fought so hard for such barren terrain.”

  • Peggy Noonan, speechwriter for Reagan 1984-9

Musical alternative to the morning’s radio news

Handel: Handel – Let The Bright Seraphim – Rowan Pierce and David Blackadder (5 minutes)

Link

Nobody sleeps at the back when this is on.

The power of a photograph to tell a story

Photograph by Fergus McGinn

Nice NYT article on the Coronavirus lockdown in Ireland.

A pint of Guinness. A half-eaten meal. And an alarm clock.

All are laid out on a table in front of an older man as he gazes out the window of a pub in Galway, Ireland, in a photo that has come to capture the nation’s coming to terms with coronavirus regulations.

The image has come to symbolize different things for different people: Some have used it to criticize the government’s restrictions on pubs, while others have applauded the man’s commitment to regulations on dining out.

Pubs that serve food have been allowed to open since the end of June. But new restrictions require that customers have a “substantial meal” costing at least 9 euros (about $10.60) if they also purchase alcohol. The rules also require patrons to leave within an hour and 45 minutes — hence the timer.

Like many countries across Europe, Ireland has seen a spike in coronavirus cases in recent weeks, with daily cases moving from the single digits during a lull in new infections in June and July to 307 new cases announced on Tuesday.

The photograph was taken by Fergus McGinn, the owner of the pub. His intention, he said, was to show a man enjoying the simple pleasure of a meal and a drink but he also hopes the picture will make people aware of the role the local pub plays for isolated members of the community in Ireland.

“Taking that away from people, that social outlet for that generation, it could be detrimental and savage on their mental health,” Mr. McGinn said.

Particularly in Ireland’s rural communities, pubs serve as a central place to connect socially, even for those who aren’t big drinkers. Yet pubs that don’t serve food have been closed since the lockdown began in March, though this week the government agreed to reopen them on September 21st.

And the significance of the timer? The government rules also require pub customers to leave within an hour and 45 minutes.

Diana Rigg RIP

A wonderful actress has passed away. There’s a nice BBC obituary up today, but there will be lots more. As the BBC obit put it, “She excelled at playing sharp-witted female characters who carried steel fists in velvet gloves.” Spot on.

As a former TV critic, though, this book is what I will remember her for.

It’s a compendium of the most wicked things theatrical folk can say about one another. I found it an indispensable fountain of ideas when I was writing about a programme or a performer that I particularly disliked.

But the interesting thing is that Rigg also included in it some of the rude things critics said about her!

For example, she reprints John Simon’s crack in New York Magazine about her nude scene in Abelard and Heloise in May 1970: “Diana Rigg is built like a brick mausoleum with insufficient flying buttresses”.

“I remember”, Rigg wrote,

“making my way to the theatre the following day, darting from doorway to doorway and praying I wouldn’t meet anyone I knew. The cast behaved with supreme tact and pretended they hadn’t read the review.”

btw: I think Simon needed to have an eye-test. Rigg was a truly beautiful woman. But she disliked doing nude scenes. “I come from Yorkshire”, she said once, “and no-one from Yorkshire takes their clothes off except on a Friday night”.

May she rest in peace.

What Bob Woodward knew (but didn’t tell — until now)

The veteran Washington reporter Bob Woodward (he of Watergate fame) has made a small industry out of conducting long interviews of US presidents and their courts when he then turns into what are (IMHO) surprisingly dull books. He’s just published the latest tombstone in this series — on the first Trump presidency.

For a less jaundiced perspective, the Columbia Journalism Review has a more detached view of the volume by Pete Vernon.

In books about presidents from Nixon to Obama, Woodward has employed a similar approach, conducting exhaustive interviews on background and using the information he gathers to write from an omniscient perspective. Woodward and Carl Bernstein, his colleague at The Washington Post, used the most famous anonymous source in American history—FBI Associate Director Mark Felt a.k.a. “Deep Throat”—to expose the cover-up behind the Watergate burglary that unraveled Nixon’s presidency. This week, Woodward told Michael Schmidt of The New York Times that “you won’t get the straight story from someone if you do it on the record. You will get a press release version of events.” But as Axios’s Jonathan Swan, one of the current masters of Washington intrigue, noted, sources “also lie on background. A lot.”

And no group of officials in recent memory has proved as willing to bend the truth as those in the Trump administration. The recent controversy over Steve Bannon’s invitation (later rescinded) to appear at The New Yorker Festival led The Washington Post’s Margaret Sullivan to declare, “Enough, already, with anything Steve Bannon has to say.” When Kellyanne Conway appears on CNN, critics question why the network gives a platform to the official who coined “alternative facts.” Yet for Woodward, reliance on the same sources is received differently: If it’s not OK for David Remnick to talk to Bannon in front of an audience, why is it OK for Woodward to use him, quite obviously, as a key source in the book?

Woodward’s approach hasn’t changed; the climate in which his sources are viewed has. Every administration is filled with people who have an agenda, who want to spin events in their favor, but the lines of credibility have shifted. In taking on the Trump presidency as his topic, Woodward is left to assemble a reliable book from unreliable sources….

Already, there are lots of controversies blowing up from specific parts of Woodward’s account. But one in particular has caught my eye. Here’s how Politico puts it:

President Donald Trump acknowledged the “deadly” nature of the coronavirus earlier this year in a series of recorded interviews with The Washington Post’s Bob Woodward, even as Trump publicly sought to dismiss the disease’s threat to Americans.

Recounting a conversation with Chinese President Xi Jinping, Trump told Woodward on Feb. 7 that the coronavirus is “more deadly than your, you know, your — even your strenuous flus.”

“This is more deadly,” he said. “This is five per — you know, this is 5 percent versus 1 percent and less than 1 percent, you know. So, this is deadly stuff.”

A few days later Trump was out in public basically claiming that this ‘Chinese flu’ was not such a big deal and he had it under control, or words to that effect.

There has been a huge hoo-hah about this, particularly the fact that Woodward knew that Trump had been lying through his teeth about the virus, but had decided not to tell anyone, let along the American people. Here’s a sample of the resulting indignation — Bess Levin in Vanity Fair:

When news broke on Wednesday that venerated reporter Bob Woodward’s forthcoming book had Donald Trump on record saying that he purposely downplayed the threat of COVID-19, despite knowing that it was “deadly stuff,” the outrage was deafening. The fury and disgust initially centered around the president’s decision to lie to the public about the fatal virus, proclaiming that it was nothing to worry about while knowing full well that it was. Later the conversation turned to Woodward’s decision to withhold crucial information, with some arguing that it was a dereliction of his duty as a journalist not to come forward and tell people, in real time, that the president was lying to their faces, as he instead saved it for his book. “There is no ethical or moral defense of Woodward’s decision to not publish these tapes as soon as they were made,” former BuzzFeed News Washington bureau chief John Stanton tweeted. “If there was any chance it could save a single life, he was obligated to do so. Bob Woodward put making money over his moral and professional duty.”

I don’t have a dog in this fight, but I was struck by Woodward’s reported response to this charge:

Speaking to the Washington Post’s Margaret Sullivan, Woodward said he waited it out because (1) Trump is a habitual liar, and he hadn’t yet done the reporting necessary to know if the president was actually telling the truth in this instance, and (2) He wanted to put the statements into context and publish them closer to the election so that people didn’t forget about them on their way to the polls:

Who knows? That might have been a shrewd judgement.

This blog is also available as a daily email. If you think this might suit you better, why not subscribe. One email a day, delivered to your inbox at 7am UK time. It’s free, and there’s a one-click unsubscribe button if you decide that your inbox is crowded enough already!

Thursday 25 June, 2020

Posted on by

New customers fill seats at Barcelona opera house concert

To mark the re-opening of Barcelona’s Gran Teatre del Liceu opera house, the UceLi Quartet played a livestreamed performance of Puccini’s I Crisantemi (Chrysanthemums).

Who (or what) was in the audience?

Answer

Can you judge a book by its (back) cover?

Well, even if you can, it makes cover-art designers (justifiably) cross.

Waterstones, the (excellent) bookselling chain, has offered its apologies to book designers after some newly reopened branches began displaying books back to front so browsers could read the blurb without picking them up.

It was understandable but slightly “heartbreaking”, said designer Anna Morrison, who mainly designs covers for literary fiction, said she could see why it was happening, but it was still “a little sad”.

“There is a real art to a book cover. It can be a real labour of love and it is a bit disappointing to think our work is being turned round.”

She’s right. One of the joys of going into a bookshop is the blaze of colour and artwork on book covers that confronts you.

Link

Facebook faces trust crisis as ad boycott grows

It’s got the trust crisis, for sure. But so what?

This from Axios

After a handful of outdoor companies like North Face, REI and Patagonia said they would stop advertising on Facebook and Instagram last week, several other advertisers have joined the movement, including Ben & Jerry’s, Eileen Fisher, Eddie Bauer, Magnolia Pictures, Upwork, HigherRing, Dashlane, TalkSpace and Arc’teryx.

Heavyweights in the ad industry have also begun pressing marketers to pull their dollars.

On Tuesday, Marc Pritchard, chief brand officer at Procter & Gamble, one of the largest advertisers in the country, threatened to pull spending if platforms didn’t take “appropriate systemic action” to address hate speech.

In an email to clients obtained by the Wall Street Journal last Friday, 360i, a digital-ad agency owned by global ad holding group Dentsu Group Inc., urged its clients to support the ad boycott being advocated by civil rights groups.

I’m sorry to say this, but it looks to me just like virtue-signalling. Just like all the sudden corporate support for “our brilliant NHS” when the Coronavirus panic started in the UK. Facebook’s targeted advertising system is just too useful to companies to be dropped.

Wrongfully Accused by an Algorithm

This seems to be the first case of its kind, but it’s the canary in the mine as far as those of us who regard facial recognition technology as toxic.

On a Thursday afternoon in January, Robert Julian-Borchak Williams was in his office at an automotive supply company when he got a call from the Detroit Police Department telling him to come to the station to be arrested. He thought at first that it was a prank.

An hour later, when he pulled into his driveway in a quiet subdivision in Farmington Hills, Mich., a police car pulled up behind, blocking him in. Two officers got out and handcuffed Mr. Williams on his front lawn, in front of his wife and two young daughters, who were distraught. The police wouldn’t say why he was being arrested, only showing him a piece of paper with his photo and the words “felony warrant” and “larceny.”

His wife, Melissa, asked where he was being taken. “Google it,” she recalls an officer replying.

The police drove Mr. Williams to a detention center. He had his mug shot, fingerprints and DNA taken, and was held overnight. Around noon on Friday, two detectives took him to an interrogation room and placed three pieces of paper on the table, face down.

“When’s the last time you went to a Shinola store?” one of the detectives asked, in Mr. Williams’s recollection. Shinola is an upscale boutique that sells watches, bicycles and leather goods in the trendy Midtown neighborhood of Detroit. Mr. Williams said he and his wife had checked it out when the store first opened in 2014.

The detective turned over the first piece of paper. It was a still image from a surveillance video, showing a heavyset man, dressed in black and wearing a red St. Louis Cardinals cap, standing in front of a watch display. Five timepieces, worth $3,800, were shoplifted.

“Is this you?” asked the detective.

The second piece of paper was a close-up. The photo was blurry, but it was clearly not Mr. Williams. He picked up the image and held it next to his face.

“No, this is not me,” Mr. Williams said. “You think all black men look alike?”

Mr. Williams knew that he had not committed the crime in question. What he could not have known, as he sat in the interrogation room, is that his case may be the first known account of an American being wrongfully arrested based on a flawed match from a facial recognition algorithm, according to experts on technology and the law.

Mr Williams had a cast-iron alibi, but the Detroit police couldn’t be bothered to check

He has since figured out what he was doing the evening the shoplifting occurred. He was driving home from work, and had posted a video to his private Instagram because a song he loved came on — 1983’s “We Are One,” by Maze and Frankie Beverly. The lyrics go:

I can’t understand
Why we treat each other in this way
Taking up time
With the silly silly games we play

Imagine a world where this stuff is everywhere, where you’re always in a police line-up.

The history of inquiries into race riots

A sobering (and depressing) piece by the Harvard historian Jill Lepore in the New Yorker.

TL;DR? (In case you’re busy, here’s the gist.)

In a 1977 study, “Commission Politics: The Processing of Racial Crisis in America,” Michael Lipsky and David J. Olson reported that, between 1917 and 1943, at least twenty-one commissions were appointed to investigate race riots, and, however sincerely their members might have been interested in structural change, none of the commissions led to any. The point of a race-riot commission, Lipsky and Olson argue, is for the government that appoints it to appear to be doing something, while actually doing nothing.

It’s the old, old story. What’s the betting the same thing will happen with Boris Johnson’s “cross-government inquiry into all aspects of racial inequality in the UK”?

Lepore’s is a fine piece, well worth reading in full. Thanks to David Vincent for alerting me to it.

Segway, the most hyped invention since the Macintosh, ends production

Very good report on what once looked like a great idea, but one that never caught on. Segways were very useful for TV cameramen and camerawomen covering golf tournaments, though.

My main regret is that I never managed to try one.

Quarantine diary — Day 96

Link

This blog is also available as a daily email. If you think this might suit you better, why not subscribe? One email a day, delivered to your inbox at 7am UK time. It’s free, and there’s a one-click unsubscribe if you decide that your inbox is full enough already!

Monday 17 February, 2020

Posted on by

Quote of the Day

In this election there are two sides. One side believes in the rule of law, the other doesn’t. Everything else, to be settled later, once the rule-of-law is re-established.

  • Dave Winer

_____________________________ 

My review of Andrew Marantz’s new book — Antisocial

On today’s Guardian. It’s a sobering read.

There has always been a dark undercurrent of white supremacism in some sectors of American culture. It was kept from public view for decades by the editorial gatekeepers of the old media ecosystem. But once the internet arrived, a sophisticated online culture of conspiracy theorists, racists and other malign discontents thrived in cyberspace. But it stayed below the radar until a fully paid-up conspiracy theorist won the Republican nomination. Trump’s candidacy and campaign had the effect of “mainstreaming” that which had previously been largely hidden from view. At which point, the innocent public began to see and experience what Marantz has closely observed, namely the remarkable capabilities of extremist “edgelords” to weaponise YouTube, Twitter and Facebook for destructive purposes.

One of the most depressing things about 2016 was the apparent inability of American journalism to deal with this pollution of the public sphere. In part, this was because they were crippled by their professional standards. It’s not always possible to be even-handed and honest. “The plain fact,” writes Marantz at one point, “was that the alt-right was a racist movement full of creeps and liars. If a newspaper’s house style didn’t allow its reporters to say so, then the house style was preventing its reporters from telling the truth.” Trump’s mastery of Twitter led the news agenda every day, faithfully followed by mainstream media, like beagles following a live trail. And his use of the “fake news” metaphor was masterly: a reminder of why, as Marantz points out, Lügenpresse – “lying press” – was also a favourite epithet of Joseph Goebbels.

Frank Ramsey

Frank Ramsey was a legend in Cambridge as one of the brightest young men of his time. He died tragically young (he was 26) in 1930, from an infection acquired from swimming in the river Cam. Now there’s a new biography of him by Cheryl Misak. Here’s part of her blurb about him:

The economist John Maynard Keynes identified Ramsey as a major talent when he was a mathematics student at Cambridge in the early 1920s. During his undergraduate days, Ramsey demolished Keynes’ theory of probability and C.H. Douglas’s social credit theory; made a valiant attempt at repairing Bertrand Russell’s Principia Mathematica; and translated Ludwig Wittgenstein’s Tractatus Logico-Philosophicus, and wrote a critique of the latter alongside a critical notice of it that still stands as one of the most challenging commentaries of that difficult and influential book.

Keynes, in an impressive show of administrative skill and sleight of hand, made the 21-year-old Ramsey a fellow of King’s College at a time when only someone who had studied there could be a fellow. (Ramsey had done his degree at Trinity).

Ramsey validated Keynes’ judgment. In 1926 he was the first to figure out how to define probability subjectively and invented the expected utility that underpins much of contemporary economics.

I’d never heard of Ramsey until I came on Keynes’s essay on him in his wonderful collection, Essays in Biography, published in 1933. (One of my favourite books, btw.) Given that Keynes himself was ferociously bright, the fact that he had such a high opinion of Ramsey was what made me sit up. Here’s an extract that conveys that:

Seeing all of Frank Ramsey’s logical essays published together, we can perceive quite clearly the direction which is mind was taking. It is a remarkable example of how the young can take up the story at the point to which the previous generation had brought it a little out of breath, and then proceed forward without taking more than about a week thoroughly to digest everything which had been done up to date, and to understand with apparent ease stuff which to anyone even 10 years older seemed hopelessly difficult. One almost has to believe that Ramsay in his nursery year near Magdalene1 was unconsciously absorbing from 1903 to 1914 everything which anyone may have been saying or writing from Trinity.

(Among the people in Trinity College at the time were Bertrand Russell, A.N. Whitehead and Ludwig Wittgenstein.)

The hacking of Jeff Bezos’s phone

Interesting (but — according to other forensic experts — incomplete) technical report into his the Amazon boss’s smartphone was hacked, presumably by someone working for the Saudi Crown Prince.

_____________________________________________ 

Where people have faith in their elections

The U.S. public’s confidence in elections is one of the worst of any wealthy democracy, according to a recently published Gallup poll. It found that a mere 40 percent of Americans have confidence in the honesty of their elections. As low as that figure is, distrust of elections is nothing new for the U.S. public.

The research found that a majority of Americans have had no confidence in the honesty of elections every year since 2012 with the share trusting the process at the ballot box sinking as low as 30 percent during the 2016 presidential campaign. Gallup stated that its 2019 data came at a time when eight U.S. intelligence agencies confirmed allegations of foreign interference in the 2016 presidential election and identified attempts to engage in similar activities during the midterms in 2018.

This chart shows how the U.S. compares to other developed OECD nations with the highest confidence scores recorded across Northern Europe and Finland, Norway and Sweden best-ranked.

Source

_________________________________________ 

David Spiegelhalter: Should We Trust Algorithms?

As the philosopher Onora O’Neill has said (O’Neill, 2013), organizations should not try to be trusted; rather they should aim to demonstrate trustworthiness, which requires honesty, competence, and reliability. This simple but powerful idea has been very influential: the revised Code of Practice for official statistics in the United Kingdom puts Trustworthiness as its first “pillar” (UK Statistics Authority, 2018).

It seems reasonable that, when confronted by an algorithm, we should expect trustworthy claims both:

  • about the system — what the developers say it can do, and how it has been evaluated, and

  • by the system — what it says about a specific case.

Terrific article

  1. Ramsey’s father was Master of Magdalene. ↩

Biased machines may be easier to fix than biased humans

Posted on by

This morning’s Observer column:

One of the things that really annoys AI researchers is how supposedly “intelligent” machines are judged by much higher standards than are humans. Take self-driving cars, they say. So far they’ve driven millions of miles with very few accidents, a tiny number of them fatal. Yet whenever an autonomous vehicle kills someone there’s a huge hoo-ha, while every year in the US nearly 40,000 people die in crashes involving conventional vehicles.

Likewise, the AI evangelists complain, everybody and his dog (this columnist included) is up in arms about algorithmic bias: the way in which automated decision-making systems embody the racial, gender and other prejudices implicit in the data sets on which they were trained. And yet society is apparently content to endure the astonishing irrationality and capriciousness of much human decision-making.

If you are a prisoner applying for parole in some jurisdictions, for example, you had better hope that the (human) judge has just eaten when your case comes up…

Read on

Bias in machine learning

Posted on by

Nice example from Daphne Keller of Google:

Another notion of bias, one that is highly relevant to my work, are cases in which an algorithm is latching onto something that is meaningless and could potentially give you very poor results. For example, imagine that you’re trying to predict fractures from X-ray images in data from multiple hospitals. If you’re not careful, the algorithm will learn to recognize which hospital generated the image. Some X-ray machines have different characteristics in the image they produce than other machines, and some hospitals have a much larger percentage of fractures than others. And so, you could actually learn to predict fractures pretty well on the data set that you were given simply by recognizing which hospital did the scan, without actually ever looking at the bone. The algorithm is doing something that appears to be good but is actually doing it for the wrong reasons. The causes are the same in the sense that these are all about how the algorithm latches onto things that it shouldn’t latch onto in making its prediction.

Addressing bias in algorithms is crucial, especially in domains like healthcare where accurate predictions are vital. One effective approach to recognizing and mitigating biases is to rigorously test the algorithm in scenarios similar to its real-world applications. Suppose a machine-learning algorithm is trained on data from specific hospitals to predict fractures from X-ray images. In this case, it may appropriately incorporate prior knowledge about patient populations in those hospitals, resulting in reliable predictions within that context. However, the challenge arises when the algorithm is intended to be used in different hospitals not present in the initial training data set. To avoid unintended biases, a robust evaluation process is essential, and the use of a mobile learning management system can prove beneficial. Such a system enables continuous monitoring and assessment of the algorithm’s performance across various hospital settings, ensuring it doesn’t latch onto irrelevant factors and provides accurate predictions based on genuine medical insights.

To recognize and address these situations, you have to make sure that you test the algorithm in a regime that is similar to how it will be used in the real world. So, if your machine-learning algorithm is one that is trained on the data from a given set of hospitals, and you will only use it in those same set of hospitals, then latching onto which hospital did the scan could well be a reasonable approach. It’s effectively letting the algorithm incorporate prior knowledge about the patient population in different hospitals. The problem really arises if you’re going to use that algorithm in the context of another hospital that wasn’t in your data set to begin with. Then, you’re asking the algorithm to use these biases that it learned on the hospitals that it trained on, on a hospital where the biases might be completely wrong.