Terrorist Watch Lists: 'Excessive, Inaccurate, Ineffective'
Surprisingly, this is news to some people:
The numbers "suggest a staggeringly high rate of false positives with respect to the identification of supposed terrorists. This really confirms the long-standing fear that this list is inaccurate and ultimately ineffective as an anti-terror tool."
The administration needs to be introduced to the Base Rate Fallacy. Bruce Schneier has more to say on this subject.
The numbers "suggest a staggeringly high rate of false positives with respect to the identification of supposed terrorists. This really confirms the long-standing fear that this list is inaccurate and ultimately ineffective as an anti-terror tool."
The administration needs to be introduced to the Base Rate Fallacy. Bruce Schneier has more to say on this subject.
posted by Management at 3:13 PM
2 Comments:
Terror Suspect List Yields Few Arrests
20,000 Detentions in '06 Rile Critics
By Ellen Nakashima
Washington Post Staff Writer
Saturday, August 25, 2007; A01
The government's terrorist screening database flagged Americans and foreigners as suspected terrorists almost 20,000 times last year. But only a small fraction of those questioned were arrested or denied entry into the United States, raising concerns among critics about privacy and the list's effectiveness.
A range of state, local and federal agencies as well as U.S. embassies overseas rely on the database to pinpoint terrorism suspects, who can be identified at borders or even during routine traffic stops. The database consolidates a dozen government watch lists, as well as a growing amount of information from various sources, including airline passenger data. The government said it was planning to expand the data-sharing to private-sector groups with a "substantial bearing on homeland security," though officials would not be more specific.
Few specifics are known about how the system operates, how many people are detained or turned back from borders, or the criteria used to identify suspects. The government will not discuss cases, nor will it confirm whether an individual's name is on its list.
Slightly more than half of the 20,000 encounters last year were logged by Customs and Border Protection officers, who turned back or handed over to authorities 550 people, most of them foreigners, Customs officials said. FBI and other officials said that they could not provide data on the number of people arrested or denied entry for the other half of the database hits. FBI officials indicated that the number of arrests was small.
The government says the database is a powerful tool for identifying and tracking suspected terrorists and for sharing intelligence, and that its purpose is not necessarily to make arrests. But the new details about the numbers, disclosed in an FBI budget document and in interviews, raise questions about the database's effectiveness and its impact on privacy, critics said. They argued that the number of hits relative to arrests was alarmingly high and indicated that the threshold for including someone on a watch list was too low, potentially violating thousands of Americans' civil liberties when they are stopped.
David Sobel, senior counsel with the Electronic Frontier Foundation, a privacy organization, said the numbers "suggest a staggeringly high rate of false positives with respect to the identification of supposed terrorists." He added that "this really confirms the long-standing fear that this list is inaccurate and ultimately ineffective as an anti-terrorism tool."
Jayson P. Ahern, deputy commissioner for U.S. Customs and Border Protection, said focusing on arrests misses "a much larger universe" of suspicious U.S. citizens.
"There are many potentially dangerous individuals who fly beneath the radar of enforceable actions and who are every bit as sinister as those we intercept," he said.
The database is maintained by the Terrorist Screening Center, a joint operation between the FBI and the Department of Homeland Security. Rick Kopel, the TSC's deputy director, called it "one of the best things the government has been able to accomplish since 9/11."
The government said private-sector entities with a "substantial bearing on homeland security" could also gain access to the data, which is kept for 99 years, according to a notice in the Federal Register this week.
The watch list includes information from the Transportation Security Administration's air passenger "no-fly" list, the State Department's Consular Lookout and Support System list and the FBI's Violent Gang and Terrorist Organizations File.
To be included in the database, a person must be "a known or suspected terrorist such as those who finance terrorist activities, are known members of a terrorist organizations, terrorist operatives, or someone that provides material support to a terrorist or terrorist organization," said Michelle Petrovich, a spokesman for the Terrorist Screening Center. According to the Justice Department's inspector general, the database contained at least 235,000 records as of last fall.
Using the database, U.S. and international authorities prevented "numerous attempts" at entry into the United States by an Egyptian citizen, Omar Ahmed Ali, who went on in 2005 to commit a suicide bombing in Qatar that killed one British citizen and injured 12, Petrovich said.
Many U.S. citizens are stopped, questioned and, if no arrest warrant is pending, released. They are not told their watch-list status. To do so, the government says, could tip off suspects that they are likely to be questioned or detained.
Some travelers who are repeatedly stopped can only speculate that they are on the watch list.
Abe Dabdoub, 39, and his wife, both U.S. citizens, live in a Cleveland suburb. He said he has been detained 21 times at Michigan's border with Canada since last August. Dabdoub, who works for an electronics manufacturing company, said he has even begun to keep a spreadsheet. The first four times, he said, he was handcuffed. Once, his wife had to plead with the agents not to handcuff him in front of their 5- and 7-year-old boys, he said. The agents know him so well by now that they call him by his first name. Every time he asks them why he is being stopped, Customs officers tell him, "We can't tell you, for national security reasons," he said.
Customs officials declined to comment on his case.
Agencies nominate names to the list based on rigorous, classified criteria, Kopel said. The TSC has created a redress unit that ensures that watch-list and source information is accurate, officials said. Since 2005, the unit has resolved more than 90 percent of the several hundred complaints it has received, including by deleting names or adjusting data.
Each watch-list hit is a "positive encounter" -- what the government says is a conclusive match against the database -- by a customs officer or other official with an American or foreigner. U.S. citizens, if there is no arrest warrant, cannot be denied entry. About half of the encounters take place at land borders, airports or seaports. Other travelers are flagged at consular offices or by state and local police.
The number of hits has surged since the second half of fiscal 2004, when the database was created. That year, the FBI reported 5,396 encounters, with some people having multiple encounters. In 2005, 15,730 hits were logged. Next year, the FBI projects 22,400 hits.
FBI officials said the rising numbers result from wider information-sharing among international, federal, state and local authorities.
"A lot of times it's not to our advantage to make an arrest," FBI spokesman Paul Bresson said. "We don't want the subject to know what we know. It doesn't mean we're not paying attention. On the contrary, it shows that we're being very proactive in trying to identify threats."
But Steven Aftergood, director of the Federation of American Scientists' Project on Government Secrecy, said growing use of this database magnifies the consequences of errors that are entered into it.
"There needs to be a reliable way to correct bad information and protect the innocent," he said.
The government's system casts too broad a net, and its definition of who should be watch-listed is too broad, said Harvey Grossman, legal director of the American Civil Liberties Union of Illinois, which has filed a class-action lawsuit against the government on behalf of 10 Muslim Americans who allege they were detained and mistreated after being placed on a watch list without grounds. People with only distant casual contact with a suspect might be listed, he said. "What you eventually get is a worthless list of people."
In rare cases, citizens have discovered they are on the watch list.
Francisco "Kiko" Martinez, a Colorado lawyer and civil-rights activist, said he was detained twice in recent years by police officers who pulled him over on traffic stops and held him in one case more than three hours, and in another, in handcuffs. Through legal proceedings, Martinez obtained police reports that revealed his watch-list status.
"A driver's license check revealed [Martinez] as a possible individual having ties with terrorism," a state trooper wrote after a 2004 stop near Chicago, according to one report.
Last year, Martinez sued the federal government, claiming that he was unlawfully detained and that he was included on a watch list as a result of his political activities.
Last month, he won a $106,500 settlement from federal, state and tribal authorities. Though the settlement did not address any of the underlying constitutional claims, Martinez asserted that it "shows that I shouldn't have been on this terrorism watch list in the first place" and that "the government is misusing this so-called war against terrorism to target its domestic political opponents."
Justice Department spokesman Charles Miller said the department declined to comment on the case.
Jim McMahon, chief of staff for the International Association of Chiefs of Police, which represents 18,000 state and local police agencies across the country, said the database helps police officers "make a better judgment" about whether to detain a person. One of the 9/11 hijackers, Ziad Samir Jarrah, was ticketed for going 95 miles per hour on Interstate 95 in Maryland two days before the attacks, he said. "Today, chances are he would have been on the list," he said.
How To Not Catch Terrorists
By Bruce Schneier
Forbes
March 26, 2007
Data mining for terrorists: It's an idea that just won't die. But it won't find any terrorists, it puts us at greater risk of crimes like identity theft, and it gives the police far too much power in a free society.
The first massive government program to collect dossiers on every American for data mining purposes was called Total Information Awareness. The public found the idea so abhorrent, and objected so forcefully, that Congress killed funding for the program in September 2003. But data mining is like a hydra--chop one head off, two more grow in its place. In May 2004, the General Accounting Office published a report that listed 122 different federal government data mining programs that used people's personal information. That didn't include classified military programs like Tangram, or state-run programs like MATRIX.
Now TIA is back with yet another name: Analysis, Dissemination, Visualization, Insight and Semantic Enhancement, or ADVISE. "It's an experiment to see how you can better analyze data that you already have, that you've already legally collected, to see if you can understand it, sort it and make use of it more readily than simply doing it manually," Homeland Security Chief Michael Chertoff told the Associated Press this month.
The names change, but the basic idea remains the same: suck up as much data as possible about everyone, sift through it with massive computers, and investigate patterns that might indicate terrorist plots. It's a compelling idea, but it's wrong. We're not going to find terrorist plots through data mining, and we're going to waste valuable resources chasing down false alarms.
Used properly, data mining is a great tool. As a result of data mining, AT&T reduces the costs of cell phone fraud, Amazon.com shows me books I might want to buy, and Google shows me advertising I'm more likely to be interested in. But it only works when there's (1) a reasonable percentage of attacks per year, (2) a well-defined profile to search for, and (3) and a low cost of false alarms.
Look at one of data mining's success stories: credit card fraud. All credit card companies data mine their transaction databases, looking for spending patterns that indicate a stolen card. About 1% of cards are stolen and fraudulently used each year in the U.S.; that's enough of a population to make searching for them effective. There are also common fraud patterns that can be computed from that data, and they're easy to search for. Additionally, the cost of a false alarm is only a phone call to the cardholder asking him to verify a couple of purchases. Cardholders don't even resent these phone calls--as long as they're not too frequent--so the cost is just a few minutes of operator time.
Terrorist plots are different. First, attacks are very rare. This means that even very accurate systems will be so flooded with false alarms that they will be useless: millions of false alarms for every one real attack, even assuming unrealistically accurate systems.
Let's look at some numbers. Assume an unrealistically optimistic system with a 1-in-100 false positive rate (99% accurate), and a 1-in-1,000 false negative rate (99.9% accurate). That is, while it will mistakenly classify something innocent as a terrorist plot one in a hundred times, it will only miss a real terrorist plot one in a thousand times. Assume one billion possible "plots" to sift through per year, about four per American citizen, and that there is one actual terrorist plot per year.
Even this unrealistically accurate system will generate 10 million false alarms for every real terrorist plot it uncovers. Every day of every year, the police will have to investigate 270,000 potential plots in order to find the one real terrorist plot per month.
In statistics, it's called the "base rate fallacy," and it applies in other domains as well. For example, even highly accurate medical tests are useless as diagnostic tools if the incidence of the disease is rare in the general population. Terrorist attacks are also rare, so any "test" is going to result in an endless stream of false alarms.
Second, there is no well defined terrorist profile. In hindsight, it was easy to connect the Sept. 11, 2001 dots and point to the warning signs, but it's much harder to do so before the fact. Certainly, there are common warning signs that many terrorist plots share, but they share them with non-terrorist events as well. We live in a "six degrees of separation" world, where everyone is connected. Add in the problems of sleeper cells, loner terrorists like the Unabomber, and billions of perfectly innocent plots like surprise birthday parties and corporate takeovers, and you have an impossible problem.
And third, the cost of these false alarms is enormous. It's not just the cost of the FBI agents running around chasing dead-end leads instead of doing things that might actually make us safer, but also the cost in civil liberties. The fundamental freedoms that make our country the envy of the world are valuable, and not something that we should throw away lightly.
There is something un-American about a government program that uses secret criteria to collect dossiers on innocent people and shares that information with various agencies, all without any oversight. It's the sort of thing you'd have expected from the former Soviet Union or East Germany, or modern-day China.
Finding terrorism plots is not a problem that lends itself to data mining. It's a needle-in-a-haystack problem, and throwing more hay on the pile doesn't make the problem any easier. Real security comes from old-fashioned investigative work: putting people in charge of investigating potential plots and letting them direct the computers, instead of putting the computers in charge and letting them decide who should be investigated. It's what caught the London liquid bombers last summer, and it's our best hope for our own security in the future.
Post a Comment
<< Home