Criminal gangs and Terrorism

In late November 2019, federal, state and local law enforcement led a series of arrests and seizures in Georgia. According to a typical news account:

A multi-agency investigation in north Georgia has led to several federal charges and 13 arrests across two states in connection with a major criminal organization…two search warrants were executed…led to arrests of people from both Georgia and Tennessee. Charges included Racketeering Influence Corrupt Organization (RICO) Act violations, violation of the Street Gang Terrorism and Prevention Act, firearms possession counts and numerous counts tied to illegal drugs.

Among the arrests were people that the ARDEO statement suggests were members of or associated with the Aryan Brotherhood, a criminal organization “that is involved in violence, firearms, and transportation of illegal narcotics.”

This case highlights an inclusion challenge for tPP. News accounts consistently refers to the Aryan Brotherhood as a CRIMINAL organization. However, the charge rhetorically labels  gang activity TERRORISM, and the Brotherhood is an organization that can produce organized political violence and has a socio-political agenda.

Do we include the case based on the group, since the Aryan Brotherhood has been known organizationally to support political violence? Do we include because state speech labels the crime as terroristic? Do we exclude because  the defendants are labeled as criminals and not social or political actors?
To make the question more complex, in recent weeks, President Trump has threatened to begin designating Mexican drugs cartels as foreign terrorist organizations (FTOs). According to the New York Times:

President Trump revived this idea [of narco-terrorism] in an interview last week with the former Fox News host Bill O’Reilly. “Are you going to designate those cartels in Mexico as terror groups and start hitting them with drones and things like that?” Mr. O’Reilly asked. “I don’t want to say what I’m going to do, but they will be designated,” Mr. Trump responded. “I have been working on that for the last 90 days.” He was referring to putting some of the cartels on the State Department’s list of foreign terrorist organizations.

If this were to occur, it would have major impacts on (not only tPP but also) US domestic policing policy, national security policy, and questions of legality and counter-terrorism measures.

Input and Output: Opportunities Afforded to tPP Team Members


This continues our series of student reflections and analysis authored by our research team.


Input and Output: Opportunities Afforded to tPP Team Members

Izzy Bielamowicz

Over the course of my time with tPP, I have been offered numerous opportunities by our executive director Dr. Michael Loadenthal and by the project itself. These opportunities have led to invaluable experiences for my professional career development. Beyond the direct impact my work with tPP has on my resume, the tangential exposure which I have garnered as a result of my involvement with the project is unequivocal. Of course, these opportunities are not handed out for free. With meticulous and quality work, I and many other tPP members have dedicated themselves to the project, leveraging a hard work ethic to further personal development. tPP is arguably the most profitable experience I have listed on my resume in terms of both personal and career development. tPP has taught me how to be an effective team member in pairs, small groups, and large groups; has led to infinite leadership opportunities; and has even given me the chance to submit my research on the dataset for publication.

In terms of teamwork, tPP has a complex hierarchy of members in which pairings and groupings evolve. At the most basic structural level, each semester has given me a new, consistent partner for coding purposes. Working one-on-one with another member of the team is difficult at times as disagreement often occurs during the coding process (specifically among contentious variables such as ideological affiliation, tactic, and completion of crime). These disagreements are unacceptable without resolution, so compromise is a necessity – an extremely crucial skill to possess, professionally.

At a more profound level, I have co-authored elaborate research papers with team members for the duration of entire semesters (about four months), learning the value of patience, negotiation, and tenacity. Furthermore, team work has permeated my time with the project in terms of planning for the team in a collaborative environment such as round-table meeting for the general body and the steering committee.

With the opportunity to be a member of the Steering Committee as the prime example, the leadership opportunities which I have been offered throughout my time with tPP are endless. As a member of the Steering Committee, I am able to see the project from a new perspective. Available to help guide and mentor new members, inclusion in important decision-making and project updates, and service as a reference are just a few of the many responsibilities associated with this position. Having this experience listed on my professional resume will open many doors for me, but over a year of loyalty and commitment to tPP granted me the opportunity to fill this roll. Action, reaction; hard work, payoff; input, output.

Other leadership opportunities bestowed onto tPP members include representing the project at conferences and other university sponsored events, and communicating with the project’s expert advisory board – building a professional network along the way. The most beneficial of all of my tPP experiences; however, are the publishing opportunities I have been awarded throughout my time with the project. Advantaging the connections I have through tPP – from Dr. Loadenthal to the expansive advisory board – I have been able to submit various works to multiple publishers, including leading journals on the subject of Critical Terrorism Studies. Receiving feedback from reviewers and editors has been a unique and insightful opportunity to improve my writing and prepare for a professional career. Having works reviewed by journals and being on a project which has produced publications is extremely impressive on a resume for undergraduates, and differentiates me and others working with the team from competition when applying and interviewing for potential job positions.

I am beyond grateful for the opportunities tPP has given me during my course of work on the project. From improving research and writing ability to affording unparalleled leadership opportunities, the weight this project has on my academic experience is insurmountable. I have gained a unique skillset as a result of my involvement with the project, and the magnitude this impact has had on my personal and professional life is uncontested. I will continue to give my best to tPP, and I rest assured that the project will continue to give back to me.

Excluding the Political Extremists 


This continues our series of student reflections and analysis authored by our research team.


Excluding the Political Extremists 

Bridget Dickens

As the Prosecution Project finds new cases, our first step is to see if they should be included. There are basic requirements, like the date range and ensuring they a felony. However, the deciding factor is if the government labeled the case as being terroristic or extremist or if the crime advances terrorism or extremism. Sometimes it is difficult to determine whether a case should be included.

Charles Dyer is a case I looked at which raises questions. Dyer was a former Marine, Tea Party member, and a member of the Oklahoma Patriot group, the Oathkeepers. He posted videos under the pseudonym “July 4 Patriot” which explained his belief system. On January 12, 2010, officers arrested Dyer for raping a seven-year-old girl and possessing an unregistered grenade. Later that year, he missed a court date and was a fugitive until he was found the following week in Houston. Dyer was eventually sentenced to 30 years in prison. Ultimately, it was determined that Dyer should be excluded. His extremist right-wing ideology was not enough to include him. 

Charles Dyer’s case brings up an important question: when people are political extremists should we always include them in our data set? The short answer to this question is no. In our manual, we specify that the crime must be “in furtherance of terrorism, extremism, or political violence.” Crime is the main focus, not the person. While political ideology can hint at beliefs, it does not always affect a person’s criminal motive. It’s important to understand what an ideology is in order to understand how why we exclude some cases.

Ideology is defined by the British author Andrew Heywood on page 10 of his book Political Ideologies: An Introduction as the following:

An ideology is a more or less coherent set of ideas that provides the basis for organized political action, whether this is intended to preserve, modify or overthrow the existing system of power. All ideologies therefore have the following feature. They: 

 

      • Offer an account of the existing order, usually in the form of a ‘world-view’
      • advance a model of a desired future, a vision of the ‘good society’ 
      • Explain how political change can and should be brought about – how to get from  (a) to (b)

This definition encapsulates the idea of ideologies being a combination of one’s personal beliefs for a utopia and the actions taken to reach this dream. Ideology is not just having anti-government beliefs as Dyer did. The person must also attempt to put these beliefs into practice, such as organizing extremist groups, sending death threats, massacres, and more. Though Dyer fits this description through his involvement in the Oath Keepers, that did not relate to him raping a child. The rape was a separate action and no sources showed that he did it in order to further his belief system. Our goal is to look is to track political extremism and terrorism are prosecuted. While Dyer’s personality fit political extremism, the crime he was prosecuted for does not.  

It is difficult to imagine the thousands of other cases we would need to include if we only focused on people’s ideology. While all of those crimes would have been committed by someone with an extremist ideology or membership to a terrorist group, not all the crimes would have been motivated by their political ideology. The discussion of whether to include or exclude these people needs to always be rooted in the physical crime. 

Works Cited 

Heywood, Andrew. Political Ideologies: An Introduction. 6th ed. London, UK: Palgrave, 2017. [Excerpt, Chapter 1: “Political Ideologies and Why They Matter”]

People vs. Property and the Ambiguity of Data


This continues our series of student reflections and analysis authored by our research team.


People vs. Property and the Ambiguity of Data

Megan Roques

When coding my first few cases with tPP, I thought “how could people think this is hard?” The cases were straightforward and simple, the offense was either charged with a clear charge or the motive was very clearly explained and outlined. More often than not, the reality is that these cases tend to be complicated because humans themselves are complicated. Not all offenders have a clear motive of why they did XYZ or if they have a clear motive they don’t have a clear plan of how to demonstrate their motive.

This was the complication I experienced when coding and learning about the David Theiring case. David Theiring is a caucasian man from Madison, Indiana who detonated two bombs: one outside a police station and one outside a judge’s house. Although one may think that these actions were in opposition of the government or a particular candidate, Theiring did not state why he did it. Coding some of the variables such as “tactic” and “completion of crime” were straightforward and simple. However, a few variables were difficult because of the lack of information and the amount of ambiguity in this case. The variable that I find the most time reflecting on was “People vs property.” 

Although no one was harmed in both detonations, was his intent to harm people or to destroy property to make a statement? If he did it to hurt people, was his discontent stemming from a decision the judge made and the police department enforced or were these targets picked randomly? It is easy to get caught up in this web of trying to understand why Theiring did this.

Additionally, it is easy to read the case file and just pick “Property” since there was only property damage. Another way to try to rush through the process is to just assume that he was intending to harm people and property because property was damaged and maybe he thought the people would be inside. However, I believe that there is beauty to the difficult process that is understanding the human mind. When coding cases, it is important to code with purpose, meaning that when two people code the same case there are no discrepancies. 

One of the most valuable things I find when I am coding a case is a surplus of information. Helen Gibson writes of the acquisition and preparation of data for investigations. She explains, “without data there is nothing to build upon, to combine, to analyze or draw conclusions from (Gibson, 69). “The ambiguity of data and information from the offender was the reason that this case was so hard to code.

Again, this is not a flaw from the judicial system in obtaining the data but just a fault of human error. Humans refuse to overshare personal details of their lives or opinions unless they feel like their opinion will be validated. Furthermore, in a criminal setting most individuals are more likely to refrain from sharing information about the reasoning behind their crime in fear of being judged. 

Additionally, the ambiguity of data in a crime is not deemed important to the judge giving out the sentence. For example, if a person pleads guilty for the felony charges he is in question for (such as David Theiring), then the judge has little interest in their actions and hands out the sentence. However, for people (like us) studying the criminal process, the motive plays a big role in the research process. I believe that there should be more information gathered from crimes which may stem from ideological roots such as this case.

Gibson mentions sources of where to find data from defendants such as social media, electoral rolls, webpages, government databases, etc… (74). Some may believe that using individual’s electronic footprint when analyzing their crime is a violation of privacy. Others may believe that the question “People vs. property” is insignificant and not enough of reason to dive into Theiring’s personal life. However, in order to gain a little more understanding in the nature of these crimes, we need to understand the people who are being charged with the crimes.

References

Gibson, Helen. “Acquisition and Preparation of Data for OSINT Investigations.” In Open Source Intelligence Investigation: From Strategy to Implementation, 69-93. Advanced Sciences and Technologies for Security Applications. Cham, Switzerland: Springer, 2017.

Scraping the Violence Project’s Mass Shooter Database (part 2 of 2)

Reflecting back on the quick scraping exercise employing the Violence Project’s Mass Shooter Database (detailed in a previous blog post), I was puzzled by one thing.

In thinking through my own mental list of mass shootings which meet the criteria for inclusion with tPP, I was not able to determine why the 2009 shooting at a US Army facility in Forth Hood in Texas was not included. Why was the perpetrator, (then Major) Nidal Hasan, not included in my initial 13 results?

I decided to investigate. Upon locating the record for Hasan in the Mass Shooter Database, I noticed that he had been excluded from my own scraping approach because his grievance/motive was coded as ‘other’, a category I had decided to exclude. Seeing what to me was such a clearly socio-political crime coded this way got me to think: Are there are crimes in this set, coded as ‘other’ but in fact meeting tPP’s definition of a socio-political motive?

To determine this, I re-ran the same set of procedures: excluding cases prior to 1990, excluding cases where the perpetrator died, then excluding all cases that remained with motives besides those coded as ‘other.’ This left only a 10 cases displayed below:

  1. Michael Vernon
  2. Andrew Golden
  3. Mitchell Johnson
  4. Emanuel Burl Patterson
  5. Nidal Hasan
  6. Riccardo McCray
  7. William Hudson
  8. Robert Thomas
  9. Jarrod Ramos
  10. Kip Kinkel

From this list, I preformed a cursory investigation into the unknown cases using a news aggregator (i.e. Nexus Uni) and a simple boolean search string (“First name Last Name” AND shooting AND year) which inserted the shooter’s name and the year of the incident. This string returned positive results for all cases, and after reading about the various incidents, none met the criteria for inclusion beyond the attack by Hasan.

In total the Mass Shooter Database provided:

  • 23 cases for initial review for possible inclusion
  • 14 of these cases met the criteria for secondary investigation and likely inclusion
  • 9 of these cases were new to tPP while 5 are used for triangulation and ensuring the reliability of our coding procedures

Although this secondary review did not yield any new results (beyond triangulation information for the Hasan case), it did confirm the thoroughness of the original process and the integrity of the Mass Shooting Database’s conceptual coding. Upon comparing the Mass Shooting Database’s coding of Hasan and tPP’s coding, I found no disagreements in the codes, further cementing the accuracy of both projects’ data.

It’s the Government’s Say (Part 1): On the Topic of Variable “State Speech Act”


This continues our series of student reflections and analysis authored by our research team.


It’s the Government’s Say (Part 1): On the Topic of Variable “State Speech Act”

Emily Ashner

The difficulty that remains most intact as I continue working on this project is to remain truly objective. Luckily, the codebook is detailed in how to dissect the information from source documents and I remain in check by my partner. Beyond obtaining the validity of the project, however, I often find myself hesitant on tagging cases with the value of “state speech act.”

By nature of the variable, state speech act is defined purely based on the government’s definition of an individual. For exact definition, the codebook spells out this variable under 3 circumstances: the individual has been identified of a member of a foreign terrorist organization (FTO) or political group and acted in this manner, the individual has been identified as part of one of these groups, even if the case does not appear to be socio-politically motivated. In summary, the government often has complete say on inclusion of cases based on a group identification. Ryan Reilly (2018) discusses the easy ability to tag someone as a terrorist if they discuss a designated foreign domestic terrorist group (FTO) on the internet, but there are many difficulties of doing so for domestic groups are as the current laws are do not give much insight into these groups as foreign ones.  

By nature of the project, there is obvious flaw in the justice system’s means of prosecution. One of the goals of this research is to identify how these specific crimes follow a pattern of “otheredness”, often driven by race or participation in certain terrorist organizations. Therefore, when coding for an individual I believe should clearly be excluded, the state gets final say in this decision. For example, I am in the processing of coding a case for Michael Bruce Messer, a man who was arrested and sentenced for two counts of knowingly possessing in and affecting commerce, a firearm and ammunition, all of which had been shipped and transported in interstate and foreign commerce. Although in this indictment there was no mention of this, the DOJ mentions his want to join ISIS in a press release on his sentencing. Now the personal hesitation of this resulted from a possibility that the weapon had no correlation with his post online. That could still be true, however, since the government made a clear connection in this article about his want to be a part of this FTO in relation to the charge, this deems a state speech act and inclusion in the project.

While there is an obvious need for the government to clearly state laws and determine aspects to correctly and systematically address crime, there are times such as this case where their say seems ambiguous. I am not arguing the fact that Messer committed crime and could possibly commit much worse based on his want to join a terrorist organization, but I do question how we define and change our opinion on the individuals we code based on simple single sentence additions and descriptions by the DOJ.  

References:

Reilly, Ryan J. “There’s A Good Reason Feds Don’t Call White Guys Terrorists, Says DOJ Domestic Terror Chief.” Huffington Post. January 11, 2018, Online edition, sec. Politics.

Scraping the Violence Project’s Mass Shooter Database (part 1 of 2)

The thoughtful Greg Reese from Miami’s Research Computing Support sent our team a link to a news story today. The email was titled, “Might have relevance to your work” and linked to a story by Vice News, “Nearly All Mass Shooters Since 1966 Have Had 4 Things in Common.”

This article presents a recent data set published by The Violence Project known as the Mass Shooter Database.

Greg was right to send this our way as there are obvious likely overlaps between tPP’s case criteria and the MSD’s. When reviewing Vice’s secondary review, they noted:

“[Mass shootings are] also increasingly motivated by racial, religious, or misogynist hatred, particularly the ones that occurred in the past five years.”

As soon as I saw this I decided to request access to the data set and promptly received a link. The meticulous and easy to navigate data set provided event data on 171 shootings. From there I sorted columns to prioritize certain variable values and trim the 171 events down to those which would likely meet the inclusion criteria for tPP.

  • I began by eliminating any shootings prior to 1990 as this is outside of tPP’s data range.
  • I then used the “on scene outcome” variable to remove all cases where the shooter died on scene, keeping only those in which the individual was apprehended. Since tPP requires the charging of a crime, only individuals who survived their attack could be included.
  • I then sorted by motive. The data set codes for 13 “grievances” and “motivations.” Using these criteria I colored all cases which displayed the following values:
    • Racial element
    • Interest in white supremacy/Notable racism/Xenophobia
    • Religious hate
    • Homophobia

I also included two cases coded as “Notable misogyny” as this is a recurrent trend in some cases we have added to our project. I then eliminated all of the cases which displayed other grievances as these would not likely meet our definition of a socio-political motive.

This process produced a final set of 13 cases which, according to my interpretations of the coding criteria as provided by The Violence Project, likely meet the criteria for inclusion in tPP. These cases will subsequently be assigned to coders to investigate, and eventually coded for inclusion or exclusion. The cases identified (prior to individual investigation) are:

  1. Kenneth French
  2. Colin Ferguson
  3. Hastings Arthur Wise
  4. Richard Baumhammers
  5. Steven Stagner
  6. Chai Vang
  7. Dylann Roof
  8. Arcan Cetin
  9. Nikolas Cruz
  10. Dimitrios Pagourtzis
  11. Jarrod Ramos
  12. Robert Bowers
  13. Patrick Crusius

Our scraping procedure for data sets requires that we first check if an incident is already included in the project. This involves searching the final data set as well as a series of ‘in progress’ sheets managed by coding teams. If the case is already included, as is the case of the 4 defendants underlined, we will evaluate our coding choices based on the new data for triangulation and possible modification. Since the data provided by The Violence Project is more detailed in certain aspects, we may be able to more accurately represent the record within tPP by exploring the other researchers’ coding decisions.

This search yielded confirmatory information on 4 cases, and 9 likely new case starters. These 9 cases will be investigated by coding teams. They will be worked through the inclusion/exclusion decision tree, and if they pass, entered into the team workflow.

Not Your Stereotypical Terrorism


This continues our series of student reflections and analysis authored by our research team.


Not Your Stereotypical Terrorism

Sarah Carrier

It is a regular day, you are drinking your morning coffee, when suddenly your phone receives a news alert. The title reads something like, “Another Terrorist Attack Strikes America”. You think to yourself, “Great, another ISIS attack, when will our government get it together and get rid of them?” However, you are mistaken. The attack was not performed by a member of ISIS, but rather a member of the Aryan Brotherhood (White Supremacists). Or, perhaps it was performed by an environmental activist against a construction site. Truth is, many people who receive a news alert like the one mentioned above will only read the title, not the story itself, and assume facts based or prior knowledge or prior prejudices. A recent study done by the Media Insight project claims that only 25% of participants went in-depth on breaking news, and 30% said they do not go in-depth on news stories. Readers can find out more about the study here.

There is a huge issue of disinformation being spread to American news consumers. Either people watch a news site that shares in their views and does not give all the facts, or Americans do not bother to go further than news headlines. As a result, their is a common misconception when it comes to terrorism. Many Americans believe terrorism to be similar to 9/11. It is large scale destruction, constitutes loss of life, and is committed by a muslim, member of ISIS, or Al-qaeda. However, the definition for terrorism is changing, there is both foreign and domestic terrorism. The FBI defines terrorism in USC Title 18 writing:

International terrorism: Violent, criminal acts committed by individuals and/or groups who are inspired by, or associated with, designated foreign terrorist organizations or nations (state-sponsored). Domestic terrorism: Violent, criminal acts committed by individuals and/or groups to further ideological goals stemming from domestic influences, such as those of a political, religious, social, racial, or environmental nature.” 

Any violent crime with the goal of influencing politics, or attacking a particular group in such as religion or race, can be a terrorist act. It is not only the Salafi/Jihadist/Islamist people who can commit a foreign terrorism act, it can also be the left or right groups that can commit domestic terrorism. However, it is more common to see Alt-Right groups, for example White Supremacists, perform an act of terrorism. A study done by the news site Quartz, with the Global Terrorism Database, found that in 2017 around ⅔ of terrorist acts in America were committed by right leaning groups. The study and a news article about the study can be found here. 

In the story Three Little Pigs, it is hard to imagine the big bad wolf who huffed and puffed and blew their house down as anything other than a wolf. However, what if another animal came along who wanted the pigs? What if it was a bear, or a lion, or even a dog? It is hard to imagine the story we have been hearing since children in any other possible way. The true can be said of terrorism. Since childhood, many Americans have been told the big bad wolf is ISIS or Al-qaeda. This was acceptable to Americans. Those groups are far away and do not affect everyday life. It is much harder to change the narrative. To believe the wolf is not a wolf at all, but a dog who lives in your backyard. This is a much scarier reality that many Americans do not want to live with. However, this truth about terrorism must be accepted. For example, in Charlottesville, James Alex Field, a member of the Alt-Right, was able to drive into a crowd and kill and injure many people. This criminal act was defined by the United States as terrorism. This is a large and well-known story. However, there are many smaller stories about domestic terrorism that do not make national news. So, next time you are drinking your morning coffee, click on the headline and read the full article. 

References

CillizzaBioBioReporter, Chris Cillizza closeChris. “Americans Read Headlines. And Not Much Else.” Washington Post. Accessed November 1, 2019. https://www.washingtonpost.com/news/the-fix/wp/2014/03/19/americans-read-headlines-and-not-much-else/.

Federal Bureau of Investigation. “Terrorism.” Folder. Accessed November 1, 2019. https://www.fbi.gov/investigate/terrorism.

Southern Poverty Law Center. “Study Shows Two-Thirds of U.S. Terrorism Tied to Right-Wing Extremists.” Accessed November 1, 2019. https://www.splcenter.org/hatewatch/2018/09/12/study-shows-two-thirds-us-terrorism-tied-right-wing-extremists.

On tPP’s 3-step Verification Process


This continues our series of student reflections and analysis authored by our research team.


On tPP’s 3-step Verification Process

Izzy Bielamowicz

More-so than in the past, this semester I have come to realize the importance of tPP’s verification process. Throughout my time as a member of tPP, I always viewed multi-person verification as useful, but it was not until this semester, when I became a member of the Steering Committee that I finally understood the cruciality of the 3-step process to its full extent.

Arguably one of the most critical methodological approaches of tPP is that before any completed case is moved into the official dataset, it is checked for accuracy by at least 3 separate coders. First, a case is “claimed” by a coding pair in the team spreadsheet, which is accessible to all team members, by coloring its row green. Once the coding pair has claimed a case, they begin the coding process exclusive of one another. Coding a case individually allows for each coder to process and interpret the facts of the case on their own accord based on the guidelines of the codebook.

Once the coders have singly coded a case, they meet to discuss their rationale behind each variable. If the coders have arrived at the same conclusions for every variable based on the facts of the case and the guidelines of the codebook, it is likely that they have correctly coded the case. Should the coders run into a variable which they have coded differently, there is opportunity for valuable discussion. In my personal experience, it is often the case that one of the coders has missed information within the source files or that they have misinterpreted the codebook. In other instances; however, I have experienced dissenting opinions based on inconsistent understanding of the material. Often, variables which offer issue for dissent are ideological target, ideological affiliation, and tactic. This may be attributed to different assumptions gathered from the known facts of the case, and the inevitable ambiguity of certain cases regarding ideology and plot. Should this disagreement occur, the coders reanalyze the case and agree upon the most suitable code. Once the coding pair has completed their case – should dissent arise or not – they move the completed case back into the team spreadsheet, coloring it white.

Once a case is completed by a coding pair and moved back into the team spreadsheet, it is reviewed again by a member of the Steering Committee, the Auditor. Going through the review process for the first time this semester was the experience which I needed to fully grasp the importance of tPP’s verification process. Assigned 10 cases to review, I began by opening the source files for each case and cross-checking the information I was gleaning with the coding which had already been done. As I checked, I realized that mistakes had been made by the initial coding pair. I proceeded to correct the errors and transfer the updated cases to the official dataset. Without having a third review of the cases I had been assigned, there would have been inaccurate codes in tPP’s dataset, decreasing the validity of the project as a whole.

Ultimately, the 3-step verification process tPP follows ensures the legitimacy of the data. While there are generally fewer cases which require a third review to correct errors than those cases which need rectifying, it is critical that every case follow the same process to maintain the strength of the data. The implementation of coding pairs guarantees that variables which necessitate discussion are discussed before a decision is made. Furthermore, because tPP is composed of circulating membership, a third review by veteran team members on the Steering Committee secures the accuracy of the codes. While not unique to tPP, the multi-person verification process which the project follows enhances the credibility and effectiveness of the data.