Proposal Final for Developing an Application in Support of Counter Terrorism
By Akshat Kapoor Qin Wang William Leannah (Group 1)
November 2, 2005
Project Aims
This group is going to create a text mining system in support of counter terrorism. To accomplish that task the group has investigated and is using a new text mining technique known as Unintended Information Revelation (UIR).1 The premise of UIR is that pieces of information that by themselves appear to be innocent may be linked together to reveal inadvertently highly sensitive data (Hidden Agendas). The goal of the project is to end up with a UIR system which produces a concept chain graph from the analyzed data. The concept chains would represent a collection of terrorist based data that was classified and linked together either directly or indirectly. An example of our final output, a concept chain graph is shown in Figure 1. The graph is describing an indirect link between two previously unlinked pieces of information (event 1) "Boarded Flight 11 for Los Angeles" and (location A) "Airport Portland, Maine". Now linked, the two pieces of information could be interpreted in such a way that could lead authorities to take some further action like elevating the country's terrorist threat level. Once produced, we expect the graphs to reveal potential threat scenarios that were previously unknown. Located below the concept chain is summary information regarding the different points on the graph. The summary information includes a description of the source and provides the specific key concepts that form the basis for the links. It also gives the user an indication for how strong the link is between the materials represented in the graph. A research team from the University of Buffalo is currently leading the development of the UIR model. Our group adopted their research as a starting point for our project. The Buffalo team has already created their own prototype system that builds concept chain graphs based on a "best path for connecting two concepts". The information retrieval end of their system sounds very similar to a Hidden Markov Model approach. Since the availability of classified data is limited in the public domain, the group has decided to use the 911 commission report as the primary training corpus for the project.2 We intend to break off small segments of the 911 commission report for use as the development set. To test the system this group will select several news articles at random from internet based sources. We will select news articles that have both a related and unrelated context to the events of September 11, 2001. Selecting both kinds of articles will allow us to prove the validity of the process. This is a system test to see if it can create links between both types of articles. The text classification and information retrieval processes will utilize natural language programming methods that have been described in this course. Specifically, we will be using Regular Expressions, Probability, N-Grams, and Hidden Markov Models. Regular expressions will be used to break up documents into usable fragments, sequences, or words. To capture the most amount of information the system will not be case-sensitive. For example, a word like "Al-Qaeda" will be the same as "alqaeda" or "AL-QAEDA". Also the title "NEW YORK" is treated same as "new york". Probability calculations provide a way to determine the best classifications for the emphasized classes of information. N-Grams help us form the basis of our concept chains because they are ideally suited to predicting the sequence of words. Different threshold levels will be used to determine if the link warrants representation during the
1 2
In War on Terrorism, New Search Engine Seeks Hidden Vulnerabilities ; Ellen Goldbaum; Site Institute; International Anti-Terrorism Website; 9-11 Commission Report
2
final output. Hidden Markov Models also seem appropriate in the context of this project because they are based on finding the best path through the state machine based on the highest probability. Tentatively, this group envisions the process of going from a text article to concept chain graph working similar to the following: 1. Using Python we load the 9-11 commission report into an N-Gram model. 2. Parse the text to retrieve keywords (events, individuals, locations) Using a custom part-of-speech tagging tag set as Dr. Struble suggested. 3. Enter keywords into database. 4. Construct concept chain graph from keywords (as shown in Figure 1 below).
Figure 1 - Concept Chain Graph
Background and Significance
Since the attacks on the United States in September 11, 2001 the monitoring and analyzing of terrorist group activities has become an important facet in the war on terror.3 Terrorist organizations like Al-Qaeda make heavy use of the internet to organize and facilitate their Anti-American messages and activities. Over the last four years U.S. intelligence agencies have been inundated with information from multiple sources such as: newspaper articles, websites, research papers, court documents, and even
3
Site Institute; International Anti-Terrorism Website; About Page; http://siteinstitute.org/mission.html
3
messages from the terrorist themselves. A growing concern in this country is how to process this large volume of unstructured information into something that could be used to stop the future plans of these groups before more innocent victims are killed.4 Processing unstructured data into predictive models is relatively new technology making the difficulty of classifying and prioritizing broad information about terrorist groups less surprising. Additionally, terrorist organizations behavior is unlike enemies that were faced in the past. These groups are often comprised of an unknown amount of members and sympathizers that carry out seemingly hidden agendas on an ongoing basis. The primary challenge facing projects like the one being described in this proposal is the reduction or elimination of false positives and false negatives.5 Too many false negatives and investigators will quickly lose confidence in the system. Meaning cleared information should have received more attention because it led to another terrorist event. Likewise, false positives could lead to the arrest of potentially innocent persons having nothing to do with terrorist organizations. Additionally, individual privacy is something that needs close consideration. The term "Unintended Information Revelation" certainly does not sound privacy friendly. It has been suggested that any unstructured information could be linked together and concocted in such a way as to produce whatever picture the author wants. In particular, government research groups like the Department of Defense created programs like Terrorism Information Awareness (TIA) to analyze large volumes of structured and unstructured in data the hope of pattern detection. Since its inception congressional views of the TIA program have turned negative subsequently eliminating funding.6 As a result, the program will likely "go black" (classified secret) and continue operating out of the spotlight of public scrutiny. There is little doubt that government agencies are still very interested in monitoring the following medium: 1. Terrorist Computer Networks. 2. Rapid identification of critical information. 3. Surveillance of the web, e-mail, or chat rooms. Although privacy concerns related to this research exist, it is beyond the scope of this group's ability to solve them. Privacy issues especially in the United States are always sensitive. Ultimately, what people will have to decide is should they live with more privacy or less terrorist attacks. What we will attempt to achieve is finding better ways to eliminate or reduce false positives and false negatives. UIR researchers have reported the need for improvement in this area. What constitutes false positives or false negatives aren't even clearly defined within the current UIR context. To add value to the current state of UIR research this group will focus its process improvement efforts within two aspects: Classifying and organizing unstructured terrorist data being stored within our database model and the linking of the classified data as it relates to preventing terrorism.
Preliminary Work
After performing some research on the internet, it was discovered that research groups in several different countries are trying to accomplish this very task utilizing similar
4 5 6
Site Institute; International Anti-Terrorism Website; About Page; http://siteinstitute.org/mission.html Data Mining, Next Generation Challenges and Future Directions; Pages 158-159. Congress Kills TIA Program; Roy Mark
4
approaches in the text mining discipline that are being covered in this class. One group out of Australia is in the process of creating an application that will run on any desktop computer which could analyze thousands of terrorist email messages and provide the user with a visual map. This map may help determine if a particular threat exists and whether or not to commit additional resources to the investigation. A similar project underway in the United States is trying to create a digital library to be used for the detection of terrorist threats. When finished these government systems will form the foundation of computer driven text mining based anti-terrorist initiatives. The two pieces of software that the group has chosen to complete this project is the Python programming language and Microsoft Access database to store the information. The first thing the group did was test to see if we could connect Python to Access and we did that successfully. Figure 2 shows a Python code sample uses the Python Windows API to connect to Access database. Once connected a tokenized article is then stored in the database with word frequency information from a test article. An example can be found in Figure 3.
Figure 2 - Connecting Python to Access.
Figure 3 - N-Gram information stored in the database.
5
It was discovered during this process that the PDF version of the 911 Commission Report was encrypted and could not be read by Python. So the group found some free tools on the internet that converted it to a text document that can now be used in our project. We have also prototyped a potential database model using Access. The database prototype stores the three classes of information that we have chosen to focus on for this project, events, locations, and people. In addition to the class tables, the schema includes help tables that facilitate the links between people and events, people and locations, and people and other people. We are using this prototype to create an example of concept chain graph using a Microsoft Access Report. At the time of this writing the prototyped report is not complete and will not be included in this proposal. Figure 4 shows an example of our prototyped schema in Access.
Figure 4 - Prototype Database Schema
To date we have not conducted any baselines studies. (Please Note: We ha...