Li Yang successfully defended his dissertation,  Re-evaluating and Exploring the Contributions of Constituent Grammar to Semantic Role Labeling, on Sept. 4.

Committee:  Steve Abney (Chair), George Michailidis, Drago Radev, Rich Thomason

Li will continue his work for Janya in Buffalo, NY, a company that develops information extraction software.

Congratulations, Dr. Yang!

Abstract:

Since the seminal work of Gildea and Jurafsky (2000), semantic role labeling (SRL) researchers have been trying to determine the appropriate syntactic/semantic knowledge and machine learning algorithms to tackle the challenges in SRL. In search of the appropriate knowledge, SRL researchers
shifted from constituency grammar to dependency grammar around 2007 due to the suspension in improvement in the systems relying on features based on constituency grammar. However, the results from the CoNLL-2008 SRL systems, all of which utilized dependency grammar-based features, did not support the hypothesis that dependency grammar was more suitable for SRL. Therefore, determining the right syntactic/semantic knowledge for SRL still remains an open question. This entails that finding the right syntactic/semantic knowledge to create features that generalize across the syntactic variations that a verb appears in and involve argument movement or displacement remains a challenge as well.

The current dissertation continues the effort to discover the appropriate syntactic/semantic knowledge for SRL. Specifically, while seeking the proper features to solve the SRL problem in general, the present work focuses on tackling the syntactic variation challenge by integrating three types of less thoroughly explored knowledge in constituency grammar-based SRL systems, including context dependence among the semantic roles of core arguments, syntactic structures involving argument movement or displacement, and dependency grammar relations. Integrating such knowledge leads to the following novel approach.

The system identifies the core and non-core semantic arguments of a verb. To classify a non-core argument, the system uses a set of generic features. For a core-argument, the system relies on the preceding types of knowledge to extract the base argument configuration (BAC) feature in which the core arguments’ positions overlap with those of an argument structure of the verb. As a result, BAC features generalize across the syntactic variations a verb appears in. Together with the two levels of backoff features dealing with unrealized core arguments and unknown verbs respectively, BAC features effectively solve the argument classification task and successfully handles the preceding challenge. However, the experimental results indicate that  the overall performance is affected by the argument identification module. The immediate future work would be to improve the identification module.

Terry Szymanski’s Qualifying Research Paper has been approved.  The paper, “Computational Approaches to Sound Change and Reconstruction, ” uses a probabilistic model of language change and diversification as the basis for a reconstruction algorithm. Given cognate word-lists from two or more cognate word lists, the goal of the algorithm is to reconstruct the form of the words in the proto-language, derive the necessary sound changes, and determine the most likely topology for linguistic phylogeny. Experiments show that the algorithm is successful on small, simulated data sets, and should be extensible for application to real language data.

Terry now advances to Doctoral candidacy.  Congratulations, Terry!

Excerpted from the National Science Foundation

The sixth International Linguistics Olympiad ended today in Slanchev Bryag, Bulgaria, and U.S. high school students captured 11 out of 33 awards, including gold medals in individual and team events. This was only the second time the U.S. has ever competed in the event. Their achievement brings a new focus on computational linguistics.

This year’s Olympiad featured 16 teams from around the world, including Bulgaria, Estonia, Germany, Latvia, the Netherlands, Poland, Russia, Sweden, South Korea and Slovenia. Each problem presented clues about the sounds, words or grammar of a language the students had never studied, such as Micmac, a Native American language spoken in Canada, the New Caledonia languages of Drehu and Cemuhi, as well as several historical Chinese dialects. They were then judged by how accurately and quickly they could untangle the clues to figure out the rules and structures of the languages to solve the problem.

Team 1 was composed of Guy Tabachnick of New York City, Jeffrey Lim of Arlington, Mass., Josh Falk of Pittsburgh, Pa, and Anand Natarajan of San Jose, Calif.

Jae-Kyu Lee of Andover, Mass., Rebecca Jacobs of Encino, Calif., Morris Alper of Palo Alto, Calif., and Hanzhi Zhu of Shrewsbury, Mass. participated as Team 2.

Team 1 claimed a silver medal in the team competition and Team 2 captured a gold. Team 2 also won a trophy for the highest combined score on the individual competition. In the individual competition, Jacobs, Lim and Tabachnick were awarded bronze medals, Alper and Natarajan won silver, and Zhu captured a gold.

The U.S. teams were led by head coach Dragomir Radev, associate professor of computer science, information, and linguistics at the University of Michigan, and associate coach Lori Levin, co-chair of NACLO and associate research professor in the Language Technologies Institute at Carnegie Mellon University. Adam Hesterberg, who achieved the highest individual score in last year’s Olympiad and is currently attending Princeton University, was present this year as an assistant coach. The team was also accompanied by National Board Certified Teacher Amy Troyan, who also serves as gifted program coordinator at Taylor Allderdice High School.

Abney, Steven. 2007. Semi-supervised Learning for Computational Linguistics. Chapman & Hall/CRC Computer Science & Data Analysis Volume: 8

From the publisher:
-Offers applications in information extraction, parsing, and word senses, such as WordNet
-Provides background material in machine learning that includes the areas of classification and clustering
-Covers a variety of methods, including co-boosting, transductive SVMs, McLachlan’s algorithm, and the EM algorithm
-Examines in detail the concept of label propagation in a graph
-Discusses spectral methods, including the definition of harmonics, the eigenvectors of matrices and graphs, spectral clustering, and the connection to label propagation
-Introduces the necessary mathematics in a just-in-time manner

The rapid advancement in the theoretical understanding of statistical and machine learning methods for semisupervised learning has made it difficult for nonspecialists to keep up to date in the field. Providing a broad, accessible treatment of the theory as well as linguistic applications, Semisupervised Learning for Computational Linguistics offers self-contained coverage of semisupervised methods that includes background material on supervised and unsupervised learning.

The book presents a brief history of semisupervised learning and its place in the spectrum of learning methods before moving on to discuss well-known natural language processing methods, such as self-training and co-training. It then centers on machine learning techniques, including the boundary-oriented methods of perceptrons, boosting, support vector machines (SVMs), and the null-category noise model. In addition, the book covers clustering, the expectation-maximization (EM) algorithm, related generative methods, and agreement methods. It concludes with the graph-based method of label propagation as well as a detailed discussion of spectral methods.

Taking an intuitive approach to the material, this lucid book facilitates the application of semisupervised learning methods to natural language processing and provides the framework and motivation for a more systematic study of machine learning.

Li Yang has just returned from the 10th Conference of the Pacific Association for Computational Linguistics (PACLING 2007) in Melbourne, where he presented a paper titled Named Entity Recognition Using Syntactic/Semantic Information, co-authored with Steven Abney.

In their paper, Yang and Abney show that combining deep syntactic knowledge with machine learning methods significantly improves the performance on the task of named entity recognition.

Deep processing is the major theme of PACLING 2007.

Drago Radev coached the US team in their first trip to the Linguistics Olympiad

From the news report:
Six American high-school students took the top honors in the 2007 International Linguistics Olympiad in St. Petersburg, Russia earlier this month. This year was the first time a delegation represented the United States at the annual competition. Their victory brings a new focus on computational linguistics.

This year’s International Olympiad featured 15 teams representing 9 different countries, including the Netherlands, Russia and Spain. Competitors were given problem sets consisting of sentences in languages most people are not familiar with, including: Tatar; Georgian; a language spoken by indigenous people in Bolivia called Movima; the Papua New Guinean language Ndom; Hawaiian; Turkish; and their English translations. With just this information, the competitors then had to translate more sentences from these languages into English. Winners were judged by how accurately and quickly they could figure out the rules and structure of the languages and complete their translations.

Read the rest