Multi-lingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model

Abstract

Current investigations in data-driven models of parsing have shifted from purely syntactic
analysis to richer semantic representations, showing that the successful recovery of the meaning
of text requires structured analyses of both its grammar and its semantics. In this article, we
report on a joint generative history-based model to predict the most likely derivation of a
dependency parser for both syntactic and semantic dependencies, in multiple languages.

Because these two dependency structures are not isomorphic, we propose a weak synchro-
nisation at the level of meaningful subsequences of the two derivations. These synchronised
subsequences encompass decisions about the left side of each individual word. We also propose
novel derivations for semantic dependency structures, which are appropriate for the relatively
unconstrained nature of these graphs.

To train a joint model of these synchronised derivations, we make use of a latent variable
model of parsing, the Incremental Sigmoid Belief Network architecture. This architecture induces
latent feature representations of the derivations, which are used to discover correlations both
within and between the two derivations, providing the first application of ISBNs to a multi-task
learning problem.

This joint model achieves competitive performance on both syntactic and semantic de-
pendency parsing for several languages. Because of the general nature of the approach, this
extension of the ISBN architecture to weakly-synchronised syntactic-semantic derivations is also
an exemplification of its applicability to other problems where two independent, but related,
representations are being learnt.