Daniel Klein

Analyzing the rate at which languages change can clarify whether similarities across languages are solely the result of cognitive biases or might be partially due to descent from a common ancestor. To demonstrate this approach, we use a simple model of language evolution to mathematically determine how long it should take for the distribution over languages to lose the influence of a common ancestor and converge to a form that is determined by constraints on language learning. We show that modeling language learning as Bayesian inference of n binary parameters or the ordering of n constraints results in convergence in a number of generations that is on the order of n log n. We relax some of the simplifying assumptions of this model to explore how different assumptions about language evolution affect predictions about the time to convergence; in general, convergence time increases as the model becomes more realistic. This allows us to characterize the assumptions about language learning (given the models that we consider) that are sufficient for convergence to have taken place on a timescale that is consistent with the origin of human languages. These results clearly identify the consequences of a set of simple models of language evolution and show how analysis of convergence rates provides a tool that can be used to explore questions about the relationship between accounts of language learning and the origins of similarities across languages

Knowledge consists of the triad: information, interpretation, and judgement. Much of modern political economy has miscarried by proceeding as though knowledge were merely “information” that is, as though interpretation were symmetric and final. Economic prosperity depends greatly on new knowledge or “discovery” of profit opportunities that translate into social betterment. These discoveries are often a transcending of the working interpretation, not merely the acquisition of new information. The evolution of interpretations is key to appreciating voluntarism as a maxim for policy. Smith, Hayek, and Kirzner taught that bad public policy is partly the result of not knowing better, not of evil or greed. Economists may justify their efforts and their salaries chiefly by teaching citizens to know better. Such is the economist’s calling.La connaissance consiste en une triade : information, interprétation, et jugement. Une bonne part de l’économie politique moderne a échoué en assimilant la connaissance à l’information, ce qui revenait à dire que l’interprétation était symétrique et ultime. La prospérité économique repose dans une grande mesure sur l’émergence de nouvelles connaissances ou la “découverte” d’occasions de profit porteuses d’améliorations sociales. Souvent ces découvertes transcendent l’interprétation courante et non seulement l’acquisition de nouvelles informations. L’évolution des interprétations est essentielle pour évaluer le volontarisme en tant que maxime pour la politique. Smith, Hayek et Kirzner nous ont enseigné que des politiques publiques inappropriées résultent en partie d’une déficience de connaissances plutôt que de mauvaises intentions ou d’avidité. Les économistes peuvent justifier leurs efforts et leurs salaires essentiellement en apprenant aux citoyens à s’enrichir de connaissances. Telle est la vocation de l’économiste.

Formal Ontology is a discipline whose business is to develop formal theories about general aspects of reality such as identity, dependence, parthood, truth-making, causality, etc. A foundational ontology is a specific consistent set of these ontological theories that support activities such as domain analysis, conceptual clarification, and meaning negotiation. A (well-founded) core ontology specifies, under a foundational ontology, the central concepts and relations of a given domain. Foundational and core ontologies can be seen as ontology engineering frameworks to systematically address the laborious task of building large (more specific) domain ontologies. However, both in research and industry, it is common that ontologies as computational artifacts are built without the aid of any framework of this kind, often yielding modeling mistakes and representation gaps. In this paper, we analyze a case in the domain of cybersecurity, namely, the case of D3FEND — an OWL knowledge graph of cybersecurity countermeasure techniques proposed by the MITRE Corporation. Based on the Reference Ontology for Security Engineering (ROSE), a core ontology of the security domain founded in the Unified Founda-tional Ontology (UFO), our investigation reveals a number of semantic issues and opportunities for improvement in D3FEND, including missing concepts, semantic overload of terms, and lacking constraints that cause an under-specification of the model. As a result of our ontological analysis, we propose several suggestions for the appropriate redesign of D3FEND to overcome those issues.

OVERVIEWWe ambitiously reexamine Smith’s moral theory in relation to Hume’s. We regard Smith's developments as glorious and important. We also see them as quite fully agreeable to Hume, as enhancement, not departure. But Smith represents matters otherwise! Why would Smith overstate disagreement with his best friend?One aspect of Smith’s enhancement, an aspect he makes very conspicuous, is that between moral approval and beneficialness there is another phase, namely, the moral judge's sense of propriety. With that phase now finding formulation, Smith, if only implicitly, generates a spiral of beneficialness and propriety, a spiral shown in Figure 7 in the present paper. We consider Figure 7, illustrating the spiral, to be the most important arrival point in the present paper; it highlights the non-foundationalism of Smith's ethics. But to arrive at the spiral, we must engage in extensive exegesis.In Part IV of The Theory of Moral Sentiments, Smith presents a foil against which he develops his own theory, a foil supposedly representing Hume. According to the foil, moral approval derives from ‘utility.’ But, in multiple ways, the foil is misleading. We provide an interpretation of Hume, notably his four-factor account of moral approval, before examining Smith's representation of Hume.One twist is that Smith used the words utility and useful differently than Hume did – Smith quietly stretched them to include species of agreeableness, thereby obscuring the importance of agreeableness in Hume’s theory.Another, more significant problem is that Smith allows the impression that in Hume moral approval derives quite determinately from beneficialness. In fact Hume conveys the interpretive and sentimental spaciousness of the operations that generate moral approval; here, Hume even speaks repeatedly of ‘proper sentiments’, thus almost using the term propriety himself.But the propriety phase in Smith opens up to a key facet of Smith’s development on Hume: He poeticizes a locus of sympathy not emphasized in Hume – namely, that between the moral judge and her own man within the breast; that locus enters the theory in addition to the sympathies emphasized in Hume, not in lieu of them. We distinguish lateral sympathy, which is important in Hume’s thought, and vertical sympathy, which is especially characteristic of Smith’s more inner-directed and allegorical thought. Smith embraces Hume’s lateral sympathy and enhances moral theory by adding formulations that elaborate vertical sympathy.Next, we come to something of a twist in the whole matter: We show that – as Smith well knew all along! – propriety is a species of agreeableness! Smith’s propriety phase represents another dimension within which such agreeableness lives: Smith’s vertical dimension thus gives rise to a spiral representing the diachronic development of the judge herself. It is a spiral of beneficialness and propriety: Each propriety phase in the next loop of the spiral engenders a species of agreeableness now as a part of beneficialness.Smith's developments on Hume, then, involve the following three facets: formulation of the propriety phase; the poetic elaboration of vertical sympathy; the diachronic spiral of propriety and beneficialness.The three facets come together, especially in Ed. 6 of The Theory of Moral Sentiments. The whole development goes beyond Hume, but, really, is agreeable to Hume – though Smith himself portrays his developments as disagreeing with features of Hume's moral theory.We speculate that Smith was more or less aware of all that that we say, including the absense of any really substantive disagreement. Why, in that case, would Smith have proceeded as he did? We address that question at the end of the piece. Our speculations suggest a method in the madness.

Table at Dimitri's Taverna : on seeking a philosophy of old age -- Old Greek's olive trees : on Epicurus's philosophy of fulfillment -- Deserted terrace : on time and worry beads -- Tasso's rain-spattered photographs : on solitary reflection -- Sirocco of youth's beauty : on existential authenticity -- Tintinnabulation of sheep bells : on mellowing to metaphysics -- Iphigenia's guest : on stoicism and old old age -- Burning boat in Kamini Harbor : on the timeliness of spirituality -- Returning home : on a mindful old age.

This paper presents empirical studies and closely corresponding theoretical models of the performance of a chart parser exhaustively parsing the Penn Treebank with the Treebank’s own CFG grammar. We show how performance is dramatically affected by rule representation and tree transformations, but little by top-down vs. bottom-up strategies. We discuss grammatical saturation, including analysis of the strongly connected components of the phrasal nonterminals in the Treebank, and model how, as sentence length increases, the effective grammar rule size increases as regions of the grammar are unlocked, yielding super-cubic observed time behavior in some configurations.

While Ç ´Ò¿ µ methods for parsing probabilistic context-free grammars (PCFGs) are well known, a tabular parsing framework for arbitrary PCFGs which allows for botton-up, topdown, and other parsing strategies, has not yet been provided. This paper presents such an algorithm, and shows its correctness and advantages over prior work. The paper finishes by bringing out the connections between the algorithm and work on hypergraphs, which permits us to extend the presented Viterbi (best parse) algorithm to an inside (total probability) algorithm

While symbolic parsers can be viewed as deduction systems, this view is less natural for probabilistic parsers. We present a view of parsing as directed hypergraph analysis which naturally covers both symbolic and probabilistic parsing. We illustrate the approach by showing how a dynamic extension of Dijkstra’s algorithm can be used to construct a probabilistic chart parser with an Ç´Ò¿µ time bound for arbitrary PCFGs, while preserving as much of the flexibility of symbolic chart parsers as allowed by the inherent ordering of probabilistic dependencies.

Unsupervised grammar induction systems commonly judge potential constituents on the basis of their effects on the likelihood of the data. Linguistic justifications of constituency, on the other hand, rely on notions such as substitutability and varying external contexts. We describe two systems for distributional grammar induction which operate on such principles, using part-of-speech tags as the contextual features. The advantages and disadvantages of these systems are examined, including precision/recall trade-offs, error analysis, and extensibility.

We present a novel generative model for natural language tree structures in which semantic (lexical dependency) and syntactic (PCFG) structures are scored with separate models. This factorization provides conceptual simplicity, straightforward opportunities for separately improving the component models, and a level of performance comparable to similar, non-factored models. Most importantly, unlike other modern parsing models, the factored model admits an extremely effective A* parsing algorithm, which enables efficient, exact inference.

We demonstrate that an unlexicalized PCFG can parse much more accurately than previously shown, by making use of simple, linguistically motivated state splits, which break down false independence assumptions latent in a vanilla treebank grammar. Indeed, its performance of 86.36% (LP/LR F1) is better than that of early lexicalized PCFG models, and surprisingly close to the current state-of-theart. This result has potential uses beyond establishing a strong lower bound on the maximum possible accuracy of unlexicalized models: an unlexicalized PCFG is much more compact, easier to replicate, and easier to interpret than more complex lexical models, and the parsing algorithms are simpler, more widely understood, of lower asymptotic complexity, and easier to optimize.

This paper separates conditional parameter estima- tion, which consistently raises test set accuracy on statistical NLP tasks, from conditional model struc- tures, such as the conditional Markov model used for maximum-entropy tagging, which tend to lower accuracy. Error analysis on part-of-speech tagging shows that the actual tagging errors made by the conditionally structured model derive not only from label bias, but also from other ways in which the independence assumptions of the conditional model structure are unsuited to linguistic sequences. The paper presents new word-sense disambiguation and POS tagging experiments, and integrates apparently conflicting reports from other recent work.

We discuss two named-entity recognition models which use characters and character n-grams either exclusively or as an important part of their data representation. The first model is a character-level HMM with minimal context information, and the second model is a maximum-entropy conditional markov model with substantially richer context features. Our best model achieves an overall F1 of 86.07% on the English test data (92.31% on the development data). This number represents a 25% error reduction over the same model without word-internal (substring) features.

We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to the basic model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task.

This paper discusses ensembles of simple but heterogeneous classifiers for word-sense disambiguation, examining the Stanford-CS224N system entered in the SENSEVAL-2 English lexical sample task. First-order classifiers are combined by a second-order classifier, which variously uses majority voting, weighted voting, or a maximum entropy model. While individual first-order classifiers perform comparably to middle-scoring teams’ systems, the combination achieves high performance. We discuss trade-offs and empirical performance. Finally, we present an analysis of the combination, examining how ensemble performance depends on error independence and task difficulty.

erative clustering. First, we show formally that the common heuristic agglomerative clustering algorithms – Ward’s method, single-link, complete-link, and a variant of group-average – are each equivalent to a hierarchical model-based method. This interpretation gives a theoretical explanation of the empirical behavior of these algorithms, as well as a principled approach to resolving practical issues, such as number of clusters or the choice of method. Second, we show how a model-based viewpoint can suggest variations on these basic agglomerative algorithms. We introduce adjusted complete-link, Mahalanobis-link, and line-link as variants, and demonstrate their utility.

A* PCFG parsing can dramatically reduce the time required to find the exact Viterbi parse by conservatively estimating outside Viterbi probabilities. We discuss various estimates and give efficient algorithms for computing them. On Penn treebank sentences, our most detailed estimate reduces the total number of edges processed to less than 3% of that required by exhaustive parsing, and even a simpler estimate which can be pre-computed in under a minute still reduces the work by a factor of 5. The algorithm extends the classic A* graph search procedure to a certain hypergraph associated with parsing. Unlike bestfirst and finite-beam methods for achieving this kind of speed-up, the A* parser is guaranteed to return the most likely parse, not just an approximation. The algorithm is also correct for a wide range of parser control strategies and maintains a worst-case cubic time bound.

We present an improved method for clustering in the presence of very limited supervisory information, given as pairwise instance constraints. By allowing instance-level constraints to have spacelevel inductive implications, we are able to successfully incorporate constraints for a wide range of data set types. Our method greatly improves on the previously studied constrained -means algorithm, generally requiring less than half as many constraints to achieve a given accuracy on a range of real-world data, while also being more robust when over-constrained. We additionally discuss an active learning algorithm which increases the value of constraints even further.