Gabriel Greenberg

Visual representations express distinctively visual content. Such content takes the form of a kind of space where objects and properties are assigned locations in relation to a viewpoint. Many have conceived of visual space as a metric three-dimensional volume, analogous to physical space. Yet this assumption, I argue, over-constrains visual content, excluding the ubiquitous phenomenon of indeterminate depth perception. In this paper, I propose that visual contents are view spaces: two-dimensional directional arrays of objects, properties and relations. View spaces prioritize visual direction as a core aspect of structure, while demoting depth to a variable feature like colour or shape. This proposal accommodates depth indeterminacy while preserving distinctive visual structure, and it aligns with the use of feature maps in vision science and computer vision. I will argue that this architectural differentiation of direction and depth is supported by a wide range of evidence from philosophy and psychology.

In this paper, we describe two film conventions for representing what a character sees: point of view (POV) and sight link. On a POV interpretation, the viewpoint of a shot represents the viewpoint of a particular character; while in sight link, a shot of a character looking off-screen is associated with a shot of what they are looking at. Our account of both treats them as spatial in nature, and relates them to similar spatial interpretative principles that generalize beyond character eyelines. We additionally observe that whether the object shot in a sight link takes a POV interpretation depends on whether the glance shot precedes or follows it. We offer a processing account of this effect of order on interpretation, which relies on the assumption that spatial coherence in film is established by incremental viewpoint grounding.

This chapter develops a semantic theory for maps and situates it within the broader geography of meaning and semiotic significance. The discussion focuses on three central aspects of map semantics: the use of space, line marking, and linguistic tags. It is argued that the treatment of space in maps must be based on geometrical projection from a viewpoint rather than the traditional analysis in terms of spatial isomorphism. The chapter then shows how to integrate the projection-based semantics of maps and the symbolic semantics of markers and tags, within the framework of possible-world semantics. The resulting account addresses issues of accuracy, viewpoint, orientation, and exhaustivity.

Tagging is the phenomenon in which regions of a picture, map, or diagram are annotated with words or other symbols, to provide descriptive information about a depicted object. The interpretive principles that govern tagged images are not well understood, due in part to the difficulty of integrating pictorial and linguistic semantic rules. Rather than directly combining these rules, I propose to use the framework of perspectival feature maps as an intermediary representation of content, in which the outputs of pictorial and linguistic interpretation may be assimilated. The result is a simple and compositional semantics for tagged images.

It is common to distinguish two great families of representation. Symbolic representations include logical and mathematical symbols, words, and complex linguistic expressions. Iconic representations include dials, diagrams, maps, pictures, 3-dimensional models, and depictive gestures. This essay describes and motivates a new way of distinguishing iconic from symbolic representation. It locates the difference not in the signs themselves, nor in the contents they express, but in the semantic rules by which signs are associated with contents. The two kinds of rule have divergent forms, occupying opposite poles on a spectrum of naturalness. Symbolic rules are composed entirely of primitive juxtapositions of sign types with contents, while iconic rules determine contents entirely by uniform natural relations with sign types. This distinction is marked explicitly in the formal semantics of familiar sign systems, both for atomic first-order representations, like words, pixel colors, and dials, and for complex second-order representations, like sentences, diagrams, and pictures.

This paper presents an analysis of face emoji (disc-shaped pictograms with stylized facial expressions) that accompany written text. We propose that there is a use of face emoji in which they comment on a target proposition expressed by the accompanying text, as opposed to making an independent contribution to discourse. Focusing on positively valenced and negatively valenced emoji (which we gloss as _happy_ and _unhappy_, respectively), we argue that the emoji comment on how the target proposition bears on a contextually provided discourse value endorsed by the author. Discourse values embody what an author desires, aspires to, wishes for, or hopes for. Our analysis derives a range of non-trivial generalizations, including (i) ordering restrictions with regards to the placement of emoji and text, (ii) cases of apparent mixed emotions, and (iii) cases where the lexical content of the accompanying text influences the acceptability of a face emoji.

This essay calls attention to a set of linguistic interactions between counterfactual conditionals, on one hand, and possibility modals like could have and might have, on the other. These data present a challenge to the popular variably strict semantics for counterfactual conditionals. Instead, they support a version of the strict conditional semantics in which counterfactuals and possibility modals share a unified quantificational domain. I’ll argue that pragmatic explanations of this evidence are not available to the variable analysis. And putative counterexamples to the unified strict analysis, on careful inspection, in fact support it. Ultimately, the semantics of conditionals and modals must be linked together more closely than has sometimes been recognized, and a unified strict semantics for conditionals and modals is the only way to fully achieve this.

A semantics of pictorial representation should provide an account of how pictorial signs are associated with the contents they express. Unlike the familiar semantics of spoken languages, this problem has a distinctively spatial cast for depiction. Pictures themselves are two-dimensional artifacts, and their contents take the form of pictorial spaces, perspectival arrangements of objects and properties in three dimensions. A basic challenge is to explain how pictures are associated with the particular pictorial spaces they express. Inspiration here comes from recent proposals that analyze depiction in terms of geometrical projection. In this essay, I will argue that, for a central class of pictures, the projection-based theory of depiction provides the best explanation for how pictures express pictorial spaces, while rival perceptual and resemblance theories fall short. Since the composition of pictorial space is itself the basis for all other aspects of pictorial content, the proposal provides a natural foundation for further pictorial semantics.

This introduction aims to familiarize readers with basic dimensions of variation among pictorial and diagrammatic representations, as we understand them, in order to serve as a backdrop to the articles in this volume. Instead of trying to canvas the vast range of representational kinds, we focus on a few important axes of difference, and a small handful of illustrative examples. We begin in Section 1 with background: the distinction between pictures and diagrams, the concept of systems of representation, and that of the properties of usage associated with signs. In Section 2 we illustrate these ideas with a case study of diagrammatic representation: the evolution from Euler diagrams to Venn diagrams. Section 3 is correspondingly devoted to pictorial representation, illustrated by the comparison between parallel and linear perspective drawing. We conclude with open questions, and then briefly summarize the articles to follow.

This essay argues for a model of pictorial representation which aims to explain the relationship between pictorial content and pictorial accuracy. Focusing on cases where pictures are intended to convey accurate information, the model distinguishes between two fundamental representational relations: on one hand, a picture expresses a content; on the other, it aims at a target scene. Such a picture is accurate when the content it expresses fits the target scene it aims at. In addition, the model follows the traditional division of content into two aspects: singular content specifies the particular individuals which a picture is of, and attributive content specifies the properties and relations which the picture ascribes to those individuals. For a picture to be accurate, both aspects must be matched in the target. I call this the Three-Part Model because it distinguishes between the triad of factors, singular content, attributive content, and target, which together determine pictorial accuracy. Though previous work on depiction has not recognized the distinctive role played by target, I will argue that it is essential in order to make sense of accuracy judgements across a range of central cases.

Because humans cannot know one another’s minds directly, every form of communication is a solution to the same basic problem: how can privately held information be made publicly accessible through manipulations of the physical environment? Language is by far the best studied response to this challenge. But there are a diversity of non-linguistic strategies for representation with external signs as well, from facial expressions and fog horns to chronological graphs and architectural renderings. The general thesis of this dissertation is that there is an impressively wide spectrum of conventional systems of representation, corresponding to the many ways that the problem of communication can be solved, and that these systems can be described and explained using the tools of contemporary mathematical semantics. As a partial corrective to the countervailing norm, this work concentrates on the class of systems arguably most different from language— those governing the interpretation of pictorial images. Such representations dominate practical communication: witness the proliferation of maps, road signs, newspaper photographs, scientific illustrations, television shows, engineering drawings, and even the fleeting imagery of manual gesture. I argue that systems of depiction and languages embody a parallel technologies of communication. Both are based on semantics: systematic and conventional mappings from signs to representational content. But I also provide evidence that these semantics are profoundly divergent. Whereas the semantics of languages are based on arbitrary associations of signs and denotations, the semantics of systems of depiction are based on rules of geometrical transformation. Drawing on recent research in computer graphics and computational vision, I go on to develop a precise theory of pictorial semantics. This in turn facilitates a detailed comparison of iconic, image-based representation, and symbolic, language-based representation. A consequence of these conclusions is that the traditional, language-centric conception of semantics must be overhauled to allow for a more general semantic theory, one which countenances the wide variety of interpretive mechanisms actually at work in human communication.

What is it for a picture to depict a scene? The most orthodox philosophical theory of pictorial representation holds that depiction is grounded in resemblance. A picture represents a scene in virtue of being similar to that scene in certain ways. This essay presents evidence against this claim: curvilinear perspective is one common style of depiction in which successful pictorial representation depends as much on a picture's systematic differences with the scene depicted as on the similarities; it cannot be analyzed in terms of similarity alone. The same problem arises for many other kinds of depiction. The essay concludes that depiction in general is not grounded in resemblance but geometrical transformation.

This paper examines the interplay of semantics and pragmatics within the domain of film. Films are made up of individual shots strung together in sequences over time. Though each shot is disconnected from the next, combinations of shots still convey coherent stories that take place in continuous space and time. How is this possible? The semantic view of film holds that film coherence is achieved in part through a kind of film language, a set of conventions which govern the relationships between shots. In this paper, we develop and defend a new version of the semantic view. We articulate it for a pair of conventions that govern spatial relations between viewpoints. One such rule is already well-known; sometimes called the "180° Rule," we term it the X-Constraint; to this we add a previously unrecorded rule, the T-Constraint. As we show, both have the effect, in different ways, of limiting the way that viewpoint can shift through space from shot to shot over the course of a film sequence. Such constraints, we contend, are analogous to relations of discourse coherence that are widely recognized in the linguistic domain. If film is to have a language, it is a language made up of rules like these.