Communicating with impact: harnessing data from nonverbal cues

Few people will refute the notion that being able to exude confidence, charm, and competence is key to attaining social and material success. And while some individuals are naturally more attuned to implicit meanings conveyed by others in social circumstances, research supports that being “pragmatically intelligent” or “socially attuned” is a skill that can be developed and honed to a significant extent.

In his book “Silent Messages”, Dr. Albert Mehrabian discusses the fact that only 7% of the meaning in human to human communication is conveyed through words (syntax & semantics). The rest comes from nonverbal means of communication such as tonality (prosody), kinesic cues (facial expressions, hand gestures, stance, posture), proxemic cues, haptic cues and others . In essence, what this research highlights is that without awareness of how others are coming across to you and how you come across to them, you’re missing out on most of the meaning conveyed through the interactions you’re participating in. Understanding these nonverbal cues and harnessing their power can help individuals communicate more efficiently and reach their goals more effectively. In other words, whether the “objective” is to close a new business deal, book a romantic date, or get a salary raise, the non-verbal signals conveyed along with the “formal message” are no less than outcome-defining

Back in 1936, Dale Carnegie wrote “How to Win Friends and Influence People”, an all-time best seller in the business and personal-development categories. People have always admired the confidence, charisma, and influence of who they perceived to be leaders of their generation in business, politics, religion, culture, music and even sports. Many wondered how they were able to command such respect? What is their secret? How do they showcase so much power and competence relative to others?

Showcasing competence, dominance and relaxed confidence are fundamental to our perception of a person’s ability to lead and overcome obstacles and are typically observed in the context of interpersonal conflicts and collisions of perspectives, also called frames. If you’re looking to move up the career/social ladder, gaining a deeper understanding of implicit social signals and how they are conveyed through nonverbal communication can give you an edge in society. 

But what if technology could help us do that by helping us “read” situations more deeply and understand the power dynamics and “true meaning” in high-stake situations? 

The idea of leveraging technology to measure and analyze emotion is an obvious one – especially as computer vision technologies evolve. In fact, Gartner highlighted thirteen different uses for emotion AI in technology. These range from Software that can help doctors with the diagnosis of diseases such as depression and dementia by using voice analysis, or technology that can be used during job interviews to understand the credibility of a candidate.

At Substrata, we have lined up some of the world’s top researchers in the fields of Language Pragmatics, Social Signal Processing and Nonverbal Communication to create a technology that helps business professionals become better dealmakers. Our platform, called Q (Beta) analyzes interaction data in email threads to help users understand what is the true sentiment of each message and who has the upper-hand in each situation along the relationship – and ultimately predict the prospect’s propensity to engage further or buy what the user is offering.

There used to be a huge blind spot between technology and social psychology. While software solutions that deal with human communications have traditionally focused on syntax (formal language), there is a huge opportunity in delving deeper into pragmatics, developing the  technologies that help us “read between the lines”. Think of the Sales Technology market: there are thousands of tools designed to help sales people prioritize, engage, earn, close, and upsell – from CRMs to management and Sales-Ops platforms. These tools help manage the structured data around the sales process, but shy away from focusing on the one thing that matters most – the seller-prospect human dynamics. Once the layer of social intelligence, intentionality and influence is added to these solutions, the entire landscape as we know it is likely to change.

Credits: Speech Acts Theory was Developed by John Austin (1962) & elaborated by John Searle (1969) | Conversational Implicature was developed by Paul Grice (1989), Professor Thomas M. Holtgraves, Ball State University

This post will delve into two of the most prominent socio-linguistic theories:

  1. Speech Acts. Developed by John Austin (published posthumously in 1961/’62) and expanded upon by John Searle (1981), this theory explicitly conceptualizes linguistic meaning as “use.” Put simply, the intended meaning of an utterance is the “use” of that utterance.
  2. Conversational Implicature. This approach to non-literal meaning was developed by Paul Grice, whose views have been extremely influential on the field of pragmatics.

John Langshaw Austin (1911 – 1960), the father of “Speech Acts” theory | Stanford Encyclopedia

Austin’s speech acts theory arose from his observation that it’s impossible to determine the “truth” value of many utterances because they are extremely contextual and depend on many dynamic factors. For example, what is the truth value of the utterance, “I promise to do it tonight”? Austin suggested that all speech acts have a dimension of meaning AND a particular force, concluding that any utterance involves the simultaneous performance of a number of different acts. These acts include: 1) a Locutionary Act, which imparts basic phonetic, syntactic, and semantic meaning; 2) an Illocutionary Act, or the “act of saying,” which includes the force of change or vector of influence on reality generated by the utterance when it is spoken, as intended by the speaker; and 3) a Perlocutionary Act, which refers to the effect(s) the utterance has on the listener/receiver of the message.

It is clear today that Austin’s speech act theory marked the beginning of a new way of viewing language that placed emphasis on language as an action rather than as an abstract system for describing reality.

Enter John Searle, who provided a framework for more precisely specifying the actions that can be accomplished with language. This framework also defines the relationships between those actions, one’s words, and the mental states of the interlocutors.

Searle systematized and extended speech theory in several directions by offering 4 Felicity Conditions for Requests:

  1. Propositional Content, which predicates future act(s) taken by the listener
  2. Preparatory Condition, in which the speaker believes the listener is able to perform an act, although he or she has not yet done so
  3. Sincerity Condition, in which the speaker sincerely wants the listener to perform an act
  4. Essential Condition, in which the utterance counts as an attempt to get the listener to perform an act

According to Searle, utterances have 5 conventionally recognized illocutionary points:

  • Directive: an attempt to get the receiver of the message to perform some future action (requesting, ordering, questioning, etc). The speaker is attempting to alter “the world” with words; in a “world-to-words” fit, the speaker attempts to bring the world in line with his words.
  • Assertive: an attempt to present an actual state of affairs, committing the speaker to something “being the case it is presented to be.” Assertives can involve concluding, asserting, informing, predicting, and reporting.
  • Commissive: an attempt to commit the speaker to a future course of action, include promising, warning, threatening, guaranteeing, etc. Commissives are also a “world-to-words” fit but, unlike directives, it is the speaker’s actions that will change the world rather than the listener’s.
  • Declarative: an attempt to bring about a change in some institutional state of affairs. These may include declaring war, performing a marriage, cancelling a person’s right to freedom due to a felony, etc.
  • Expressive: an attempt to express a psychological state. Examples of expressives include thanking, complaining, greeting, apologizing, etc.

Speech Acts and Intentions

Paul Grice contributed to the field in 1957 and 1989 by making important distinctions between signs and signals.

Signs convey information, whether with or without the recognition of the speaker’s intention, and are perceived as relatively “objective” in nature.

Signals, on the other hand, convey non-natural meaning (meaning-nn) and are communicative acts that achieve their ends by virtue of words, the listener’s recognition of those words, and the speaker’s intention to fulfill their meaning.

Grice’s Conversational Implicature

Herbert Paul Grice (1913-1988), a British philosopher of language whose work on meaning influenced the study of semantics and pragmatics

While Grice’s theory is considered brief and not completely “worked out,” its insights are profound and have had a tremendous impact on many areas of research. The theory is concerned with what today is called “language pragmatics,” and involves understanding how people convey non-literal meaning; in other words, how we mean more than (or something completely different from) what we say. Grice coined the term “Cooperative Principle” (CP), defined as follows: “One should make his conversational contribution such as is required, at the state which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged.”

Simply put, people should communicate in a rational and efficient manner, say what they mean, and stick to the facts without letting their conversation ramble. Grice further clarified these requirements in the following four “conversational maxims”:

  • Quantity – make your contribution as informative as required (neither more nor less)
  • Quality – make your contribution as factual (evidence-based) and truthful as possible
  • Manner – be clear; avoid ambiguity and obscurity
  • Relation – make your contribution relevant to the exchange

While clearly many (if not most) conversationalists do not abide by these guidelines, it is generally assumed that they represent a framework for understanding other people’s utterances. Consider the example,”Can you pass the salt?” The pragmatic meaning of this sentence is, “Please pass me the salt,” and it is a classic instance of implicature that arises to preserve adherence to a specific conversational maxim.

The more explicit an intended deviation from a maxim—e.g., abrupt topic changes such as “I hope it stops raining soon” in response to a personal question like “How’s everything going with your wife?”—the farther away the pragmatical/contextual meaning is from the literal (syntactic/lexical) one. It’s interesting to note that, if the speaker violating the maxim holds a mutually agreed upon superior social status rank, it is likely that, to avoid a conflict, the listener will supplicate by re-legitimizing the violating utterance with another utterance that is general enough to put the answer in a logical perspective.

Grice’s theory can be considered an important adjunct to Speech Act theory. It provides a framework by which speakers convey—and listeners recognize—illocutionary/pragmatic forces.

Grice’s Conversational Implicature is also importance because a person’s intended meaning (illocutionary force) cannot be derived from semantics alone. Context (including the two parties’ mutually assumed knowledge) is crucial in determining a speaker’s true and full meaning.

Grice’s baseline assumption, as reflected by his maxims, is that perfectly “clean” communication requires complete cooperativeness. Violations, therefore, bring in the pragmatic “noise” of signals.

I would argue that a “perfectly cooperative” communication (or an attempt to maintain such) is likely to occur between a mutually perceived inferior subject and a mutually perceived superior one, and only in that direction. A mutually perceived superior subject is expected to violate the maxims (in one or more forms of syntax) to maintain the “broadcasting” of his or her social value. In fact, I tend to believe that the higher the social status gap between the two subjects, the higher the expected level of violation.

This “violation vector” is hard-wired into our brains from early childhood. The classic parent-child relationship, for example, is very asymmetrical. What’s more interesting is that once we reach maturity, we attribute authority to violation and violation to authority. Theoretically speaking, a previously mutually perceived inferior subject can “level the playing field” to an extent by signaling superiority via moderate violations. As long as these violations remain explicit and moderate, the perceived social status gap (maintained by the superior) will start to shrink and the inferior will gain a social advantage in the situation.