---
title: "Beware the Rationalization Trap! : A sketchnote"
slug: beware-the-rationalization-trap-a-sketchnote
canonical_url: https://nlpurr.github.io/posts/beware-the-rationalization-trap-a-sketchnote/
collection: Sketchnotes
published_at: 2022-07-28T00:00:00.000Z
updated_at: 2022-07-28T00:00:00.000Z
tags: 
  - Explanation
  - LLMs
author: NLPurr
---

## Navigation Context

- Canonical URL: https://nlpurr.github.io/posts/beware-the-rationalization-trap-a-sketchnote/
- You are here: Home > Posts > Sketchnotes > Beware the Rationalization Trap! : A sketchnote

### Useful Next Links
- [Home](https://nlpurr.github.io/)
- [Posts](https://nlpurr.github.io/collections/posts/)
- [Sketchnotes](https://nlpurr.github.io/collections/sketchnotes/)

As I mentioned before, I am really glad to be able to merge my love for NLP with that of sketchnoting for science communication.  
  
Today's paper is "Beware the Rationalization Trap!" by [@RSevastjanova](https://www.twitter.com/RSevastjanova) and [@manunna\_91](https://www.twitter.com/manunna_91)

Explanations can either match or be on odds with any user's mental model. A good explanation should be truthful, have no missing reasonings, and account for the users' mental models.

[![Image uploaded to Notion](https://nlpurr.github.io/_astro/Untitled.DkIquGAb_XTth7.webp)](https://nlpurr.github.io/_astro/Untitled.DkIquGAb_XTth7.webp)

Saliency, tuple & concepts are the main explanation types used, though, counterfactual explanations are preferred. Humans decide explanation goal, judge sensibility, &, develop mental models.  
  
Mental models, amongst other things, should be personal, evolving & contrastive.

[![Image uploaded to Notion](https://nlpurr.github.io/_astro/Untitled.Dg9UQnPL_ZcrQkQ.webp)](https://nlpurr.github.io/_astro/Untitled.Dg9UQnPL_ZcrQkQ.webp)

When comparing the explanation to the LLM's behavior, we either end up with high or low fidelity explanations. Depending on the alignment with a user's mental model, a person can leave with either a truthful understanding, an untruthful understanding, or no understanding.

[![Image uploaded to Notion](https://nlpurr.github.io/_astro/Untitled.mAWDrHto_Z1yN6qJ.webp)](https://nlpurr.github.io/_astro/Untitled.mAWDrHto_Z1yN6qJ.webp)

One of the main aims of any model explanation decision, should be supporting mental model calibration through: (a) no false generalization, (b) human-like explanations, (c) catch pitfalls, (d) contrastive explanations, (e) adaptive process, and, (f) collaborative learning.

[![Image uploaded to Notion](https://nlpurr.github.io/_astro/Untitled.CyDbTIQJ_Z2jpBpd.webp)](https://nlpurr.github.io/_astro/Untitled.CyDbTIQJ_Z2jpBpd.webp)

Here is the complete graphical abstract for the paper.

[![Image uploaded to Notion](https://nlpurr.github.io/_astro/Untitled.D7g06RxM_hBfg3.webp)](https://nlpurr.github.io/_astro/Untitled.D7g06RxM_hBfg3.webp)