Understanding Semantics

Understanding Semantics

Learn how TrueTrends uses semantic search to understand meaning, not just keywords, delivering insights that reflect how people actually think and communicate.

The Core Difference: Keywords vs. Meaning

Traditional Keyword Search (Google Style)

When you search for "cheese" and "fries" separately, traditional systems just count exact matches:

  • cheese = raw count of posts containing "cheese"
  • fries = raw count of posts containing "fries"
  • cheese fries = cheese + fries (simple addition)

Semantic Search (TrueTrends Style)

We don't look for exact keywords. Instead, we analyze meanings captured in mathematical representations called embeddings:

  • cheese = posts discussing cheese-related concepts (cheddar, mozzarella, cheesy, dairy)
  • fries = posts about fries-related concepts (french fries, chips, crispy potatoes)
  • cheese fries = posts about the combined concept (loaded fries, cheesy chips, chili cheese fries)

Real Example: The Cheese Fries Phenomenon

Let's examine actual search results to understand why semantic counts differ:

The Numbers

  • Cheese = 23,225 posts
  • Fries = 12,012 posts
  • Cheese fries = 24,068 posts

Why Cheese Fries ≠ Cheese + Fries

The semantic count for "cheese fries" (24,068) is not the sum of cheese (23,225) + fries (12,012). Here's why:

Different Conversation Types

  1. "I love cheese" → counted in cheese only
  2. "I had fries today" → counted in fries only
  3. "I had cheese fries at Shake Shack" → counted in cheese fries
  4. "I had loaded fries" → counted in cheese fries (semantic similarity)

The last group reveals why counts differ: "cheese fries" represents a distinct concept, not just two separate ingredients mentioned together.

Simple Analogy

Think of it like asking:

  • "How many people like bread?" → 50 people
  • "How many people like butter?" → 40 people
  • "How many people like bread and butter?" → Could be 70 people

Why? Because "bread and butter" is its own cultural concept with its own following, separate from people who just like bread or just like butter.

Visual Understanding

Semantic Search Venn Diagram

Not every mention of "cheese" contributes to "cheese fries," and not every mention of "fries" contributes to "cheese fries." The overlap represents a unique semantic space.

Technical Deep Dive

Every word or phrase converts into a vector (a list of numbers that represents meaning):

cheese → v₁ = [0.8, 0.1, 0.5]
fries → v₂ = [0.2, 0.9, 0.4]
cheese fries → v₃ = [0.7, 0.8, 0.6]

Note: These are simplified 3D vectors for illustration. Actual embeddings use hundreds of dimensions.

How Qdrant Vector Database Works

  1. Storage: All social media posts convert to vectors and store in our vector database
  2. Updates: New posts continuously convert to vectors during data updates
  3. Search Process:
    • Query "cheese fries" converts to vector v₃
    • Qdrant compares v₃ against all stored post vectors
    • Returns posts with highest similarity scores

Cosine Similarity: The Magic Behind Meaning

Cosine similarity measures the angle between vector "arrows" in mathematical space:

Cosine Similarity Formula

Understanding the Scale

  • Similarity = 1: Arrows point same direction (very similar meaning)
  • Similarity = 0: Arrows at 90° angle (unrelated concepts)
  • Similarity = -1: Arrows point opposite directions (opposing meanings)

Why This Matters

  • Direction over Magnitude: Similarity cares about what words mean, not how often they appear
  • Contextual Understanding: "Loaded fries" and "cheese fries" point in similar directions
  • Semantic Neighborhoods: Related concepts cluster together in vector space

Why Counts Diverge

When you search different terms, you're exploring different regions of semantic space:

Vector Search Visualization

  • "cheese" → finds posts near vector v₁
  • "fries" → finds posts near vector v₂
  • "cheese fries" → finds posts near vector v₃ (a distinct location)

The result: 23,225 + 12,012 ≠ 24,068

The embedding for "cheese fries" occupies its own neighborhood where posts about the combined concept naturally cluster.

Why This Approach Matters

Business Intelligence Benefits

  • Trend Discovery: Find emerging concepts before they become obvious
  • Market Understanding: Capture how customers actually talk about products
  • Competitive Analysis: Understand semantic associations around brand mentions
  • Content Strategy: Align messaging with natural language patterns

Real-World Applications

  • A restaurant chain discovers "loaded fries" trends without explicitly searching for it
  • A food brand identifies cultural shifts in comfort food preferences
  • Marketers understand the semantic space around their product category
  • Analysts capture sentiment toward concept clusters, not just individual keywords

Key Takeaways

  1. Semantic search finds meaning, not just keyword matches
  2. Embeddings create mathematical maps of language and concepts
  3. Vector similarity discovers relationships between ideas
  4. Counts reflect semantic neighborhoods, not arithmetic operations
  5. This approach mirrors human understanding better than traditional search

Understanding these principles helps you interpret TrueTrends data more effectively and discover insights that keyword-based tools miss.

TrueTrends Help Center - Documentation & Support