Why JSON Is Suddenly a Problem in the Age of AI?

For years, JSON has been our go-to format for structured data. APIs love it. Databases accept it. Developers practically breathe it. It's simple, predictable, and universal.

But here’s the twist: what worked perfectly fine for traditional software doesn’t always translate well to AI models, especially large language models (LLMs) like GPT, Claude, or Gemini.

The culprit? Every single character

For those who are new to JSON

JSON (JavaScript Object Notation) is a lightweight text format used to represent data as key-value pairs.

It’s easy for humans to read and easy for machines to parse, which is why it became the standard for APIs, configuration files, and web applications.

Its clean structure and wide support made it the backbone of modern software systems.

Example JSON

{
  "users": [
    { "id": 1, "firstName": "Dipendra", "interests": ["music", "tech"] },
    { "id": 2, "firstName": "Ajay", "interests": ["coding", "books"] }
  ]
}

Why JSON Becomes Expensive Inside LLMs?

When you send JSON to an LLM, it doesn’t just “read” it. It tokenizes it, i.e, it turns every brace, every comma, every space, every quotation mark into tokens.

tokens = money + latency

A verbose JSON payload can:

• Burn more tokens than necessary
• Becomes slower for the LLM to interpret
• Lower accuracy due to extra noise
• Increase your operational costs (dramatically, in some cases)

So while JSON is great for APIs, it’s not necessarily great for LLM prompts.

What is TOON?

TOON (Token-Oriented Object Notation) is a format designed specifically for LLMs. Think of it as JSON’s minimalist cousin who cares deeply about efficiency.

Instead of braces and quotes everywhere, TOON uses a mix of YAML-style indentation and CSV-like simplicity to express structure.

The result? A clean, compact format that sends less fluff and more signal to the model.

Why TOON Is a Game-Changer?

Here’s what TOON brings to the table:

• 30-60% fewer tokens compared to JSON
• Faster LLM responses because there’s less text to process
• Lower costs (this gets very real at scale)
• Surprisingly human-readable
• Especially powerful for large, flat datasets

If you’re building AI features that rely on structured inputs, this is huge.

Let’s Compare: JSON vs TOON

Here’s a classic example.

JSON Version

{
 "courses": [
 { "id": 101, "name": "APA Mastery", "price": 99, "rating": "4.9" },
 { "id": 102, "name": "AI & Automation", "price": 79, "rating": "4.3" },
 { "id": 103, "name": "AI for Leaders", "price": 89, "rating": "4.5" }
 ]
}

TOON Version

courses[3]{id, name, price, rating}:
101, APA Mastery, 99, 4.9
102, AI & Automation, 79, 4.3
103, AI for Leaders, 89, 4.5

No curly brackets. No quotes. No repetition.

Just clean, tabular-style data that still keeps structure intact without wasting tokens.

Real-World Results

In performance tests highlighted in the TOON introduction video:

• A dataset that costs 10,000 tokens in JSON
• Required only ~4,500 tokens in TOON

That’s a 55% reduction straight off the top.

But a small caveat: With highly nested data, TOON can become less efficient than JSON. So if your data is deeply hierarchical, flatten it first.

How to Start Using TOON in Your AI Applications?

Good news: You don’t have to abandon JSON entirely.

Here’s the simple workflow:

1. Keep your data in JSON internally (APIs, DBs, business logic).
2. Flatten your structure as much as reasonably possible.
3. Run it through a TOON encoder (there’s a TypeScript reference implementation).
4. Pass the TOON output to your LLM instead of the full JSON.

No major rewrites. No new database schema. Just a more efficient format at the very last mile.

When does TOON shine?

TOON works exceptionally well in:

• LLM system prompts
• Structured prompt engineering
• Large flat datasets
• Readable logs and debugging contexts
• Workflow agents pulling external data

Basically, anywhere JSON feels “too heavy.”

Where TOON Isn’t Ideal?

To keep things realistic:

• Deeply nested JSON may expand instead of shrink
• Not meant for long-term storage
• Not a replacement for your existing API contracts

Think of TOON as a transport optimization just for LLMs, not a universal data format.

Final Thoughts

TOON is new, but it’s gaining traction fast, especially among AI builders who are hitting token limits, speed issues, and prompt-cost bottlenecks.

If your app relies on sending structured data to LLMs, give TOON a shot. Flatten your JSON, encode it, and compare the token usage yourself.

Chances are, you’ll see a noticeable boost in:

• Speed
• Clarity
• Cost savings

And once you start paying attention to token efficiency, you realize how many other parts of your AI stack can be optimized.

Happy TOONing!