Smartypants corrupts Unicode punctuation when rendering MDX content collections via render() on Windows

[HikeAndMap]

Astro Info

Astro                    v7.1.6
Vite                     v8.1.5
Node                     v24.18.1
System                   Windows (x64)
Package Manager          npm
Output                   static
Adapter                  none
Integrations             @astrojs/mdx (v7.0.5)

If this issue only occurs in one browser, which browser is a problem?

Happens in FireFox and Chrome, didn't test others

Describe the Bug

Describe the bug

When rendering an MDX content collection entry using render() from astro:content, enabling markdown.smartypants causes typographic punctuation to be corrupted into mojibake on Windows.

For example:

• Don’t becomes Don’
• “Hello” becomes “Hello�

This only occurs when rendering an entry from a content collection using render(). A direct .mdx page renders correctly.

Environment

• Astro: 7.1.6
• @astrojs/mdx: 7.0.5
• @mdx-js/mdx: 3.1.1
• Node.js: 24.18.1 (LTS)
• npm
• Windows 11

What I tested

I spent quite some time isolating the issue.

✅ Source file encoding

The .mdx file is saved as UTF-8.

I verified this by:

• Saving the file explicitly as UTF-8.
• Confirming the encoding in Notepad++.
• Inspecting the file with Format-Hex.

The UTF-8 bytes are correct:

• ASCII apostrophe (') → 27
• Right single quotation mark (’) → E2 80 99

The source file itself is not corrupted.

✅ Content collection

getCollection() returns the correct content.

For example:

console.log(lesson.body);

prints:

I don’t like it

which proves the content is read correctly before rendering.

✅ Direct MDX pages

A page such as:

src/pages/direct.mdx

renders perfectly:

Don’t

“I don’t like it”

No corruption occurs.

❌ Rendering through astro:content

Rendering the same content using:

const { Content } = await render(lesson);

produces HTML containing:

Don’t

“I don’t like it�

Viewing the generated HTML source confirms the mojibake is already present before the browser renders the page.

✅ Disabling Smartypants fixes the problem

Changing:

markdown: {
  smartypants: false,
}

immediately fixes the issue.

The same content then renders correctly:

Don't

"I don't like it"

Summary

The issue appears to occur only when all of the following are true:

• Rendering an MDX content collection entry using render() from astro:content
• markdown.smartypants is enabled
• Running on Windows

Direct .mdx pages do not exhibit this behavior, and the content returned by getCollection() is already correct before rendering.

This suggests the corruption occurs somewhere during the rendering pipeline rather than while reading the source file.

What's the expected result?

When rendering an MDX content collection entry using render() from astro:content, enabling markdown.smartypants should produce valid typographic punctuation without corrupting the output.

For example, the following Markdown:

Don't

Don’t

"I don't like it"

"I don’t like it"

should render as:

Don't

Don’t

“I don't like it”

“I don’t like it”

or, if Smartypants is expected to convert straight quotes:

Don’t

Don’t

“I don’t like it”

“I don’t like it”

In either case, the rendered HTML should contain valid Unicode punctuation (’, “, ”) and should never contain mojibake such as ’, “, or �.

Link to Minimal Reproducible Example

I will create that later - right now not in a position to do

Participation

• I am willing to submit a pull request for this issue.

[astrobot-houston]

• Reproduced: Skipped — Windows-specific issue, cannot reproduce in Linux sandbox
• Exploration: No
• Unit Test: No — could not reproduce the issue on Linux
• Priority: P3: minor bug. The issue only affects Windows users rendering MDX content collections with smartypants enabled, and has a trivial workaround (disabling smartypants). Direct .mdx pages are unaffected.

The reported corruption pattern (’ for ', “ for ") is a classic sign of UTF-8 bytes being decoded as Windows-1252/Latin-1. The UTF-8 encoding for ' (U+2019) is bytes E2 80 99, and when interpreted as Latin-1 those three bytes produce ’. This only manifests when rendering MDX content collection entries via render() with smartypants enabled on Windows — direct .mdx pages and getCollection() body content are both correct, suggesting the encoding mismatch occurs somewhere in the content collection rendering pipeline after smartypants inserts the curly quote characters.

I was unable to determine the cause of this issue. The bug could not be reproduced on Linux — both content collection rendering and direct MDX pages produce correctly-encoded UTF-8 curly quotes. A Windows CI environment would be needed to reproduce and pinpoint where in the rendering pipeline the encoding is being mishandled. Areas to investigate include how the MDX compilation output is read/written during content collection rendering (potentially in packages/astro/src/content/ or packages/integrations/mdx/src/), and whether any file I/O in that path uses a platform-default encoding instead of explicitly specifying UTF-8.

Workaround: Set markdown: { smartypants: false } in astro.config.mjs to avoid the corruption.

Full Triage Report

Issue #17555: Smartypants corrupts Unicode punctuation when rendering MDX content collections via render() on Windows

Original Issue

Author: HikeAndMap
URL: #17555

Description

When rendering an MDX content collection entry using render() from astro:content, enabling markdown.smartypants causes typographic punctuation to be corrupted into mojibake on Windows.

Examples of corruption:

• Don't becomes Don’
• "Hello" becomes “Helloâ€

The mojibake pattern (’ for ', “ for ") is the classic symptom of UTF-8 bytes being decoded as Windows-1252/Latin-1. The UTF-8 encoding for ' (U+2019) is E2 80 99, and when those three bytes are interpreted as Latin-1, they produce ’.

Reporter's Environment

• Astro: 7.1.6
• @astrojs/mdx: 7.0.5
• @mdx-js/mdx: 3.1.1
• Node.js: 24.18.1
• npm
• Windows 11

Reporter's Findings

• Source .mdx files are correctly encoded as UTF-8
• getCollection() returns correct content (no corruption at read time)
• Direct .mdx pages render correctly (no corruption)
• Content collection entries rendered via render() show corruption
• Disabling smartypants: false fixes the issue
• Only occurs on Windows

Reproduction Attempt

Setup

Created a minimal reproduction project in triage/gh-17555 with:

1. An MDX content collection with a test entry containing straight quotes/apostrophes
2. A page rendering the content collection entry via render()
3. A direct MDX page with the same content for comparison
4. markdown.smartypants: true in the Astro config

Content Used

---
title: Test Lesson
---

Don't

Don't

"I don't like it"

"I don't like it"

Results (Linux)

Build succeeded. Both the content collection page (/index.html) and direct MDX page (/direct/index.html) rendered correctly with proper Unicode curly quotes.

Content collection page output:

<p>Don't</p>
<p>Don't</p>
<p>"I don't like it"</p>
<p>"I don't like it"</p>

Hex inspection confirmed proper UTF-8 encoding:

• e2 80 99 = ' (RIGHT SINGLE QUOTATION MARK)
• e2 80 9c = " (LEFT DOUBLE QUOTATION MARK)

No corruption observed on Linux.

Conclusion

Skipped — host-specific (Windows). The issue explicitly requires Windows to reproduce. The reporter confirmed the bug only occurs on Windows 11, and our Linux-based sandbox produces correct output for both content collection rendering and direct MDX pages.

The encoding corruption pattern (UTF-8 bytes read as Latin-1/Windows-1252) suggests the issue may be in how Node.js or the rendering pipeline handles file encoding on Windows, possibly related to system locale or default encoding settings that differ between Windows and Linux.

This report was made by an LLM. The analysis may be wrong, and the potential fix might not work, but is intended as a starting point for exploring the issue.