Is cross-posting duplicate content? I measured it before deleting anything

An AdSense review came back with the “low value content” label and no indication of which pages triggered it. Looking for a plausible cause in my own setup, I landed on the fact that the same articles go out to Zenn, Qiita and dev.to. I wrote a remediation doc: unpublish the five live cross-posts, set canonical everywhere, resubmit.

Before running any of it, I counted how much text actually overlaps. Across 11 articles and 22 files, three prose lines matched — and two of those were horizontal rules. The remediation doc went in the bin, and the work turned into adding canonical URLs on the platforms that accept them.

Measure the overlap before you delete a channel

If you write each syndicated version separately rather than pasting the original, the lines that match will be code, tables and headings. Prose barely survives the rewrite. A single overlap percentage hides that completely, so split the match by line type before drawing any conclusion.

The fix, when there is one, is a canonical URL rather than a deletion. dev.to, Hashnode and Medium all accept one through their APIs. Qiita, Zenn and note do not, which means the option of pointing at an original was never available there in the first place.

What the setup looks like

Aulvem runs a hub-and-spoke layout. The site is the hub; the platforms are spokes that stand on their own and link back. Same subject, different reader and different depth, written from scratch on each side.

Layer	Where	Body file
Hub	aulvem.com	src/content/blog/{en,ja}/<slug>.mdx
Spoke	dev.to / Hashnode	body.en.md
Spoke	Zenn / Qiita	body.ja.md

The measured set covers everything published between 2026-05-17 and 2026-08-08: 11 articles, counted as 22 pairs because English and Japanese are separate files. The syndicated bodies carry no frontmatter and the hub files are MDX, so both sides get their frontmatter stripped before comparison.

The script is about thirty lines

Normalize whitespace, drop blank lines, put the hub’s lines in a Set, and walk the syndicated file. The only part that needs care is the classification.

import fs from "node:fs";

const norm = (s) => s.replace(/\s+/g, " ").trim();

function bodyOf(file) {
  let t = fs.readFileSync(file, "utf8");
  if (t.startsWith("---")) t = t.slice(t.indexOf("\n---", 3) + 4); // strip frontmatter
  return t;
}

// Split into [normalized line, kind]. Everything inside a fence is code.
function classify(text) {
  const out = [];
  let inFence = false;
  for (const raw of text.split("\n")) {
    const line = norm(raw);
    if (/^```/.test(line)) { inFence = !inFence; out.push([line, "code"]); continue; }
    if (!line) continue;
    if (inFence) { out.push([line, "code"]); continue; }
    if (/^#{1,6}\s/.test(line)) { out.push([line, "heading"]); continue; }
    if (/^\|/.test(line)) { out.push([line, "table"]); continue; }
    out.push([line, "prose"]);
  }
  return out;
}

const hub = new Set(classify(bodyOf(hubFile)).map(([l]) => l));
const ext = classify(bodyOf(syndicatedFile));
const matched = ext.filter(([l]) => hub.has(l));

Toggling inFence matters more than it looks. Shell snippets start lines with #, SQL and YAML samples start lines with |, and mermaid blocks are full of both. Classify by leading character alone and a large share of the code lands in the prose bucket, which is the exact number you were hoping to trust.

Dropping blank lines matters too. They always match, so leaving them in inflates both sides of the ratio.

47.9% overlap, and 762 lines of it were code

Printing those three prose lines took a second: two are --- rules, and the third is a single English bullet from the Zod schema article. Nothing else in 393 lines of prose survived from one version to the other.

Per article the pattern holds. High overlap tracks code density, not copying.

Code repeats because both versions describe the same implementation. Rewriting CREATE UNIQUE INDEX uq_outbox_dedup for the syndicated copy would make one of the two articles wrong. Shared snippets are a weak basis for calling your own site a duplicate.

Where a canonical URL fits, and where it does not

flowchart LR
  Devto["dev.to<br/>canonical_url"] --> HP["aulvem.com<br/>(original)"]
  Hashnode["Hashnode<br/>originalArticleURL"] --> HP
  Medium["Medium<br/>canonicalUrl"] --> HP
  Qiita["Qiita<br/>not supported"] -.-> HP
  Zenn["Zenn<br/>not supported"] -.-> HP
  Note["note<br/>not supported"] -.-> HP

The Forem API exposes canonical_url on the article object, so syndicating with an original declared costs one line in the payload.

body: JSON.stringify({
  article: {
    title: fm.title,
    body_markdown: body,
    published: fm.published ?? false,
    tags,
    canonical_url: fm.canonical_url,   // here
  },
}),

Hashnode has a trap worth knowing about. originalArticleURL exists on the publishPost input but not on createDraft. If your pipeline creates a draft and you press publish in the dashboard, the canonical never gets attached.

const input = publish
  ? { title, contentMarkdown, publicationId, tags, originalArticleURL: fm.canonical_url }
  : { title, contentMarkdown, publicationId, tags };   // no place for it on a draft

Qiita’s API v2 docs list title, body, tags, private, tweet and organization_url_name for item creation, with nothing that declares an external original. Zenn’s article frontmatter is title, emoji, type, topics, published and publication_name — same story.

I added canonical_url: https://aulvem.com/blog/<slug>/ to all 11 dev.yaml drafts. Because the value is fixed at post time, the five already-published articles do not pick it up; those need PUT /articles/{id} or a manual edit. That part is still open.

”Temporarily unpublish” is not a reversible operation

On a platform with no canonical field, the way back from unpublishing is to post again — new URL, views reset, inbound links broken. What reads as a temporary measure is a deletion with extra steps.

The traffic makes the trade obvious.

The article I published on 8 August took 196 views on Qiita in its first eight days, with a 221-second average read time. The same article on my own site: 10 page views over the same window. Removing the syndicated copies would have shut down the side where people read, to protect the side where they do not, in exchange for an AdSense placement worth a few hundred yen a month.

The two things I had conflated

The first was treating an AdSense decision as evidence about search-side duplicate handling. AdSense reviews a site against its program policies; search-side duplicate URL consolidation is a normalization step driven by canonical and hreflang. One does not explain the other.

The second was reading “Qiita’s canonical points at Qiita” as a symptom. Every platform without canonical support emits a self-referencing canonical. It is the default state, not a signal about my site.

While those two ideas were in play I had also drafted 17 gadget reviews from 2021 out of the site. Those are back. Hiding writing I produced myself is a strange move when the concern on the table is originality.

Wrapping up

The useful step was counting, not deleting. Thirty lines of script, one run, and the answer — three matching prose lines out of 393 — removed every task that came after it.

For canonical, the rule I settled on is simple: set it where the platform accepts it, accept its absence where it does not. Qiita, Zenn and note give me no way to point at an original, and that is not a reason to stop syndicating to them.

What stayed with me is how close I came to running the remediation first. The same instinct — make breakage visible before acting on a hunch — shows up in the Zod schema enforcement post and in the .xlsx silent breakage post.

FAQ

How do I check whether my cross-posts count as duplicate content?

Normalize both bodies line by line, put the canonical article’s lines in a set, and count how many of the syndicated article’s lines are in it. The part that decides the answer is tracking code fences, because a fenced block is full of lines starting with # or | that a naive classifier reads as prose. Across my 11 articles and 22 files, 762 of the 795 matching lines sat inside fences and only 3 of 393 prose lines matched.

Which platforms let me declare a canonical URL?

dev.to takes article.canonical_url on the article endpoint, Hashnode takes originalArticleURL on the publishPost mutation, and Medium takes canonicalUrl on the post endpoint. Qiita’s items API has no equivalent parameter, Zenn’s article frontmatter has no such key, and note’s editor has no field for it. On those three the option to point at an original simply does not exist.

Is unpublishing the syndicated copies a safe temporary fix?

Not on platforms without canonical support, because the only way back is to post again under a new URL, which drops the views and any inbound links. My syndicated totals are 2,240 views on Qiita, 309 on dev.to and 182 impressions on Zenn, against 7 search clicks to aulvem.com over 28 days. Deleting the copies removes the channel where the readers actually are.

Does an AdSense rejection tell me anything about search duplicate handling?

No. AdSense reviews a site against its program policies, while search-side duplicate handling is URL consolidation driven by canonical and hreflang signals. Reasoning from one to the other is how I ended up drafting a fix for a problem I had never measured.