DN digha nikāya, PTS Rhys David Eng. translation aligned pali+eng, for s.4nt.org

DN 2 is a good sample:

https://s.4nt.org/dn/dn2/index.html

This took AI about 6 hours to mostly autonomously process DN (34 long suttas).

Compared to MN, this one aligned poorly (line by line...).

partly because PTS fills out pali source ...pe... elisions with long paragraphs of repetitions.

But the one guarantee of s.4nt.org is the aligned translations,

even if they're off on line by line,

they are as an entire sutta a verbatim diff of the original source (minus some white space and punctuation mark normalization). 

PTS DN Can be aligned very well with some human guidance and better section marks identifying chunk points for large suttas

This would require volunteer help from someone technically savvy in ai, programming,

and competent in pali.

competent in those areas enough to not take any meaningful amount of my time.

contact me if interested.

I have too much on my plate to bring PTS DN up to par with the other aligned nikayas.

Summary of AI session

 

PTS Dīgha Nikāya — True Pointer-Alignment

Claude Code session · thread 20260621_pts_dn_align · result 34/34 verbatim · committed · locked

userassistanttool callresult / successdecision

frank

do pts true align for dn. the trick with the pts series is — from doing mn, we learned the epub had wrongly truncated many suttas. so we have to check the epub for dn against buddhadust.net, or the httrack scrape at C:\My Web Sites\https___buddhadust.net_

Let me get oriented — the project layout, the prior PTS-MN thread, and where PTS/DN data live.

Read memory/thanissaro-pipeline-epub-free.md · pts-splitter-sentinel-dedup.md

Bash list project root · data/pts_epub · grep memory for "pts"

_concat_pts_raw.py _deepseek_align.py data/pts_epub/dn-pts-v2.0.epub archive/20260613_pts_mn_align (the MN precedent thread)

Read kamma/archive/20260613_pts_mn_align/{spec,plan,handoff}.md

MN lesson: the SC epub truncated 20 suttas → re-harvested from BuddhaDust; formalized --pts-source <zip|dir>; reuse for DN/SN/AN.

Key finding: DN PTS is T. W. & C. A. F. Rhys Davids (dn.NN.rhyt.pts.html) — not Horner like MN. BuddhaDust files are .html (the code expects .htm), and there are _contents/_preface files to skip. Let me run the real harvester on both sources to measure truncation.

Bash apples-to-apples: harvester(epub) vs harvester(BuddhaDust), all 34 DN

dn15 ratio 0.60 <== epub SHORTER dn20 ratio 0.77 <== epub SHORTER …the other 32 match BuddhaDust ~1.00

So only dn15 & dn20 are truncated (far milder than MN's 20). But the committed DN raw predates the MN splitter fix, so all 34 are also under-split — a uniform BuddhaDust re-harvest is the clean move. A few decisions are genuinely yours:

decision · source

Which source for the DN raw re-harvest?

All 34 from BuddhaDust — authoritative, fixes truncation, uniform provenance

Minimal: epub for 32 + BuddhaDust for dn15/dn20 only

decision · eyeball gate

Human review of the concatenated raw before aligning (like MN)?

Yes — pause for review

No — proceed straight through

decision · process

How to track and run this?

As a /kamma thread (plan · do · review · finalize)

Direct, no kamma overhead

frank

at some point rename the PTS translators to use "PTS" as a prefix for the actual translator name per book — so "PTS Rhys Davids", "PTS I. B. Horner", etc. otherwise, good. continue

Noted — folding the "PTS <translator>" convention into the plan (prompt name + column label). Scaffolding the thread now.

Skill kamma:1-plan

Write spec.md · plan.md → kamma/threads/20260621_pts_dn_align/

7 phases: harvest · eyeball · naming · align · validate · build · lock

frank

cont. you're using deepseek rather than sonnet?

Good catch — my "Sonnet subagents" wording was carried over from the older MN notes. The most recent runs (SN, Snp) used _deepseek_align.py. Switching the plan to the DeepSeek recipe and confirming the tooling takes --coll dn --translator pts.

PHASE 1Wire BuddhaDust source + re-harvest all 34

Edit fake_align_pts.py — accept .html in _DirSource / _name_map / file regexes; skip _contents/_preface

Bash gen-fake-pts-dn --pts-source "<scrape>" --force

34 DN suttas → dist/dn dn15 529→842 lines · dn20 175→222 (truncation fixed; siblings untouched)

PHASE 2Concat + under-split scan + EYEBALL GATE

The eyeball pass uncovered 4 real harvest bugs the old fake-align edition silently shipped (fake-align does no verbatim check, so they were invisible):

Bash inspect tails of all 34

dn1 tail: "] [#8. Kassapa-Sīhanāda Suttanta: …" ← inter-sutta TOC nav leaked

Edit ×4 — strip ctr nav rows · break at first note-def · 6 note-id schemes · single-h1 nidāna intro-strip

bug 1 trailing TOC nav fixed bug 2 footnote apparatus (defs+continuations+p.p. boxes) fixed bug 3 note-id schemes n/edn/itn/in/inn/ni (markers carry 'f') fixed bug 4 single-h1 intro leak — dn16 had lost its WHOLE translation fixed

Bash re-harvest + verify all 34 heads & tails

32/34 clean; 2 editorial remnants left for frank (dn7 head, dn28 tail)

decision · dn7 bracket

dn7 (a cross-reference sutta) opens with an editorial bracket note before the nidāna. Keep or strip?

Strip the bracket note

Keep it

decision · dn28 tail

dn28 ends with one editorial line in the main-text class. Keep or strip?

Keep it

Strip it

decision · next

How to proceed after these two?

Proceed now to alignment

I'll eyeball the concat first

PHASE 3PTS naming "PTS <translator>"

Edit align_pointer.py — pts_label(coll) + display_name() · build_dn.py · build.py

prompts & DN column now read "PTS Rhys Davids" · MN → "PTS I. B. Horner"

Bash pytest tests/ -q

658 passed · 5 failed = known baseline reds (no new regressions)

PHASE 4Alignment — the method pivot

Bash _deepseek_align flash@16k → @32k (34 suttas, background)

flash@16k → 2/34 · flash@32k → 26/34 responded but only 7 verbatim-clean front-collapse: dn16 referenced 9% of sentences

Bash escalate to deepseek-v4-pro@32k

0/8 — empty replies: reasoning_tokens=32000, reply len 0 (pro burns the whole budget reasoning)

DeepSeek worked for SN/Snp because those are short verse. DN's long prose triggers front-collapse (flash) and reasoning-overflow (pro). The older Sonnet-subagent method handles long prose reliably and runs in parallel. You steered me to DeepSeek before this surfaced — your call:

decision · aligner

DeepSeek front-collapses on DN prose (7/34 clean; pro empty). How should I align?

Switch to Sonnet subagents — keep the 7 good + monotonize near-clean; dn16 chunked

Continue with chunked DeepSeek (~1.5–2 hrs)

Sonnet for all 34 (cleanest)

Bash _monotonize_answer dn1 dn11 dn17 … (12 near-clean)

9 → verbatim=True · 5 still false (duplicate indices) → re-align

Agent ×8 sonnet — align dn2 dn4 dn5 dn6 dn7 dn9 dn10 dn12 (wave 1)

dn2: 1073/1075 unique, monotonic, full span · dn4: all 954 once

Agent ×8 sonnet — dn14 dn15 dn18 dn19 dn20 dn21 dn23 dn24 (wave 2)

Agent ×9 sonnet — dn1 dn8 dn26 dn27 dn28 dn31 dn34 dn33 + dn14 (wave 3)

dn33 (1167 keys) single-pass: 1491/1567, 0 dups, full span

Agent ×2 sonnet — dn16 (1664 keys) + dn1

dn16 single-pass: all 3001 sentences, monotonic, spans all 6 chapters

Bash ingest-all --coll dn --translator pts

# 34/34 ingested OK · 33 verbatim=True · dn9 false (1 inversion)

Bash _monotonize_answer dn9 → re-ingest

# 34/34 verbatim=True

PHASE 5Validate — drift

Bash _drift_check --coll dn --translator pts

14 flagged · 2 tail-collapse: dn13 (0.39) dn29 (0.35) — front-collapsed DeepSeek holdovers

Agent ×2 sonnet — re-align dn13, dn29 (span the whole key list)

dn13 last key 0.39→0.97 · dn29 0.35→0.996

Bash re-ingest + re-drift

34/34 verbatim · 0 tail-collapse · 0 monotonic violations 13 benign drift flags (Rhys Davids abbreviates repeated sīla — tail ≈ 1.0)

PHASE 6Build pages + drop the fake-align badge

Edit build_dn.py — remove warn=PTS_WARNING · Bash build-dn

34 rebuilt · 0 DN pages with PTS fake-align badge · 34 show "PTS Rhys Davids" (Walshe stays correctly badged)

PHASE 7Commit + lock — gated

decision · commit + lock

Ready to commit + lock all 34 DN PTS?

Yes — commit + lock all 34

Commit only, hold the locks

Wait — I'll eyeball pages first

Bash git add (path-limited) — verify zero unrelated staged

178 files staged · nothing outside expected paths

Bash git commit → e9bc5f14

179 files changed — 6 code/config · 135 dist/dn pts data · 34 responses · 3 thread docs

Bash for n in 1..34: lock dn$n --translator pts

committed: lock(pts): dn1 … dn34 — locked & proofread

Bash lock-status --translator pts

pts: 186 locked — 186 clean, 0 drifted (152 MN + 34 DN)

DONEPTS Dīgha Nikāya fully true-aligned

metric	result
Alignment	34/34 verbatim=True · cov 1.000 · 0 hard-fails
Commit	e9bc5f14 + 34 lock commits (path-limited, 0 unrelated)
Locks	186 locked · 186 clean · 0 drifted
Badge	dropped on all 34 DN pages · column "PTS Rhys Davids"
Tests	657 pass · only the 5 known baseline reds · tree clean

Your truncation-check instinct was right: the old fake-align DN had silently shipped 4 BuddhaDust harvest bugs — dn16 had lost its entire translation. Recorded the reusable harvest fixes + the DeepSeek-vs-Sonnet lesson to memory for the next PTS jobs (SN, AN — still fake).