diff --git a/docs/plans/2026-06-20-spine-org-data-model.md b/docs/plans/2026-06-20-spine-org-data-model.md new file mode 100644 index 0000000..6635bbc --- /dev/null +++ b/docs/plans/2026-06-20-spine-org-data-model.md @@ -0,0 +1,737 @@ +# Spine Data Model & Index Page — Implementation Plan + +**Goal:** Build the model layer (`spine-books`) and index page view so the app reads a real `spine.org` file and displays books grouped by status. + +**Architecture:** Model/view split. `spine-books` parses `spine.org` into plists via `org-element` + `org-entry-get`. `spine-render-index` takes those plists and an optional selected ID, returns HTML matching mockup A's class structure. A new `/` handler replaces the `/hello` smoke test. A `spine-skip-server-start` guard lets tests load `spine.el` without starting the HTTP server. + +**Tech Stack:** Emacs Lisp, `org-element` (built-in), `org-entry-get` (built-in), `ert` (built-in testing). + +--- + +## File structure + +| File | Responsibility | +|------|---------------| +| `sample-books.org` | Test data: 5 books across all statuses | +| `spine.el` | Modified: new functions, handler, skip-server-start guard | +| `test/spine-books-test.el` | ERT tests for `spine-books` | + +--- + +### Task 1: Create sample test data + +**Files:** +- Create: `sample-books.org` + +- [ ] **Step 1: Write `sample-books.org`** + +Five books covering all four statuses, plus an entry with minimal properties. + +```org +#+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a) + +* WANT The Left Hand of Darkness +:PROPERTIES: +:AUTHOR: Ursula K. Le Guin +:ISBN: 978-0441478125 +:COVER: covers/left-hand.jpg +:FORMAT: ebook +:REC_BY: Alex +:REC_NOTE: Le Guin at her most human +:RATING: +:ADDED: [2026-04-01] +:ID: 9a2b-lhd +:END: + +* WANT Dune +:PROPERTIES: +:AUTHOR: Frank Herbert +:ISBN: 978-0441172719 +:COVER: covers/dune.jpg +:FORMAT: hardcover +:REC_BY: Everyone +:REC_NOTE: +:RATING: +:ADDED: [2026-05-10] +:ID: 7f3c-dun +:END: + +* WANT Piranesi +:PROPERTIES: +:AUTHOR: Susanna Clarke +:ISBN: +:COVER: +:FORMAT: +:REC_BY: +:REC_NOTE: +:RATING: +:ADDED: [2026-06-01] +:ID: 5e8d-pir +:END: + +* READING Use of Weapons +:PROPERTIES: +:AUTHOR: Iain M. Banks +:ISBN: 978-0316029193 +:COVER: covers/use-of-weapons.jpg +:FORMAT: audiobook +:REC_BY: Priya +:REC_NOTE: If you liked Player of Games, this one will wreck you +:RATING: +:ADDED: [2026-02-20] +:ID: 8c1e-uow +:END: +:LOGBOOK: +- State "READING" from "WANT" [2026-03-01] +:END: +- [2026-03-02] The two-track structure is doing something I can't name yet. +- [2026-03-07] Zakalwe's competence reads as a wound. +- [2026-03-09] The chapter numbering. Oh. + +* READ Ancillary Justice +:PROPERTIES: +:AUTHOR: Ann Leckie +:ISBN: 978-0316246620 +:COVER: covers/ancillary.jpg +:FORMAT: ebook +:REC_BY: +:REC_NOTE: +:RATING: 5 +:ADDED: [2025-11-15] +:ID: 3a1b-aj +:END: +:LOGBOOK: +- State "READ" from "READING" [2026-01-10] +- State "READING" from "WANT" [2025-11-20] +:END: +- [2025-11-22] The pronoun game is a genuinely fresh take on identity. +- [2025-12-05] Justice of Toren's multi-body perspective is unsettling. +- [2026-01-08] The tea set. Perfect ending. +``` + +This file exercises: +- **WANT books**: 3 (Left Hand, Dune, Piranesi) — tests group header, count, different property densities +- **READING books**: 1 (Use of Weapons) — tests notes extraction, recommendation trail +- **READ books**: 1 (Ancillary Justice) — tests rating, LOGBOOK +- **ABANDONED books**: 0 — tests empty group omission +- **Minimal properties**: Piranesi has no ISBN, cover, format, recommendation, or notes + +- [ ] **Step 2: Verify file exists** + +Run: `cat sample-books.org | head -3` +Expected: `#+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a)` + +- [ ] **Step 3: Commit** + +```bash +git add sample-books.org +git commit -m "test: add sample-books.org test data" +``` + +--- + +### Task 2: Add server-start guard to spine.el + +**Files:** +- Modify: `spine.el` + +- [ ] **Step 1: Add `spine-skip-server-start` variable and wrap server start** + +Insert after the template rendering section (after line 44, before `;; --- handlers`): + +```elisp +(defvar spine-skip-server-start nil + "When non-nil, do not start the HTTP server. +Set by test harnesses that only need the model functions.") +``` + +Change the server start block (lines 53–58) to: + +```elisp +;; --- start ------------------------------------------------------------ +(unless spine-skip-server-start + (setq httpd-host spine-host) + (setq httpd-port spine-port) + (httpd-start) + (message "Spine listening on http://%s:%d" + (or spine-host "localhost") spine-port)) +``` + +- [ ] **Step 2: Verify batch load with guard** + +Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el` +Expected: exits cleanly, no "Spine listening" message, no errors. + +- [ ] **Step 3: Verify server still works normally without guard** + +Run the normal `scripts/run` flow and curl `/hello` — `/hello` still exists at this point since the handler hasn't been replaced yet. + +```bash +scripts/run 8080 +sleep 3 +curl -s http://localhost:8080/hello +scripts/stop +``` + +Expected: HTML containing `

Hello from Emacs.

`. +- [ ] **Step 4: Commit** + +```bash +git add spine.el +git commit -m "feat: add spine-skip-server-start guard for testing" +``` + +--- + +### Task 3: Implement spine-books model function + +**Files:** +- Modify: `spine.el` + +- [ ] **Step 1: Add `cl-lib` require** + +At the top of `spine.el`, after `(require 'ht)`, add: + +```elisp +(require 'cl-lib) +``` + +`cl-lib` is built into Emacs and provides `cl-return-from`. + +- [ ] **Step 2: Add `spine-org-file` config variable** + +After `spine-port` and `spine-host` defvars (after line 27), add: + +```elisp +(defvar spine-org-file "~/spine/books.org" + "Path to the spine.org data file.") +``` + +- [ ] **Step 3: Add `spine--h` HTML escaping helper** + +After `spine-render`, add: + +```elisp +(defun spine--h (text) + "HTML-escape TEXT." + (when text + (let ((s text)) + (setq s (replace-regexp-in-string "&" "&" s)) + (setq s (replace-regexp-in-string "<" "<" s)) + (setq s (replace-regexp-in-string ">" ">" s)) + (setq s (replace-regexp-in-string "\"" """ s)) + s))) +``` + +- [ ] **Step 4: Add `spine--extract-notes` helper** + +After `spine--h`, add: + +```elisp +(defun spine--extract-notes (hl-begin hl-end) + "Extract reading notes from headline body between HL-BEGIN and HL-END. +Returns list of (date-string text-string) pairs." + (let ((notes nil)) + (save-excursion + (goto-char hl-begin) + ;; Skip past PROPERTIES and LOGBOOK drawers + (while (re-search-forward + "^[ \t]*:\\(PROPERTIES\\|LOGBOOK\\):" hl-end t) + (re-search-forward "^[ \t]*:END:" hl-end t)) + ;; Collect note lines: "- [YYYY-MM-DD] text" + (while (re-search-forward + "^- \\[\\([0-9]\\{4\\}-[0-9]\\{2\\}-[0-9]\\{2\\}\\)\\] \\(.*\\)" + hl-end t) + (push (list (match-string 1) (match-string 2)) notes))) + (nreverse notes))) +``` + +- [ ] **Step 5: Add `spine-books` function** + +After `spine--extract-notes`, add: + +```elisp +(defun spine-books () + "Return a list of plists, one per top-level headline in `spine-org-file'. +Returns nil if the file is missing or cannot be parsed." + (unless (file-exists-p spine-org-file) + (message "spine-books: file not found: %s" spine-org-file) + (cl-return-from spine-books nil)) + (condition-case err + (with-temp-buffer + (insert-file-contents spine-org-file) + (org-mode) + (let ((books nil)) + (org-element-map (org-element-parse-buffer 'headline) 'headline + (lambda (hl) + (when (= (org-element-property :level hl) 1) + (let ((pos (org-element-property :begin hl))) + (push (list :id (save-excursion + (goto-char pos) + (org-entry-get nil "ID")) + :title (org-element-property :title hl) + :status (org-element-property :todo-keyword hl) + :author (save-excursion + (goto-char pos) + (org-entry-get nil "AUTHOR")) + :isbn (save-excursion + (goto-char pos) + (org-entry-get nil "ISBN")) + :cover (save-excursion + (goto-char pos) + (org-entry-get nil "COVER")) + :format (save-excursion + (goto-char pos) + (org-entry-get nil "FORMAT")) + :rec_by (save-excursion + (goto-char pos) + (org-entry-get nil "REC_BY")) + :rec_note (save-excursion + (goto-char pos) + (org-entry-get nil "REC_NOTE")) + :rating (save-excursion + (goto-char pos) + (org-entry-get nil "RATING")) + :added (save-excursion + (goto-char pos) + (org-entry-get nil "ADDED")) + :tags (org-element-property :tags hl) + :notes (spine--extract-notes + (org-element-property :begin hl) + (org-element-property :end hl))) + books))))) + (nreverse books))) + (error + (message "spine-books: failed to parse %s: %s" + spine-org-file (error-message-string err)) + nil))) +``` + +- [ ] **Step 6: Verify batch load still works** + +Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el` +Expected: exits cleanly. + +- [ ] **Step 7: Verify spine-books returns data from sample file** + +Run: +```bash +emacs --quick --batch \ + --eval "(setq spine-skip-server-start t spine-org-file \"sample-books.org\")" \ + --load spine.el \ + --eval "(progn (let ((books (spine-books))) (message \"Found %d books\" (length books))))" +``` + +Expected: `Found 5 books` in output. + +- [ ] **Step 8: Commit** + +```bash +git add spine.el +git commit -m "feat: add spine-books model function" +``` + +--- + +### Task 4: Test spine-books with ERT + +**Files:** +- Create: `test/spine-books-test.el` + +- [ ] **Step 1: Create `test/` directory** + +```bash +mkdir -p test +``` + +- [ ] **Step 2: Write `test/spine-books-test.el`** + +```elisp +;;; spine-books-test.el — ERT tests for spine-books + +(require 'ert) +(require 'cl-lib) + +;; Load spine.el without starting the server +(setq spine-skip-server-start t) +(setq spine-org-file (expand-file-name "sample-books.org" + (file-name-directory load-file-name))) +(load-file (expand-file-name "spine.el" + (file-name-directory load-file-name))) + +(ert-deftest spine-books-parses-sample () + "spine-books returns correct plists for sample-books.org." + (let ((books (spine-books))) + (should books) + (should (= (length books) 5)))) + +(ert-deftest spine-books-first-book-has-all-fields () + "Use of Weapons entry has all expected fields." + (let* ((books (spine-books)) + (uow (cl-find "8c1e-uow" books + :key (lambda (b) (plist-get b :id)) + :test #'equal))) + (should uow) + (should (equal (plist-get uow :title) "Use of Weapons")) + (should (equal (plist-get uow :status) "READING")) + (should (equal (plist-get uow :author) "Iain M. Banks")) + (should (equal (plist-get uow :isbn) "978-0316029193")) + (should (equal (plist-get uow :cover) "covers/use-of-weapons.jpg")) + (should (equal (plist-get uow :format) "audiobook")) + (should (equal (plist-get uow :rec_by) "Priya")) + (should (equal (plist-get uow :rec_note) "If you liked Player of Games, this one will wreck you")) + (should (equal (plist-get uow :tags) '("scifi" "literary"))) + (should (equal (plist-get uow :notes) + '(("2026-03-02" "The two-track structure is doing something I can't name yet.") + ("2026-03-07" "Zakalwe's competence reads as a wound.") + ("2026-03-09" "The chapter numbering. Oh.")))))) + +(ert-deftest spine-books-missing-properties-are-nil () + "Piranesi entry has nil for non-present properties." + (let* ((books (spine-books)) + (pir (cl-find "5e8d-pir" books + :key (lambda (b) (plist-get b :id)) + :test #'equal))) + (should pir) + (should (equal (plist-get pir :title) "Piranesi")) + (should (equal (plist-get pir :status) "WANT")) + (should (equal (plist-get pir :author) "Susanna Clarke")) + ;; ISBN, cover, format, rec_by, rec_note should all be nil or "" + (should-not (plist-get pir :isbn)) + (should-not (plist-get pir :cover)) + (should-not (plist-get pir :format)) + (should-not (plist-get pir :rec_by)) + (should-not (plist-get pir :rec_note)) + ;; Notes should be empty + (should-not (plist-get pir :notes)))) + +(ert-deftest spine-books-read-book-has-rating () + "Ancillary Justice has rating 5." + (let* ((books (spine-books)) + (aj (cl-find "3a1b-aj" books + :key (lambda (b) (plist-get b :id)) + :test #'equal))) + (should aj) + (should (equal (plist-get aj :status) "READ")) + (should (equal (plist-get aj :rating) "5")) + (should (= (length (plist-get aj :notes)) 3)))) + +(ert-deftest spine-books-empty-file-returns-empty-list () + "Empty Org file returns empty list, not nil." + (let* ((tmp-file (make-temp-file "spine-test-" nil ".org")) + (spine-org-file tmp-file)) + (unwind-protect + (let ((books (spine-books))) + (should (listp books)) + (should (= (length books) 0))) + (delete-file tmp-file)))) + +(ert-deftest spine-books-missing-file-returns-nil () + "Non-existent file returns nil." + (let ((spine-org-file "/tmp/spine-nonexistent-12345.org")) + (should-not (spine-books)))) +``` + +- [ ] **Step 3: Run the tests** + +```bash +emacs --quick --batch \ + --directory "$(pwd)" \ + --load test/spine-books-test.el \ + -f ert-run-tests-batch-and-exit +``` + +Expected: all 6 tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add test/spine-books-test.el +git commit -m "test: add ERT tests for spine-books" +``` + +--- + +### Task 5: Implement view functions + +**Files:** +- Modify: `spine.el` + +- [ ] **Step 1: Add `spine-render-index` function** + +After `spine-books`, add: + +```elisp +(defun spine-render-index (books &optional selected-id) + "Render the index page HTML for BOOKS, expanding the book matching SELECTED-ID." + (let ((order '("WANT" "READING" "READ" "ABANDONED")) + (grouped (make-hash-table :test 'equal))) + ;; Group books by status + (dolist (book books) + (let* ((status (or (plist-get book :status) "WANT"))) + (push book (gethash status grouped)))) + ;; Render + (with-temp-buffer + (insert "\n\n\n") + (insert "\n") + (insert "\n") + (insert "Spine\n\n\n") + (insert "
\n") + (insert "
Spine
\n") + ;; Each group in order + (dolist (status order) + (let ((group-books (nreverse (gethash status grouped)))) + (when group-books + (insert (format "
%s (%d)
\n" + status (length group-books))) + (dolist (book group-books) + (let ((selected (equal (plist-get book :id) selected-id))) + (insert (format "
\n" + (if selected " selected" ""))) + ;; Format glyph + (let ((fmt (plist-get book :format))) + (insert (format "%s\n" + (cond ((equal fmt "hardcover") "[H]") + ((equal fmt "ebook") "[E]") + ((equal fmt "audiobook") "[A]") + (t ""))))) + ;; Title + author + (insert "") + (insert (spine--h (plist-get book :title))) + (let ((author (plist-get book :author))) + (when author + (insert (format " %s" + (spine--h author))))) + (insert "\n") + ;; Tags + (let ((tags (plist-get book :tags))) + (when tags + (insert (format "%s\n" + (mapconcat (lambda (t) (concat ":" t ":")) + tags " "))))) + ;; Recommendation trail + (let ((rec-by (plist-get book :rec_by)) + (rec-note (plist-get book :rec_note))) + (when (and rec-by (> (length rec-by) 0)) + (insert + (format "%s%s\n" + (spine--h rec-by) + (if (and rec-note (> (length rec-note) 0)) + (concat ": " (spine--h rec-note)) + ""))))) + ;; Status pill + (let ((status (or (plist-get book :status) "WANT"))) + (insert (format "%s\n" + (downcase status) status))) + ;; Expanded detail + (when selected + (insert "
\n") + (dolist (note (plist-get book :notes)) + (insert (format + "
[%s] %s
\n" + (spine--h (car note)) + (spine--h (cadr note))))) + (let ((rating (plist-get book :rating))) + (when (and rating (> (length rating) 0)) + (insert (format "
Rating: %s/5
\n" + (spine--h rating))))) + (let ((isbn (plist-get book :isbn))) + (when (and isbn (> (length isbn) 0)) + (insert (format "
ISBN: %s
\n" + (spine--h isbn))))) + (let ((fmt (plist-get book :format))) + (when (and fmt (> (length fmt) 0)) + (insert (format "
Format: %s
\n" + (spine--h fmt))))) + (let ((added (plist-get book :added))) + (when (and added (> (length added) 0)) + (insert (format "
Added: %s
\n" + (spine--h added))))) + (insert "
\n")) + (insert "
\n")))))) + (insert "
\n\n\n") + (buffer-string)))) +``` + +- [ ] **Step 2: Add `spine-render-empty-state` function** + +After `spine-render-index`, add: + +```elisp +(defun spine-render-empty-state () + "Render a page indicating no books file was found." + (concat "\n\n\n" + "\n" + "\n" + "Spine\n\n\n" + "
\n" + "
Spine
\n" + "
No books yet.
\n" + "
\n\n\n")) +``` + +- [ ] **Step 3: Verify batch load** + +Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el` +Expected: exits cleanly, no errors. + +- [ ] **Step 4: Commit** + +```bash +git add spine.el +git commit -m "feat: add spine-render-index and empty-state view functions" +``` + +--- + +### Task 6: Wire up the index handler + +**Files:** +- Modify: `spine.el` + +- [ ] **Step 1: Replace `/hello` handler with `/` index handler** + +Replace the `hello` handler (lines 48–51) with: + +```elisp +(defservlet index text/html () + (let ((books (spine-books))) + (if books + (insert (spine-render-index books + (ht-get (ht-from-query-string (httpd-query-string)) "id"))) + (insert (spine-render-empty-state))))) +``` + +- [ ] **Step 2: Verify batch load** + +Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el` +Expected: exits cleanly. + +- [ ] **Step 3: Commit** + +```bash +git add spine.el +git commit -m "feat: wire up / index handler replacing /hello" +``` + +--- + +### Task 7: Smoke test end-to-end + +**Files:** +- No changes — verification only. + +- [ ] **Step 1: Start server with sample data** + +```bash +emacs --quick \ + --eval "(setq spine-org-file \"$(pwd)/sample-books.org\")" \ + --load spine.el & +EMACS_PID=$! +sleep 5 +``` + +- [ ] **Step 2: Verify index page returns HTML with real data** + +```bash +curl -s http://localhost:8080/ | head -20 +``` + +Expected: HTML document with: +- `Spine` +- `
WANT (3)
` +- `
READING (1)
` +- `
READ (1)
` +- No `ABANDONED` group (zero books) +- "Use of Weapons" visible in output +- Recommendation trail "Priya: If you liked..." + +- [ ] **Step 3: Verify expanded detail via ?id=** + +```bash +curl -s "http://localhost:8080/?id=8c1e-uow" | grep -A5 "detail" +``` + +Expected: expanded detail section with reading notes for Use of Weapons: +- `[2026-03-02] The two-track structure...` +- `[2026-03-07] Zakalwe's competence...` +- `[2026-03-09] The chapter numbering. Oh.` + +- [ ] **Step 4: Verify bogus ?id= does not crash** + +```bash +curl -s "http://localhost:8080/?id=nonexistent" +``` + +Expected: normal index page, no expanded detail, HTTP 200. + +- [ ] **Step 5: Verify empty state when file is missing** + +```bash +kill $EMACS_PID 2>/dev/null +sleep 1 +emacs --quick \ + --eval "(setq spine-org-file \"/tmp/nonexistent-books.org\")" \ + --load spine.el & +EMACS_PID=$! +sleep 5 +curl -s http://localhost:8080/ +``` + +Expected: HTML containing `No books yet.` + +- [ ] **Step 6: Stop server** + +```bash +kill $EMACS_PID 2>/dev/null +``` + +- [ ] **Step 7: Verify with scripts/run** + +```bash +# Override spine-org-file via the run script (pass it as an extra eval) +SPINE_ORG="$(pwd)/sample-books.org" +emacs --quick --daemon=spine \ + --eval "(setq spine-org-file \"$SPINE_ORG\")" \ + --load spine.el +sleep 3 +curl -s http://localhost:8080/ | grep -c "WANT" +# Expected: 1 (the group header "WANT (3)") +emacsclient --socket-name=spine --eval '(kill-emacs)' +``` + +The output should contain `1`. + +--- + +### Task 8: Final verification and cleanup + +**Files:** +- No changes — verification only. + +- [ ] **Step 1: Run full ERT test suite** + +```bash +emacs --quick --batch \ + --directory "$(pwd)" \ + --load test/spine-books-test.el \ + -f ert-run-tests-batch-and-exit +``` + +Expected: all 6 tests pass. + +- [ ] **Step 2: Verify git status is clean** + +```bash +git status +``` + +Expected: `nothing to commit, working tree clean` + +- [ ] **Step 3: Review commit log** + +```bash +git log --oneline +``` + +Expected: ~7 commits covering sample data, server guard, model, tests, view, handler.