docs: add data model and index page implementation plan

This commit is contained in:
2026-06-20 16:54:53 -04:00
parent de64d71936
commit d476a4eacc
@@ -0,0 +1,737 @@
# Spine Data Model & Index Page — Implementation Plan
**Goal:** Build the model layer (`spine-books`) and index page view so the app reads a real `spine.org` file and displays books grouped by status.
**Architecture:** Model/view split. `spine-books` parses `spine.org` into plists via `org-element` + `org-entry-get`. `spine-render-index` takes those plists and an optional selected ID, returns HTML matching mockup A's class structure. A new `/` handler replaces the `/hello` smoke test. A `spine-skip-server-start` guard lets tests load `spine.el` without starting the HTTP server.
**Tech Stack:** Emacs Lisp, `org-element` (built-in), `org-entry-get` (built-in), `ert` (built-in testing).
---
## File structure
| File | Responsibility |
|------|---------------|
| `sample-books.org` | Test data: 5 books across all statuses |
| `spine.el` | Modified: new functions, handler, skip-server-start guard |
| `test/spine-books-test.el` | ERT tests for `spine-books` |
---
### Task 1: Create sample test data
**Files:**
- Create: `sample-books.org`
- [ ] **Step 1: Write `sample-books.org`**
Five books covering all four statuses, plus an entry with minimal properties.
```org
#+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a)
* WANT The Left Hand of Darkness
:PROPERTIES:
:AUTHOR: Ursula K. Le Guin
:ISBN: 978-0441478125
:COVER: covers/left-hand.jpg
:FORMAT: ebook
:REC_BY: Alex
:REC_NOTE: Le Guin at her most human
:RATING:
:ADDED: [2026-04-01]
:ID: 9a2b-lhd
:END:
* WANT Dune
:PROPERTIES:
:AUTHOR: Frank Herbert
:ISBN: 978-0441172719
:COVER: covers/dune.jpg
:FORMAT: hardcover
:REC_BY: Everyone
:REC_NOTE:
:RATING:
:ADDED: [2026-05-10]
:ID: 7f3c-dun
:END:
* WANT Piranesi
:PROPERTIES:
:AUTHOR: Susanna Clarke
:ISBN:
:COVER:
:FORMAT:
:REC_BY:
:REC_NOTE:
:RATING:
:ADDED: [2026-06-01]
:ID: 5e8d-pir
:END:
* READING Use of Weapons
:PROPERTIES:
:AUTHOR: Iain M. Banks
:ISBN: 978-0316029193
:COVER: covers/use-of-weapons.jpg
:FORMAT: audiobook
:REC_BY: Priya
:REC_NOTE: If you liked Player of Games, this one will wreck you
:RATING:
:ADDED: [2026-02-20]
:ID: 8c1e-uow
:END:
:LOGBOOK:
- State "READING" from "WANT" [2026-03-01]
:END:
- [2026-03-02] The two-track structure is doing something I can't name yet.
- [2026-03-07] Zakalwe's competence reads as a wound.
- [2026-03-09] The chapter numbering. Oh.
* READ Ancillary Justice
:PROPERTIES:
:AUTHOR: Ann Leckie
:ISBN: 978-0316246620
:COVER: covers/ancillary.jpg
:FORMAT: ebook
:REC_BY:
:REC_NOTE:
:RATING: 5
:ADDED: [2025-11-15]
:ID: 3a1b-aj
:END:
:LOGBOOK:
- State "READ" from "READING" [2026-01-10]
- State "READING" from "WANT" [2025-11-20]
:END:
- [2025-11-22] The pronoun game is a genuinely fresh take on identity.
- [2025-12-05] Justice of Toren's multi-body perspective is unsettling.
- [2026-01-08] The tea set. Perfect ending.
```
This file exercises:
- **WANT books**: 3 (Left Hand, Dune, Piranesi) — tests group header, count, different property densities
- **READING books**: 1 (Use of Weapons) — tests notes extraction, recommendation trail
- **READ books**: 1 (Ancillary Justice) — tests rating, LOGBOOK
- **ABANDONED books**: 0 — tests empty group omission
- **Minimal properties**: Piranesi has no ISBN, cover, format, recommendation, or notes
- [ ] **Step 2: Verify file exists**
Run: `cat sample-books.org | head -3`
Expected: `#+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a)`
- [ ] **Step 3: Commit**
```bash
git add sample-books.org
git commit -m "test: add sample-books.org test data"
```
---
### Task 2: Add server-start guard to spine.el
**Files:**
- Modify: `spine.el`
- [ ] **Step 1: Add `spine-skip-server-start` variable and wrap server start**
Insert after the template rendering section (after line 44, before `;; --- handlers`):
```elisp
(defvar spine-skip-server-start nil
"When non-nil, do not start the HTTP server.
Set by test harnesses that only need the model functions.")
```
Change the server start block (lines 5358) to:
```elisp
;; --- start ------------------------------------------------------------
(unless spine-skip-server-start
(setq httpd-host spine-host)
(setq httpd-port spine-port)
(httpd-start)
(message "Spine listening on http://%s:%d"
(or spine-host "localhost") spine-port))
```
- [ ] **Step 2: Verify batch load with guard**
Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el`
Expected: exits cleanly, no "Spine listening" message, no errors.
- [ ] **Step 3: Verify server still works normally without guard**
Run the normal `scripts/run` flow and curl `/hello``/hello` still exists at this point since the handler hasn't been replaced yet.
```bash
scripts/run 8080
sleep 3
curl -s http://localhost:8080/hello
scripts/stop
```
Expected: HTML containing `<h1>Hello from Emacs.</h1>`.
- [ ] **Step 4: Commit**
```bash
git add spine.el
git commit -m "feat: add spine-skip-server-start guard for testing"
```
---
### Task 3: Implement spine-books model function
**Files:**
- Modify: `spine.el`
- [ ] **Step 1: Add `cl-lib` require**
At the top of `spine.el`, after `(require 'ht)`, add:
```elisp
(require 'cl-lib)
```
`cl-lib` is built into Emacs and provides `cl-return-from`.
- [ ] **Step 2: Add `spine-org-file` config variable**
After `spine-port` and `spine-host` defvars (after line 27), add:
```elisp
(defvar spine-org-file "~/spine/books.org"
"Path to the spine.org data file.")
```
- [ ] **Step 3: Add `spine--h` HTML escaping helper**
After `spine-render`, add:
```elisp
(defun spine--h (text)
"HTML-escape TEXT."
(when text
(let ((s text))
(setq s (replace-regexp-in-string "&" "&amp;" s))
(setq s (replace-regexp-in-string "<" "&lt;" s))
(setq s (replace-regexp-in-string ">" "&gt;" s))
(setq s (replace-regexp-in-string "\"" "&quot;" s))
s)))
```
- [ ] **Step 4: Add `spine--extract-notes` helper**
After `spine--h`, add:
```elisp
(defun spine--extract-notes (hl-begin hl-end)
"Extract reading notes from headline body between HL-BEGIN and HL-END.
Returns list of (date-string text-string) pairs."
(let ((notes nil))
(save-excursion
(goto-char hl-begin)
;; Skip past PROPERTIES and LOGBOOK drawers
(while (re-search-forward
"^[ \t]*:\\(PROPERTIES\\|LOGBOOK\\):" hl-end t)
(re-search-forward "^[ \t]*:END:" hl-end t))
;; Collect note lines: "- [YYYY-MM-DD] text"
(while (re-search-forward
"^- \\[\\([0-9]\\{4\\}-[0-9]\\{2\\}-[0-9]\\{2\\}\\)\\] \\(.*\\)"
hl-end t)
(push (list (match-string 1) (match-string 2)) notes)))
(nreverse notes)))
```
- [ ] **Step 5: Add `spine-books` function**
After `spine--extract-notes`, add:
```elisp
(defun spine-books ()
"Return a list of plists, one per top-level headline in `spine-org-file'.
Returns nil if the file is missing or cannot be parsed."
(unless (file-exists-p spine-org-file)
(message "spine-books: file not found: %s" spine-org-file)
(cl-return-from spine-books nil))
(condition-case err
(with-temp-buffer
(insert-file-contents spine-org-file)
(org-mode)
(let ((books nil))
(org-element-map (org-element-parse-buffer 'headline) 'headline
(lambda (hl)
(when (= (org-element-property :level hl) 1)
(let ((pos (org-element-property :begin hl)))
(push (list :id (save-excursion
(goto-char pos)
(org-entry-get nil "ID"))
:title (org-element-property :title hl)
:status (org-element-property :todo-keyword hl)
:author (save-excursion
(goto-char pos)
(org-entry-get nil "AUTHOR"))
:isbn (save-excursion
(goto-char pos)
(org-entry-get nil "ISBN"))
:cover (save-excursion
(goto-char pos)
(org-entry-get nil "COVER"))
:format (save-excursion
(goto-char pos)
(org-entry-get nil "FORMAT"))
:rec_by (save-excursion
(goto-char pos)
(org-entry-get nil "REC_BY"))
:rec_note (save-excursion
(goto-char pos)
(org-entry-get nil "REC_NOTE"))
:rating (save-excursion
(goto-char pos)
(org-entry-get nil "RATING"))
:added (save-excursion
(goto-char pos)
(org-entry-get nil "ADDED"))
:tags (org-element-property :tags hl)
:notes (spine--extract-notes
(org-element-property :begin hl)
(org-element-property :end hl)))
books)))))
(nreverse books)))
(error
(message "spine-books: failed to parse %s: %s"
spine-org-file (error-message-string err))
nil)))
```
- [ ] **Step 6: Verify batch load still works**
Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el`
Expected: exits cleanly.
- [ ] **Step 7: Verify spine-books returns data from sample file**
Run:
```bash
emacs --quick --batch \
--eval "(setq spine-skip-server-start t spine-org-file \"sample-books.org\")" \
--load spine.el \
--eval "(progn (let ((books (spine-books))) (message \"Found %d books\" (length books))))"
```
Expected: `Found 5 books` in output.
- [ ] **Step 8: Commit**
```bash
git add spine.el
git commit -m "feat: add spine-books model function"
```
---
### Task 4: Test spine-books with ERT
**Files:**
- Create: `test/spine-books-test.el`
- [ ] **Step 1: Create `test/` directory**
```bash
mkdir -p test
```
- [ ] **Step 2: Write `test/spine-books-test.el`**
```elisp
;;; spine-books-test.el — ERT tests for spine-books
(require 'ert)
(require 'cl-lib)
;; Load spine.el without starting the server
(setq spine-skip-server-start t)
(setq spine-org-file (expand-file-name "sample-books.org"
(file-name-directory load-file-name)))
(load-file (expand-file-name "spine.el"
(file-name-directory load-file-name)))
(ert-deftest spine-books-parses-sample ()
"spine-books returns correct plists for sample-books.org."
(let ((books (spine-books)))
(should books)
(should (= (length books) 5))))
(ert-deftest spine-books-first-book-has-all-fields ()
"Use of Weapons entry has all expected fields."
(let* ((books (spine-books))
(uow (cl-find "8c1e-uow" books
:key (lambda (b) (plist-get b :id))
:test #'equal)))
(should uow)
(should (equal (plist-get uow :title) "Use of Weapons"))
(should (equal (plist-get uow :status) "READING"))
(should (equal (plist-get uow :author) "Iain M. Banks"))
(should (equal (plist-get uow :isbn) "978-0316029193"))
(should (equal (plist-get uow :cover) "covers/use-of-weapons.jpg"))
(should (equal (plist-get uow :format) "audiobook"))
(should (equal (plist-get uow :rec_by) "Priya"))
(should (equal (plist-get uow :rec_note) "If you liked Player of Games, this one will wreck you"))
(should (equal (plist-get uow :tags) '("scifi" "literary")))
(should (equal (plist-get uow :notes)
'(("2026-03-02" "The two-track structure is doing something I can't name yet.")
("2026-03-07" "Zakalwe's competence reads as a wound.")
("2026-03-09" "The chapter numbering. Oh."))))))
(ert-deftest spine-books-missing-properties-are-nil ()
"Piranesi entry has nil for non-present properties."
(let* ((books (spine-books))
(pir (cl-find "5e8d-pir" books
:key (lambda (b) (plist-get b :id))
:test #'equal)))
(should pir)
(should (equal (plist-get pir :title) "Piranesi"))
(should (equal (plist-get pir :status) "WANT"))
(should (equal (plist-get pir :author) "Susanna Clarke"))
;; ISBN, cover, format, rec_by, rec_note should all be nil or ""
(should-not (plist-get pir :isbn))
(should-not (plist-get pir :cover))
(should-not (plist-get pir :format))
(should-not (plist-get pir :rec_by))
(should-not (plist-get pir :rec_note))
;; Notes should be empty
(should-not (plist-get pir :notes))))
(ert-deftest spine-books-read-book-has-rating ()
"Ancillary Justice has rating 5."
(let* ((books (spine-books))
(aj (cl-find "3a1b-aj" books
:key (lambda (b) (plist-get b :id))
:test #'equal)))
(should aj)
(should (equal (plist-get aj :status) "READ"))
(should (equal (plist-get aj :rating) "5"))
(should (= (length (plist-get aj :notes)) 3))))
(ert-deftest spine-books-empty-file-returns-empty-list ()
"Empty Org file returns empty list, not nil."
(let* ((tmp-file (make-temp-file "spine-test-" nil ".org"))
(spine-org-file tmp-file))
(unwind-protect
(let ((books (spine-books)))
(should (listp books))
(should (= (length books) 0)))
(delete-file tmp-file))))
(ert-deftest spine-books-missing-file-returns-nil ()
"Non-existent file returns nil."
(let ((spine-org-file "/tmp/spine-nonexistent-12345.org"))
(should-not (spine-books))))
```
- [ ] **Step 3: Run the tests**
```bash
emacs --quick --batch \
--directory "$(pwd)" \
--load test/spine-books-test.el \
-f ert-run-tests-batch-and-exit
```
Expected: all 6 tests pass.
- [ ] **Step 4: Commit**
```bash
git add test/spine-books-test.el
git commit -m "test: add ERT tests for spine-books"
```
---
### Task 5: Implement view functions
**Files:**
- Modify: `spine.el`
- [ ] **Step 1: Add `spine-render-index` function**
After `spine-books`, add:
```elisp
(defun spine-render-index (books &optional selected-id)
"Render the index page HTML for BOOKS, expanding the book matching SELECTED-ID."
(let ((order '("WANT" "READING" "READ" "ABANDONED"))
(grouped (make-hash-table :test 'equal)))
;; Group books by status
(dolist (book books)
(let* ((status (or (plist-get book :status) "WANT")))
(push book (gethash status grouped))))
;; Render
(with-temp-buffer
(insert "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n")
(insert "<meta charset=\"utf-8\">\n")
(insert "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n")
(insert "<title>Spine</title>\n</head>\n<body>\n")
(insert "<div class=\"frame\">\n")
(insert "<div class=\"titlebar\">Spine</div>\n")
;; Each group in order
(dolist (status order)
(let ((group-books (nreverse (gethash status grouped))))
(when group-books
(insert (format "<div class=\"group\">%s (%d)</div>\n"
status (length group-books)))
(dolist (book group-books)
(let ((selected (equal (plist-get book :id) selected-id)))
(insert (format "<div class=\"row%s\">\n"
(if selected " selected" "")))
;; Format glyph
(let ((fmt (plist-get book :format)))
(insert (format "<span class=\"glyph\">%s</span>\n"
(cond ((equal fmt "hardcover") "[H]")
((equal fmt "ebook") "[E]")
((equal fmt "audiobook") "[A]")
(t "")))))
;; Title + author
(insert "<span class=\"title\">")
(insert (spine--h (plist-get book :title)))
(let ((author (plist-get book :author)))
(when author
(insert (format " <span class=\"muted\">%s</span>"
(spine--h author)))))
(insert "</span>\n")
;; Tags
(let ((tags (plist-get book :tags)))
(when tags
(insert (format "<span class=\"tags\">%s</span>\n"
(mapconcat (lambda (t) (concat ":" t ":"))
tags " ")))))
;; Recommendation trail
(let ((rec-by (plist-get book :rec_by))
(rec-note (plist-get book :rec_note)))
(when (and rec-by (> (length rec-by) 0))
(insert
(format "<span class=\"trail\">%s%s</span>\n"
(spine--h rec-by)
(if (and rec-note (> (length rec-note) 0))
(concat ": " (spine--h rec-note))
"")))))
;; Status pill
(let ((status (or (plist-get book :status) "WANT")))
(insert (format "<span class=\"pill %s\">%s</span>\n"
(downcase status) status)))
;; Expanded detail
(when selected
(insert "<div class=\"detail\">\n")
(dolist (note (plist-get book :notes))
(insert (format
"<div class=\"logline\"><span class=\"logdate\">[%s]</span> %s</div>\n"
(spine--h (car note))
(spine--h (cadr note)))))
(let ((rating (plist-get book :rating)))
(when (and rating (> (length rating) 0))
(insert (format "<div>Rating: %s/5</div>\n"
(spine--h rating)))))
(let ((isbn (plist-get book :isbn)))
(when (and isbn (> (length isbn) 0))
(insert (format "<div>ISBN: %s</div>\n"
(spine--h isbn)))))
(let ((fmt (plist-get book :format)))
(when (and fmt (> (length fmt) 0))
(insert (format "<div>Format: %s</div>\n"
(spine--h fmt)))))
(let ((added (plist-get book :added)))
(when (and added (> (length added) 0))
(insert (format "<div>Added: %s</div>\n"
(spine--h added)))))
(insert "</div>\n"))
(insert "</div>\n"))))))
(insert "</div>\n</body>\n</html>\n")
(buffer-string))))
```
- [ ] **Step 2: Add `spine-render-empty-state` function**
After `spine-render-index`, add:
```elisp
(defun spine-render-empty-state ()
"Render a page indicating no books file was found."
(concat "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n"
"<meta charset=\"utf-8\">\n"
"<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n"
"<title>Spine</title>\n</head>\n<body>\n"
"<div class=\"frame\">\n"
"<div class=\"titlebar\">Spine</div>\n"
"<div class=\"row\">No books yet.</div>\n"
"</div>\n</body>\n</html>\n"))
```
- [ ] **Step 3: Verify batch load**
Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el`
Expected: exits cleanly, no errors.
- [ ] **Step 4: Commit**
```bash
git add spine.el
git commit -m "feat: add spine-render-index and empty-state view functions"
```
---
### Task 6: Wire up the index handler
**Files:**
- Modify: `spine.el`
- [ ] **Step 1: Replace `/hello` handler with `/` index handler**
Replace the `hello` handler (lines 4851) with:
```elisp
(defservlet index text/html ()
(let ((books (spine-books)))
(if books
(insert (spine-render-index books
(ht-get (ht-from-query-string (httpd-query-string)) "id")))
(insert (spine-render-empty-state)))))
```
- [ ] **Step 2: Verify batch load**
Run: `emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el`
Expected: exits cleanly.
- [ ] **Step 3: Commit**
```bash
git add spine.el
git commit -m "feat: wire up / index handler replacing /hello"
```
---
### Task 7: Smoke test end-to-end
**Files:**
- No changes — verification only.
- [ ] **Step 1: Start server with sample data**
```bash
emacs --quick \
--eval "(setq spine-org-file \"$(pwd)/sample-books.org\")" \
--load spine.el &
EMACS_PID=$!
sleep 5
```
- [ ] **Step 2: Verify index page returns HTML with real data**
```bash
curl -s http://localhost:8080/ | head -20
```
Expected: HTML document with:
- `<title>Spine</title>`
- `<div class="group">WANT (3)</div>`
- `<div class="group">READING (1)</div>`
- `<div class="group">READ (1)</div>`
- No `ABANDONED` group (zero books)
- "Use of Weapons" visible in output
- Recommendation trail "Priya: If you liked..."
- [ ] **Step 3: Verify expanded detail via ?id=**
```bash
curl -s "http://localhost:8080/?id=8c1e-uow" | grep -A5 "detail"
```
Expected: expanded detail section with reading notes for Use of Weapons:
- `[2026-03-02] The two-track structure...`
- `[2026-03-07] Zakalwe's competence...`
- `[2026-03-09] The chapter numbering. Oh.`
- [ ] **Step 4: Verify bogus ?id= does not crash**
```bash
curl -s "http://localhost:8080/?id=nonexistent"
```
Expected: normal index page, no expanded detail, HTTP 200.
- [ ] **Step 5: Verify empty state when file is missing**
```bash
kill $EMACS_PID 2>/dev/null
sleep 1
emacs --quick \
--eval "(setq spine-org-file \"/tmp/nonexistent-books.org\")" \
--load spine.el &
EMACS_PID=$!
sleep 5
curl -s http://localhost:8080/
```
Expected: HTML containing `No books yet.`
- [ ] **Step 6: Stop server**
```bash
kill $EMACS_PID 2>/dev/null
```
- [ ] **Step 7: Verify with scripts/run**
```bash
# Override spine-org-file via the run script (pass it as an extra eval)
SPINE_ORG="$(pwd)/sample-books.org"
emacs --quick --daemon=spine \
--eval "(setq spine-org-file \"$SPINE_ORG\")" \
--load spine.el
sleep 3
curl -s http://localhost:8080/ | grep -c "WANT"
# Expected: 1 (the group header "WANT (3)")
emacsclient --socket-name=spine --eval '(kill-emacs)'
```
The output should contain `1`.
---
### Task 8: Final verification and cleanup
**Files:**
- No changes — verification only.
- [ ] **Step 1: Run full ERT test suite**
```bash
emacs --quick --batch \
--directory "$(pwd)" \
--load test/spine-books-test.el \
-f ert-run-tests-batch-and-exit
```
Expected: all 6 tests pass.
- [ ] **Step 2: Verify git status is clean**
```bash
git status
```
Expected: `nothing to commit, working tree clean`
- [ ] **Step 3: Review commit log**
```bash
git log --oneline
```
Expected: ~7 commits covering sample data, server guard, model, tests, view, handler.