Files

T

jbrechtel d476a4eacc docs: add data model and index page implementation plan

2026-06-20 16:54:53 -04:00

23 KiB

Raw Permalink Blame History

Spine Data Model & Index Page — Implementation Plan

Goal: Build the model layer (spine-books) and index page view so the app reads a real spine.org file and displays books grouped by status.

Architecture: Model/view split. spine-books parses spine.org into plists via org-element + org-entry-get. spine-render-index takes those plists and an optional selected ID, returns HTML matching mockup A's class structure. A new / handler replaces the /hello smoke test. A spine-skip-server-start guard lets tests load spine.el without starting the HTTP server.

Tech Stack: Emacs Lisp, org-element (built-in), org-entry-get (built-in), ert (built-in testing).

File structure

File	Responsibility
`sample-books.org`	Test data: 5 books across all statuses
`spine.el`	Modified: new functions, handler, skip-server-start guard
`test/spine-books-test.el`	ERT tests for `spine-books`

Task 1: Create sample test data

Files:

Create: sample-books.org
Step 1: Write sample-books.org

Five books covering all four statuses, plus an entry with minimal properties.

#+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a)

* WANT The Left Hand of Darkness
:PROPERTIES:
:AUTHOR:  Ursula K. Le Guin
:ISBN:    978-0441478125
:COVER:   covers/left-hand.jpg
:FORMAT:  ebook
:REC_BY:  Alex
:REC_NOTE: Le Guin at her most human
:RATING:  
:ADDED:   [2026-04-01]
:ID:      9a2b-lhd
:END:

* WANT Dune
:PROPERTIES:
:AUTHOR:  Frank Herbert
:ISBN:    978-0441172719
:COVER:   covers/dune.jpg
:FORMAT:  hardcover
:REC_BY:  Everyone
:REC_NOTE: 
:RATING:  
:ADDED:   [2026-05-10]
:ID:      7f3c-dun
:END:

* WANT Piranesi
:PROPERTIES:
:AUTHOR:  Susanna Clarke
:ISBN:    
:COVER:   
:FORMAT:  
:REC_BY:  
:REC_NOTE: 
:RATING:  
:ADDED:   [2026-06-01]
:ID:      5e8d-pir
:END:

* READING Use of Weapons
:PROPERTIES:
:AUTHOR:  Iain M. Banks
:ISBN:    978-0316029193
:COVER:   covers/use-of-weapons.jpg
:FORMAT:  audiobook
:REC_BY:  Priya
:REC_NOTE: If you liked Player of Games, this one will wreck you
:RATING:  
:ADDED:   [2026-02-20]
:ID:      8c1e-uow
:END:
:LOGBOOK:
- State "READING" from "WANT" [2026-03-01]
:END:
- [2026-03-02] The two-track structure is doing something I can't name yet.
- [2026-03-07] Zakalwe's competence reads as a wound.
- [2026-03-09] The chapter numbering. Oh.

* READ Ancillary Justice
:PROPERTIES:
:AUTHOR:  Ann Leckie
:ISBN:    978-0316246620
:COVER:   covers/ancillary.jpg
:FORMAT:  ebook
:REC_BY:  
:REC_NOTE: 
:RATING:  5
:ADDED:   [2025-11-15]
:ID:      3a1b-aj
:END:
:LOGBOOK:
- State "READ" from "READING" [2026-01-10]
- State "READING" from "WANT" [2025-11-20]
:END:
- [2025-11-22] The pronoun game is a genuinely fresh take on identity.
- [2025-12-05] Justice of Toren's multi-body perspective is unsettling.
- [2026-01-08] The tea set. Perfect ending.

This file exercises:

WANT books: 3 (Left Hand, Dune, Piranesi) — tests group header, count, different property densities
READING books: 1 (Use of Weapons) — tests notes extraction, recommendation trail
READ books: 1 (Ancillary Justice) — tests rating, LOGBOOK
ABANDONED books: 0 — tests empty group omission
Minimal properties: Piranesi has no ISBN, cover, format, recommendation, or notes
Step 2: Verify file exists

Run: cat sample-books.org | head -3 Expected: #+TODO: WANT(w) READING(r) | READ(d) ABANDONED(a)

Step 3: Commit

git add sample-books.org
git commit -m "test: add sample-books.org test data"

Task 2: Add server-start guard to spine.el

Files:

Modify: spine.el
Step 1: Add spine-skip-server-start variable and wrap server start

Insert after the template rendering section (after line 44, before ;; --- handlers):

(defvar spine-skip-server-start nil
  "When non-nil, do not start the HTTP server.
Set by test harnesses that only need the model functions.")

Change the server start block (lines 53–58) to:

;; --- start ------------------------------------------------------------
(unless spine-skip-server-start
  (setq httpd-host spine-host)
  (setq httpd-port spine-port)
  (httpd-start)
  (message "Spine listening on http://%s:%d"
           (or spine-host "localhost") spine-port))

Step 2: Verify batch load with guard

Run: emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el Expected: exits cleanly, no "Spine listening" message, no errors.

Step 3: Verify server still works normally without guard

Run the normal scripts/run flow and curl /hello — /hello still exists at this point since the handler hasn't been replaced yet.

scripts/run 8080
sleep 3
curl -s http://localhost:8080/hello
scripts/stop

Expected: HTML containing <h1>Hello from Emacs.</h1>.

Step 4: Commit

git add spine.el
git commit -m "feat: add spine-skip-server-start guard for testing"

Task 3: Implement spine-books model function

Files:

Modify: spine.el
Step 1: Add cl-lib require

At the top of spine.el, after (require 'ht), add:

(require 'cl-lib)

cl-lib is built into Emacs and provides cl-return-from.

Step 2: Add spine-org-file config variable

After spine-port and spine-host defvars (after line 27), add:

(defvar spine-org-file "~/spine/books.org"
  "Path to the spine.org data file.")

Step 3: Add spine--h HTML escaping helper

After spine-render, add:

(defun spine--h (text)
  "HTML-escape TEXT."
  (when text
    (let ((s text))
      (setq s (replace-regexp-in-string "&" "&amp;" s))
      (setq s (replace-regexp-in-string "<" "&lt;" s))
      (setq s (replace-regexp-in-string ">" "&gt;" s))
      (setq s (replace-regexp-in-string "\"" "&quot;" s))
      s)))

Step 4: Add spine--extract-notes helper

After spine--h, add:

(defun spine--extract-notes (hl-begin hl-end)
  "Extract reading notes from headline body between HL-BEGIN and HL-END.
Returns list of (date-string text-string) pairs."
  (let ((notes nil))
    (save-excursion
      (goto-char hl-begin)
      ;; Skip past PROPERTIES and LOGBOOK drawers
      (while (re-search-forward
              "^[ \t]*:\\(PROPERTIES\\|LOGBOOK\\):" hl-end t)
        (re-search-forward "^[ \t]*:END:" hl-end t))
      ;; Collect note lines: "- [YYYY-MM-DD] text"
      (while (re-search-forward
              "^- \\[\\([0-9]\\{4\\}-[0-9]\\{2\\}-[0-9]\\{2\\}\\)\\] \\(.*\\)"
              hl-end t)
        (push (list (match-string 1) (match-string 2)) notes)))
    (nreverse notes)))

Step 5: Add spine-books function

After spine--extract-notes, add:

(defun spine-books ()
  "Return a list of plists, one per top-level headline in `spine-org-file'.
Returns nil if the file is missing or cannot be parsed."
  (unless (file-exists-p spine-org-file)
    (message "spine-books: file not found: %s" spine-org-file)
    (cl-return-from spine-books nil))
  (condition-case err
      (with-temp-buffer
        (insert-file-contents spine-org-file)
        (org-mode)
        (let ((books nil))
          (org-element-map (org-element-parse-buffer 'headline) 'headline
            (lambda (hl)
              (when (= (org-element-property :level hl) 1)
                (let ((pos (org-element-property :begin hl)))
                  (push (list :id       (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "ID"))
                              :title    (org-element-property :title hl)
                              :status   (org-element-property :todo-keyword hl)
                              :author   (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "AUTHOR"))
                              :isbn     (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "ISBN"))
                              :cover    (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "COVER"))
                              :format   (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "FORMAT"))
                              :rec_by   (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "REC_BY"))
                              :rec_note (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "REC_NOTE"))
                              :rating   (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "RATING"))
                              :added    (save-excursion
                                          (goto-char pos)
                                          (org-entry-get nil "ADDED"))
                              :tags     (org-element-property :tags hl)
                              :notes    (spine--extract-notes
                                         (org-element-property :begin hl)
                                         (org-element-property :end hl)))
                        books)))))
          (nreverse books)))
    (error
     (message "spine-books: failed to parse %s: %s"
              spine-org-file (error-message-string err))
     nil)))

Step 6: Verify batch load still works

Run: emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el Expected: exits cleanly.

Step 7: Verify spine-books returns data from sample file

Run:

emacs --quick --batch \
  --eval "(setq spine-skip-server-start t spine-org-file \"sample-books.org\")" \
  --load spine.el \
  --eval "(progn (let ((books (spine-books))) (message \"Found %d books\" (length books))))"

Expected: Found 5 books in output.

Step 8: Commit

git add spine.el
git commit -m "feat: add spine-books model function"

Task 4: Test spine-books with ERT

Files:

Create: test/spine-books-test.el
Step 1: Create test/ directory

mkdir -p test

Step 2: Write test/spine-books-test.el

;;; spine-books-test.el — ERT tests for spine-books

(require 'ert)
(require 'cl-lib)

;; Load spine.el without starting the server
(setq spine-skip-server-start t)
(setq spine-org-file (expand-file-name "sample-books.org"
                                       (file-name-directory load-file-name)))
(load-file (expand-file-name "spine.el"
                             (file-name-directory load-file-name)))

(ert-deftest spine-books-parses-sample ()
  "spine-books returns correct plists for sample-books.org."
  (let ((books (spine-books)))
    (should books)
    (should (= (length books) 5))))

(ert-deftest spine-books-first-book-has-all-fields ()
  "Use of Weapons entry has all expected fields."
  (let* ((books (spine-books))
         (uow (cl-find "8c1e-uow" books
                       :key (lambda (b) (plist-get b :id))
                       :test #'equal)))
    (should uow)
    (should (equal (plist-get uow :title) "Use of Weapons"))
    (should (equal (plist-get uow :status) "READING"))
    (should (equal (plist-get uow :author) "Iain M. Banks"))
    (should (equal (plist-get uow :isbn) "978-0316029193"))
    (should (equal (plist-get uow :cover) "covers/use-of-weapons.jpg"))
    (should (equal (plist-get uow :format) "audiobook"))
    (should (equal (plist-get uow :rec_by) "Priya"))
    (should (equal (plist-get uow :rec_note) "If you liked Player of Games, this one will wreck you"))
    (should (equal (plist-get uow :tags) '("scifi" "literary")))
    (should (equal (plist-get uow :notes)
                   '(("2026-03-02" "The two-track structure is doing something I can't name yet.")
                     ("2026-03-07" "Zakalwe's competence reads as a wound.")
                     ("2026-03-09" "The chapter numbering. Oh."))))))

(ert-deftest spine-books-missing-properties-are-nil ()
  "Piranesi entry has nil for non-present properties."
  (let* ((books (spine-books))
         (pir (cl-find "5e8d-pir" books
                       :key (lambda (b) (plist-get b :id))
                       :test #'equal)))
    (should pir)
    (should (equal (plist-get pir :title) "Piranesi"))
    (should (equal (plist-get pir :status) "WANT"))
    (should (equal (plist-get pir :author) "Susanna Clarke"))
    ;; ISBN, cover, format, rec_by, rec_note should all be nil or ""
    (should-not (plist-get pir :isbn))
    (should-not (plist-get pir :cover))
    (should-not (plist-get pir :format))
    (should-not (plist-get pir :rec_by))
    (should-not (plist-get pir :rec_note))
    ;; Notes should be empty
    (should-not (plist-get pir :notes))))

(ert-deftest spine-books-read-book-has-rating ()
  "Ancillary Justice has rating 5."
  (let* ((books (spine-books))
         (aj (cl-find "3a1b-aj" books
                      :key (lambda (b) (plist-get b :id))
                      :test #'equal)))
    (should aj)
    (should (equal (plist-get aj :status) "READ"))
    (should (equal (plist-get aj :rating) "5"))
    (should (= (length (plist-get aj :notes)) 3))))

(ert-deftest spine-books-empty-file-returns-empty-list ()
  "Empty Org file returns empty list, not nil."
  (let* ((tmp-file (make-temp-file "spine-test-" nil ".org"))
         (spine-org-file tmp-file))
    (unwind-protect
        (let ((books (spine-books)))
          (should (listp books))
          (should (= (length books) 0)))
      (delete-file tmp-file))))

(ert-deftest spine-books-missing-file-returns-nil ()
  "Non-existent file returns nil."
  (let ((spine-org-file "/tmp/spine-nonexistent-12345.org"))
    (should-not (spine-books))))

Step 3: Run the tests

emacs --quick --batch \
  --directory "$(pwd)" \
  --load test/spine-books-test.el \
  -f ert-run-tests-batch-and-exit

Expected: all 6 tests pass.

Step 4: Commit

git add test/spine-books-test.el
git commit -m "test: add ERT tests for spine-books"

Task 5: Implement view functions

Files:

Modify: spine.el
Step 1: Add spine-render-index function

After spine-books, add:

(defun spine-render-index (books &optional selected-id)
  "Render the index page HTML for BOOKS, expanding the book matching SELECTED-ID."
  (let ((order '("WANT" "READING" "READ" "ABANDONED"))
        (grouped (make-hash-table :test 'equal)))
    ;; Group books by status
    (dolist (book books)
      (let* ((status (or (plist-get book :status) "WANT")))
        (push book (gethash status grouped))))
    ;; Render
    (with-temp-buffer
      (insert "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n")
      (insert "<meta charset=\"utf-8\">\n")
      (insert "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n")
      (insert "<title>Spine</title>\n</head>\n<body>\n")
      (insert "<div class=\"frame\">\n")
      (insert "<div class=\"titlebar\">Spine</div>\n")
      ;; Each group in order
      (dolist (status order)
        (let ((group-books (nreverse (gethash status grouped))))
          (when group-books
            (insert (format "<div class=\"group\">%s (%d)</div>\n"
                            status (length group-books)))
            (dolist (book group-books)
              (let ((selected (equal (plist-get book :id) selected-id)))
                (insert (format "<div class=\"row%s\">\n"
                                (if selected " selected" "")))
                ;; Format glyph
                (let ((fmt (plist-get book :format)))
                  (insert (format "<span class=\"glyph\">%s</span>\n"
                                  (cond ((equal fmt "hardcover") "[H]")
                                        ((equal fmt "ebook")    "[E]")
                                        ((equal fmt "audiobook") "[A]")
                                        (t                      "")))))
                ;; Title + author
                (insert "<span class=\"title\">")
                (insert (spine--h (plist-get book :title)))
                (let ((author (plist-get book :author)))
                  (when author
                    (insert (format " <span class=\"muted\">%s</span>"
                                    (spine--h author)))))
                (insert "</span>\n")
                ;; Tags
                (let ((tags (plist-get book :tags)))
                  (when tags
                    (insert (format "<span class=\"tags\">%s</span>\n"
                                    (mapconcat (lambda (t) (concat ":" t ":"))
                                               tags " ")))))
                ;; Recommendation trail
                (let ((rec-by (plist-get book :rec_by))
                      (rec-note (plist-get book :rec_note)))
                  (when (and rec-by (> (length rec-by) 0))
                    (insert
                     (format "<span class=\"trail\">%s%s</span>\n"
                             (spine--h rec-by)
                             (if (and rec-note (> (length rec-note) 0))
                                 (concat ": " (spine--h rec-note))
                               "")))))
                ;; Status pill
                (let ((status (or (plist-get book :status) "WANT")))
                  (insert (format "<span class=\"pill %s\">%s</span>\n"
                                  (downcase status) status)))
                ;; Expanded detail
                (when selected
                  (insert "<div class=\"detail\">\n")
                  (dolist (note (plist-get book :notes))
                    (insert (format
                             "<div class=\"logline\"><span class=\"logdate\">[%s]</span> %s</div>\n"
                             (spine--h (car note))
                             (spine--h (cadr note)))))
                  (let ((rating (plist-get book :rating)))
                    (when (and rating (> (length rating) 0))
                      (insert (format "<div>Rating: %s/5</div>\n"
                                      (spine--h rating)))))
                  (let ((isbn (plist-get book :isbn)))
                    (when (and isbn (> (length isbn) 0))
                      (insert (format "<div>ISBN: %s</div>\n"
                                      (spine--h isbn)))))
                  (let ((fmt (plist-get book :format)))
                    (when (and fmt (> (length fmt) 0))
                      (insert (format "<div>Format: %s</div>\n"
                                      (spine--h fmt)))))
                  (let ((added (plist-get book :added)))
                    (when (and added (> (length added) 0))
                      (insert (format "<div>Added: %s</div>\n"
                                      (spine--h added)))))
                  (insert "</div>\n"))
                (insert "</div>\n"))))))
      (insert "</div>\n</body>\n</html>\n")
      (buffer-string))))

Step 2: Add spine-render-empty-state function

After spine-render-index, add:

(defun spine-render-empty-state ()
  "Render a page indicating no books file was found."
  (concat "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n"
          "<meta charset=\"utf-8\">\n"
          "<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n"
          "<title>Spine</title>\n</head>\n<body>\n"
          "<div class=\"frame\">\n"
          "<div class=\"titlebar\">Spine</div>\n"
          "<div class=\"row\">No books yet.</div>\n"
          "</div>\n</body>\n</html>\n"))

Step 3: Verify batch load

Run: emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el Expected: exits cleanly, no errors.

Step 4: Commit

git add spine.el
git commit -m "feat: add spine-render-index and empty-state view functions"

Task 6: Wire up the index handler

Files:

Modify: spine.el
Step 1: Replace /hello handler with / index handler

Replace the hello handler (lines 48–51) with:

(defservlet index text/html ()
  (let ((books (spine-books)))
    (if books
        (insert (spine-render-index books
                  (ht-get (ht-from-query-string (httpd-query-string)) "id")))
      (insert (spine-render-empty-state)))))

Step 2: Verify batch load

Run: emacs --quick --batch --eval "(setq spine-skip-server-start t)" --load spine.el Expected: exits cleanly.

Step 3: Commit

git add spine.el
git commit -m "feat: wire up / index handler replacing /hello"

Task 7: Smoke test end-to-end

Files:

No changes — verification only.
Step 1: Start server with sample data

emacs --quick \
  --eval "(setq spine-org-file \"$(pwd)/sample-books.org\")" \
  --load spine.el &
EMACS_PID=$!
sleep 5

Step 2: Verify index page returns HTML with real data

curl -s http://localhost:8080/ | head -20

Expected: HTML document with:

<title>Spine</title>
<div class="group">WANT (3)</div>
<div class="group">READING (1)</div>
<div class="group">READ (1)</div>
No ABANDONED group (zero books)
"Use of Weapons" visible in output
Recommendation trail "Priya: If you liked..."
Step 3: Verify expanded detail via ?id=

curl -s "http://localhost:8080/?id=8c1e-uow" | grep -A5 "detail"

Expected: expanded detail section with reading notes for Use of Weapons:

[2026-03-02] The two-track structure...
[2026-03-07] Zakalwe's competence...
[2026-03-09] The chapter numbering. Oh.
Step 4: Verify bogus ?id= does not crash

curl -s "http://localhost:8080/?id=nonexistent"

Expected: normal index page, no expanded detail, HTTP 200.

Step 5: Verify empty state when file is missing

kill $EMACS_PID 2>/dev/null
sleep 1
emacs --quick \
  --eval "(setq spine-org-file \"/tmp/nonexistent-books.org\")" \
  --load spine.el &
EMACS_PID=$!
sleep 5
curl -s http://localhost:8080/

Expected: HTML containing No books yet.

Step 6: Stop server

kill $EMACS_PID 2>/dev/null

Step 7: Verify with scripts/run

# Override spine-org-file via the run script (pass it as an extra eval)
SPINE_ORG="$(pwd)/sample-books.org"
emacs --quick --daemon=spine \
  --eval "(setq spine-org-file \"$SPINE_ORG\")" \
  --load spine.el
sleep 3
curl -s http://localhost:8080/ | grep -c "WANT"
# Expected: 1 (the group header "WANT (3)")
emacsclient --socket-name=spine --eval '(kill-emacs)'

The output should contain 1.

Task 8: Final verification and cleanup

Files:

No changes — verification only.
Step 1: Run full ERT test suite

emacs --quick --batch \
  --directory "$(pwd)" \
  --load test/spine-books-test.el \
  -f ert-run-tests-batch-and-exit

Expected: all 6 tests pass.

Step 2: Verify git status is clean

git status

Expected: nothing to commit, working tree clean

Step 3: Review commit log

git log --oneline

Expected: ~7 commits covering sample data, server guard, model, tests, view, handler.

23 KiB Raw Permalink Blame History Unescape Escape

Spine Data Model & Index Page — Implementation Plan

File structure

Task 1: Create sample test data

Task 2: Add server-start guard to spine.el

Task 3: Implement spine-books model function

Task 4: Test spine-books with ERT

Task 5: Implement view functions

Task 6: Wire up the index handler

Task 7: Smoke test end-to-end

Task 8: Final verification and cleanup

23 KiB

Raw Permalink Blame History