Add implementation plan for live recipe watching

2026-05-19 22:58:07 -04:00
parent c7bff79371
commit 042bcc9e51
1 changed files with 473 additions and 0 deletions
@@ -0,0 +1,473 @@
+# Live Recipe Watching Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Watch the recipe directory for `.cook` file changes and update the in-memory recipe index without a server restart.
+
+**Architecture:** Replace the `[RecipeInfo]` closure with an `IORef (Map FilePath RecipeInfo)`. A `forkIO`'d watcher thread using `fsnotify` subscribes to filesystem events, debounces within a 200ms window, and updates the `IORef` via `atomicModifyIORef'`. HTTP handlers read from the `IORef` on each request.
+
+**Tech Stack:** Haskell, `fsnotify-0.4.4.0`, `Data.IORef`, `Data.Map.Strict`, `Data.Time.Clock`, `Control.Concurrent`.
+
+---
+
+## File Map
+
+| File | Action | Purpose |
+|---|---|---|
+| `stack.yaml` | Modify | Add `fsnotify` as extra-dep (not in LTS-24.38) |
+| `package.yaml` | Modify | Add `fsnotify` to library dependencies |
+| `roux-server.cabal` | Regenerate | Via `./hs hpack` |
+| `src/Roux/Server.hs` | Modify | State type, watcher thread, handler updates |
+
+No new files needed — all changes are contained in `Server.hs`.
+
+---
+
+### Task 1: Add `fsnotify` dependency
+
+**Files:**
+- Modify: `package.yaml`, `stack.yaml`
+- Regenerate: `roux-server.cabal`
+
+- [ ] **Step 1: Add fsnotify to stack.yaml extra-deps**
+
+Since `fsnotify` is not in LTS-24.38, add it as an extra-dep in `stack.yaml`:
+
+```yaml
+extra-deps:
+  - fsnotify-0.4.4.0
+```
+
+- [ ] **Step 2: Add fsnotify to package.yaml library dependencies**
+
+```yaml
+  - containers
+  - directory
+  - filepath
+  - fsnotify
+  - http-types
+```
+
+- [ ] **Step 3: Regenerate roux-server.cabal**
+
+Run: `./hs hpack`
+Expected: no errors.
+
+- [ ] **Step 4: Verify stack can resolve fsnotify**
+
+Run: `./hs stack build --no-run-tests 2>&1 | tail -10`
+Expected: builds successfully (or at least resolves deps).
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add package.yaml stack.yaml roux-server.cabal
+git commit -m "chore: add fsnotify dependency for live recipe watching"
+```
+
+---
+
+### Task 2: Refactor `Server.hs` for live watching
+
+**Files:**
+- Modify: `src/Roux/Server.hs`
+
+This is the main implementation task. All changes are in one file.
+
+- [ ] **Step 1: Rewrite imports**
+
+Replace all imports in `src/Roux/Server.hs` with:
+
+```haskell
+{-# LANGUAGE OverloadedStrings #-}
+
+{- | WAI application for the Roux recipe server.
+
+Handles HTTP requests directly via wai — no framework.
+Recipes are loaded from disk at startup, then watched live via fsnotify.
+-}
+module Roux.Server (
+    app,
+) where
+
+import Control.Concurrent (forkIO)
+import Control.Concurrent.Chan (Chan, newChan, readChan)
+import Control.Monad (when)
+import Data.ByteString.Lazy (ByteString)
+import Data.Char (toLower)
+import Data.IORef (IORef, atomicModifyIORef', newIORef, readIORef)
+import Data.Map.Strict (Map)
+import qualified Data.Map.Strict as Map
+import Data.Time.Clock (UTCTime, getCurrentTime, diffUTCTime)
+import qualified Network.HTTP.Types as HTTP
+import qualified Network.Wai as Wai
+import System.Directory (doesFileExist)
+import System.FilePath ((</>), takeBaseName)
+import System.FSNotify (Event (..), EventIsDirectory (..), StopListening,
+                        WatchConfig (..), WatchMode (..), withManagerConf,
+                        watchTreeChan)
+
+import qualified Roux.Html as Html
+import qualified Roux.RecipeIndex as Idx
+```
+
+- [ ] **Step 2: Rewrite `app` to use `IORef` and fork the watcher**
+
+Replace the entire `app` function body:
+
+```haskell
+-- | Build the WAI 'Application' given a path to the recipe directory.
+--
+-- Recipes are scanned at startup and then watched live via fsnotify.
+app :: FilePath -> IO Wai.Application
+app recipeDir = do
+    putStrLn $ "[roux] scanning recipes from " <> recipeDir
+    recipes <- Idx.scanRecipes recipeDir
+    let count = length recipes
+        ok = length [() | r <- recipes, isRight (Idx.riRecipe r)]
+        errs = count - ok
+    putStrLn $ "[roux] found " <> show count <> " recipe(s)"
+    when (errs > 0) $
+        putStrLn $
+            "[roux] " <> show errs <> " had parse errors"
+    -- Build initial map keyed by lowercased filename
+    let recipeMap = Map.fromList [(map toLower (Idx.riFilename r), r) | r <- recipes]
+    state <- newIORef recipeMap
+    -- Start the filesystem watcher
+    forkIO (watchRecipes recipeDir state)
+    pure (handleRequest state)
+  where
+    isRight (Right _) = True
+    isRight (Left _) = False
+```
+
+Note: `isRight` was previously a top-level function. It's now local to `app`. That's fine.
+
+- [ ] **Step 3: Add the `watchRecipes` function**
+
+Add after `app` (before `handleRequest`):
+
+```haskell
+-- | Watch a directory for .cook file changes and update the shared state.
+-- Implements debouncing: events for the same path within 200ms are skipped.
+watchRecipes :: FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
+watchRecipes dir state = do
+    putStrLn $ "[roux] watching " <> dir <> " for changes"
+    let cfg = defaultConfig{confWatchMode = WatchModeOS}
+    withManagerConf cfg $ \mgr -> do
+        chan <- newChan
+        _stopListening <- watchTreeChan mgr dir (const True) chan
+        -- lastSeen tracks when we last processed each path (for debouncing)
+        debounceLoop chan state dir Map.empty
+  where
+    debounceLoop :: Chan Event -> IORef (Map FilePath Idx.RecipeInfo) -> FilePath -> Map FilePath UTCTime -> IO ()
+    debounceLoop chan state dir lastSeen = do
+        ev <- readChan chan
+        -- Only process file events (skip directory events)
+        let isDir = eventIsDirectory ev
+        now <- getCurrentTime
+        if isDir == IsDirectory
+            then debounceLoop chan state dir lastSeen
+            else do
+                let path = eventPath ev
+                let key = map toLower path
+                let lastTime = Map.lookup key lastSeen
+                case lastTime of
+                    Just t | diffUTCTime now t < 0.2 -> do
+                        -- Debounced: skip, but pass through lastSeen
+                        debounceLoop chan state dir lastSeen
+                    _ -> do
+                        -- Process the event
+                        reReadRecipe dir path state
+                        debounceLoop chan state dir (Map.insert key now lastSeen)
+```
+
+Key points:
+- Uses `defaultConfig` with `WatchModeOS` (inotify on Linux)
+- `watchTreeChan` takes an `ActionPredicate` (`const True` means watch everything)
+- `readChan` blocks until events arrive
+- Debouncing: if the same path was processed within 0.2 seconds, skip
+- Directory events are filtered out
+
+- [ ] **Step 4: Add the `reReadRecipe` helper**
+
+Add after `watchRecipes`:
+
+```haskell
+-- | Re-read a single recipe file and update the shared state.
+-- On read/parse errors, logs the error and keeps the previous version.
+reReadRecipe :: FilePath -> FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
+reReadRecipe dir path state = do
+    let key = map toLower path
+    let fullPath = dir </> path
+    exists <- doesFileExist fullPath
+    if not exists
+        then do
+            atomicModifyIORef' state $ \m ->
+                (Map.delete key m, ())
+            putStrLn $ "[roux] removed " <> path
+        else do
+            content <- readFile fullPath
+            let parsed = parseCookFile (T.pack content)
+                title = case parsed of
+                    Right r -> extractTitle r path
+                    Left _ -> T.pack (takeBaseName path)
+            case parsed of
+                Left err -> do
+                    putStrLn $ "[roux] parse error in " <> path <> ": " <> err
+                    putStrLn $ "[roux] keeping previous version"
+                Right _ -> do
+                    let info =
+                            Idx.RecipeInfo
+                                { Idx.riFilename = path
+                                , Idx.riTitle = title
+                                , Idx.riRecipe = parsed
+                                }
+                    atomicModifyIORef' state $ \m ->
+                        (Map.insert key info m, ())
+                    putStrLn $ "[roux] updated " <> path
+  where
+    extractTitle recipe filename =
+        case metaTitle (recipeMetadata recipe) of
+            Just t -> t
+            Nothing -> T.pack (takeBaseName filename)
+    metaTitle (Data.CookLang.Metadata m) = m ... -- WRONG, need to check Data.CookLang
+```
+
+**Wait — the `extractTitle` function already exists in `Roux.RecipeIndex`**. Let me check what's exported.
+
+`Idx.extractTitle` is a module-internal function (not exported from `Roux.RecipeIndex`). The `Idx.loadRecipe` function does the same logic internally.
+
+I have two options:
+1. Export `extractTitle` from `Roux.RecipeIndex`
+2. Import `Data.CookLang` and extract title inline
+
+Option 1 is cleaner. I'll add `extractTitle` to the export list of `RecipeIndex.hs`.
+
+Actually wait — looking at the `Roux.RecipeIndex` module:
+
+```haskell
+module Roux.RecipeIndex (
+    RecipeInfo (..),
+    RecipeSearchEntry (..),
+    scanRecipes,
+    toSearchEntry,
+) where
+```
+
+`extractTitle` is defined but not exported. I have a few choices:
+
+Option A: Add `extractTitle` to the export list.
+Option B: Use the existing `loadRecipe` pattern but that requires access to the raw recipe's metadata via `Data.CookLang`.
+Option C: Write a local helper in Server.hs.
+
+The simplest is Option A or creating a small helper in `Roux.RecipeIndex` that can load a single file (like `loadRecipe`).
+
+Actually, the cleanest approach: export `loadRecipe` from `RecipeIndex`. It's already defined there. Then I can call `Idx.reloadRecipe` or reuse `loadRecipe` directly.
+
+Looking at `loadRecipe`:
+
+```haskell
+loadRecipe :: FilePath -> FilePath -> IO RecipeInfo
+loadRecipe dir filename = do
+    content <- readFile (dir </> filename)
+    let parsed = parseCookFile (T.pack content)
+        title = case parsed of
+            Right r -> extractTitle r filename
+            Left _err -> T.pack (takeBaseName filename)
+    return RecipeInfo{ riFilename = filename, riTitle = title, riRecipe = parsed }
+```
+
+Perfect. I can either:
+- Export `loadRecipe` from `RecipeIndex` and call it from `Server.hs`
+- Or keep the local `reReadRecipe` with its own inline logic
+
+Since loadRecipe is already tested and working, exporting it is cleaner. But actually, the plan says to keep changes minimal. Let me just add `loadRecipe` to the exports of `RecipeIndex.hs` and use it in `Server.hs`.
+
+Let me update the Step 4 code:
+
+```haskell
+reReadRecipe :: FilePath -> FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
+reReadRecipe dir path state = do
+    let fullPath = dir </> path
+    exists <- doesFileExist fullPath
+    if not exists
+        then do
+            let key = map toLower path
+            atomicModifyIORef' state $ \m ->
+                (Map.delete key m, ())
+            putStrLn $ "[roux] removed " <> path
+        else do
+            -- Use Idx.loadRecipe to read and parse
+            info <- Idx.loadRecipe dir path
+            let key = map toLower path
+            case Idx.riRecipe info of
+                Left err -> do
+                    putStrLn $ "[roux] parse error in " <> path <> ": " <> err
+                    putStrLn $ "[roux] keeping previous version"
+                Right _ -> do
+                    atomicModifyIORef' state $ \m ->
+                        (Map.insert key info m, ())
+                    putStrLn $ "[roux] updated " <> path
+```
+
+This is much cleaner. I need to export `loadRecipe` from `RecipeIndex.hs`.
+
+- [ ] **Step 5: Update `handleRequest` to accept `IORef`**
+
+Replace `handleRequest`:
+
+```haskell
+-- | Route an incoming request to the appropriate handler.
+handleRequest :: IORef (Map FilePath Idx.RecipeInfo) -> Wai.Application
+handleRequest state request respond = do
+    recipes <- readIORef state
+    let recipeList = Map.elems recipes
+    case Wai.pathInfo request of
+        -- Index page (default: alphabetical)
+        [] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
+        -- Index page (sorted by tags) — sort mode ignored, driven client-side
+        ["sorted", "tags"] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
+        -- Index page (sorted by course/category) — sort mode ignored, driven client-side
+        ["sorted", "course"] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
+        -- Recipe detail page
+        ["recipes", rawPath] ->
+            case lookupRecipe (urlDecodePath rawPath) recipes of
+                Just info -> respond (htmlResponse (Html.recipePage info))
+                Nothing -> respond notFound
+        _ -> respond notFound
+```
+
+`handleRequest` is now an `IO` action — it reads from the `IORef` on each request. `Wai.Application` is `Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived`, so returning `IO` is fine.
+
+- [ ] **Step 6: Simplify `lookupRecipe` to use `Map.lookup`**
+
+Replace `lookupRecipe`:
+
+```haskell
+-- | Look up a recipe by filename (case-insensitive, .cook extension optional).
+lookupRecipe :: FilePath -> Map FilePath Idx.RecipeInfo -> Maybe Idx.RecipeInfo
+lookupRecipe target recipes =
+    Map.lookup (map toLower (stripCookExt target)) recipes
+```
+
+- [ ] **Step 7: Remove `stripCookExt` and `urlDecodePath` helpers**
+
+These are unchanged. Keep them.
+
+- [ ] **Step 8: Remove unused imports**
+
+Remove from imports:
+- `Data.List (find)` — no longer used
+
+Keep:
+- `Data.Char (toLower)` — still used in `lookupRecipe` and `reReadRecipe`
+
+- [ ] **Step 9: Export `loadRecipe` from `RecipeIndex.hs`**
+
+In `src/Roux/RecipeIndex.hs`, add `loadRecipe` to the export list:
+
+```haskell
+module Roux.RecipeIndex (
+    RecipeInfo (..),
+    RecipeSearchEntry (..),
+    loadRecipe,
+    scanRecipes,
+    toSearchEntry,
+) where
+```
+
+- [ ] **Step 10: Build to verify compilation**
+
+Run: `./hs stack build`
+Expected: compiles without errors.
+
+- [ ] **Step 11: Run tests to verify nothing broke**
+
+Run: `./hs stack test`
+Expected: all tests pass.
+
+- [ ] **Step 12: Commit**
+
+```bash
+git add src/Roux/Server.hs src/Roux/RecipeIndex.hs
+git commit -m "feat: watch recipe directory for live updates via fsnotify"
+```
+
+---
+
+### Task 3: Manual smoke test
+
+**Files:** (none, manual testing)
+
+- [ ] **Step 1: Start the server in the background**
+
+```bash
+./hs stack exec roux-server -- --recipe-dir recipes &
+sleep 2
+```
+
+- [ ] **Step 2: Verify a recipe loads**
+
+```bash
+curl -s http://localhost:8080/ | grep -i "recipe" | head -5
+```
+Expected: index page HTML with recipe titles.
+
+- [ ] **Step 3: Check server logs for watching confirmation**
+
+Should see: `[roux] watching recipes for changes`
+
+- [ ] **Step 4: Modify a recipe file and verify the change is picked up**
+
+```bash
+# Pick a recipe, temporarily replace content
+RECIPE=$(ls recipes/*.cook | head -1)
+RECIPE_NAME=$(basename "$RECIPE" .cook)
+echo "---" > /tmp/test_change.cook
+echo "title: Test Live Update" >> /tmp/test_change.cook
+echo "---" >> /tmp/test_change.cook
+echo "Testing live file watching." >> /tmp/test_change.cook
+cp "$RECIPE" /tmp/backup.cook
+cp /tmp/test_change.cook "$RECIPE"
+sleep 1
+curl -s "http://localhost:8080/recipes/$RECIPE_NAME" | grep "Test Live Update"
+# Restore original
+cp /tmp/backup.cook "$RECIPE"
+```
+Expected: grep finds "Test Live Update" in the response.
+
+- [ ] **Step 5: Delete a recipe file and verify 404**
+
+```bash
+RECIPE2=$(ls recipes/*.cook | grep -v "$(basename "$RECIPE")" | head -1)
+RECIPE2_NAME=$(basename "$RECIPE2" .cook)
+cp "$RECIPE2" /tmp/backup2.cook
+rm "$RECIPE2"
+sleep 1
+curl -s -o /dev/null -w "%{http_code}" "http://localhost:8080/recipes/$RECIPE2_NAME"
+# Restore
+cp /tmp/backup2.cook "$RECIPE2"
+```
+Expected: HTTP 404.
+
+- [ ] **Step 6: Add a new recipe file and verify it appears**
+
+```bash
+echo "---" > /tmp/new_recipe.cook
+echo "title: New Test Recipe" >> /tmp/new_recipe.cook
+echo "---" >> /tmp/new_recipe.cook
+echo "A brand new recipe." >> /tmp/new_recipe.cook
+cp /tmp/new_recipe.cook recipes/new_test_recipe.cook
+sleep 1
+curl -s "http://localhost:8080/recipes/new_test_recipe" | grep "New Test Recipe"
+# Cleanup
+rm recipes/new_test_recipe.cook
+```
+Expected: grep finds "New Test Recipe" in the response.
+
+- [ ] **Step 7: Kill the server**
+
+```bash
+kill %1 2>/dev/null; wait 2>/dev/null
+```