Add implementation plan for live recipe watching

2026-05-19 22:58:07 -04:00
parent c7bff79371
commit 042bcc9e51
1 changed files with 473 additions and 0 deletions
@@ -0,0 +1,473 @@
 # Live Recipe Watching Implementation Plan
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 **Goal:** Watch the recipe directory for `.cook` file changes and update the in-memory recipe index without a server restart.
 **Architecture:** Replace the `[RecipeInfo]` closure with an `IORef (Map FilePath RecipeInfo)`. A `forkIO`'d watcher thread using `fsnotify` subscribes to filesystem events, debounces within a 200ms window, and updates the `IORef` via `atomicModifyIORef'`. HTTP handlers read from the `IORef` on each request.
 **Tech Stack:** Haskell, `fsnotify-0.4.4.0`, `Data.IORef`, `Data.Map.Strict`, `Data.Time.Clock`, `Control.Concurrent`.
 ---
 ## File Map
 | File | Action | Purpose |
 |---|---|---|
 | `stack.yaml` | Modify | Add `fsnotify` as extra-dep (not in LTS-24.38) |
 | `package.yaml` | Modify | Add `fsnotify` to library dependencies |
 | `roux-server.cabal` | Regenerate | Via `./hs hpack` |
 | `src/Roux/Server.hs` | Modify | State type, watcher thread, handler updates |
 No new files needed — all changes are contained in `Server.hs`.
 ---
 ### Task 1: Add `fsnotify` dependency
 **Files:**
 - Modify: `package.yaml`, `stack.yaml`
 - Regenerate: `roux-server.cabal`
 - [ ] **Step 1: Add fsnotify to stack.yaml extra-deps**
 Since `fsnotify` is not in LTS-24.38, add it as an extra-dep in `stack.yaml`:
 ```yaml
 extra-deps:
  - fsnotify-0.4.4.0
 ```
 - [ ] **Step 2: Add fsnotify to package.yaml library dependencies**
 ```yaml
  - containers
  - directory
  - filepath
  - fsnotify
  - http-types
 ```
 - [ ] **Step 3: Regenerate roux-server.cabal**
 Run: `./hs hpack`
 Expected: no errors.
 - [ ] **Step 4: Verify stack can resolve fsnotify**
 Run: `./hs stack build --no-run-tests 2>&1 | tail -10`
 Expected: builds successfully (or at least resolves deps).
 - [ ] **Step 5: Commit**
 ```bash
 git add package.yaml stack.yaml roux-server.cabal
 git commit -m "chore: add fsnotify dependency for live recipe watching"
 ```
 ---
 ### Task 2: Refactor `Server.hs` for live watching
 **Files:**
 - Modify: `src/Roux/Server.hs`
 This is the main implementation task. All changes are in one file.
 - [ ] **Step 1: Rewrite imports**
 Replace all imports in `src/Roux/Server.hs` with:
 ```haskell
 {-# LANGUAGE OverloadedStrings #-}
 {- | WAI application for the Roux recipe server.
 Handles HTTP requests directly via wai — no framework.
 Recipes are loaded from disk at startup, then watched live via fsnotify.
 -}
 module Roux.Server (
    app,
 ) where
 import Control.Concurrent (forkIO)
 import Control.Concurrent.Chan (Chan, newChan, readChan)
 import Control.Monad (when)
 import Data.ByteString.Lazy (ByteString)
 import Data.Char (toLower)
 import Data.IORef (IORef, atomicModifyIORef', newIORef, readIORef)
 import Data.Map.Strict (Map)
 import qualified Data.Map.Strict as Map
 import Data.Time.Clock (UTCTime, getCurrentTime, diffUTCTime)
 import qualified Network.HTTP.Types as HTTP
 import qualified Network.Wai as Wai
 import System.Directory (doesFileExist)
 import System.FilePath ((</>), takeBaseName)
 import System.FSNotify (Event (..), EventIsDirectory (..), StopListening,
                        WatchConfig (..), WatchMode (..), withManagerConf,
                        watchTreeChan)
 import qualified Roux.Html as Html
 import qualified Roux.RecipeIndex as Idx
 ```
 - [ ] **Step 2: Rewrite `app` to use `IORef` and fork the watcher**
 Replace the entire `app` function body:
 ```haskell
 -- | Build the WAI 'Application' given a path to the recipe directory.
 --
 -- Recipes are scanned at startup and then watched live via fsnotify.
 app :: FilePath -> IO Wai.Application
 app recipeDir = do
    putStrLn $ "[roux] scanning recipes from " <> recipeDir
    recipes <- Idx.scanRecipes recipeDir
    let count = length recipes
        ok = length [() | r <- recipes, isRight (Idx.riRecipe r)]
        errs = count - ok
    putStrLn $ "[roux] found " <> show count <> " recipe(s)"
    when (errs > 0) $
        putStrLn $
            "[roux] " <> show errs <> " had parse errors"
    -- Build initial map keyed by lowercased filename
    let recipeMap = Map.fromList [(map toLower (Idx.riFilename r), r) | r <- recipes]
    state <- newIORef recipeMap
    -- Start the filesystem watcher
    forkIO (watchRecipes recipeDir state)
    pure (handleRequest state)
  where
    isRight (Right _) = True
    isRight (Left _) = False
 ```
 Note: `isRight` was previously a top-level function. It's now local to `app`. That's fine.
 - [ ] **Step 3: Add the `watchRecipes` function**
 Add after `app` (before `handleRequest`):
 ```haskell
 -- | Watch a directory for .cook file changes and update the shared state.
 -- Implements debouncing: events for the same path within 200ms are skipped.
 watchRecipes :: FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
 watchRecipes dir state = do
    putStrLn $ "[roux] watching " <> dir <> " for changes"
    let cfg = defaultConfig{confWatchMode = WatchModeOS}
    withManagerConf cfg $ \mgr -> do
        chan <- newChan
        _stopListening <- watchTreeChan mgr dir (const True) chan
        -- lastSeen tracks when we last processed each path (for debouncing)
        debounceLoop chan state dir Map.empty
  where
    debounceLoop :: Chan Event -> IORef (Map FilePath Idx.RecipeInfo) -> FilePath -> Map FilePath UTCTime -> IO ()
    debounceLoop chan state dir lastSeen = do
        ev <- readChan chan
        -- Only process file events (skip directory events)
        let isDir = eventIsDirectory ev
        now <- getCurrentTime
        if isDir == IsDirectory
            then debounceLoop chan state dir lastSeen
            else do
                let path = eventPath ev
                let key = map toLower path
                let lastTime = Map.lookup key lastSeen
                case lastTime of
                    Just t | diffUTCTime now t < 0.2 -> do
                        -- Debounced: skip, but pass through lastSeen
                        debounceLoop chan state dir lastSeen
                    _ -> do
                        -- Process the event
                        reReadRecipe dir path state
                        debounceLoop chan state dir (Map.insert key now lastSeen)
 ```
 Key points:
 - Uses `defaultConfig` with `WatchModeOS` (inotify on Linux)
 - `watchTreeChan` takes an `ActionPredicate` (`const True` means watch everything)
 - `readChan` blocks until events arrive
 - Debouncing: if the same path was processed within 0.2 seconds, skip
 - Directory events are filtered out
 - [ ] **Step 4: Add the `reReadRecipe` helper**
 Add after `watchRecipes`:
 ```haskell
 -- | Re-read a single recipe file and update the shared state.
 -- On read/parse errors, logs the error and keeps the previous version.
 reReadRecipe :: FilePath -> FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
 reReadRecipe dir path state = do
    let key = map toLower path
    let fullPath = dir </> path
    exists <- doesFileExist fullPath
    if not exists
        then do
            atomicModifyIORef' state $ \m ->
                (Map.delete key m, ())
            putStrLn $ "[roux] removed " <> path
        else do
            content <- readFile fullPath
            let parsed = parseCookFile (T.pack content)
                title = case parsed of
                    Right r -> extractTitle r path
                    Left _ -> T.pack (takeBaseName path)
            case parsed of
                Left err -> do
                    putStrLn $ "[roux] parse error in " <> path <> ": " <> err
                    putStrLn $ "[roux] keeping previous version"
                Right _ -> do
                    let info =
                            Idx.RecipeInfo
                                { Idx.riFilename = path
                                , Idx.riTitle = title
                                , Idx.riRecipe = parsed
                                }
                    atomicModifyIORef' state $ \m ->
                        (Map.insert key info m, ())
                    putStrLn $ "[roux] updated " <> path
  where
    extractTitle recipe filename =
        case metaTitle (recipeMetadata recipe) of
            Just t -> t
            Nothing -> T.pack (takeBaseName filename)
    metaTitle (Data.CookLang.Metadata m) = m ... -- WRONG, need to check Data.CookLang
 ```
 **Wait — the `extractTitle` function already exists in `Roux.RecipeIndex`**. Let me check what's exported.
 `Idx.extractTitle` is a module-internal function (not exported from `Roux.RecipeIndex`). The `Idx.loadRecipe` function does the same logic internally.
 I have two options:
 1. Export `extractTitle` from `Roux.RecipeIndex`
 2. Import `Data.CookLang` and extract title inline
 Option 1 is cleaner. I'll add `extractTitle` to the export list of `RecipeIndex.hs`.
 Actually wait — looking at the `Roux.RecipeIndex` module:
 ```haskell
 module Roux.RecipeIndex (
    RecipeInfo (..),
    RecipeSearchEntry (..),
    scanRecipes,
    toSearchEntry,
 ) where
 ```
 `extractTitle` is defined but not exported. I have a few choices:
 Option A: Add `extractTitle` to the export list.
 Option B: Use the existing `loadRecipe` pattern but that requires access to the raw recipe's metadata via `Data.CookLang`.
 Option C: Write a local helper in Server.hs.
 The simplest is Option A or creating a small helper in `Roux.RecipeIndex` that can load a single file (like `loadRecipe`).
 Actually, the cleanest approach: export `loadRecipe` from `RecipeIndex`. It's already defined there. Then I can call `Idx.reloadRecipe` or reuse `loadRecipe` directly.
 Looking at `loadRecipe`:
 ```haskell
 loadRecipe :: FilePath -> FilePath -> IO RecipeInfo
 loadRecipe dir filename = do
    content <- readFile (dir </> filename)
    let parsed = parseCookFile (T.pack content)
        title = case parsed of
            Right r -> extractTitle r filename
            Left _err -> T.pack (takeBaseName filename)
    return RecipeInfo{ riFilename = filename, riTitle = title, riRecipe = parsed }
 ```
 Perfect. I can either:
 - Export `loadRecipe` from `RecipeIndex` and call it from `Server.hs`
 - Or keep the local `reReadRecipe` with its own inline logic
 Since loadRecipe is already tested and working, exporting it is cleaner. But actually, the plan says to keep changes minimal. Let me just add `loadRecipe` to the exports of `RecipeIndex.hs` and use it in `Server.hs`.
 Let me update the Step 4 code:
 ```haskell
 reReadRecipe :: FilePath -> FilePath -> IORef (Map FilePath Idx.RecipeInfo) -> IO ()
 reReadRecipe dir path state = do
    let fullPath = dir </> path
    exists <- doesFileExist fullPath
    if not exists
        then do
            let key = map toLower path
            atomicModifyIORef' state $ \m ->
                (Map.delete key m, ())
            putStrLn $ "[roux] removed " <> path
        else do
            -- Use Idx.loadRecipe to read and parse
            info <- Idx.loadRecipe dir path
            let key = map toLower path
            case Idx.riRecipe info of
                Left err -> do
                    putStrLn $ "[roux] parse error in " <> path <> ": " <> err
                    putStrLn $ "[roux] keeping previous version"
                Right _ -> do
                    atomicModifyIORef' state $ \m ->
                        (Map.insert key info m, ())
                    putStrLn $ "[roux] updated " <> path
 ```
 This is much cleaner. I need to export `loadRecipe` from `RecipeIndex.hs`.
 - [ ] **Step 5: Update `handleRequest` to accept `IORef`**
 Replace `handleRequest`:
 ```haskell
 -- | Route an incoming request to the appropriate handler.
 handleRequest :: IORef (Map FilePath Idx.RecipeInfo) -> Wai.Application
 handleRequest state request respond = do
    recipes <- readIORef state
    let recipeList = Map.elems recipes
    case Wai.pathInfo request of
        -- Index page (default: alphabetical)
        [] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
        -- Index page (sorted by tags) — sort mode ignored, driven client-side
        ["sorted", "tags"] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
        -- Index page (sorted by course/category) — sort mode ignored, driven client-side
        ["sorted", "course"] -> respond (htmlResponse (Html.indexPage Html.AlphaSort recipeList))
        -- Recipe detail page
        ["recipes", rawPath] ->
            case lookupRecipe (urlDecodePath rawPath) recipes of
                Just info -> respond (htmlResponse (Html.recipePage info))
                Nothing -> respond notFound
        _ -> respond notFound
 ```
 `handleRequest` is now an `IO` action — it reads from the `IORef` on each request. `Wai.Application` is `Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived`, so returning `IO` is fine.
 - [ ] **Step 6: Simplify `lookupRecipe` to use `Map.lookup`**
 Replace `lookupRecipe`:
 ```haskell
 -- | Look up a recipe by filename (case-insensitive, .cook extension optional).
 lookupRecipe :: FilePath -> Map FilePath Idx.RecipeInfo -> Maybe Idx.RecipeInfo
 lookupRecipe target recipes =
    Map.lookup (map toLower (stripCookExt target)) recipes
 ```
 - [ ] **Step 7: Remove `stripCookExt` and `urlDecodePath` helpers**
 These are unchanged. Keep them.
 - [ ] **Step 8: Remove unused imports**
 Remove from imports:
 - `Data.List (find)` — no longer used
 Keep:
 - `Data.Char (toLower)` — still used in `lookupRecipe` and `reReadRecipe`
 - [ ] **Step 9: Export `loadRecipe` from `RecipeIndex.hs`**
 In `src/Roux/RecipeIndex.hs`, add `loadRecipe` to the export list:
 ```haskell
 module Roux.RecipeIndex (
    RecipeInfo (..),
    RecipeSearchEntry (..),
    loadRecipe,
    scanRecipes,
    toSearchEntry,
 ) where
 ```
 - [ ] **Step 10: Build to verify compilation**
 Run: `./hs stack build`
 Expected: compiles without errors.
 - [ ] **Step 11: Run tests to verify nothing broke**
 Run: `./hs stack test`
 Expected: all tests pass.
 - [ ] **Step 12: Commit**
 ```bash
 git add src/Roux/Server.hs src/Roux/RecipeIndex.hs
 git commit -m "feat: watch recipe directory for live updates via fsnotify"
 ```
 ---
 ### Task 3: Manual smoke test
 **Files:** (none, manual testing)
 - [ ] **Step 1: Start the server in the background**
 ```bash
 ./hs stack exec roux-server -- --recipe-dir recipes &
 sleep 2
 ```
 - [ ] **Step 2: Verify a recipe loads**
 ```bash
 curl -s http://localhost:8080/ | grep -i "recipe" | head -5
 ```
 Expected: index page HTML with recipe titles.
 - [ ] **Step 3: Check server logs for watching confirmation**
 Should see: `[roux] watching recipes for changes`
 - [ ] **Step 4: Modify a recipe file and verify the change is picked up**
 ```bash
 # Pick a recipe, temporarily replace content
 RECIPE=$(ls recipes/*.cook | head -1)
 RECIPE_NAME=$(basename "$RECIPE" .cook)
 echo "---" > /tmp/test_change.cook
 echo "title: Test Live Update" >> /tmp/test_change.cook
 echo "---" >> /tmp/test_change.cook
 echo "Testing live file watching." >> /tmp/test_change.cook
 cp "$RECIPE" /tmp/backup.cook
 cp /tmp/test_change.cook "$RECIPE"
 sleep 1
 curl -s "http://localhost:8080/recipes/$RECIPE_NAME" | grep "Test Live Update"
 # Restore original
 cp /tmp/backup.cook "$RECIPE"
 ```
 Expected: grep finds "Test Live Update" in the response.
 - [ ] **Step 5: Delete a recipe file and verify 404**
 ```bash
 RECIPE2=$(ls recipes/*.cook | grep -v "$(basename "$RECIPE")" | head -1)
 RECIPE2_NAME=$(basename "$RECIPE2" .cook)
 cp "$RECIPE2" /tmp/backup2.cook
 rm "$RECIPE2"
 sleep 1
 curl -s -o /dev/null -w "%{http_code}" "http://localhost:8080/recipes/$RECIPE2_NAME"
 # Restore
 cp /tmp/backup2.cook "$RECIPE2"
 ```
 Expected: HTTP 404.
 - [ ] **Step 6: Add a new recipe file and verify it appears**
 ```bash
 echo "---" > /tmp/new_recipe.cook
 echo "title: New Test Recipe" >> /tmp/new_recipe.cook
 echo "---" >> /tmp/new_recipe.cook
 echo "A brand new recipe." >> /tmp/new_recipe.cook
 cp /tmp/new_recipe.cook recipes/new_test_recipe.cook
 sleep 1
 curl -s "http://localhost:8080/recipes/new_test_recipe" | grep "New Test Recipe"
 # Cleanup
 rm recipes/new_test_recipe.cook
 ```
 Expected: grep finds "New Test Recipe" in the response.
 - [ ] **Step 7: Kill the server**
 ```bash
 kill %1 2>/dev/null; wait 2>/dev/null
 ```