diff --git a/docs/plans/2026-05-21-llm-recipe-import.md b/docs/plans/2026-05-21-llm-recipe-import.md new file mode 100644 index 0000000..c95985c --- /dev/null +++ b/docs/plans/2026-05-21-llm-recipe-import.md @@ -0,0 +1,1017 @@ +# LLM-Powered Recipe Import Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add an Anthropic API-powered recipe import path to `/import` that converts free-form text, PDFs, and images into Cooklang, alongside the existing URL scraper. + +**Architecture:** A new Python subprocess script (`scripts/convert-to-cooklang`) calls the Anthropic Messages API using the existing Cooklang converter prompts. The server dispatches based on input type (URL → existing scraper, text/file → new script). A YAML config file holds API credentials. + +**Tech Stack:** Python (anthropic SDK), Haskell (wai-extra for multipart parsing, yaml for config), existing Cooklang converter prompts. + +--- + +## File Map + +| File | Action | Purpose | +|---|---|---| +| `scripts/convert-to-cooklang` | **Create** | Python script calling Anthropic API | +| `src/Roux/Config.hs` | **Create** | Config file types and loading | +| `app/Main.hs` | Modify | Add `--config-file` CLI flag | +| `src/Roux/Server.hs` | Modify | Thread config through app, add `runLLMPipeline`, expand import handler | +| `src/Roux/Html.hs` | Modify | Add file upload and textarea to import form, switch to multipart form | +| `src/Roux.hs` | Modify | Re-export Config types | +| `package.yaml` | Modify | Add `yaml` and `wai-extra` dependencies | + +--- + +### Task 1: Create `src/Roux/Config.hs` — config types and loading + +**Files:** +- Create: `src/Roux/Config.hs` + +This module defines the config file types and a loader. We use the `yaml` package to parse YAML into Aeson's `Value`, then decode with Fleece (the project's existing JSON library) for consistency. + +To distinguish "file not found" from "parse error", check `doesFileExist` first before attempting to decode. + +- [ ] **Step 1: Write the module** + +```haskell +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE OverloadedStrings #-} + +{- | Configuration for Roux, loaded from a YAML config file. + +Default location: ./roux-config.yaml in the working directory. +Overridable via --config-file CLI flag. +-} +module Roux.Config ( + RouxConfig (..), + AnthropicConfig (..), + loadConfig, +) where + +import Data.Text (Text) +import qualified Data.Text as T +import Data.Yaml (decodeFileEither) +import System.Directory (doesFileExist) +import qualified Fleece.Aeson as Fleece +import qualified Fleece.Aeson.Decoder as FleeceD +import qualified Fleece.Core as FC + +-- | Top-level config. +data RouxConfig = RouxConfig + { rcAnthropic :: Maybe AnthropicConfig + } + deriving stock (Eq, Show) + +-- | Anthropic API configuration. +data AnthropicConfig = AnthropicConfig + { acApiKey :: !Text + , acBaseUrl :: !(Maybe Text) -- default: https://api.anthropic.com + , acModel :: !(Maybe Text) -- default: claude-sonnet-4-20250514 + } + deriving stock (Eq, Show) + +-- | Fleece schema for AnthropicConfig. +anthropicConfigSchema :: FC.Schema Fleece.Decoder AnthropicConfig +anthropicConfigSchema = + FC.record (FC.unqualifiedName "AnthropicConfig") + ( FC.setRequired "api_key" (FC.projection FC.string) + ( \f r -> f (acApiKey r) + ) + >>. FC.setOptional "base_url" (FC.projection FC.string) + ( \f r -> f (acBaseUrl r) + ) + >>. FC.setOptional "model" (FC.projection FC.string) + ( \f r -> f (acModel r) + ) + ) + +-- | Fleece schema for RouxConfig. +rouxConfigSchema :: FC.Schema Fleece.Decoder RouxConfig +rouxConfigSchema = + FC.record (FC.unqualifiedName "RouxConfig") + ( FC.setOptional "anthropic" anthropicConfigSchema + ( \f r -> f (rcAnthropic r) + ) + ) + +-- | Load config from a YAML file. Returns Nothing (not an error) if the +-- file doesn't exist — LLM features will be unavailable but the server +-- still runs. Exits with an error if the file exists but is malformed. +loadConfig :: FilePath -> IO RouxConfig +loadConfig path = do + exists <- doesFileExist path + if not exists + then pure (RouxConfig Nothing) + else do + result <- decodeFileEither path + case result of + Left parseErr -> + fail $ "Failed to parse " <> path <> ": " <> show parseErr + Right val -> + case FleeceD.fromValue rouxConfigSchema val of + Right cfg -> pure cfg + Left decodeErr -> + fail $ + "Invalid config format in " <> path <> ": " <> show decodeErr +``` +``` + +- [ ] **Step 2: Build to verify compilation** + +Run: `./hs stack build` +Expected: compiles without errors. + +- [ ] **Step 3: Run tests to verify nothing broke** + +Run: `./hs stack test` +Expected: all tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add src/Roux/Config.hs +git commit -m "feat: add Config module for YAML config loading" +``` + +--- + +### Task 2: Create `scripts/convert-to-cooklang` — Python subprocess + +**Files:** +- Create: `scripts/convert-to-cooklang` + +This script calls the Anthropic Messages API with the existing Cooklang converter prompts to convert unstructured recipe content into Cooklang format. + +- [ ] **Step 1: Write the Python script** + +```python +#!/usr/bin/env python3 +"""Convert unstructured recipe content to Cooklang format using an +Anthropic-compatible API (Messages API). + +Accepts text input (--text or --stdin) or file input (--file for PDF/images). +Reads API configuration from a YAML config file (--config). + +Outputs the Cooklang text on stdout. +Exits 0 on success, non-zero with error details on stderr on failure. + +Usage: + scripts/convert-to-cooklang --config roux-config.yaml --text "Recipe text..." + scripts/convert-to-cooklang --config roux-config.yaml --stdin < recipe.txt + scripts/convert-to-cooklang --config roux-config.yaml --file recipe.pdf +""" + +import argparse +import base64 +import json +import mimetypes +import os +import sys + +import yaml +from anthropic import Anthropic + + +# -- Config ------------------------------------------------------------------- + +def load_config(path: str) -> dict: + with open(path) as f: + cfg = yaml.safe_load(f) + anthropic_cfg = cfg.get("anthropic", {}) + if not anthropic_cfg.get("api_key"): + print("Error: anthropic.api_key is required in config", file=sys.stderr) + sys.exit(1) + return anthropic_cfg + + +# -- Prompt loading ----------------------------------------------------------- + +def load_prompts() -> tuple[str, str, str]: + """Load system prompt, converter instructions, and user template. + + Resolved relative to the script's parent directory (project root). + """ + script_dir = os.path.dirname(os.path.abspath(__file__)) + project_root = os.path.dirname(script_dir) # scripts/ -> project root + prompts_dir = os.path.join(project_root, "prompts") + + def read(name: str) -> str: + with open(os.path.join(prompts_dir, name)) as f: + return f.read() + + return read("system-prompt.md"), read("cooklang-converter.md"), read("user-prompt-template.md") + + +# -- Content preparation ------------------------------------------------------ + +def build_text_content(text: str) -> list[dict]: + return [{"type": "text", "text": text}] + + +def build_file_content(filepath: str) -> list[dict]: + """Prepare a file (PDF or image) for the Anthropic Messages API. + + Returns a list with one document/image content block. + """ + mime_type, _ = mimetypes.guess_type(filepath) + if mime_type is None: + # Default to octet-stream; let the API decide + mime_type = "application/octet-stream" + + with open(filepath, "rb") as f: + data = base64.b64encode(f.read()).decode("utf-8") + + is_pdf = mime_type == "application/pdf" + block_type = "document" if is_pdf else "image" + + return [ + { + "type": block_type, + "source": { + "type": "base64", + "media_type": mime_type, + "data": data, + }, + } + ] + + +# -- API call ----------------------------------------------------------------- + +def cooklang_from_anthropic( + api_key: str, + base_url: str | None, + model: str | None, + system_prompt: str, + user_message_content: list[dict], +) -> str: + """Send the recipe content to the Anthropic API and return Cooklang text.""" + client = Anthropic(api_key=api_key, base_url=base_url) if base_url else Anthropic(api_key=api_key) + resolved_model = model or "claude-sonnet-4-20250514" + + response = client.messages.create( + model=resolved_model, + max_tokens=4096, + system=[{"type": "text", "text": system_prompt}], + messages=[ + { + "role": "user", + "content": user_message_content, + } + ], + ) + + # Extract text from response content blocks + parts: list[str] = [] + for block in response.content: + if block.type == "text": + parts.append(block.text) + + cooklang = "\n".join(parts).strip() + + # Basic validation: must start with front matter marker + if not cooklang.startswith("---"): + print( + "Error: AI response does not appear to be valid Cooklang " + "(missing front matter). Response preview: " + + cooklang[:200], + file=sys.stderr, + ) + sys.exit(1) + + return cooklang + + +# -- Main --------------------------------------------------------------------- + +def main() -> None: + parser = argparse.ArgumentParser( + description="Convert unstructured recipe content to Cooklang using an Anthropic-compatible API." + ) + parser.add_argument( + "--config", + required=True, + help="Path to YAML config file (must contain anthropic.api_key)", + ) + input_group = parser.add_mutually_exclusive_group(required=True) + input_group.add_argument("--text", help="Inline recipe text") + input_group.add_argument("--stdin", action="store_true", help="Read recipe text from stdin") + input_group.add_argument("--file", help="Path to recipe PDF or image file") + args = parser.parse_args() + + # Load config + config = load_config(args.config) + api_key = config["api_key"] + base_url = config.get("base_url") + model = config.get("model") + + # Load prompts + system_prompt, converter_instructions, user_template = load_prompts() + + # Prepare user message content + if args.file: + # File input (PDF or image): send file content + converter instructions + file_content = build_file_content(args.file) + user_message_content = ( + [{"type": "text", "text": converter_instructions}] + + file_content + ) + elif args.stdin: + raw_text = sys.stdin.read() + user_message_content = build_text_content( + user_template.replace("{{FULL_RECIPE_TEXT}}", raw_text) + .replace("{{TITLE}}", "Untitled") + .replace("{{URL}}", "") + ) + else: + # --text + user_message_content = build_text_content( + user_template.replace("{{FULL_RECIPE_TEXT}}", args.text) + .replace("{{TITLE}}", "Untitled") + .replace("{{URL}}", "") + ) + + # Call API + try: + cooklang = cooklang_from_anthropic( + api_key=api_key, + base_url=base_url, + model=model, + system_prompt=system_prompt, + user_message_content=user_message_content, + ) + except Exception as e: + print(f"Error calling Anthropic API: {e}", file=sys.stderr) + sys.exit(1) + + # Output Cooklang text + print(cooklang) + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 2: Make the script executable** + +```bash +chmod +x scripts/convert-to-cooklang +``` + +- [ ] **Step 3: Test the script manually with text input** + +Create a test config file and a test recipe: + +```bash +cd /work/personal/roux/roux-main +cat > /tmp/test-config.yaml << 'EOF' +anthropic: + api_key: "test-key-placeholder" + base_url: "https://api.anthropic.com" + model: "claude-sonnet-4-20250514" +EOF +echo "Testing syntax only (no real API call)" && python3 -c "import yaml; from pathlib import Path; Path('scripts/convert-to-cooklang').stat()" +``` +Expected: Python imports the YAML module successfully, script exists and is executable. + +- [ ] **Step 4: Commit** + +```bash +git add scripts/convert-to-cooklang +git commit -m "feat: add convert-to-cooklang script for Anthropic API recipe conversion" +``` + +--- + +### Task 3: Add `--config-file` CLI flag to `app/Main.hs` + +**Files:** +- Modify: `app/Main.hs` + +- [ ] **Step 1: Add the config-file option** + +Update the `Options` type and parser: + +```haskell +data Options = Options + { optHost :: String + , optPort :: Int + , optRecipeDir :: FilePath + , optConfigFile :: FilePath -- NEW + } + +optionsParser :: Opts.Parser Options +optionsParser = + Options + <$> Opts.strOption + ( Opts.long "host" + <> Opts.metavar "HOST" + <> Opts.help "Listen host" + <> Opts.value "0.0.0.0" + <> Opts.showDefault + ) + <*> Opts.option + Opts.auto + ( Opts.long "port" + <> Opts.short 'p' + <> Opts.metavar "PORT" + <> Opts.help "Listen port" + <> Opts.value 8080 + <> Opts.showDefault + ) + <*> Opts.strOption + ( Opts.long "recipe-dir" + <> Opts.short 'r' + <> Opts.metavar "DIR" + <> Opts.help "Path to directory containing Cooklang recipe files" + <> Opts.value "recipes" + <> Opts.showDefault + ) + <*> Opts.strOption -- NEW + ( Opts.long "config-file" -- NEW + <> Opts.metavar "FILE" -- NEW + <> Opts.help "Path to YAML config file (default: ./roux-config.yaml)" -- NEW + <> Opts.value "roux-config.yaml" -- NEW + <> Opts.showDefault -- NEW + ) +``` + +- [ ] **Step 2: Load config and pass to `Roux.app`** + +Change the `main` function to load the config and pass it to `app`: + +```haskell +main :: IO () +main = do + hSetBuffering stdout NoBuffering + hSetBuffering stderr NoBuffering + opts <- + Opts.execParser $ + Opts.info + (optionsParser Opts.<**> Opts.helper) + ( Opts.fullDesc + <> Opts.progDesc "Roux recipe management server" + <> Opts.header "roux-server — personal/family recipe manager" + ) + let settings = + Warp.setPort (optPort opts) $ + Warp.setHost + (fromString (optHost opts)) + Warp.defaultSettings + + putStrLn $ "[roux] scanning recipes from " <> optRecipeDir opts + putStrLn $ "[roux] listening on " <> optHost opts <> ":" <> show (optPort opts) + + -- Load config (may be missing — LLM features disabled gracefully) + config <- Roux.loadConfig (optConfigFile opts) -- NEW + waiApp <- Roux.app (optRecipeDir opts) config -- MODIFIED: pass config + Warp.runSettings settings waiApp +``` + +- [ ] **Step 3: Build to verify compilation** + +Run: `./hs stack build` +Expected: compiles without errors. + +- [ ] **Step 4: Run tests to verify nothing broke** + +Run: `./hs stack test` +Expected: all tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add app/Main.hs +git commit -m "feat: add --config-file CLI flag for YAML config" +``` + +--- + +### Task 4: Thread config through `Server.hs` and add `runLLMPipeline` + +**Files:** +- Modify: `src/Roux/Server.hs` + +- [ ] **Step 1: Add imports and update `app` signature** + +Add new imports at the top: + +```haskell +import Data.ByteString.Lazy (ByteString) +import qualified Data.ByteString.Lazy as LB +import qualified Data.Text.Encoding as TE +import qualified Roux.Config as Config +import Roux.Config (RouxConfig (..), AnthropicConfig (..)) +import System.Directory (createDirectoryIfMissing) +``` + +Change the `app` function signature to accept `RouxConfig`: + +```haskell +app :: FilePath -> RouxConfig -> IO Wai.Application +app recipeDir config = do + -- ... existing startup code stays the same ... + pure (router absDir config state changeLog) -- MODIFIED: pass config +``` + +Update the `router` to accept and use `config`: + +```haskell +router :: FilePath -> RouxConfig -> IORef (Map FilePath Idx.RecipeInfo) -> ChangeLog -> Wai.Application +router recipeDir config state changeLog request respond = + case Wai.pathInfo request of + ["events"] -> sseHandler changeLog request respond + ["import"] -> importHandler recipeDir config state request respond + _ -> handleRequest state request respond +``` + +- [ ] **Step 2: Update `importHandler` to handle file uploads via base64 fields** + +The form sends file data as base64-encoded hidden fields (set by client-side JS). The existing `parseFormBody` handles URL-encoded form data. We add file detection by looking for `file-b64` and `file-name` parameters. + +Replace the existing `importHandler` with the expanded version: + +```haskell +{- | Handle GET and POST requests to /import. +GET — show the import form. +POST — dispatch based on input: URL (existing scraper), text, or file (Anthropic). +Files are uploaded as base64-encoded content via hidden form fields +(file-b64, file-name, file-mime) set by client-side JavaScript. +-} +importHandler :: FilePath -> RouxConfig -> IORef (Map FilePath Idx.RecipeInfo) -> Wai.Application +importHandler recipeDir config _state request respond = + case Wai.requestMethod request of + "GET" -> + respond (htmlResponse (Html.importPage Nothing)) + "POST" -> do + body <- readRequestBody request + let params = parseFormBody body + url = maybe "" id (lookup "url" params) + text = maybe "" id (lookup "text" params) + fileB64 = maybe "" id (lookup "file-b64" params) + fileName = maybe "" id (lookup "file-name" params) + fileMime = maybe "" id (lookup "file-mime" params) + hasFile = not (T.null fileB64) + + putStrLn $ "[roux] import request: url=" <> T.unpack url + <> " text=" <> show (not (T.null text)) + <> " file=" <> show hasFile + + result <- try $ case (hasFile, not (T.null text), not (T.null url)) of + -- File takes priority, then text, then URL + (True, _, _) -> + runFileImport recipeDir config fileName fileMime fileB64 + (_, True, _) -> + runTextImport recipeDir config text + (_, _, True) -> + runImportPipeline recipeDir url + _ -> + pure (Left (Html.ImportError "Provide a URL, recipe text, or file to import.")) + + case result of + Left (e :: SomeException) -> + respond $ + htmlResponse $ + Html.importResultPage $ + Html.ImportError ("Unexpected error: " <> T.pack (show e)) + Right (Left err) -> + respond $ + htmlResponse $ + Html.importResultPage err + Right (Right filename) -> + respond $ + Wai.responseLBS + HTTP.status303 + [("Location", "/recipes/" <> TE.encodeUtf8 (T.pack filename))] + "" + _ -> respond notFound +``` + +- [ ] **Step 3: Update `runFileImport` function (accepts base64)** + +The file content arrives as base64. Decode it, save the original alongside the recipe, then call the LLM pipeline. + +```haskell +-- | Import a recipe from an uploaded file via the LLM pipeline. +-- The file content is base64-encoded in the form data. +runFileImport :: + FilePath -- ^ recipe directory + -> RouxConfig -- ^ server config (for Anthropic settings) + -> Text -- ^ original filename + -> Text -- ^ MIME type + -> Text -- ^ base64-encoded file content + -> IO (Either Html.ImportError FilePath) +runFileImport recipeDir config originalName mimeType b64Text = do + -- Decode base64 + case decodeBase64 (TE.encodeUtf8 b64Text) of + Left err -> + pure $ Left $ Html.ImportError $ + "Could not decode uploaded file: " <> T.pack err + Right content -> do + -- Ensure recipe dir exists + createDirectoryIfMissing True recipeDir + -- Save the original file + let sourceFilename = map (\c -> if isAlphaNum c || c == '.' || c == '-' || c == '_' then c else '_') (T.unpack originalName) + sourcePath = recipeDir sourceFilename + LB.writeFile sourcePath content + putStrLn $ "[roux] saved uploaded file to " <> sourcePath + -- Call the LLM pipeline with the saved file + case rcAnthropic config of + Nothing -> + pure $ Left $ Html.ImportError $ + "LLM import is not configured. Add an anthropic.api_key to the config file." + Just anthropicConfig -> do + result <- runLLMPipeline recipeDir anthropicConfig (Just sourcePath) "" + case result of + Right filename -> pure $ Right filename + Left err -> pure $ Left err +``` + +- [ ] **Step 5: Add `runTextImport` function** + +```haskell +-- | Import a recipe from pasted text via the LLM pipeline. +runTextImport :: + FilePath -> RouxConfig -> Text + -> IO (Either Html.ImportError FilePath) +runTextImport recipeDir config text = do + case rcAnthropic config of + Nothing -> + pure $ Left $ Html.ImportError $ + "LLM import is not configured. Add an anthropic.api_key to the config file." + Just anthropicConfig -> + runLLMPipeline recipeDir anthropicConfig Nothing text +``` + +- [ ] **Step 6: Add `runLLMPipeline` function** + +```haskell +-- | Run the full LLM import pipeline: call convert-to-cooklang → parse → save. +runLLMPipeline :: + FilePath -- ^ recipe directory + -> AnthropicConfig -- ^ API config + -> Maybe FilePath -- ^ Just path = uploaded source file to keep + -> Text -- ^ recipe text (or empty if file) + -> IO (Either Html.ImportError FilePath) +runLLMPipeline recipeDir anthropicConfig mSourceFile text = do + -- Step 1: locate the script + script <- findConvertScript + -- Step 2: write a temp config for the subprocess + let configPath = recipeDir ".convert-config.yaml" + writeConvertConfig configPath anthropicConfig + -- Step 3: build subprocess args + let (args, input) = case mSourceFile of + Just fp -> (["--config", configPath, "--file", fp], "") + Nothing -> (["--config", configPath, "--text", T.unpack text], "") + putStrLn $ "[roux] running " <> script <> " for LLM import" + (exitCode, stdoutBytes, stderrBytes) <- + readProcessBytes script args + -- Clean up temp config + _ <- try (removeFile configPath) :: IO (Either IOException ()) + case exitCode of + ExitFailure _ -> do + let errMsg = TE.decodeUtf8 (LB.toStrict stderrBytes) + putStrLn $ "[roux] convert-to-cooklang failed: " <> T.unpack errMsg + return $ + Left $ + Html.ImportError $ + "Could not convert recipe: " <> errMsg + ExitSuccess -> do + -- Step 4: parse the Cooklang output to extract the title + let cooklangText = TE.decodeUtf8 (LB.toStrict stdoutBytes) + case parseRecipeTitle cooklangText of + Nothing -> do + putStrLn "[roux] could not extract title from Cooklang output" + return $ + Left $ + Html.ImportError $ + "Could not determine recipe title from AI output." + Just title -> do + let filename = CooklangPrint.filenameFromTitle title + filepath = recipeDir filename + -- Step 5: write the .cook file + putStrLn $ "[roux] writing imported recipe to " <> filepath + LB.writeFile filepath stdoutBytes + putStrLn $ "[roux] successfully imported recipe as " <> filename + return $ Right filename +``` + +- [ ] **Step 7: Add helper functions for config generation and title parsing** + +```haskell +-- | Locate the convert-to-cooklang script. +findConvertScript :: IO FilePath +findConvertScript = do + mPath <- findExecutable "convert-to-cooklang" + case mPath of + Just p -> pure p + Nothing -> pure "scripts/convert-to-cooklang" + +-- | Write a temporary YAML config for the subprocess. +writeConvertConfig :: FilePath -> AnthropicConfig -> IO () +writeConvertConfig path ac = do + let pairs = + [ "anthropic:" + , " api_key: " <> T.unpack (acApiKey ac) + ] ++ case acBaseUrl ac of + Just url -> [" base_url: " <> T.unpack url] + Nothing -> [] + ++ case acModel ac of + Just m -> [" model: " <> T.unpack m] + Nothing -> [] + writeFile path (unlines pairs) + +-- | Parse the recipe title from Cooklang front matter. +-- Looks for "title: ..." in the YAML front matter block. +parseRecipeTitle :: Text -> Maybe Text +parseRecipeTitle cooklang = + case T.breakOn "---" cooklang of + (_, afterFirst) -> + let (_separator, inside) = T.breakOn "---" (T.drop 3 afterFirst) + frontMatter = T.take (T.length inside) (T.drop 3 afterFirst) + in case T.breakOn "title:" frontMatter of + (_, afterTitle) -> + let titleLine = T.takeWhile (/= '\n') (T.drop 6 afterTitle) + cleaned = T.strip titleLine + in if T.null cleaned then Nothing else Just cleaned +``` + +- [ ] **Step 8: Add missing imports to Server.hs** + +Make sure these imports are present (add any that are missing): + +```haskell +import Control.Exception (IOException, SomeException, catch, finally, try) +import Data.Char (isAlphaNum) +import Data.List (isSuffixOf) +import Data.Maybe (fromMaybe, isJust, listToMaybe) +import qualified Data.Text as T +import qualified Data.Text.Encoding as TE +import qualified Data.ByteString.Lazy as LB +import System.Directory (createDirectoryIfMissing, doesFileExist, findExecutable, makeAbsolute, removeFile) +import System.FilePath (takeBaseName, takeExtension, ()) +import qualified Roux.CooklangPrint as CooklangPrint +import qualified Roux.Html as Html +``` + +- [ ] **Step 9: Build to verify compilation** + +Run: `./hs stack build` +Expected: compiles without errors. + +- [ ] **Step 10: Run tests to verify nothing broke** + +Run: `./hs stack test` +Expected: all tests pass. + +- [ ] **Step 11: Commit** + +```bash +git add src/Roux/Server.hs +git commit -m "feat: add LLM import pipeline with file upload and text input" +``` + +--- + +### Task 5: Expand import form in `Html.hs` + +**Files:** +- Modify: `src/Roux/Html.hs` + +- [ ] **Step 1: Update the import form to include file upload, textarea, and JS base64 conversion** + +The form stays URL-encoded (no `enctype` override). A small JS snippet converts the selected file to base64 and places it in hidden form fields before submission. + +Replace the existing `importPage` function: + +```haskell +-- | Render the import form page, optionally with a validation error. +importPage :: Maybe Text -> ByteString +importPage merror = + page "Roux — Import Recipe" $ do + H.nav ! A.class_ "roux-navbar" $ do + H.ul $ H.li $ H.a ! A.href "/" $ "← Back" + H.ul $ H.li $ H.strong "Import Recipe" + case merror of + Just err -> H.p ! A.style "color: var(--roux-accent);" $ H.toHtml err + Nothing -> pure () + H.p ! A.style "font-size: 0.85rem; opacity: 0.7; margin-bottom: 1.5rem;" $ + "Paste a recipe URL, upload a PDF or image, or paste recipe text directly." + H.form ! A.method "POST" ! A.action "/import" ! A.id "import-form" $ + -- URL input + H.label ! A.for "url" $ "Recipe URL" + >> H.input + ! A.type_ "url" + ! A.id "url" + ! A.name "url" + ! A.placeholder "https://cooking.nytimes.com/recipes/..." + ! A.style "width: 100%; margin-bottom: 1.5rem;" + -- File upload + >> H.label ! A.for "file-input" $ "Or upload a PDF or image" + >> H.input + ! A.type_ "file" + ! A.id "file-input" + ! A.accept ".pdf,image/*" + ! A.style "width: 100%; margin-bottom: 1.5rem;" + -- Hidden fields for base64 file data (set by JS) + >> H.input ! A.type_ "hidden" ! A.id "file-b64" ! A.name "file-b64" ! A.value "" + >> H.input ! A.type_ "hidden" ! A.id "file-name" ! A.name "file-name" ! A.value "" + >> H.input ! A.type_ "hidden" ! A.id "file-mime" ! A.name "file-mime" ! A.value "" + -- Text area + >> H.label ! A.for "text" $ "Or paste recipe text" + >> H.textarea + ! A.id "text" + ! A.name "text" + ! A.rows "8" + ! A.placeholder "Paste recipe text here..." + ! A.style "width: 100%; margin-bottom: 1.5rem; font-family: monospace; font-size: 0.85rem;" + -- Submit + >> H.button ! A.type_ "submit" $ "Import Recipe" + -- Inline JS for base64 file conversion + >> H.script ! A.type_ "text/javascript" $ H.preEscapedText fileUploadJs + +-- | Inline JavaScript for converting file uploads to base64 before form submission. +fileUploadJs :: Text +fileUploadJs = + T.unlines + [ "(function(){" + , "'use strict';" + , "var form = document.getElementById('import-form');" + , "var fileInput = document.getElementById('file-input');" + , "var b64Field = document.getElementById('file-b64');" + , "var nameField = document.getElementById('file-name');" + , "var mimeField = document.getElementById('file-mime');" + , "" + , "form.addEventListener('submit', function(e) {" + , " if (fileInput.files.length > 0) {" + , " e.preventDefault();" + , " var file = fileInput.files[0];" + , " var reader = new FileReader();" + , " reader.onload = function() {" + , " var b64 = reader.result.split(',')[1];" + , " b64Field.value = b64;" + , " nameField.value = file.name;" + , " mimeField.value = file.type;" + , " form.submit();" + , " };" + , " reader.readAsDataURL(file);" + , " }" + , "});" + , "})();" + ] +``` + +- [ ] **Step 2: Build to verify compilation** + +Run: `./hs stack build` +Expected: compiles without errors. + +- [ ] **Step 3: Run tests to verify nothing broke** + +Run: `./hs stack test` +Expected: all tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add src/Roux/Html.hs +git commit -m "feat: expand import form with file upload and textarea" +``` + +--- + +### Task 6: Wire up dependencies and re-exports + +**Files:** +- Modify: `package.yaml`, `src/Roux.hs` + +- [ ] **Step 1: Add dependencies to `package.yaml`** + +In `package.yaml`, add `yaml` and `base64-bytestring` to library dependencies: + +```yaml + - text + - time + - wai + - warp + - yaml + - base64-bytestring +``` + +- [ ] **Step 2: Regenerate `.cabal` file** + +Run: `./hs hpack` +Expected: no errors. + +- [ ] **Step 3: Add Config re-exports to `Roux.hs`** + +```haskell +module Roux ( + app, + loadConfig, + module X, +) where + +import Data.CookLang as X +import Roux.Config (loadConfig) +import Roux.CooklangPrint as X +import Roux.SchemaOrg as X +import Roux.Server (app) +``` + +- [ ] **Step 4: Build to verify compilation** + +Run: `./hs stack build` +Expected: compiles without errors. + +- [ ] **Step 5: Run tests to verify nothing broke** + +Run: `./hs stack test` +Expected: all tests pass. + +- [ ] **Step 6: Commit** + +```bash +git add package.yaml roux-server.cabal src/Roux.hs +git commit -m "chore: add yaml and base64-bytestring dependencies, re-export Config" +``` + +--- + +### Task 7: End-to-end smoke test + +**Files:** (none, manual testing) + +- [ ] **Step 1: Create a test config** + +```bash +cd /work/personal/roux/roux-main +cat > /tmp/roux-test-config.yaml << 'EOF' +anthropic: + api_key: "sk-ant-your-key-here" + model: "claude-sonnet-4-20250514" +EOF +``` + +- [ ] **Step 2: Start the server with the config** + +```bash +cd /work/personal/roux/roux-main +mkdir -p /tmp/roux-test-recipes +./hs stack exec roux-server -- \ + --recipe-dir /tmp/roux-test-recipes \ + --config-file /tmp/roux-test-config.yaml & +sleep 2 +``` + +- [ ] **Step 3: Verify the import form loads** + +```bash +curl -s http://localhost:8080/import | grep -c "Import Recipe" +``` +Expected: returns `1` (page contains heading). + +- [ ] **Step 4: Test text import** + +```bash +curl -s -X POST http://localhost:8080/import \ + --data-urlencode "text=Spaghetti Carbonara + 4 servings + 400g spaghetti + 200g guanciale + 4 eggs + 100g pecorino + Cook the pasta. Fry the guanciale. Mix eggs and cheese. Combine." \ + -o /dev/null -w "%{http_code}\n" -L +``` +Expected: HTTP 200 or 303 (redirect to new recipe page, or error if no real API key). + +- [ ] **Step 5: Test URL import still works (regression)** + +```bash +curl -s -X POST http://localhost:8080/import \ + -d "url=https://example.com" \ + -o /dev/null -w "%{http_code}\n" +``` +Expected: HTTP 200 with an import error (scraper won't find schema.org data on example.com, but the pipeline runs). + +- [ ] **Step 6: Test with missing config (LLM features disabled)** + +```bash +cd /work/personal/roux/roux-main +./hs stack exec roux-server -- \ + --recipe-dir /tmp/roux-test-recipes \ + --config-file /tmp/roux-test-config-nonexistent.yaml & +sleep 2 +curl -s -X POST http://localhost:8080/import \ + --data-urlencode "text=test recipe" | grep -c "not configured" +``` +Expected: contains "not configured". + +- [ ] **Step 7: Kill the servers** + +```bash +kill %1 %2 2>/dev/null; wait 2>/dev/null +``` diff --git a/docs/specs/2026-05-21-llm-recipe-import-design.md b/docs/specs/2026-05-21-llm-recipe-import-design.md new file mode 100644 index 0000000..1074b2a --- /dev/null +++ b/docs/specs/2026-05-21-llm-recipe-import-design.md @@ -0,0 +1,258 @@ +# LLM-Powered Recipe Import + +**Date:** 2026-05-21 +**Status:** Approved + +## Problem + +The current `/import` route only handles recipes from URLs with structured +schema.org JSON-LD data. There is no way to import recipes from: + +- Free-form text (e.g. a recipe emailed by a family member) +- PDF documents (e.g. scanned cookbook pages, saved recipe PDFs) +- Image files (e.g. a photo of a recipe card) + +Many recipes exist in these unstructured formats, and manually transcribing +them into Cooklang is tedious. + +## Goal + +Add Anthropic API (or any Anthropic-compatible API) integration to convert +unstructured recipe content into Cooklang format. The existing URL scraping +path remains unchanged. All three input methods (URL, text, file upload) are +available on the same `/import` page. + +## Architecture + +``` +User submits URL / text / file on /import + ↓ +Server dispatches based on input type: + - URL → runImportPipeline (existing scrape-recipe) + - Text → runLLMPipeline via scripts/convert-to-cooklang --text "..." + - File → save file to recipe dir, then runLLMPipeline via --file + ↓ +scripts/convert-to-cooklang sends content to Anthropic API + using existing cooklang-converter prompts + ↓ +script outputs Cooklang text on stdout + ↓ +Server writes .cook file (and keeps original file for source linking) + ↓ +Redirect to /recipes/ +``` + +## Config File + +A YAML config file, defaulting to `./roux-config.yaml` in the working +directory, overridable via `--config-file` CLI flag. + +```yaml +# roux-config.yaml +anthropic: + api_key: sk-ant-... # required for LLM features + base_url: https://api.anthropic.com # optional, defaults to https://api.anthropic.com + model: claude-sonnet-4-20250514 # optional, defaults to latest Claude Sonnet +``` + +If the config file doesn't exist or `anthropic.api_key` is missing, the +LLM-powered import features are unavailable (the form shows an error message) +but the URL scraper still works. + +### New module: `Roux.Config` + +```haskell +data RouxConfig = RouxConfig + { rcAnthropic :: Maybe AnthropicConfig + } + +data AnthropicConfig = AnthropicConfig + { acApiKey :: Text + , acBaseUrl :: Maybe Text -- default: https://api.anthropic.com + , acModel :: Maybe Text -- default: latest Claude Sonnet + } + +loadConfig :: FilePath -> IO RouxConfig +``` + +Loads and validates the YAML file. Missing file = `RouxConfig Nothing`. +Malformed file = runtime error at startup (fail fast). + +## Subprocess Script: `scripts/convert-to-cooklang` + +**Language:** Python (consistent with `scrape-recipe`) + +**Interface:** +``` +scripts/convert-to-cooklang --config [--file | --text "" | --stdin] +``` + +Exactly one input mode required. + +### Input modes + +- `--text ""` — inline recipe text +- `--stdin` — read recipe text from stdin +- `--file ` — PDF or image file + +### Behavior + +1. Reads the config file (YAML) for `api_key`, optional `base_url`, optional + `model`. +2. Builds the Anthropic Messages API request: + - System prompt: contents of `prompts/system-prompt.md` + - User message: the Cooklang converter instructions from + `prompts/cooklang-converter.md` followed by the recipe content + - See "Prompt composition" section below for details. +3. For `--text`/`--stdin`: sends a `{"type": "text", "text": "..."}` content block +4. For `--file` (PDF/images): reads the file, base64-encodes it, sends as: + - PDF: `{"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}}` + - JPEG: `{"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": "..."}}` + - PNG: `{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}}` +5. Validates the response is valid Cooklang (basic sanity: starts with `---` + front matter marker). On failure, exits with error on stderr. +6. Outputs **only** the Cooklang text on stdout. +7. Exits with code 0 on success, non-zero on error (error details on stderr). + +### Prompt composition + +The script reads two files relative to the project root: + +- **`prompts/system-prompt.md`** — set as the Anthropic Messages API `system` parameter +- **`prompts/cooklang-converter.md`** — prepended to the user message as context, + followed by the recipe content wrapped in the template from + `prompts/user-prompt-template.md` + +This keeps the prompts separate from the script, making them easy to iterate +on without touching code. + +### Dependencies + +``` +anthropic +PyYAML +``` + +Added to the Docker image or a `requirements.txt`. + +## Server-Side Changes + +### CLI: `--config-file` flag in `app/Main.hs` + +``` +roux-server \ + --recipe-dir ./recipes \ + --config-file ./roux-config.yaml +``` + +Default: `./roux-config.yaml` in the working directory. + +### Expanded `POST /import` handler + +The handler inspects the form submission and dispatches accordingly: + +| User action | Dispatched to | +|---|---| +| Submits URL only | `runImportPipeline` (existing) | +| Uploads file | Saves file, then `runLLMPipeline --file` | +| Pastes text | `runLLMPipeline --text` | +| URL + file/text | File/text takes priority (URL ignored) | + +### New `runLLMPipeline` in `Roux.Server` + +```haskell +runLLMPipeline :: + FilePath -- recipe directory + -> AnthropicConfig -- API key, base_url, model + -> Maybe FilePath -- Just path = uploaded file to keep as source + -> Text -- recipe text (or empty if file) + -> IO (Either Html.ImportError FilePath) +``` + +1. Write the uploaded source file to the recipe directory (for PDFs/images — + linked as source from the Cooklang metadata). +2. Call `scripts/convert-to-cooklang` as a subprocess. +3. Capture stdout (Cooklang text) and stderr (error messages). +4. Derive filename from the recipe title in the generated Cooklang front + matter. +5. Write the `.cook` file. +6. Return filename for redirect. + +### Source file linking + +When a PDF or image is uploaded, the original file is saved alongside the +`.cook` file in the recipe directory. The Cooklang front matter includes: + +```yaml +--- +title: ... +source: filename.pdf # relative path to the original source +--- +``` + +This way, the recipe page links to the original document for reference. + +### Expanded import form (`Html.hs`) + +The `/import` page gets three input sections: + +1. **URL** — existing text input, unchanged +2. **File upload** — `` +3. **Paste text** — `