1018 lines
34 KiB
Markdown
1018 lines
34 KiB
Markdown
# LLM-Powered Recipe Import Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add an Anthropic API-powered recipe import path to `/import` that converts free-form text, PDFs, and images into Cooklang, alongside the existing URL scraper.
|
|
|
|
**Architecture:** A new Python subprocess script (`scripts/convert-to-cooklang`) calls the Anthropic Messages API using the existing Cooklang converter prompts. The server dispatches based on input type (URL → existing scraper, text/file → new script). A YAML config file holds API credentials.
|
|
|
|
**Tech Stack:** Python (anthropic SDK), Haskell (wai-extra for multipart parsing, yaml for config), existing Cooklang converter prompts.
|
|
|
|
---
|
|
|
|
## File Map
|
|
|
|
| File | Action | Purpose |
|
|
|---|---|---|
|
|
| `scripts/convert-to-cooklang` | **Create** | Python script calling Anthropic API |
|
|
| `src/Roux/Config.hs` | **Create** | Config file types and loading |
|
|
| `app/Main.hs` | Modify | Add `--config-file` CLI flag |
|
|
| `src/Roux/Server.hs` | Modify | Thread config through app, add `runLLMPipeline`, expand import handler |
|
|
| `src/Roux/Html.hs` | Modify | Add file upload and textarea to import form, switch to multipart form |
|
|
| `src/Roux.hs` | Modify | Re-export Config types |
|
|
| `package.yaml` | Modify | Add `yaml` and `wai-extra` dependencies |
|
|
|
|
---
|
|
|
|
### Task 1: Create `src/Roux/Config.hs` — config types and loading
|
|
|
|
**Files:**
|
|
- Create: `src/Roux/Config.hs`
|
|
|
|
This module defines the config file types and a loader. We use the `yaml` package to parse YAML into Aeson's `Value`, then decode with Fleece (the project's existing JSON library) for consistency.
|
|
|
|
To distinguish "file not found" from "parse error", check `doesFileExist` first before attempting to decode.
|
|
|
|
- [ ] **Step 1: Write the module**
|
|
|
|
```haskell
|
|
{-# LANGUAGE DataKinds #-}
|
|
{-# LANGUAGE OverloadedStrings #-}
|
|
|
|
{- | Configuration for Roux, loaded from a YAML config file.
|
|
|
|
Default location: ./roux-config.yaml in the working directory.
|
|
Overridable via --config-file CLI flag.
|
|
-}
|
|
module Roux.Config (
|
|
RouxConfig (..),
|
|
AnthropicConfig (..),
|
|
loadConfig,
|
|
) where
|
|
|
|
import Data.Text (Text)
|
|
import qualified Data.Text as T
|
|
import Data.Yaml (decodeFileEither)
|
|
import System.Directory (doesFileExist)
|
|
import qualified Fleece.Aeson as Fleece
|
|
import qualified Fleece.Aeson.Decoder as FleeceD
|
|
import qualified Fleece.Core as FC
|
|
|
|
-- | Top-level config.
|
|
data RouxConfig = RouxConfig
|
|
{ rcAnthropic :: Maybe AnthropicConfig
|
|
}
|
|
deriving stock (Eq, Show)
|
|
|
|
-- | Anthropic API configuration.
|
|
data AnthropicConfig = AnthropicConfig
|
|
{ acApiKey :: !Text
|
|
, acBaseUrl :: !(Maybe Text) -- default: https://api.anthropic.com
|
|
, acModel :: !(Maybe Text) -- default: claude-sonnet-4-20250514
|
|
}
|
|
deriving stock (Eq, Show)
|
|
|
|
-- | Fleece schema for AnthropicConfig.
|
|
anthropicConfigSchema :: FC.Schema Fleece.Decoder AnthropicConfig
|
|
anthropicConfigSchema =
|
|
FC.record (FC.unqualifiedName "AnthropicConfig")
|
|
( FC.setRequired "api_key" (FC.projection FC.string)
|
|
( \f r -> f (acApiKey r)
|
|
)
|
|
>>. FC.setOptional "base_url" (FC.projection FC.string)
|
|
( \f r -> f (acBaseUrl r)
|
|
)
|
|
>>. FC.setOptional "model" (FC.projection FC.string)
|
|
( \f r -> f (acModel r)
|
|
)
|
|
)
|
|
|
|
-- | Fleece schema for RouxConfig.
|
|
rouxConfigSchema :: FC.Schema Fleece.Decoder RouxConfig
|
|
rouxConfigSchema =
|
|
FC.record (FC.unqualifiedName "RouxConfig")
|
|
( FC.setOptional "anthropic" anthropicConfigSchema
|
|
( \f r -> f (rcAnthropic r)
|
|
)
|
|
)
|
|
|
|
-- | Load config from a YAML file. Returns Nothing (not an error) if the
|
|
-- file doesn't exist — LLM features will be unavailable but the server
|
|
-- still runs. Exits with an error if the file exists but is malformed.
|
|
loadConfig :: FilePath -> IO RouxConfig
|
|
loadConfig path = do
|
|
exists <- doesFileExist path
|
|
if not exists
|
|
then pure (RouxConfig Nothing)
|
|
else do
|
|
result <- decodeFileEither path
|
|
case result of
|
|
Left parseErr ->
|
|
fail $ "Failed to parse " <> path <> ": " <> show parseErr
|
|
Right val ->
|
|
case FleeceD.fromValue rouxConfigSchema val of
|
|
Right cfg -> pure cfg
|
|
Left decodeErr ->
|
|
fail $
|
|
"Invalid config format in " <> path <> ": " <> show decodeErr
|
|
```
|
|
```
|
|
|
|
- [ ] **Step 2: Build to verify compilation**
|
|
|
|
Run: `./hs stack build`
|
|
Expected: compiles without errors.
|
|
|
|
- [ ] **Step 3: Run tests to verify nothing broke**
|
|
|
|
Run: `./hs stack test`
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/Roux/Config.hs
|
|
git commit -m "feat: add Config module for YAML config loading"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Create `scripts/convert-to-cooklang` — Python subprocess
|
|
|
|
**Files:**
|
|
- Create: `scripts/convert-to-cooklang`
|
|
|
|
This script calls the Anthropic Messages API with the existing Cooklang converter prompts to convert unstructured recipe content into Cooklang format.
|
|
|
|
- [ ] **Step 1: Write the Python script**
|
|
|
|
```python
|
|
#!/usr/bin/env python3
|
|
"""Convert unstructured recipe content to Cooklang format using an
|
|
Anthropic-compatible API (Messages API).
|
|
|
|
Accepts text input (--text or --stdin) or file input (--file for PDF/images).
|
|
Reads API configuration from a YAML config file (--config).
|
|
|
|
Outputs the Cooklang text on stdout.
|
|
Exits 0 on success, non-zero with error details on stderr on failure.
|
|
|
|
Usage:
|
|
scripts/convert-to-cooklang --config roux-config.yaml --text "Recipe text..."
|
|
scripts/convert-to-cooklang --config roux-config.yaml --stdin < recipe.txt
|
|
scripts/convert-to-cooklang --config roux-config.yaml --file recipe.pdf
|
|
"""
|
|
|
|
import argparse
|
|
import base64
|
|
import json
|
|
import mimetypes
|
|
import os
|
|
import sys
|
|
|
|
import yaml
|
|
from anthropic import Anthropic
|
|
|
|
|
|
# -- Config -------------------------------------------------------------------
|
|
|
|
def load_config(path: str) -> dict:
|
|
with open(path) as f:
|
|
cfg = yaml.safe_load(f)
|
|
anthropic_cfg = cfg.get("anthropic", {})
|
|
if not anthropic_cfg.get("api_key"):
|
|
print("Error: anthropic.api_key is required in config", file=sys.stderr)
|
|
sys.exit(1)
|
|
return anthropic_cfg
|
|
|
|
|
|
# -- Prompt loading -----------------------------------------------------------
|
|
|
|
def load_prompts() -> tuple[str, str, str]:
|
|
"""Load system prompt, converter instructions, and user template.
|
|
|
|
Resolved relative to the script's parent directory (project root).
|
|
"""
|
|
script_dir = os.path.dirname(os.path.abspath(__file__))
|
|
project_root = os.path.dirname(script_dir) # scripts/ -> project root
|
|
prompts_dir = os.path.join(project_root, "prompts")
|
|
|
|
def read(name: str) -> str:
|
|
with open(os.path.join(prompts_dir, name)) as f:
|
|
return f.read()
|
|
|
|
return read("system-prompt.md"), read("cooklang-converter.md"), read("user-prompt-template.md")
|
|
|
|
|
|
# -- Content preparation ------------------------------------------------------
|
|
|
|
def build_text_content(text: str) -> list[dict]:
|
|
return [{"type": "text", "text": text}]
|
|
|
|
|
|
def build_file_content(filepath: str) -> list[dict]:
|
|
"""Prepare a file (PDF or image) for the Anthropic Messages API.
|
|
|
|
Returns a list with one document/image content block.
|
|
"""
|
|
mime_type, _ = mimetypes.guess_type(filepath)
|
|
if mime_type is None:
|
|
# Default to octet-stream; let the API decide
|
|
mime_type = "application/octet-stream"
|
|
|
|
with open(filepath, "rb") as f:
|
|
data = base64.b64encode(f.read()).decode("utf-8")
|
|
|
|
is_pdf = mime_type == "application/pdf"
|
|
block_type = "document" if is_pdf else "image"
|
|
|
|
return [
|
|
{
|
|
"type": block_type,
|
|
"source": {
|
|
"type": "base64",
|
|
"media_type": mime_type,
|
|
"data": data,
|
|
},
|
|
}
|
|
]
|
|
|
|
|
|
# -- API call -----------------------------------------------------------------
|
|
|
|
def cooklang_from_anthropic(
|
|
api_key: str,
|
|
base_url: str | None,
|
|
model: str | None,
|
|
system_prompt: str,
|
|
user_message_content: list[dict],
|
|
) -> str:
|
|
"""Send the recipe content to the Anthropic API and return Cooklang text."""
|
|
client = Anthropic(api_key=api_key, base_url=base_url) if base_url else Anthropic(api_key=api_key)
|
|
resolved_model = model or "claude-sonnet-4-20250514"
|
|
|
|
response = client.messages.create(
|
|
model=resolved_model,
|
|
max_tokens=4096,
|
|
system=[{"type": "text", "text": system_prompt}],
|
|
messages=[
|
|
{
|
|
"role": "user",
|
|
"content": user_message_content,
|
|
}
|
|
],
|
|
)
|
|
|
|
# Extract text from response content blocks
|
|
parts: list[str] = []
|
|
for block in response.content:
|
|
if block.type == "text":
|
|
parts.append(block.text)
|
|
|
|
cooklang = "\n".join(parts).strip()
|
|
|
|
# Basic validation: must start with front matter marker
|
|
if not cooklang.startswith("---"):
|
|
print(
|
|
"Error: AI response does not appear to be valid Cooklang "
|
|
"(missing front matter). Response preview: "
|
|
+ cooklang[:200],
|
|
file=sys.stderr,
|
|
)
|
|
sys.exit(1)
|
|
|
|
return cooklang
|
|
|
|
|
|
# -- Main ---------------------------------------------------------------------
|
|
|
|
def main() -> None:
|
|
parser = argparse.ArgumentParser(
|
|
description="Convert unstructured recipe content to Cooklang using an Anthropic-compatible API."
|
|
)
|
|
parser.add_argument(
|
|
"--config",
|
|
required=True,
|
|
help="Path to YAML config file (must contain anthropic.api_key)",
|
|
)
|
|
input_group = parser.add_mutually_exclusive_group(required=True)
|
|
input_group.add_argument("--text", help="Inline recipe text")
|
|
input_group.add_argument("--stdin", action="store_true", help="Read recipe text from stdin")
|
|
input_group.add_argument("--file", help="Path to recipe PDF or image file")
|
|
args = parser.parse_args()
|
|
|
|
# Load config
|
|
config = load_config(args.config)
|
|
api_key = config["api_key"]
|
|
base_url = config.get("base_url")
|
|
model = config.get("model")
|
|
|
|
# Load prompts
|
|
system_prompt, converter_instructions, user_template = load_prompts()
|
|
|
|
# Prepare user message content
|
|
if args.file:
|
|
# File input (PDF or image): send file content + converter instructions
|
|
file_content = build_file_content(args.file)
|
|
user_message_content = (
|
|
[{"type": "text", "text": converter_instructions}]
|
|
+ file_content
|
|
)
|
|
elif args.stdin:
|
|
raw_text = sys.stdin.read()
|
|
user_message_content = build_text_content(
|
|
user_template.replace("{{FULL_RECIPE_TEXT}}", raw_text)
|
|
.replace("{{TITLE}}", "Untitled")
|
|
.replace("{{URL}}", "")
|
|
)
|
|
else:
|
|
# --text
|
|
user_message_content = build_text_content(
|
|
user_template.replace("{{FULL_RECIPE_TEXT}}", args.text)
|
|
.replace("{{TITLE}}", "Untitled")
|
|
.replace("{{URL}}", "")
|
|
)
|
|
|
|
# Call API
|
|
try:
|
|
cooklang = cooklang_from_anthropic(
|
|
api_key=api_key,
|
|
base_url=base_url,
|
|
model=model,
|
|
system_prompt=system_prompt,
|
|
user_message_content=user_message_content,
|
|
)
|
|
except Exception as e:
|
|
print(f"Error calling Anthropic API: {e}", file=sys.stderr)
|
|
sys.exit(1)
|
|
|
|
# Output Cooklang text
|
|
print(cooklang)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
- [ ] **Step 2: Make the script executable**
|
|
|
|
```bash
|
|
chmod +x scripts/convert-to-cooklang
|
|
```
|
|
|
|
- [ ] **Step 3: Test the script manually with text input**
|
|
|
|
Create a test config file and a test recipe:
|
|
|
|
```bash
|
|
cd /work/personal/roux/roux-main
|
|
cat > /tmp/test-config.yaml << 'EOF'
|
|
anthropic:
|
|
api_key: "test-key-placeholder"
|
|
base_url: "https://api.anthropic.com"
|
|
model: "claude-sonnet-4-20250514"
|
|
EOF
|
|
echo "Testing syntax only (no real API call)" && python3 -c "import yaml; from pathlib import Path; Path('scripts/convert-to-cooklang').stat()"
|
|
```
|
|
Expected: Python imports the YAML module successfully, script exists and is executable.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add scripts/convert-to-cooklang
|
|
git commit -m "feat: add convert-to-cooklang script for Anthropic API recipe conversion"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Add `--config-file` CLI flag to `app/Main.hs`
|
|
|
|
**Files:**
|
|
- Modify: `app/Main.hs`
|
|
|
|
- [ ] **Step 1: Add the config-file option**
|
|
|
|
Update the `Options` type and parser:
|
|
|
|
```haskell
|
|
data Options = Options
|
|
{ optHost :: String
|
|
, optPort :: Int
|
|
, optRecipeDir :: FilePath
|
|
, optConfigFile :: FilePath -- NEW
|
|
}
|
|
|
|
optionsParser :: Opts.Parser Options
|
|
optionsParser =
|
|
Options
|
|
<$> Opts.strOption
|
|
( Opts.long "host"
|
|
<> Opts.metavar "HOST"
|
|
<> Opts.help "Listen host"
|
|
<> Opts.value "0.0.0.0"
|
|
<> Opts.showDefault
|
|
)
|
|
<*> Opts.option
|
|
Opts.auto
|
|
( Opts.long "port"
|
|
<> Opts.short 'p'
|
|
<> Opts.metavar "PORT"
|
|
<> Opts.help "Listen port"
|
|
<> Opts.value 8080
|
|
<> Opts.showDefault
|
|
)
|
|
<*> Opts.strOption
|
|
( Opts.long "recipe-dir"
|
|
<> Opts.short 'r'
|
|
<> Opts.metavar "DIR"
|
|
<> Opts.help "Path to directory containing Cooklang recipe files"
|
|
<> Opts.value "recipes"
|
|
<> Opts.showDefault
|
|
)
|
|
<*> Opts.strOption -- NEW
|
|
( Opts.long "config-file" -- NEW
|
|
<> Opts.metavar "FILE" -- NEW
|
|
<> Opts.help "Path to YAML config file (default: ./roux-config.yaml)" -- NEW
|
|
<> Opts.value "roux-config.yaml" -- NEW
|
|
<> Opts.showDefault -- NEW
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 2: Load config and pass to `Roux.app`**
|
|
|
|
Change the `main` function to load the config and pass it to `app`:
|
|
|
|
```haskell
|
|
main :: IO ()
|
|
main = do
|
|
hSetBuffering stdout NoBuffering
|
|
hSetBuffering stderr NoBuffering
|
|
opts <-
|
|
Opts.execParser $
|
|
Opts.info
|
|
(optionsParser Opts.<**> Opts.helper)
|
|
( Opts.fullDesc
|
|
<> Opts.progDesc "Roux recipe management server"
|
|
<> Opts.header "roux-server — personal/family recipe manager"
|
|
)
|
|
let settings =
|
|
Warp.setPort (optPort opts) $
|
|
Warp.setHost
|
|
(fromString (optHost opts))
|
|
Warp.defaultSettings
|
|
|
|
putStrLn $ "[roux] scanning recipes from " <> optRecipeDir opts
|
|
putStrLn $ "[roux] listening on " <> optHost opts <> ":" <> show (optPort opts)
|
|
|
|
-- Load config (may be missing — LLM features disabled gracefully)
|
|
config <- Roux.loadConfig (optConfigFile opts) -- NEW
|
|
waiApp <- Roux.app (optRecipeDir opts) config -- MODIFIED: pass config
|
|
Warp.runSettings settings waiApp
|
|
```
|
|
|
|
- [ ] **Step 3: Build to verify compilation**
|
|
|
|
Run: `./hs stack build`
|
|
Expected: compiles without errors.
|
|
|
|
- [ ] **Step 4: Run tests to verify nothing broke**
|
|
|
|
Run: `./hs stack test`
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add app/Main.hs
|
|
git commit -m "feat: add --config-file CLI flag for YAML config"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Thread config through `Server.hs` and add `runLLMPipeline`
|
|
|
|
**Files:**
|
|
- Modify: `src/Roux/Server.hs`
|
|
|
|
- [ ] **Step 1: Add imports and update `app` signature**
|
|
|
|
Add new imports at the top:
|
|
|
|
```haskell
|
|
import Data.ByteString.Lazy (ByteString)
|
|
import qualified Data.ByteString.Lazy as LB
|
|
import qualified Data.Text.Encoding as TE
|
|
import qualified Roux.Config as Config
|
|
import Roux.Config (RouxConfig (..), AnthropicConfig (..))
|
|
import System.Directory (createDirectoryIfMissing)
|
|
```
|
|
|
|
Change the `app` function signature to accept `RouxConfig`:
|
|
|
|
```haskell
|
|
app :: FilePath -> RouxConfig -> IO Wai.Application
|
|
app recipeDir config = do
|
|
-- ... existing startup code stays the same ...
|
|
pure (router absDir config state changeLog) -- MODIFIED: pass config
|
|
```
|
|
|
|
Update the `router` to accept and use `config`:
|
|
|
|
```haskell
|
|
router :: FilePath -> RouxConfig -> IORef (Map FilePath Idx.RecipeInfo) -> ChangeLog -> Wai.Application
|
|
router recipeDir config state changeLog request respond =
|
|
case Wai.pathInfo request of
|
|
["events"] -> sseHandler changeLog request respond
|
|
["import"] -> importHandler recipeDir config state request respond
|
|
_ -> handleRequest state request respond
|
|
```
|
|
|
|
- [ ] **Step 2: Update `importHandler` to handle file uploads via base64 fields**
|
|
|
|
The form sends file data as base64-encoded hidden fields (set by client-side JS). The existing `parseFormBody` handles URL-encoded form data. We add file detection by looking for `file-b64` and `file-name` parameters.
|
|
|
|
Replace the existing `importHandler` with the expanded version:
|
|
|
|
```haskell
|
|
{- | Handle GET and POST requests to /import.
|
|
GET — show the import form.
|
|
POST — dispatch based on input: URL (existing scraper), text, or file (Anthropic).
|
|
Files are uploaded as base64-encoded content via hidden form fields
|
|
(file-b64, file-name, file-mime) set by client-side JavaScript.
|
|
-}
|
|
importHandler :: FilePath -> RouxConfig -> IORef (Map FilePath Idx.RecipeInfo) -> Wai.Application
|
|
importHandler recipeDir config _state request respond =
|
|
case Wai.requestMethod request of
|
|
"GET" ->
|
|
respond (htmlResponse (Html.importPage Nothing))
|
|
"POST" -> do
|
|
body <- readRequestBody request
|
|
let params = parseFormBody body
|
|
url = maybe "" id (lookup "url" params)
|
|
text = maybe "" id (lookup "text" params)
|
|
fileB64 = maybe "" id (lookup "file-b64" params)
|
|
fileName = maybe "" id (lookup "file-name" params)
|
|
fileMime = maybe "" id (lookup "file-mime" params)
|
|
hasFile = not (T.null fileB64)
|
|
|
|
putStrLn $ "[roux] import request: url=" <> T.unpack url
|
|
<> " text=" <> show (not (T.null text))
|
|
<> " file=" <> show hasFile
|
|
|
|
result <- try $ case (hasFile, not (T.null text), not (T.null url)) of
|
|
-- File takes priority, then text, then URL
|
|
(True, _, _) ->
|
|
runFileImport recipeDir config fileName fileMime fileB64
|
|
(_, True, _) ->
|
|
runTextImport recipeDir config text
|
|
(_, _, True) ->
|
|
runImportPipeline recipeDir url
|
|
_ ->
|
|
pure (Left (Html.ImportError "Provide a URL, recipe text, or file to import."))
|
|
|
|
case result of
|
|
Left (e :: SomeException) ->
|
|
respond $
|
|
htmlResponse $
|
|
Html.importResultPage $
|
|
Html.ImportError ("Unexpected error: " <> T.pack (show e))
|
|
Right (Left err) ->
|
|
respond $
|
|
htmlResponse $
|
|
Html.importResultPage err
|
|
Right (Right filename) ->
|
|
respond $
|
|
Wai.responseLBS
|
|
HTTP.status303
|
|
[("Location", "/recipes/" <> TE.encodeUtf8 (T.pack filename))]
|
|
""
|
|
_ -> respond notFound
|
|
```
|
|
|
|
- [ ] **Step 3: Update `runFileImport` function (accepts base64)**
|
|
|
|
The file content arrives as base64. Decode it, save the original alongside the recipe, then call the LLM pipeline.
|
|
|
|
```haskell
|
|
-- | Import a recipe from an uploaded file via the LLM pipeline.
|
|
-- The file content is base64-encoded in the form data.
|
|
runFileImport ::
|
|
FilePath -- ^ recipe directory
|
|
-> RouxConfig -- ^ server config (for Anthropic settings)
|
|
-> Text -- ^ original filename
|
|
-> Text -- ^ MIME type
|
|
-> Text -- ^ base64-encoded file content
|
|
-> IO (Either Html.ImportError FilePath)
|
|
runFileImport recipeDir config originalName mimeType b64Text = do
|
|
-- Decode base64
|
|
case decodeBase64 (TE.encodeUtf8 b64Text) of
|
|
Left err ->
|
|
pure $ Left $ Html.ImportError $
|
|
"Could not decode uploaded file: " <> T.pack err
|
|
Right content -> do
|
|
-- Ensure recipe dir exists
|
|
createDirectoryIfMissing True recipeDir
|
|
-- Save the original file
|
|
let sourceFilename = map (\c -> if isAlphaNum c || c == '.' || c == '-' || c == '_' then c else '_') (T.unpack originalName)
|
|
sourcePath = recipeDir </> sourceFilename
|
|
LB.writeFile sourcePath content
|
|
putStrLn $ "[roux] saved uploaded file to " <> sourcePath
|
|
-- Call the LLM pipeline with the saved file
|
|
case rcAnthropic config of
|
|
Nothing ->
|
|
pure $ Left $ Html.ImportError $
|
|
"LLM import is not configured. Add an anthropic.api_key to the config file."
|
|
Just anthropicConfig -> do
|
|
result <- runLLMPipeline recipeDir anthropicConfig (Just sourcePath) ""
|
|
case result of
|
|
Right filename -> pure $ Right filename
|
|
Left err -> pure $ Left err
|
|
```
|
|
|
|
- [ ] **Step 5: Add `runTextImport` function**
|
|
|
|
```haskell
|
|
-- | Import a recipe from pasted text via the LLM pipeline.
|
|
runTextImport ::
|
|
FilePath -> RouxConfig -> Text
|
|
-> IO (Either Html.ImportError FilePath)
|
|
runTextImport recipeDir config text = do
|
|
case rcAnthropic config of
|
|
Nothing ->
|
|
pure $ Left $ Html.ImportError $
|
|
"LLM import is not configured. Add an anthropic.api_key to the config file."
|
|
Just anthropicConfig ->
|
|
runLLMPipeline recipeDir anthropicConfig Nothing text
|
|
```
|
|
|
|
- [ ] **Step 6: Add `runLLMPipeline` function**
|
|
|
|
```haskell
|
|
-- | Run the full LLM import pipeline: call convert-to-cooklang → parse → save.
|
|
runLLMPipeline ::
|
|
FilePath -- ^ recipe directory
|
|
-> AnthropicConfig -- ^ API config
|
|
-> Maybe FilePath -- ^ Just path = uploaded source file to keep
|
|
-> Text -- ^ recipe text (or empty if file)
|
|
-> IO (Either Html.ImportError FilePath)
|
|
runLLMPipeline recipeDir anthropicConfig mSourceFile text = do
|
|
-- Step 1: locate the script
|
|
script <- findConvertScript
|
|
-- Step 2: write a temp config for the subprocess
|
|
let configPath = recipeDir </> ".convert-config.yaml"
|
|
writeConvertConfig configPath anthropicConfig
|
|
-- Step 3: build subprocess args
|
|
let (args, input) = case mSourceFile of
|
|
Just fp -> (["--config", configPath, "--file", fp], "")
|
|
Nothing -> (["--config", configPath, "--text", T.unpack text], "")
|
|
putStrLn $ "[roux] running " <> script <> " for LLM import"
|
|
(exitCode, stdoutBytes, stderrBytes) <-
|
|
readProcessBytes script args
|
|
-- Clean up temp config
|
|
_ <- try (removeFile configPath) :: IO (Either IOException ())
|
|
case exitCode of
|
|
ExitFailure _ -> do
|
|
let errMsg = TE.decodeUtf8 (LB.toStrict stderrBytes)
|
|
putStrLn $ "[roux] convert-to-cooklang failed: " <> T.unpack errMsg
|
|
return $
|
|
Left $
|
|
Html.ImportError $
|
|
"Could not convert recipe: " <> errMsg
|
|
ExitSuccess -> do
|
|
-- Step 4: parse the Cooklang output to extract the title
|
|
let cooklangText = TE.decodeUtf8 (LB.toStrict stdoutBytes)
|
|
case parseRecipeTitle cooklangText of
|
|
Nothing -> do
|
|
putStrLn "[roux] could not extract title from Cooklang output"
|
|
return $
|
|
Left $
|
|
Html.ImportError $
|
|
"Could not determine recipe title from AI output."
|
|
Just title -> do
|
|
let filename = CooklangPrint.filenameFromTitle title
|
|
filepath = recipeDir </> filename
|
|
-- Step 5: write the .cook file
|
|
putStrLn $ "[roux] writing imported recipe to " <> filepath
|
|
LB.writeFile filepath stdoutBytes
|
|
putStrLn $ "[roux] successfully imported recipe as " <> filename
|
|
return $ Right filename
|
|
```
|
|
|
|
- [ ] **Step 7: Add helper functions for config generation and title parsing**
|
|
|
|
```haskell
|
|
-- | Locate the convert-to-cooklang script.
|
|
findConvertScript :: IO FilePath
|
|
findConvertScript = do
|
|
mPath <- findExecutable "convert-to-cooklang"
|
|
case mPath of
|
|
Just p -> pure p
|
|
Nothing -> pure "scripts/convert-to-cooklang"
|
|
|
|
-- | Write a temporary YAML config for the subprocess.
|
|
writeConvertConfig :: FilePath -> AnthropicConfig -> IO ()
|
|
writeConvertConfig path ac = do
|
|
let pairs =
|
|
[ "anthropic:"
|
|
, " api_key: " <> T.unpack (acApiKey ac)
|
|
] ++ case acBaseUrl ac of
|
|
Just url -> [" base_url: " <> T.unpack url]
|
|
Nothing -> []
|
|
++ case acModel ac of
|
|
Just m -> [" model: " <> T.unpack m]
|
|
Nothing -> []
|
|
writeFile path (unlines pairs)
|
|
|
|
-- | Parse the recipe title from Cooklang front matter.
|
|
-- Looks for "title: ..." in the YAML front matter block.
|
|
parseRecipeTitle :: Text -> Maybe Text
|
|
parseRecipeTitle cooklang =
|
|
case T.breakOn "---" cooklang of
|
|
(_, afterFirst) ->
|
|
let (_separator, inside) = T.breakOn "---" (T.drop 3 afterFirst)
|
|
frontMatter = T.take (T.length inside) (T.drop 3 afterFirst)
|
|
in case T.breakOn "title:" frontMatter of
|
|
(_, afterTitle) ->
|
|
let titleLine = T.takeWhile (/= '\n') (T.drop 6 afterTitle)
|
|
cleaned = T.strip titleLine
|
|
in if T.null cleaned then Nothing else Just cleaned
|
|
```
|
|
|
|
- [ ] **Step 8: Add missing imports to Server.hs**
|
|
|
|
Make sure these imports are present (add any that are missing):
|
|
|
|
```haskell
|
|
import Control.Exception (IOException, SomeException, catch, finally, try)
|
|
import Data.Char (isAlphaNum)
|
|
import Data.List (isSuffixOf)
|
|
import Data.Maybe (fromMaybe, isJust, listToMaybe)
|
|
import qualified Data.Text as T
|
|
import qualified Data.Text.Encoding as TE
|
|
import qualified Data.ByteString.Lazy as LB
|
|
import System.Directory (createDirectoryIfMissing, doesFileExist, findExecutable, makeAbsolute, removeFile)
|
|
import System.FilePath (takeBaseName, takeExtension, (</>))
|
|
import qualified Roux.CooklangPrint as CooklangPrint
|
|
import qualified Roux.Html as Html
|
|
```
|
|
|
|
- [ ] **Step 9: Build to verify compilation**
|
|
|
|
Run: `./hs stack build`
|
|
Expected: compiles without errors.
|
|
|
|
- [ ] **Step 10: Run tests to verify nothing broke**
|
|
|
|
Run: `./hs stack test`
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 11: Commit**
|
|
|
|
```bash
|
|
git add src/Roux/Server.hs
|
|
git commit -m "feat: add LLM import pipeline with file upload and text input"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Expand import form in `Html.hs`
|
|
|
|
**Files:**
|
|
- Modify: `src/Roux/Html.hs`
|
|
|
|
- [ ] **Step 1: Update the import form to include file upload, textarea, and JS base64 conversion**
|
|
|
|
The form stays URL-encoded (no `enctype` override). A small JS snippet converts the selected file to base64 and places it in hidden form fields before submission.
|
|
|
|
Replace the existing `importPage` function:
|
|
|
|
```haskell
|
|
-- | Render the import form page, optionally with a validation error.
|
|
importPage :: Maybe Text -> ByteString
|
|
importPage merror =
|
|
page "Roux — Import Recipe" $ do
|
|
H.nav ! A.class_ "roux-navbar" $ do
|
|
H.ul $ H.li $ H.a ! A.href "/" $ "← Back"
|
|
H.ul $ H.li $ H.strong "Import Recipe"
|
|
case merror of
|
|
Just err -> H.p ! A.style "color: var(--roux-accent);" $ H.toHtml err
|
|
Nothing -> pure ()
|
|
H.p ! A.style "font-size: 0.85rem; opacity: 0.7; margin-bottom: 1.5rem;" $
|
|
"Paste a recipe URL, upload a PDF or image, or paste recipe text directly."
|
|
H.form ! A.method "POST" ! A.action "/import" ! A.id "import-form" $
|
|
-- URL input
|
|
H.label ! A.for "url" $ "Recipe URL"
|
|
>> H.input
|
|
! A.type_ "url"
|
|
! A.id "url"
|
|
! A.name "url"
|
|
! A.placeholder "https://cooking.nytimes.com/recipes/..."
|
|
! A.style "width: 100%; margin-bottom: 1.5rem;"
|
|
-- File upload
|
|
>> H.label ! A.for "file-input" $ "Or upload a PDF or image"
|
|
>> H.input
|
|
! A.type_ "file"
|
|
! A.id "file-input"
|
|
! A.accept ".pdf,image/*"
|
|
! A.style "width: 100%; margin-bottom: 1.5rem;"
|
|
-- Hidden fields for base64 file data (set by JS)
|
|
>> H.input ! A.type_ "hidden" ! A.id "file-b64" ! A.name "file-b64" ! A.value ""
|
|
>> H.input ! A.type_ "hidden" ! A.id "file-name" ! A.name "file-name" ! A.value ""
|
|
>> H.input ! A.type_ "hidden" ! A.id "file-mime" ! A.name "file-mime" ! A.value ""
|
|
-- Text area
|
|
>> H.label ! A.for "text" $ "Or paste recipe text"
|
|
>> H.textarea
|
|
! A.id "text"
|
|
! A.name "text"
|
|
! A.rows "8"
|
|
! A.placeholder "Paste recipe text here..."
|
|
! A.style "width: 100%; margin-bottom: 1.5rem; font-family: monospace; font-size: 0.85rem;"
|
|
-- Submit
|
|
>> H.button ! A.type_ "submit" $ "Import Recipe"
|
|
-- Inline JS for base64 file conversion
|
|
>> H.script ! A.type_ "text/javascript" $ H.preEscapedText fileUploadJs
|
|
|
|
-- | Inline JavaScript for converting file uploads to base64 before form submission.
|
|
fileUploadJs :: Text
|
|
fileUploadJs =
|
|
T.unlines
|
|
[ "(function(){"
|
|
, "'use strict';"
|
|
, "var form = document.getElementById('import-form');"
|
|
, "var fileInput = document.getElementById('file-input');"
|
|
, "var b64Field = document.getElementById('file-b64');"
|
|
, "var nameField = document.getElementById('file-name');"
|
|
, "var mimeField = document.getElementById('file-mime');"
|
|
, ""
|
|
, "form.addEventListener('submit', function(e) {"
|
|
, " if (fileInput.files.length > 0) {"
|
|
, " e.preventDefault();"
|
|
, " var file = fileInput.files[0];"
|
|
, " var reader = new FileReader();"
|
|
, " reader.onload = function() {"
|
|
, " var b64 = reader.result.split(',')[1];"
|
|
, " b64Field.value = b64;"
|
|
, " nameField.value = file.name;"
|
|
, " mimeField.value = file.type;"
|
|
, " form.submit();"
|
|
, " };"
|
|
, " reader.readAsDataURL(file);"
|
|
, " }"
|
|
, "});"
|
|
, "})();"
|
|
]
|
|
```
|
|
|
|
- [ ] **Step 2: Build to verify compilation**
|
|
|
|
Run: `./hs stack build`
|
|
Expected: compiles without errors.
|
|
|
|
- [ ] **Step 3: Run tests to verify nothing broke**
|
|
|
|
Run: `./hs stack test`
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/Roux/Html.hs
|
|
git commit -m "feat: expand import form with file upload and textarea"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Wire up dependencies and re-exports
|
|
|
|
**Files:**
|
|
- Modify: `package.yaml`, `src/Roux.hs`
|
|
|
|
- [ ] **Step 1: Add dependencies to `package.yaml`**
|
|
|
|
In `package.yaml`, add `yaml` and `base64-bytestring` to library dependencies:
|
|
|
|
```yaml
|
|
- text
|
|
- time
|
|
- wai
|
|
- warp
|
|
- yaml
|
|
- base64-bytestring
|
|
```
|
|
|
|
- [ ] **Step 2: Regenerate `.cabal` file**
|
|
|
|
Run: `./hs hpack`
|
|
Expected: no errors.
|
|
|
|
- [ ] **Step 3: Add Config re-exports to `Roux.hs`**
|
|
|
|
```haskell
|
|
module Roux (
|
|
app,
|
|
loadConfig,
|
|
module X,
|
|
) where
|
|
|
|
import Data.CookLang as X
|
|
import Roux.Config (loadConfig)
|
|
import Roux.CooklangPrint as X
|
|
import Roux.SchemaOrg as X
|
|
import Roux.Server (app)
|
|
```
|
|
|
|
- [ ] **Step 4: Build to verify compilation**
|
|
|
|
Run: `./hs stack build`
|
|
Expected: compiles without errors.
|
|
|
|
- [ ] **Step 5: Run tests to verify nothing broke**
|
|
|
|
Run: `./hs stack test`
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add package.yaml roux-server.cabal src/Roux.hs
|
|
git commit -m "chore: add yaml and base64-bytestring dependencies, re-export Config"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: End-to-end smoke test
|
|
|
|
**Files:** (none, manual testing)
|
|
|
|
- [ ] **Step 1: Create a test config**
|
|
|
|
```bash
|
|
cd /work/personal/roux/roux-main
|
|
cat > /tmp/roux-test-config.yaml << 'EOF'
|
|
anthropic:
|
|
api_key: "sk-ant-your-key-here"
|
|
model: "claude-sonnet-4-20250514"
|
|
EOF
|
|
```
|
|
|
|
- [ ] **Step 2: Start the server with the config**
|
|
|
|
```bash
|
|
cd /work/personal/roux/roux-main
|
|
mkdir -p /tmp/roux-test-recipes
|
|
./hs stack exec roux-server -- \
|
|
--recipe-dir /tmp/roux-test-recipes \
|
|
--config-file /tmp/roux-test-config.yaml &
|
|
sleep 2
|
|
```
|
|
|
|
- [ ] **Step 3: Verify the import form loads**
|
|
|
|
```bash
|
|
curl -s http://localhost:8080/import | grep -c "Import Recipe"
|
|
```
|
|
Expected: returns `1` (page contains heading).
|
|
|
|
- [ ] **Step 4: Test text import**
|
|
|
|
```bash
|
|
curl -s -X POST http://localhost:8080/import \
|
|
--data-urlencode "text=Spaghetti Carbonara
|
|
4 servings
|
|
400g spaghetti
|
|
200g guanciale
|
|
4 eggs
|
|
100g pecorino
|
|
Cook the pasta. Fry the guanciale. Mix eggs and cheese. Combine." \
|
|
-o /dev/null -w "%{http_code}\n" -L
|
|
```
|
|
Expected: HTTP 200 or 303 (redirect to new recipe page, or error if no real API key).
|
|
|
|
- [ ] **Step 5: Test URL import still works (regression)**
|
|
|
|
```bash
|
|
curl -s -X POST http://localhost:8080/import \
|
|
-d "url=https://example.com" \
|
|
-o /dev/null -w "%{http_code}\n"
|
|
```
|
|
Expected: HTTP 200 with an import error (scraper won't find schema.org data on example.com, but the pipeline runs).
|
|
|
|
- [ ] **Step 6: Test with missing config (LLM features disabled)**
|
|
|
|
```bash
|
|
cd /work/personal/roux/roux-main
|
|
./hs stack exec roux-server -- \
|
|
--recipe-dir /tmp/roux-test-recipes \
|
|
--config-file /tmp/roux-test-config-nonexistent.yaml &
|
|
sleep 2
|
|
curl -s -X POST http://localhost:8080/import \
|
|
--data-urlencode "text=test recipe" | grep -c "not configured"
|
|
```
|
|
Expected: contains "not configured".
|
|
|
|
- [ ] **Step 7: Kill the servers**
|
|
|
|
```bash
|
|
kill %1 %2 2>/dev/null; wait 2>/dev/null
|
|
```
|