refactor: return Either from loadConfig, add Config tests
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,258 @@
|
|||||||
|
# LLM-Powered Recipe Import
|
||||||
|
|
||||||
|
**Date:** 2026-05-21
|
||||||
|
**Status:** Approved
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The current `/import` route only handles recipes from URLs with structured
|
||||||
|
schema.org JSON-LD data. There is no way to import recipes from:
|
||||||
|
|
||||||
|
- Free-form text (e.g. a recipe emailed by a family member)
|
||||||
|
- PDF documents (e.g. scanned cookbook pages, saved recipe PDFs)
|
||||||
|
- Image files (e.g. a photo of a recipe card)
|
||||||
|
|
||||||
|
Many recipes exist in these unstructured formats, and manually transcribing
|
||||||
|
them into Cooklang is tedious.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Add Anthropic API (or any Anthropic-compatible API) integration to convert
|
||||||
|
unstructured recipe content into Cooklang format. The existing URL scraping
|
||||||
|
path remains unchanged. All three input methods (URL, text, file upload) are
|
||||||
|
available on the same `/import` page.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
User submits URL / text / file on /import
|
||||||
|
↓
|
||||||
|
Server dispatches based on input type:
|
||||||
|
- URL → runImportPipeline (existing scrape-recipe)
|
||||||
|
- Text → runLLMPipeline via scripts/convert-to-cooklang --text "..."
|
||||||
|
- File → save file to recipe dir, then runLLMPipeline via --file <path>
|
||||||
|
↓
|
||||||
|
scripts/convert-to-cooklang sends content to Anthropic API
|
||||||
|
using existing cooklang-converter prompts
|
||||||
|
↓
|
||||||
|
script outputs Cooklang text on stdout
|
||||||
|
↓
|
||||||
|
Server writes .cook file (and keeps original file for source linking)
|
||||||
|
↓
|
||||||
|
Redirect to /recipes/<filename>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config File
|
||||||
|
|
||||||
|
A YAML config file, defaulting to `./roux-config.yaml` in the working
|
||||||
|
directory, overridable via `--config-file` CLI flag.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# roux-config.yaml
|
||||||
|
anthropic:
|
||||||
|
api_key: sk-ant-... # required for LLM features
|
||||||
|
base_url: https://api.anthropic.com # optional, defaults to https://api.anthropic.com
|
||||||
|
model: claude-sonnet-4-20250514 # optional, defaults to latest Claude Sonnet
|
||||||
|
```
|
||||||
|
|
||||||
|
If the config file doesn't exist or `anthropic.api_key` is missing, the
|
||||||
|
LLM-powered import features are unavailable (the form shows an error message)
|
||||||
|
but the URL scraper still works.
|
||||||
|
|
||||||
|
### New module: `Roux.Config`
|
||||||
|
|
||||||
|
```haskell
|
||||||
|
data RouxConfig = RouxConfig
|
||||||
|
{ rcAnthropic :: Maybe AnthropicConfig
|
||||||
|
}
|
||||||
|
|
||||||
|
data AnthropicConfig = AnthropicConfig
|
||||||
|
{ acApiKey :: Text
|
||||||
|
, acBaseUrl :: Maybe Text -- default: https://api.anthropic.com
|
||||||
|
, acModel :: Maybe Text -- default: latest Claude Sonnet
|
||||||
|
}
|
||||||
|
|
||||||
|
loadConfig :: FilePath -> IO RouxConfig
|
||||||
|
```
|
||||||
|
|
||||||
|
Loads and validates the YAML file. Missing file = `RouxConfig Nothing`.
|
||||||
|
Malformed file = runtime error at startup (fail fast).
|
||||||
|
|
||||||
|
## Subprocess Script: `scripts/convert-to-cooklang`
|
||||||
|
|
||||||
|
**Language:** Python (consistent with `scrape-recipe`)
|
||||||
|
|
||||||
|
**Interface:**
|
||||||
|
```
|
||||||
|
scripts/convert-to-cooklang --config <path> [--file <path> | --text "<text>" | --stdin]
|
||||||
|
```
|
||||||
|
|
||||||
|
Exactly one input mode required.
|
||||||
|
|
||||||
|
### Input modes
|
||||||
|
|
||||||
|
- `--text "<recipe text>"` — inline recipe text
|
||||||
|
- `--stdin` — read recipe text from stdin
|
||||||
|
- `--file <path>` — PDF or image file
|
||||||
|
|
||||||
|
### Behavior
|
||||||
|
|
||||||
|
1. Reads the config file (YAML) for `api_key`, optional `base_url`, optional
|
||||||
|
`model`.
|
||||||
|
2. Builds the Anthropic Messages API request:
|
||||||
|
- System prompt: contents of `prompts/system-prompt.md`
|
||||||
|
- User message: the Cooklang converter instructions from
|
||||||
|
`prompts/cooklang-converter.md` followed by the recipe content
|
||||||
|
- See "Prompt composition" section below for details.
|
||||||
|
3. For `--text`/`--stdin`: sends a `{"type": "text", "text": "..."}` content block
|
||||||
|
4. For `--file` (PDF/images): reads the file, base64-encodes it, sends as:
|
||||||
|
- PDF: `{"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}}`
|
||||||
|
- JPEG: `{"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": "..."}}`
|
||||||
|
- PNG: `{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}}`
|
||||||
|
5. Validates the response is valid Cooklang (basic sanity: starts with `---`
|
||||||
|
front matter marker). On failure, exits with error on stderr.
|
||||||
|
6. Outputs **only** the Cooklang text on stdout.
|
||||||
|
7. Exits with code 0 on success, non-zero on error (error details on stderr).
|
||||||
|
|
||||||
|
### Prompt composition
|
||||||
|
|
||||||
|
The script reads two files relative to the project root:
|
||||||
|
|
||||||
|
- **`prompts/system-prompt.md`** — set as the Anthropic Messages API `system` parameter
|
||||||
|
- **`prompts/cooklang-converter.md`** — prepended to the user message as context,
|
||||||
|
followed by the recipe content wrapped in the template from
|
||||||
|
`prompts/user-prompt-template.md`
|
||||||
|
|
||||||
|
This keeps the prompts separate from the script, making them easy to iterate
|
||||||
|
on without touching code.
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
```
|
||||||
|
anthropic
|
||||||
|
PyYAML
|
||||||
|
```
|
||||||
|
|
||||||
|
Added to the Docker image or a `requirements.txt`.
|
||||||
|
|
||||||
|
## Server-Side Changes
|
||||||
|
|
||||||
|
### CLI: `--config-file` flag in `app/Main.hs`
|
||||||
|
|
||||||
|
```
|
||||||
|
roux-server \
|
||||||
|
--recipe-dir ./recipes \
|
||||||
|
--config-file ./roux-config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Default: `./roux-config.yaml` in the working directory.
|
||||||
|
|
||||||
|
### Expanded `POST /import` handler
|
||||||
|
|
||||||
|
The handler inspects the form submission and dispatches accordingly:
|
||||||
|
|
||||||
|
| User action | Dispatched to |
|
||||||
|
|---|---|
|
||||||
|
| Submits URL only | `runImportPipeline` (existing) |
|
||||||
|
| Uploads file | Saves file, then `runLLMPipeline --file` |
|
||||||
|
| Pastes text | `runLLMPipeline --text` |
|
||||||
|
| URL + file/text | File/text takes priority (URL ignored) |
|
||||||
|
|
||||||
|
### New `runLLMPipeline` in `Roux.Server`
|
||||||
|
|
||||||
|
```haskell
|
||||||
|
runLLMPipeline ::
|
||||||
|
FilePath -- recipe directory
|
||||||
|
-> AnthropicConfig -- API key, base_url, model
|
||||||
|
-> Maybe FilePath -- Just path = uploaded file to keep as source
|
||||||
|
-> Text -- recipe text (or empty if file)
|
||||||
|
-> IO (Either Html.ImportError FilePath)
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Write the uploaded source file to the recipe directory (for PDFs/images —
|
||||||
|
linked as source from the Cooklang metadata).
|
||||||
|
2. Call `scripts/convert-to-cooklang` as a subprocess.
|
||||||
|
3. Capture stdout (Cooklang text) and stderr (error messages).
|
||||||
|
4. Derive filename from the recipe title in the generated Cooklang front
|
||||||
|
matter.
|
||||||
|
5. Write the `.cook` file.
|
||||||
|
6. Return filename for redirect.
|
||||||
|
|
||||||
|
### Source file linking
|
||||||
|
|
||||||
|
When a PDF or image is uploaded, the original file is saved alongside the
|
||||||
|
`.cook` file in the recipe directory. The Cooklang front matter includes:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
title: ...
|
||||||
|
source: filename.pdf # relative path to the original source
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
This way, the recipe page links to the original document for reference.
|
||||||
|
|
||||||
|
### Expanded import form (`Html.hs`)
|
||||||
|
|
||||||
|
The `/import` page gets three input sections:
|
||||||
|
|
||||||
|
1. **URL** — existing text input, unchanged
|
||||||
|
2. **File upload** — `<input type="file" accept=".pdf,image/*">`
|
||||||
|
3. **Paste text** — `<textarea>` for free-form recipe text
|
||||||
|
|
||||||
|
A note at the top explains the three options. A small JS snippet clears the
|
||||||
|
other inputs when one is filled (optional UX polish).
|
||||||
|
|
||||||
|
### Error handling
|
||||||
|
|
||||||
|
Follows the same pattern as `runImportPipeline`:
|
||||||
|
|
||||||
|
| Step | Failure mode | User sees |
|
||||||
|
|------|-------------|-----------|
|
||||||
|
| Config missing | No API key configured | "LLM import is not configured. Add an API key to the config file." |
|
||||||
|
| Subprocess | Script error, API error | "Could not convert recipe: {stderr}" |
|
||||||
|
| Output validation | Output is not valid Cooklang | "The AI returned invalid Cooklang: {preview}" |
|
||||||
|
| File write | Permission denied, disk full | "Could not save recipe: {IO error}" |
|
||||||
|
|
||||||
|
## Files Changed
|
||||||
|
|
||||||
|
### New
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `scripts/convert-to-cooklang` | Python script for Anthropic API conversion |
|
||||||
|
| `src/Roux/Config.hs` | Config file loading and types |
|
||||||
|
|
||||||
|
### Modified
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
|---|---|
|
||||||
|
| `app/Main.hs` | Add `--config-file` CLI flag |
|
||||||
|
| `src/Roux/Server.hs` | Thread config through app, add `runLLMPipeline`, expand import handler |
|
||||||
|
| `src/Roux/Html.hs` | Add file upload and textarea to import form |
|
||||||
|
| `src/Roux.hs` | Re-export Config types |
|
||||||
|
| `package.yaml` | Add `yaml` dependency |
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- **Unit test:** Write a test recipe text to a temp file, run
|
||||||
|
`convert-to-cooklang` manually, verify Cooklang output with valid front
|
||||||
|
matter.
|
||||||
|
- **Manual:** Visit `/import`, paste recipe text, verify the recipe appears
|
||||||
|
with correct metadata.
|
||||||
|
- **Manual:** Upload a recipe PDF, verify the `.cook` and original PDF are
|
||||||
|
saved to the recipe directory.
|
||||||
|
- **Manual:** Visit `/import` without a config file, verify the UI shows a
|
||||||
|
helpful message about LLM features being unavailable.
|
||||||
|
- **Manual:** Enter an invalid URL, verify URL scraper still returns a useful
|
||||||
|
error (regression test).
|
||||||
|
|
||||||
|
## Future Considerations
|
||||||
|
|
||||||
|
- **Image support** — once Claude API supports images in the same Messages API
|
||||||
|
format, no script changes needed; just add `image/*` to the accept attribute.
|
||||||
|
Actually, Claude already supports this — so it's included from day one.
|
||||||
|
- **Prompt iteration** — the prompts in `prompts/` can be tuned independently
|
||||||
|
of the code.
|
||||||
|
- **Multiple API providers** — the `base_url` config makes it easy to swap
|
||||||
|
providers (Ollama, Google Gemini via Anthropic-compatible proxy, etc.).
|
||||||
@@ -83,3 +83,4 @@ tests:
|
|||||||
- tasty-golden
|
- tasty-golden
|
||||||
- tasty-hspec
|
- tasty-hspec
|
||||||
- tasty-hunit
|
- tasty-hunit
|
||||||
|
- temporary
|
||||||
|
|||||||
@@ -143,6 +143,7 @@ test-suite roux-server-test
|
|||||||
type: exitcode-stdio-1.0
|
type: exitcode-stdio-1.0
|
||||||
main-is: Spec.hs
|
main-is: Spec.hs
|
||||||
other-modules:
|
other-modules:
|
||||||
|
Roux.ConfigSpec
|
||||||
Roux.NYTimesSpec
|
Roux.NYTimesSpec
|
||||||
Roux.ParserSpec
|
Roux.ParserSpec
|
||||||
Roux.SchemaOrgSpec
|
Roux.SchemaOrgSpec
|
||||||
@@ -179,6 +180,7 @@ test-suite roux-server-test
|
|||||||
, tasty-golden
|
, tasty-golden
|
||||||
, tasty-hspec
|
, tasty-hspec
|
||||||
, tasty-hunit
|
, tasty-hunit
|
||||||
|
, temporary
|
||||||
, text
|
, text
|
||||||
, time
|
, time
|
||||||
, wai
|
, wai
|
||||||
|
|||||||
+11
-9
@@ -50,23 +50,25 @@ rouxConfigSchema =
|
|||||||
constructor RouxConfig
|
constructor RouxConfig
|
||||||
#+ optionalNullable FC.EmitNull "anthropic" rcAnthropic anthropicConfigSchema
|
#+ optionalNullable FC.EmitNull "anthropic" rcAnthropic anthropicConfigSchema
|
||||||
|
|
||||||
{- | Load config from a YAML file. Returns config with nothing set if the file
|
{- | Load config from a YAML file. Returns 'Right' with config using 'Nothing' for all
|
||||||
doesn't exist (LLM features will be unavailable but the server still runs).
|
fields if the file doesn't exist (LLM features will be unavailable but the server still
|
||||||
Exits with an error if the file exists but is malformed.
|
runs). Returns 'Left' with an error message if the file exists but cannot be parsed or
|
||||||
|
decoded.
|
||||||
-}
|
-}
|
||||||
loadConfig :: FilePath -> IO RouxConfig
|
loadConfig :: FilePath -> IO (Either String RouxConfig)
|
||||||
loadConfig path = do
|
loadConfig path = do
|
||||||
exists <- doesFileExist path
|
exists <- doesFileExist path
|
||||||
if not exists
|
if not exists
|
||||||
then pure (RouxConfig Nothing)
|
then pure (Right (RouxConfig Nothing))
|
||||||
else do
|
else do
|
||||||
result <- Y.decodeFileEither path
|
result <- Y.decodeFileEither path
|
||||||
case result of
|
case result of
|
||||||
Left parseErr ->
|
Left parseErr ->
|
||||||
fail $ "Failed to parse " <> path <> ": " <> show parseErr
|
pure $ Left $ "Failed to parse " <> path <> ": " <> show parseErr
|
||||||
Right val ->
|
Right val ->
|
||||||
case FleeceD.fromValue rouxConfigSchema val of
|
case FleeceD.fromValue rouxConfigSchema val of
|
||||||
Right cfg -> pure cfg
|
Right cfg -> pure (Right cfg)
|
||||||
Left decodeErr ->
|
Left decodeErr ->
|
||||||
fail $
|
pure $
|
||||||
"Invalid config format in " <> path <> ": " <> show decodeErr
|
Left $
|
||||||
|
"Invalid config format in " <> path <> ": " <> show decodeErr
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
module Roux.ConfigSpec (spec) where
|
||||||
|
|
||||||
|
import System.FilePath ((</>))
|
||||||
|
import System.IO.Temp (withSystemTempDirectory)
|
||||||
|
import Test.Hspec (Spec, describe, expectationFailure, it, shouldBe)
|
||||||
|
|
||||||
|
import Roux.Config
|
||||||
|
|
||||||
|
spec :: Spec
|
||||||
|
spec = describe "loadConfig" $ do
|
||||||
|
it "returns Nothing config when file doesn't exist" $ do
|
||||||
|
result <- loadConfig "/tmp/nonexistent-config-file-for-roux-test.yaml"
|
||||||
|
result `shouldBe` Right (RouxConfig Nothing)
|
||||||
|
|
||||||
|
it "parses a minimal config with api_key" $ do
|
||||||
|
withSystemTempDirectory "roux-test" $ \dir -> do
|
||||||
|
let path = dir </> "config.yaml"
|
||||||
|
writeFile path "anthropic:\n api_key: sk-test-123\n"
|
||||||
|
result <- loadConfig path
|
||||||
|
case result of
|
||||||
|
Right (RouxConfig (Just ac)) -> do
|
||||||
|
acApiKey ac `shouldBe` "sk-test-123"
|
||||||
|
acBaseUrl ac `shouldBe` Nothing
|
||||||
|
acModel ac `shouldBe` Nothing
|
||||||
|
_ -> expectationFailure "Expected Right config"
|
||||||
|
|
||||||
|
it "parses a full config with all fields" $ do
|
||||||
|
withSystemTempDirectory "roux-test" $ \dir -> do
|
||||||
|
let path = dir </> "config.yaml"
|
||||||
|
writeFile path "anthropic:\n api_key: sk-test-456\n base_url: http://localhost:11434/v1\n model: local-model\n"
|
||||||
|
result <- loadConfig path
|
||||||
|
case result of
|
||||||
|
Right (RouxConfig (Just ac)) -> do
|
||||||
|
acApiKey ac `shouldBe` "sk-test-456"
|
||||||
|
acBaseUrl ac `shouldBe` Just "http://localhost:11434/v1"
|
||||||
|
acModel ac `shouldBe` Just "local-model"
|
||||||
|
_ -> expectationFailure "Expected Right config"
|
||||||
|
|
||||||
|
it "returns Left for malformed YAML" $ do
|
||||||
|
withSystemTempDirectory "roux-test" $ \dir -> do
|
||||||
|
let path = dir </> "config.yaml"
|
||||||
|
writeFile path "{{{{ invalid yaml }}}}\n"
|
||||||
|
result <- loadConfig path
|
||||||
|
case result of
|
||||||
|
Left _ -> pure () -- expected
|
||||||
|
Right _ -> expectationFailure "Expected Left error"
|
||||||
|
|
||||||
|
it "returns Left when api_key is missing" $ do
|
||||||
|
withSystemTempDirectory "roux-test" $ \dir -> do
|
||||||
|
let path = dir </> "config.yaml"
|
||||||
|
writeFile path "anthropic:\n model: foo\n"
|
||||||
|
result <- loadConfig path
|
||||||
|
case result of
|
||||||
|
Left _ -> pure () -- expected
|
||||||
|
Right _ -> expectationFailure "Expected Left error"
|
||||||
+4
-1
@@ -3,19 +3,22 @@ module Main (main) where
|
|||||||
import Test.Tasty (defaultMain, testGroup)
|
import Test.Tasty (defaultMain, testGroup)
|
||||||
import Test.Tasty.Hspec (testSpec)
|
import Test.Tasty.Hspec (testSpec)
|
||||||
|
|
||||||
|
import qualified Roux.ConfigSpec
|
||||||
import qualified Roux.NYTimesSpec
|
import qualified Roux.NYTimesSpec
|
||||||
import qualified Roux.ParserSpec
|
import qualified Roux.ParserSpec
|
||||||
import qualified Roux.SchemaOrgSpec
|
import qualified Roux.SchemaOrgSpec
|
||||||
|
|
||||||
main :: IO ()
|
main :: IO ()
|
||||||
main = do
|
main = do
|
||||||
|
configTests <- testSpec "Config" Roux.ConfigSpec.spec
|
||||||
parserTests <- testSpec "Parser" Roux.ParserSpec.spec
|
parserTests <- testSpec "Parser" Roux.ParserSpec.spec
|
||||||
nytTests <- testSpec "NYTimes" Roux.NYTimesSpec.spec
|
nytTests <- testSpec "NYTimes" Roux.NYTimesSpec.spec
|
||||||
schemaOrgTests <- testSpec "SchemaOrg" Roux.SchemaOrgSpec.spec
|
schemaOrgTests <- testSpec "SchemaOrg" Roux.SchemaOrgSpec.spec
|
||||||
defaultMain $
|
defaultMain $
|
||||||
testGroup
|
testGroup
|
||||||
"roux-server"
|
"roux-server"
|
||||||
[ parserTests
|
[ configTests
|
||||||
|
, parserTests
|
||||||
, nytTests
|
, nytTests
|
||||||
, schemaOrgTests
|
, schemaOrgTests
|
||||||
]
|
]
|
||||||
|
|||||||
Reference in New Issue
Block a user