refactor: return Either from loadConfig, add Config tests
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,258 @@
|
||||
# LLM-Powered Recipe Import
|
||||
|
||||
**Date:** 2026-05-21
|
||||
**Status:** Approved
|
||||
|
||||
## Problem
|
||||
|
||||
The current `/import` route only handles recipes from URLs with structured
|
||||
schema.org JSON-LD data. There is no way to import recipes from:
|
||||
|
||||
- Free-form text (e.g. a recipe emailed by a family member)
|
||||
- PDF documents (e.g. scanned cookbook pages, saved recipe PDFs)
|
||||
- Image files (e.g. a photo of a recipe card)
|
||||
|
||||
Many recipes exist in these unstructured formats, and manually transcribing
|
||||
them into Cooklang is tedious.
|
||||
|
||||
## Goal
|
||||
|
||||
Add Anthropic API (or any Anthropic-compatible API) integration to convert
|
||||
unstructured recipe content into Cooklang format. The existing URL scraping
|
||||
path remains unchanged. All three input methods (URL, text, file upload) are
|
||||
available on the same `/import` page.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
User submits URL / text / file on /import
|
||||
↓
|
||||
Server dispatches based on input type:
|
||||
- URL → runImportPipeline (existing scrape-recipe)
|
||||
- Text → runLLMPipeline via scripts/convert-to-cooklang --text "..."
|
||||
- File → save file to recipe dir, then runLLMPipeline via --file <path>
|
||||
↓
|
||||
scripts/convert-to-cooklang sends content to Anthropic API
|
||||
using existing cooklang-converter prompts
|
||||
↓
|
||||
script outputs Cooklang text on stdout
|
||||
↓
|
||||
Server writes .cook file (and keeps original file for source linking)
|
||||
↓
|
||||
Redirect to /recipes/<filename>
|
||||
```
|
||||
|
||||
## Config File
|
||||
|
||||
A YAML config file, defaulting to `./roux-config.yaml` in the working
|
||||
directory, overridable via `--config-file` CLI flag.
|
||||
|
||||
```yaml
|
||||
# roux-config.yaml
|
||||
anthropic:
|
||||
api_key: sk-ant-... # required for LLM features
|
||||
base_url: https://api.anthropic.com # optional, defaults to https://api.anthropic.com
|
||||
model: claude-sonnet-4-20250514 # optional, defaults to latest Claude Sonnet
|
||||
```
|
||||
|
||||
If the config file doesn't exist or `anthropic.api_key` is missing, the
|
||||
LLM-powered import features are unavailable (the form shows an error message)
|
||||
but the URL scraper still works.
|
||||
|
||||
### New module: `Roux.Config`
|
||||
|
||||
```haskell
|
||||
data RouxConfig = RouxConfig
|
||||
{ rcAnthropic :: Maybe AnthropicConfig
|
||||
}
|
||||
|
||||
data AnthropicConfig = AnthropicConfig
|
||||
{ acApiKey :: Text
|
||||
, acBaseUrl :: Maybe Text -- default: https://api.anthropic.com
|
||||
, acModel :: Maybe Text -- default: latest Claude Sonnet
|
||||
}
|
||||
|
||||
loadConfig :: FilePath -> IO RouxConfig
|
||||
```
|
||||
|
||||
Loads and validates the YAML file. Missing file = `RouxConfig Nothing`.
|
||||
Malformed file = runtime error at startup (fail fast).
|
||||
|
||||
## Subprocess Script: `scripts/convert-to-cooklang`
|
||||
|
||||
**Language:** Python (consistent with `scrape-recipe`)
|
||||
|
||||
**Interface:**
|
||||
```
|
||||
scripts/convert-to-cooklang --config <path> [--file <path> | --text "<text>" | --stdin]
|
||||
```
|
||||
|
||||
Exactly one input mode required.
|
||||
|
||||
### Input modes
|
||||
|
||||
- `--text "<recipe text>"` — inline recipe text
|
||||
- `--stdin` — read recipe text from stdin
|
||||
- `--file <path>` — PDF or image file
|
||||
|
||||
### Behavior
|
||||
|
||||
1. Reads the config file (YAML) for `api_key`, optional `base_url`, optional
|
||||
`model`.
|
||||
2. Builds the Anthropic Messages API request:
|
||||
- System prompt: contents of `prompts/system-prompt.md`
|
||||
- User message: the Cooklang converter instructions from
|
||||
`prompts/cooklang-converter.md` followed by the recipe content
|
||||
- See "Prompt composition" section below for details.
|
||||
3. For `--text`/`--stdin`: sends a `{"type": "text", "text": "..."}` content block
|
||||
4. For `--file` (PDF/images): reads the file, base64-encodes it, sends as:
|
||||
- PDF: `{"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}}`
|
||||
- JPEG: `{"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": "..."}}`
|
||||
- PNG: `{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}}`
|
||||
5. Validates the response is valid Cooklang (basic sanity: starts with `---`
|
||||
front matter marker). On failure, exits with error on stderr.
|
||||
6. Outputs **only** the Cooklang text on stdout.
|
||||
7. Exits with code 0 on success, non-zero on error (error details on stderr).
|
||||
|
||||
### Prompt composition
|
||||
|
||||
The script reads two files relative to the project root:
|
||||
|
||||
- **`prompts/system-prompt.md`** — set as the Anthropic Messages API `system` parameter
|
||||
- **`prompts/cooklang-converter.md`** — prepended to the user message as context,
|
||||
followed by the recipe content wrapped in the template from
|
||||
`prompts/user-prompt-template.md`
|
||||
|
||||
This keeps the prompts separate from the script, making them easy to iterate
|
||||
on without touching code.
|
||||
|
||||
### Dependencies
|
||||
|
||||
```
|
||||
anthropic
|
||||
PyYAML
|
||||
```
|
||||
|
||||
Added to the Docker image or a `requirements.txt`.
|
||||
|
||||
## Server-Side Changes
|
||||
|
||||
### CLI: `--config-file` flag in `app/Main.hs`
|
||||
|
||||
```
|
||||
roux-server \
|
||||
--recipe-dir ./recipes \
|
||||
--config-file ./roux-config.yaml
|
||||
```
|
||||
|
||||
Default: `./roux-config.yaml` in the working directory.
|
||||
|
||||
### Expanded `POST /import` handler
|
||||
|
||||
The handler inspects the form submission and dispatches accordingly:
|
||||
|
||||
| User action | Dispatched to |
|
||||
|---|---|
|
||||
| Submits URL only | `runImportPipeline` (existing) |
|
||||
| Uploads file | Saves file, then `runLLMPipeline --file` |
|
||||
| Pastes text | `runLLMPipeline --text` |
|
||||
| URL + file/text | File/text takes priority (URL ignored) |
|
||||
|
||||
### New `runLLMPipeline` in `Roux.Server`
|
||||
|
||||
```haskell
|
||||
runLLMPipeline ::
|
||||
FilePath -- recipe directory
|
||||
-> AnthropicConfig -- API key, base_url, model
|
||||
-> Maybe FilePath -- Just path = uploaded file to keep as source
|
||||
-> Text -- recipe text (or empty if file)
|
||||
-> IO (Either Html.ImportError FilePath)
|
||||
```
|
||||
|
||||
1. Write the uploaded source file to the recipe directory (for PDFs/images —
|
||||
linked as source from the Cooklang metadata).
|
||||
2. Call `scripts/convert-to-cooklang` as a subprocess.
|
||||
3. Capture stdout (Cooklang text) and stderr (error messages).
|
||||
4. Derive filename from the recipe title in the generated Cooklang front
|
||||
matter.
|
||||
5. Write the `.cook` file.
|
||||
6. Return filename for redirect.
|
||||
|
||||
### Source file linking
|
||||
|
||||
When a PDF or image is uploaded, the original file is saved alongside the
|
||||
`.cook` file in the recipe directory. The Cooklang front matter includes:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: ...
|
||||
source: filename.pdf # relative path to the original source
|
||||
---
|
||||
```
|
||||
|
||||
This way, the recipe page links to the original document for reference.
|
||||
|
||||
### Expanded import form (`Html.hs`)
|
||||
|
||||
The `/import` page gets three input sections:
|
||||
|
||||
1. **URL** — existing text input, unchanged
|
||||
2. **File upload** — `<input type="file" accept=".pdf,image/*">`
|
||||
3. **Paste text** — `<textarea>` for free-form recipe text
|
||||
|
||||
A note at the top explains the three options. A small JS snippet clears the
|
||||
other inputs when one is filled (optional UX polish).
|
||||
|
||||
### Error handling
|
||||
|
||||
Follows the same pattern as `runImportPipeline`:
|
||||
|
||||
| Step | Failure mode | User sees |
|
||||
|------|-------------|-----------|
|
||||
| Config missing | No API key configured | "LLM import is not configured. Add an API key to the config file." |
|
||||
| Subprocess | Script error, API error | "Could not convert recipe: {stderr}" |
|
||||
| Output validation | Output is not valid Cooklang | "The AI returned invalid Cooklang: {preview}" |
|
||||
| File write | Permission denied, disk full | "Could not save recipe: {IO error}" |
|
||||
|
||||
## Files Changed
|
||||
|
||||
### New
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `scripts/convert-to-cooklang` | Python script for Anthropic API conversion |
|
||||
| `src/Roux/Config.hs` | Config file loading and types |
|
||||
|
||||
### Modified
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `app/Main.hs` | Add `--config-file` CLI flag |
|
||||
| `src/Roux/Server.hs` | Thread config through app, add `runLLMPipeline`, expand import handler |
|
||||
| `src/Roux/Html.hs` | Add file upload and textarea to import form |
|
||||
| `src/Roux.hs` | Re-export Config types |
|
||||
| `package.yaml` | Add `yaml` dependency |
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit test:** Write a test recipe text to a temp file, run
|
||||
`convert-to-cooklang` manually, verify Cooklang output with valid front
|
||||
matter.
|
||||
- **Manual:** Visit `/import`, paste recipe text, verify the recipe appears
|
||||
with correct metadata.
|
||||
- **Manual:** Upload a recipe PDF, verify the `.cook` and original PDF are
|
||||
saved to the recipe directory.
|
||||
- **Manual:** Visit `/import` without a config file, verify the UI shows a
|
||||
helpful message about LLM features being unavailable.
|
||||
- **Manual:** Enter an invalid URL, verify URL scraper still returns a useful
|
||||
error (regression test).
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- **Image support** — once Claude API supports images in the same Messages API
|
||||
format, no script changes needed; just add `image/*` to the accept attribute.
|
||||
Actually, Claude already supports this — so it's included from day one.
|
||||
- **Prompt iteration** — the prompts in `prompts/` can be tuned independently
|
||||
of the code.
|
||||
- **Multiple API providers** — the `base_url` config makes it easy to swap
|
||||
providers (Ollama, Google Gemini via Anthropic-compatible proxy, etc.).
|
||||
Reference in New Issue
Block a user