tests: SSE parsing + relay event handling, update AGENTS.md
- New test module test/RelaySpec.hs (20 tests):
- extractLines: line splitting, CRLF, partial lines, empty input
- foldLines: single/two push events, non-push ignored, comments, no-space colon,
partial state carry-over across batches
- parseEvent: valid push, non-push, malformed JSON, multi-line data
- handleSSEEvent: pull on matching repo, ignore wrong gitea_repo/branch,
skip pull when merge/rebase/operation in progress
- Exported PushEvent(..), extractLines, foldLines, parseEvent, handleSSEEvent
- Fixed data line order (append instead of prepend+reverse) and event ordering
- Updated AGENTS.md: relay architecture, Go component, new deps, file layout
This commit is contained in:
@@ -34,6 +34,25 @@ All core logic lives in the single `Converge` module (`src/Converge.hs`). Key se
|
||||
- **Sync cycle** (`syncRepo`): guarded by `isInMiddleOfOperation`, then stage + commit + pull + conflict check
|
||||
- **Git operations**: shell out to `git` via `System.Process.readProcessWithExitCode` — no libgit2 dependency
|
||||
- **Notifications**: shell out to `notify-send` via `System.Process.spawnProcess`
|
||||
- **Relay listener**: `connectRelay` connects to a converge-relay SSE endpoint, parses push events, and triggers pulls on matching repos (by `gitea_repo` + branch). Runs as a background thread with exponential backoff reconnection.
|
||||
|
||||
### Relay architecture
|
||||
|
||||
```
|
||||
┌─────────────┐ POST /webhook ┌──────────────────┐ GET /events (SSE) ┌──────────────────┐
|
||||
│ Gitea │ ────────────────> │ converge-relay │ ───────────────────> │ converge (local) │
|
||||
│ Server │ (push event) │ (Go, hosted) │ event: push │ (Haskell) │
|
||||
└─────────────┘ └──────────────────┘ └─────────┬─────────┘
|
||||
│
|
||||
matches gitea_repo
|
||||
→ git pull
|
||||
```
|
||||
|
||||
The Go relay is a zero-dependency pub/sub server. Key design:
|
||||
- `POST /webhook` — receives Gitea push events, optional HMAC-SHA256 verification
|
||||
- `GET /events` — SSE stream, clients connect and hold the connection open
|
||||
- Channel-based broker: `Subscribe()` → channel, `Unsubscribe(ch)`, `Publish(event)` — non-blocking send to all subscribers
|
||||
- Slow subscribers are silently dropped (channel buffer = 16)
|
||||
|
||||
### Git operations
|
||||
|
||||
@@ -62,6 +81,8 @@ All core logic lives in the single `Converge` module (`src/Converge.hs`). Key se
|
||||
|
||||
## Build / test / run
|
||||
|
||||
### Haskell (converge)
|
||||
|
||||
```bash
|
||||
stack build # Build library + executable + tests
|
||||
stack test # Build and run the test suite
|
||||
@@ -71,6 +92,15 @@ stack exec converge -- [options] # Run the tool directly
|
||||
./scripts/run # Run the tool (uses Docker via ./hs)
|
||||
```
|
||||
|
||||
### Go (converge-relay)
|
||||
|
||||
```bash
|
||||
cd converge-relay
|
||||
./scripts/build # Format + test + build (uses Docker via ./goctl)
|
||||
./scripts/test # Run tests (uses Docker via ./goctl)
|
||||
./scripts/run --listen :8080 # Run the relay (uses Docker via ./goctl)
|
||||
```
|
||||
|
||||
## Test conventions
|
||||
|
||||
Tests use **hspec** and create real temporary git repositories — no mocking of git commands. This avoids complexity and tests actual git semantics.
|
||||
@@ -105,6 +135,7 @@ Anytime new behavior is added the `SPEC.md` should be updated and a correspondin
|
||||
| Pulls from remote origin | `GitPullSpec` | Fast-forward pull scenario |
|
||||
| Rebases on pull | `GitPullSpec` | Divergent history → linear history, no merge commits |
|
||||
| No-op during operations | `GitSafetySpec` | All 7 sentinel types + clean repo |
|
||||
| `RelaySpec` | SSE line parsing, event folding, JSON parsing, repo matching, operation-in-progress skip |
|
||||
|
||||
## Key conventions
|
||||
|
||||
@@ -117,17 +148,29 @@ Anytime new behavior is added the `SPEC.md` should be updated and a correspondin
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Haskell
|
||||
|
||||
| Package | Purpose |
|
||||
|---|---|
|
||||
| `async` | Concurrent repo watching |
|
||||
| `fsnotify` | File system event monitoring |
|
||||
| `optparse-applicative` | CLI argument parsing |
|
||||
| `yaml` | Config file parsing |
|
||||
| `http-client` | HTTP client for SSE relay connection |
|
||||
| `http-client-tls` | TLS support for HTTPS relay endpoints |
|
||||
| `http-types` | HTTP status codes |
|
||||
| `aeson` | JSON parsing (push events from relay) |
|
||||
| `bytestring` | Efficient binary data for SSE stream parsing |
|
||||
| `temporary` (test only) | Temp directories for test repos |
|
||||
| `hspec` (test only) | Test framework |
|
||||
|
||||
### Go (converge-relay)
|
||||
|
||||
Zero external dependencies — stdlib only (`net/http`, `log/slog`, `crypto/hmac`, `encoding/json`, `sync`).
|
||||
|
||||
## External requirements
|
||||
|
||||
- `git` must be on PATH (no libgit2 dependency)
|
||||
- `notify-send` for desktop notifications (optional — only called on conflict)
|
||||
- `notify-send` for desktop notifications (optional — only called on conflict/pull)
|
||||
- `hostname` command for commit message generation
|
||||
- For realtime pull: a running `converge-relay` instance reachable via HTTP(S)
|
||||
|
||||
+10
-3
@@ -37,7 +37,12 @@ module Converge (
|
||||
isGitIgnored,
|
||||
|
||||
-- * Relay listener (SSE client)
|
||||
PushEvent (..),
|
||||
connectRelay,
|
||||
extractLines,
|
||||
foldLines,
|
||||
parseEvent,
|
||||
handleSSEEvent,
|
||||
)
|
||||
where
|
||||
|
||||
@@ -633,7 +638,9 @@ stripTrailingCr bs
|
||||
|
||||
-- | Fold a list of SSE lines into accumulated state and parsed events.
|
||||
foldLines :: Maybe Text -> [BS.ByteString] -> [BS.ByteString] -> (Maybe Text, [BS.ByteString], [PushEvent])
|
||||
foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLines
|
||||
foldLines evtType0 dLines0 sseLines =
|
||||
let (evt, dl, evts) = foldl' step (evtType0, dLines0, []) sseLines
|
||||
in (evt, dl, reverse evts)
|
||||
where
|
||||
step (evt, dl, events) line
|
||||
| BS.null line =
|
||||
@@ -645,7 +652,7 @@ foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLin
|
||||
| Just rest <- pick "event: " line <|> pick "event:" line =
|
||||
(Just (decodeUtf8 (BS.dropWhile (== space) rest)), dl, events)
|
||||
| Just rest <- pick "data: " line <|> pick "data:" line =
|
||||
(evt, BS.dropWhile (== space) rest : dl, events)
|
||||
(evt, dl ++ [BS.dropWhile (== space) rest], events)
|
||||
| otherwise = (evt, dl, events)
|
||||
pick = BS.stripPrefix
|
||||
space = 32 -- ' '
|
||||
@@ -653,7 +660,7 @@ foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLin
|
||||
-- | Parse accumulated SSE data lines into a 'PushEvent' (push events only).
|
||||
parseEvent :: Maybe Text -> [BS.ByteString] -> Maybe PushEvent
|
||||
parseEvent (Just "push") dataLines =
|
||||
decode (BL.fromStrict (BS.intercalate "\n" (reverse dataLines)))
|
||||
decode (BL.fromStrict (BS.intercalate "\n" dataLines))
|
||||
parseEvent _ _ = Nothing
|
||||
|
||||
-- | Dispatch a parsed push event to matching repos (by 'rcGiteaRepo' and 'rcBranch').
|
||||
|
||||
@@ -0,0 +1,304 @@
|
||||
module RelaySpec (spec) where
|
||||
|
||||
import Converge
|
||||
( PushEvent (..),
|
||||
RepoConfig (..),
|
||||
extractLines,
|
||||
foldLines,
|
||||
handleSSEEvent,
|
||||
parseEvent,
|
||||
)
|
||||
import qualified Data.ByteString.Char8 as BSC
|
||||
import qualified Data.Text as T
|
||||
import System.FilePath ((</>))
|
||||
import TestHelper (commitCount, runGitIn, testLogger, withTempGitRepoWithRemote)
|
||||
import Test.Hspec
|
||||
|
||||
spec :: Spec
|
||||
spec = do
|
||||
extractLinesSpec
|
||||
foldLinesSpec
|
||||
parseEventSpec
|
||||
handleSSEEventSpec
|
||||
|
||||
----------------------------------------------------------------------
|
||||
-- extractLines
|
||||
----------------------------------------------------------------------
|
||||
|
||||
extractLinesSpec :: Spec
|
||||
extractLinesSpec = describe "extractLines" $ do
|
||||
it "splits a single line" $ do
|
||||
let (lines', leftover) = extractLines (bsc "\n")
|
||||
lines' `shouldBe` [bsc ""]
|
||||
leftover `shouldBe` bsc ""
|
||||
|
||||
it "splits multiple lines" $ do
|
||||
let (lines', leftover) = extractLines (bsc "hello\nworld\n")
|
||||
lines' `shouldBe` [bsc "hello", bsc "world"]
|
||||
leftover `shouldBe` bsc ""
|
||||
|
||||
it "handles CRLF line endings" $ do
|
||||
let (lines', leftover) = extractLines (bsc "hello\r\nworld\r\n")
|
||||
lines' `shouldBe` [bsc "hello", bsc "world"]
|
||||
leftover `shouldBe` bsc ""
|
||||
|
||||
it "returns partial line as leftover" $ do
|
||||
let (lines', leftover) = extractLines (bsc "hello\nwor")
|
||||
lines' `shouldBe` [bsc "hello"]
|
||||
leftover `shouldBe` bsc "wor"
|
||||
|
||||
it "returns empty for empty input" $ do
|
||||
let (lines', leftover) = extractLines (bsc "")
|
||||
lines' `shouldBe` []
|
||||
leftover `shouldBe` bsc ""
|
||||
|
||||
it "handles line with only CR" $ do
|
||||
let (lines', leftover) = extractLines (bsc "\r\n")
|
||||
lines' `shouldBe` [bsc ""]
|
||||
leftover `shouldBe` bsc ""
|
||||
|
||||
----------------------------------------------------------------------
|
||||
-- foldLines
|
||||
----------------------------------------------------------------------
|
||||
|
||||
foldLinesSpec :: Spec
|
||||
foldLinesSpec = describe "foldLines" $ do
|
||||
it "parses a single push event" $ do
|
||||
let lines' =
|
||||
[ bsc "event: push",
|
||||
bsc "data: {\"full_name\":\"myorg/myrepo\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
|
||||
bsc ""
|
||||
]
|
||||
(finalEvtType, finalDLines, events) = foldLines Nothing [] lines'
|
||||
events `shouldBe`
|
||||
[ PushEvent
|
||||
{ peFullName = "myorg/myrepo",
|
||||
peBranch = "main",
|
||||
peRef = "refs/heads/main"
|
||||
}
|
||||
]
|
||||
finalEvtType `shouldBe` Nothing
|
||||
finalDLines `shouldBe` []
|
||||
|
||||
it "parses two push events" $ do
|
||||
let lines' =
|
||||
[ bsc "event: push",
|
||||
bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
|
||||
bsc "",
|
||||
bsc "event: push",
|
||||
bsc "data: {\"full_name\":\"c/d\",\"branch\":\"dev\",\"ref\":\"refs/heads/dev\"}",
|
||||
bsc ""
|
||||
]
|
||||
(_, _, events) = foldLines Nothing [] lines'
|
||||
length events `shouldBe` 2
|
||||
peFullName (events !! 0) `shouldBe` "a/b"
|
||||
peFullName (events !! 1) `shouldBe` "c/d"
|
||||
|
||||
it "ignores non-push events" $ do
|
||||
let lines' =
|
||||
[ bsc "event: ping",
|
||||
bsc "data: hello",
|
||||
bsc ""
|
||||
]
|
||||
(_, _, events) = foldLines Nothing [] lines'
|
||||
events `shouldBe` []
|
||||
|
||||
it "ignores comment lines (starting with colon)" $ do
|
||||
let lines' =
|
||||
[ bsc ": this is a comment",
|
||||
bsc "event: push",
|
||||
bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
|
||||
bsc ""
|
||||
]
|
||||
(_, _, events) = foldLines Nothing [] lines'
|
||||
length events `shouldBe` 1
|
||||
|
||||
it "handles event: without space after colon" $ do
|
||||
let lines' =
|
||||
[ bsc "event:push",
|
||||
bsc "data:{\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
|
||||
bsc ""
|
||||
]
|
||||
(_, _, events) = foldLines Nothing [] lines'
|
||||
length events `shouldBe` 1
|
||||
|
||||
it "carries partial state across calls" $ do
|
||||
-- First batch: incomplete event (missing blank line)
|
||||
let batch1 = [bsc "event: push", bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
|
||||
(evtType1, dLines1, events1) = foldLines Nothing [] batch1
|
||||
events1 `shouldBe` []
|
||||
evtType1 `shouldBe` Just "push"
|
||||
dLines1 `shouldBe` [bsc "{\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
|
||||
|
||||
-- Second batch: completes the event
|
||||
let batch2 = [bsc ""]
|
||||
(_, _, events2) = foldLines evtType1 dLines1 batch2
|
||||
length events2 `shouldBe` 1
|
||||
|
||||
----------------------------------------------------------------------
|
||||
-- parseEvent
|
||||
----------------------------------------------------------------------
|
||||
|
||||
parseEventSpec :: Spec
|
||||
parseEventSpec = describe "parseEvent" $ do
|
||||
it "parses a valid push event" $ do
|
||||
let evt = parseEvent (Just "push") [bsc "{\"full_name\":\"org/repo\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
|
||||
evt `shouldBe` Just
|
||||
PushEvent
|
||||
{ peFullName = "org/repo",
|
||||
peBranch = "main",
|
||||
peRef = "refs/heads/main"
|
||||
}
|
||||
|
||||
it "returns Nothing for non-push event type" $ do
|
||||
let evt = parseEvent (Just "ping") [bsc "{}"]
|
||||
evt `shouldBe` Nothing
|
||||
|
||||
it "returns Nothing for malformed JSON" $ do
|
||||
let evt = parseEvent (Just "push") [bsc "not json"]
|
||||
evt `shouldBe` Nothing
|
||||
|
||||
it "handles multiple data lines joined" $ do
|
||||
let evt = parseEvent (Just "push")
|
||||
[ bsc "{\"full_name\":\"x\"",
|
||||
bsc ",\"branch\":\"y\"",
|
||||
bsc ",\"ref\":\"z\"}"
|
||||
]
|
||||
evt `shouldBe` Just
|
||||
PushEvent
|
||||
{ peFullName = "x",
|
||||
peBranch = "y",
|
||||
peRef = "z"
|
||||
}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
-- handleSSEEvent (integration — uses real git repos)
|
||||
----------------------------------------------------------------------
|
||||
|
||||
handleSSEEventSpec :: Spec
|
||||
handleSSEEventSpec = describe "handleSSEEvent" $ do
|
||||
it "pulls on a matching repo when a push event arrives" $
|
||||
withTempGitRepoWithRemote $ \localRepo -> do
|
||||
logger <- testLogger
|
||||
-- Simulate someone else pushing: commit locally, push, then reset local behind
|
||||
writeFile (localRepo </> "remote-change.txt") "pushed from elsewhere"
|
||||
_ <- runGitIn localRepo ["add", "remote-change.txt"]
|
||||
_ <- runGitIn localRepo ["commit", "-m", "simulated remote push"]
|
||||
_ <- runGitIn localRepo ["push", "origin", "main"]
|
||||
|
||||
-- Verify remote has the commit
|
||||
(_, remoteLog, _) <- runGitIn localRepo ["log", "--oneline", "origin/main"]
|
||||
let remoteCount = length (filter (not . T.null) (T.lines remoteLog))
|
||||
|
||||
-- Reset local repo to one commit behind
|
||||
_ <- runGitIn localRepo ["reset", "--hard", "HEAD~1"]
|
||||
|
||||
-- Verify local is behind
|
||||
localBefore <- commitCount localRepo
|
||||
localBefore `shouldBe` (remoteCount - 1)
|
||||
|
||||
-- Create a repo config with matching gitea_repo
|
||||
let repoCfg =
|
||||
(defaultRepoCfg localRepo)
|
||||
{ rcGiteaRepo = Just "test-org/test-repo"
|
||||
}
|
||||
|
||||
-- Send a matching push event
|
||||
let evt =
|
||||
PushEvent
|
||||
{ peFullName = "test-org/test-repo",
|
||||
peBranch = "main",
|
||||
peRef = "refs/heads/main"
|
||||
}
|
||||
handleSSEEvent logger [repoCfg] evt
|
||||
|
||||
-- Local should now have the remote's commit
|
||||
localAfter <- commitCount localRepo
|
||||
localAfter `shouldBe` remoteCount
|
||||
|
||||
it "ignores events for repos with non-matching gitea_repo" $
|
||||
withTempGitRepoWithRemote $ \localRepo -> do
|
||||
logger <- testLogger
|
||||
let repoCfg =
|
||||
(defaultRepoCfg localRepo)
|
||||
{ rcGiteaRepo = Just "test-org/test-repo"
|
||||
}
|
||||
let evt =
|
||||
PushEvent
|
||||
{ peFullName = "other-org/other-repo",
|
||||
peBranch = "main",
|
||||
peRef = "refs/heads/main"
|
||||
}
|
||||
-- This should not crash or pull — it just ignores
|
||||
handleSSEEvent logger [repoCfg] evt
|
||||
-- Test passes if no exception is thrown
|
||||
|
||||
it "ignores events for repos with non-matching branch" $
|
||||
withTempGitRepoWithRemote $ \localRepo -> do
|
||||
logger <- testLogger
|
||||
let repoCfg =
|
||||
(defaultRepoCfg localRepo)
|
||||
{ rcGiteaRepo = Just "test-org/test-repo",
|
||||
rcBranch = "main"
|
||||
}
|
||||
let evt =
|
||||
PushEvent
|
||||
{ peFullName = "test-org/test-repo",
|
||||
peBranch = "dev",
|
||||
peRef = "refs/heads/dev"
|
||||
}
|
||||
commitsBefore <- commitCount localRepo
|
||||
handleSSEEvent logger [repoCfg] evt
|
||||
commitsAfter <- commitCount localRepo
|
||||
commitsAfter `shouldBe` commitsBefore -- no pull should have happened
|
||||
|
||||
it "skips pull when an operation is in progress" $
|
||||
withTempGitRepoWithRemote $ \localRepo -> do
|
||||
logger <- testLogger
|
||||
-- Create a MERGE_HEAD sentinel to simulate a merge in progress
|
||||
(_, gitDirOut, _) <- runGitIn localRepo ["rev-parse", "--git-dir"]
|
||||
let sentinelFile = localRepo </> T.unpack (T.strip gitDirOut) </> "MERGE_HEAD"
|
||||
writeFile sentinelFile "fake merge head"
|
||||
|
||||
-- Push a new commit to remote
|
||||
writeFile (localRepo </> "during-merge.txt") "change during merge"
|
||||
_ <- runGitIn localRepo ["add", "during-merge.txt"]
|
||||
_ <- runGitIn localRepo ["commit", "-m", "commit during merge"]
|
||||
_ <- runGitIn localRepo ["push", "origin", "main"]
|
||||
_ <- runGitIn localRepo ["reset", "--hard", "HEAD~1"]
|
||||
|
||||
commitsBefore <- commitCount localRepo
|
||||
|
||||
let repoCfg =
|
||||
(defaultRepoCfg localRepo)
|
||||
{ rcGiteaRepo = Just "test-org/test-repo"
|
||||
}
|
||||
let evt =
|
||||
PushEvent
|
||||
{ peFullName = "test-org/test-repo",
|
||||
peBranch = "main",
|
||||
peRef = "refs/heads/main"
|
||||
}
|
||||
handleSSEEvent logger [repoCfg] evt
|
||||
|
||||
commitsAfter <- commitCount localRepo
|
||||
commitsAfter `shouldBe` commitsBefore -- pull should be skipped
|
||||
|
||||
----------------------------------------------------------------------
|
||||
-- Helpers
|
||||
----------------------------------------------------------------------
|
||||
|
||||
-- | Shortcut for ByteString from ASCII.
|
||||
bsc :: String -> BSC.ByteString
|
||||
bsc = BSC.pack
|
||||
|
||||
-- | Create a default RepoConfig for a given path (for testing).
|
||||
defaultRepoCfg :: FilePath -> RepoConfig
|
||||
defaultRepoCfg path =
|
||||
RepoConfig
|
||||
{ rcPath = path,
|
||||
rcRemote = "origin",
|
||||
rcBranch = "main",
|
||||
rcNotifyOnPull = False,
|
||||
rcGiteaRepo = Nothing
|
||||
}
|
||||
@@ -3,6 +3,7 @@ module Main (main) where
|
||||
import qualified GitCommitSpec
|
||||
import qualified GitPullSpec
|
||||
import qualified GitSafetySpec
|
||||
import qualified RelaySpec
|
||||
import Test.Hspec
|
||||
|
||||
main :: IO ()
|
||||
@@ -10,3 +11,4 @@ main = hspec $ do
|
||||
GitCommitSpec.spec
|
||||
GitPullSpec.spec
|
||||
GitSafetySpec.spec
|
||||
RelaySpec.spec
|
||||
|
||||
Reference in New Issue
Block a user