jbrechtel/converge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests: SSE parsing + relay event handling, update AGENTS.md

Browse Source 
- New test module test/RelaySpec.hs (20 tests):
  - extractLines: line splitting, CRLF, partial lines, empty input
  - foldLines: single/two push events, non-push ignored, comments, no-space colon,
    partial state carry-over across batches
  - parseEvent: valid push, non-push, malformed JSON, multi-line data
  - handleSSEEvent: pull on matching repo, ignore wrong gitea_repo/branch,
    skip pull when merge/rebase/operation in progress
- Exported PushEvent(..), extractLines, foldLines, parseEvent, handleSSEEvent
- Fixed data line order (append instead of prepend+reverse) and event ordering
- Updated AGENTS.md: relay architecture, Go component, new deps, file layout
This commit is contained in:
James Brechtel
2026-05-14 14:40:47 -04:00
parent 7feff5abfa
commit 662854a977
4 changed files with 360 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+44 -1
View File
@@ -34,6 +34,25 @@ All core logic lives in the single `Converge` module (`src/Converge.hs`). Key se
- **Sync cycle** (`syncRepo`): guarded by `isInMiddleOfOperation`, then stage + commit + pull + conflict check
- **Git operations**: shell out to `git` via `System.Process.readProcessWithExitCode` — no libgit2 dependency
- **Notifications**: shell out to `notify-send` via `System.Process.spawnProcess`
- **Relay listener**: `connectRelay` connects to a converge-relay SSE endpoint, parses push events, and triggers pulls on matching repos (by `gitea_repo` + branch). Runs as a background thread with exponential backoff reconnection.
### Relay architecture
```
┌─────────────┐ POST /webhook ┌──────────────────┐ GET /events (SSE) ┌──────────────────┐
│ Gitea │ ────────────────> │ converge-relay │ ───────────────────> │ converge (local) │
│ Server │ (push event) │ (Go, hosted) │ event: push │ (Haskell) │
└─────────────┘ └──────────────────┘ └─────────┬─────────┘
│
matches gitea_repo
→ git pull
```
The Go relay is a zero-dependency pub/sub server. Key design:
- `POST /webhook` — receives Gitea push events, optional HMAC-SHA256 verification
- `GET /events` — SSE stream, clients connect and hold the connection open
- Channel-based broker: `Subscribe()` → channel, `Unsubscribe(ch)`, `Publish(event)` — non-blocking send to all subscribers
- Slow subscribers are silently dropped (channel buffer = 16)
### Git operations
@@ -62,6 +81,8 @@ All core logic lives in the single `Converge` module (`src/Converge.hs`). Key se
## Build / test / run
### Haskell (converge)
```bash
stack build # Build library + executable + tests
stack test # Build and run the test suite
@@ -71,6 +92,15 @@ stack exec converge -- [options] # Run the tool directly
./scripts/run # Run the tool (uses Docker via ./hs)
```
### Go (converge-relay)
```bash
cd converge-relay
./scripts/build # Format + test + build (uses Docker via ./goctl)
./scripts/test # Run tests (uses Docker via ./goctl)
./scripts/run --listen :8080 # Run the relay (uses Docker via ./goctl)
```
## Test conventions
Tests use **hspec** and create real temporary git repositories — no mocking of git commands. This avoids complexity and tests actual git semantics.
@@ -105,6 +135,7 @@ Anytime new behavior is added the `SPEC.md` should be updated and a correspondin
| Pulls from remote origin | `GitPullSpec` | Fast-forward pull scenario |
| Rebases on pull | `GitPullSpec` | Divergent history → linear history, no merge commits |
| No-op during operations | `GitSafetySpec` | All 7 sentinel types + clean repo |
| `RelaySpec` | SSE line parsing, event folding, JSON parsing, repo matching, operation-in-progress skip |
## Key conventions
@@ -117,17 +148,29 @@ Anytime new behavior is added the `SPEC.md` should be updated and a correspondin
## Dependencies
### Haskell
| Package | Purpose |
|---|---|
| `async` | Concurrent repo watching |
| `fsnotify` | File system event monitoring |
| `optparse-applicative` | CLI argument parsing |
| `yaml` | Config file parsing |
| `http-client` | HTTP client for SSE relay connection |
| `http-client-tls` | TLS support for HTTPS relay endpoints |
| `http-types` | HTTP status codes |
| `aeson` | JSON parsing (push events from relay) |
| `bytestring` | Efficient binary data for SSE stream parsing |
| `temporary` (test only) | Temp directories for test repos |
| `hspec` (test only) | Test framework |
### Go (converge-relay)
Zero external dependencies — stdlib only (`net/http`, `log/slog`, `crypto/hmac`, `encoding/json`, `sync`).
## External requirements
- `git` must be on PATH (no libgit2 dependency)
- `notify-send` for desktop notifications (optional — only called on conflict)
- `notify-send` for desktop notifications (optional — only called on conflict/pull)
- `hostname` command for commit message generation
- For realtime pull: a running `converge-relay` instance reachable via HTTP(S)
src/Converge.hs
+10 -3
View File
@@ -37,7 +37,12 @@ module Converge (
isGitIgnored,
-- * Relay listener (SSE client)
PushEvent (..),
connectRelay,
extractLines,
foldLines,
parseEvent,
handleSSEEvent,
)
where
@@ -633,7 +638,9 @@ stripTrailingCr bs
-- | Fold a list of SSE lines into accumulated state and parsed events.
foldLines :: Maybe Text -> [BS.ByteString] -> [BS.ByteString] -> (Maybe Text, [BS.ByteString], [PushEvent])
foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLines
foldLines evtType0 dLines0 sseLines =
let (evt, dl, evts) = foldl' step (evtType0, dLines0, []) sseLines
in (evt, dl, reverse evts)
where
step (evt, dl, events) line
| BS.null line =
@@ -645,7 +652,7 @@ foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLin
| Just rest <- pick "event: " line <|> pick "event:" line =
(Just (decodeUtf8 (BS.dropWhile (== space) rest)), dl, events)
| Just rest <- pick "data: " line <|> pick "data:" line =
(evt, BS.dropWhile (== space) rest : dl, events)
(evt, dl ++ [BS.dropWhile (== space) rest], events)
| otherwise = (evt, dl, events)
pick = BS.stripPrefix
space = 32 -- ' '
@@ -653,7 +660,7 @@ foldLines evtType0 dLines0 sseLines = foldl' step (evtType0, dLines0, []) sseLin
-- | Parse accumulated SSE data lines into a 'PushEvent' (push events only).
parseEvent :: Maybe Text -> [BS.ByteString] -> Maybe PushEvent
parseEvent (Just "push") dataLines =
decode (BL.fromStrict (BS.intercalate "\n" (reverse dataLines)))
decode (BL.fromStrict (BS.intercalate "\n" dataLines))
parseEvent _ _ = Nothing
-- | Dispatch a parsed push event to matching repos (by 'rcGiteaRepo' and 'rcBranch').
test/RelaySpec.hs
+304
View File
@@ -0,0 +1,304 @@
module RelaySpec (spec) where
import Converge
( PushEvent (..),
RepoConfig (..),
extractLines,
foldLines,
handleSSEEvent,
parseEvent,
)
import qualified Data.ByteString.Char8 as BSC
import qualified Data.Text as T
import System.FilePath ((</>))
import TestHelper (commitCount, runGitIn, testLogger, withTempGitRepoWithRemote)
import Test.Hspec
spec :: Spec
spec = do
extractLinesSpec
foldLinesSpec
parseEventSpec
handleSSEEventSpec
----------------------------------------------------------------------
-- extractLines
----------------------------------------------------------------------
extractLinesSpec :: Spec
extractLinesSpec = describe "extractLines" $ do
it "splits a single line" $ do
let (lines', leftover) = extractLines (bsc "\n")
lines' `shouldBe` [bsc ""]
leftover `shouldBe` bsc ""
it "splits multiple lines" $ do
let (lines', leftover) = extractLines (bsc "hello\nworld\n")
lines' `shouldBe` [bsc "hello", bsc "world"]
leftover `shouldBe` bsc ""
it "handles CRLF line endings" $ do
let (lines', leftover) = extractLines (bsc "hello\r\nworld\r\n")
lines' `shouldBe` [bsc "hello", bsc "world"]
leftover `shouldBe` bsc ""
it "returns partial line as leftover" $ do
let (lines', leftover) = extractLines (bsc "hello\nwor")
lines' `shouldBe` [bsc "hello"]
leftover `shouldBe` bsc "wor"
it "returns empty for empty input" $ do
let (lines', leftover) = extractLines (bsc "")
lines' `shouldBe` []
leftover `shouldBe` bsc ""
it "handles line with only CR" $ do
let (lines', leftover) = extractLines (bsc "\r\n")
lines' `shouldBe` [bsc ""]
leftover `shouldBe` bsc ""
----------------------------------------------------------------------
-- foldLines
----------------------------------------------------------------------
foldLinesSpec :: Spec
foldLinesSpec = describe "foldLines" $ do
it "parses a single push event" $ do
let lines' =
[ bsc "event: push",
bsc "data: {\"full_name\":\"myorg/myrepo\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
bsc ""
]
(finalEvtType, finalDLines, events) = foldLines Nothing [] lines'
events `shouldBe`
[ PushEvent
{ peFullName = "myorg/myrepo",
peBranch = "main",
peRef = "refs/heads/main"
}
]
finalEvtType `shouldBe` Nothing
finalDLines `shouldBe` []
it "parses two push events" $ do
let lines' =
[ bsc "event: push",
bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
bsc "",
bsc "event: push",
bsc "data: {\"full_name\":\"c/d\",\"branch\":\"dev\",\"ref\":\"refs/heads/dev\"}",
bsc ""
]
(_, _, events) = foldLines Nothing [] lines'
length events `shouldBe` 2
peFullName (events !! 0) `shouldBe` "a/b"
peFullName (events !! 1) `shouldBe` "c/d"
it "ignores non-push events" $ do
let lines' =
[ bsc "event: ping",
bsc "data: hello",
bsc ""
]
(_, _, events) = foldLines Nothing [] lines'
events `shouldBe` []
it "ignores comment lines (starting with colon)" $ do
let lines' =
[ bsc ": this is a comment",
bsc "event: push",
bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
bsc ""
]
(_, _, events) = foldLines Nothing [] lines'
length events `shouldBe` 1
it "handles event: without space after colon" $ do
let lines' =
[ bsc "event:push",
bsc "data:{\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}",
bsc ""
]
(_, _, events) = foldLines Nothing [] lines'
length events `shouldBe` 1
it "carries partial state across calls" $ do
-- First batch: incomplete event (missing blank line)
let batch1 = [bsc "event: push", bsc "data: {\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
(evtType1, dLines1, events1) = foldLines Nothing [] batch1
events1 `shouldBe` []
evtType1 `shouldBe` Just "push"
dLines1 `shouldBe` [bsc "{\"full_name\":\"a/b\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
-- Second batch: completes the event
let batch2 = [bsc ""]
(_, _, events2) = foldLines evtType1 dLines1 batch2
length events2 `shouldBe` 1
----------------------------------------------------------------------
-- parseEvent
----------------------------------------------------------------------
parseEventSpec :: Spec
parseEventSpec = describe "parseEvent" $ do
it "parses a valid push event" $ do
let evt = parseEvent (Just "push") [bsc "{\"full_name\":\"org/repo\",\"branch\":\"main\",\"ref\":\"refs/heads/main\"}"]
evt `shouldBe` Just
PushEvent
{ peFullName = "org/repo",
peBranch = "main",
peRef = "refs/heads/main"
}
it "returns Nothing for non-push event type" $ do
let evt = parseEvent (Just "ping") [bsc "{}"]
evt `shouldBe` Nothing
it "returns Nothing for malformed JSON" $ do
let evt = parseEvent (Just "push") [bsc "not json"]
evt `shouldBe` Nothing
it "handles multiple data lines joined" $ do
let evt = parseEvent (Just "push")
[ bsc "{\"full_name\":\"x\"",
bsc ",\"branch\":\"y\"",
bsc ",\"ref\":\"z\"}"
]
evt `shouldBe` Just
PushEvent
{ peFullName = "x",
peBranch = "y",
peRef = "z"
}
----------------------------------------------------------------------
-- handleSSEEvent (integration — uses real git repos)
----------------------------------------------------------------------
handleSSEEventSpec :: Spec
handleSSEEventSpec = describe "handleSSEEvent" $ do
it "pulls on a matching repo when a push event arrives" $
withTempGitRepoWithRemote $ \localRepo -> do
logger <- testLogger
-- Simulate someone else pushing: commit locally, push, then reset local behind
writeFile (localRepo </> "remote-change.txt") "pushed from elsewhere"
_ <- runGitIn localRepo ["add", "remote-change.txt"]
_ <- runGitIn localRepo ["commit", "-m", "simulated remote push"]
_ <- runGitIn localRepo ["push", "origin", "main"]
-- Verify remote has the commit
(_, remoteLog, _) <- runGitIn localRepo ["log", "--oneline", "origin/main"]
let remoteCount = length (filter (not . T.null) (T.lines remoteLog))
-- Reset local repo to one commit behind
_ <- runGitIn localRepo ["reset", "--hard", "HEAD~1"]
-- Verify local is behind
localBefore <- commitCount localRepo
localBefore `shouldBe` (remoteCount - 1)
-- Create a repo config with matching gitea_repo
let repoCfg =
(defaultRepoCfg localRepo)
{ rcGiteaRepo = Just "test-org/test-repo"
}
-- Send a matching push event
let evt =
PushEvent
{ peFullName = "test-org/test-repo",
peBranch = "main",
peRef = "refs/heads/main"
}
handleSSEEvent logger [repoCfg] evt
-- Local should now have the remote's commit
localAfter <- commitCount localRepo
localAfter `shouldBe` remoteCount
it "ignores events for repos with non-matching gitea_repo" $
withTempGitRepoWithRemote $ \localRepo -> do
logger <- testLogger
let repoCfg =
(defaultRepoCfg localRepo)
{ rcGiteaRepo = Just "test-org/test-repo"
}
let evt =
PushEvent
{ peFullName = "other-org/other-repo",
peBranch = "main",
peRef = "refs/heads/main"
}
-- This should not crash or pull — it just ignores
handleSSEEvent logger [repoCfg] evt
-- Test passes if no exception is thrown
it "ignores events for repos with non-matching branch" $
withTempGitRepoWithRemote $ \localRepo -> do
logger <- testLogger
let repoCfg =
(defaultRepoCfg localRepo)
{ rcGiteaRepo = Just "test-org/test-repo",
rcBranch = "main"
}
let evt =
PushEvent
{ peFullName = "test-org/test-repo",
peBranch = "dev",
peRef = "refs/heads/dev"
}
commitsBefore <- commitCount localRepo
handleSSEEvent logger [repoCfg] evt
commitsAfter <- commitCount localRepo
commitsAfter `shouldBe` commitsBefore -- no pull should have happened
it "skips pull when an operation is in progress" $
withTempGitRepoWithRemote $ \localRepo -> do
logger <- testLogger
-- Create a MERGE_HEAD sentinel to simulate a merge in progress
(_, gitDirOut, _) <- runGitIn localRepo ["rev-parse", "--git-dir"]
let sentinelFile = localRepo </> T.unpack (T.strip gitDirOut) </> "MERGE_HEAD"
writeFile sentinelFile "fake merge head"
-- Push a new commit to remote
writeFile (localRepo </> "during-merge.txt") "change during merge"
_ <- runGitIn localRepo ["add", "during-merge.txt"]
_ <- runGitIn localRepo ["commit", "-m", "commit during merge"]
_ <- runGitIn localRepo ["push", "origin", "main"]
_ <- runGitIn localRepo ["reset", "--hard", "HEAD~1"]
commitsBefore <- commitCount localRepo
let repoCfg =
(defaultRepoCfg localRepo)
{ rcGiteaRepo = Just "test-org/test-repo"
}
let evt =
PushEvent
{ peFullName = "test-org/test-repo",
peBranch = "main",
peRef = "refs/heads/main"
}
handleSSEEvent logger [repoCfg] evt
commitsAfter <- commitCount localRepo
commitsAfter `shouldBe` commitsBefore -- pull should be skipped
----------------------------------------------------------------------
-- Helpers
----------------------------------------------------------------------
-- | Shortcut for ByteString from ASCII.
bsc :: String -> BSC.ByteString
bsc = BSC.pack
-- | Create a default RepoConfig for a given path (for testing).
defaultRepoCfg :: FilePath -> RepoConfig
defaultRepoCfg path =
RepoConfig
{ rcPath = path,
rcRemote = "origin",
rcBranch = "main",
rcNotifyOnPull = False,
rcGiteaRepo = Nothing
}
test/Spec.hs
+2
View File
@@ -3,6 +3,7 @@ module Main (main) where
import qualified GitCommitSpec
import qualified GitPullSpec
import qualified GitSafetySpec
import qualified RelaySpec
import Test.Hspec
main :: IO ()
@@ -10,3 +11,4 @@ main = hspec $ do
GitCommitSpec.spec
GitPullSpec.spec
GitSafetySpec.spec
RelaySpec.spec