Files

T

jbrechtel 7cdd7aecad docs: add final report for SSE migration

2026-06-11 12:55:17 -04:00

19 KiB

Raw Blame History

SSE Migration Implementation Plan

Note

This document may not reflect the current implementation. See the final report for up-to-date state: Final Report

For agentic workers: REQUIRED SUB-SKILL: Use compose:subagent (recommended) or compose:execute to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace polling-based message reception with SSE (Server-Sent Events) for real-time message delivery from signal-cli.

Architecture: Create a new noise-sse.el module that implements an SSE client using Emacs process filters. The client connects to signal-cli's /api/v1/events endpoint, parses incoming SSE events, and feeds them to the existing envelope processing pipeline. Includes automatic reconnection with exponential backoff.

Tech Stack: Emacs Lisp, make-network-process, url.el for URL parsing, existing noise-receive.el envelope processing.

File Structure

File	Action	Purpose
`noise-sse.el`	Create	SSE client with process filter, reconnection logic
`noise-receive.el`	Modify	Remove polling, add SSE start/stop functions
`noise.el`	Modify	Update `noise-mode` to use SSE
`test/noise-sse-test.el`	Create	Unit tests for SSE parsing and reconnection
`test/noise-receive-test.el`	Modify	Update tests for SSE integration

Task 1: Create SSE Client Core

Covers: S1 (SSE connection management)

Files:

Create: noise-sse.el
Test: test/noise-sse-test.el
Step 1: Write failing test for SSE event parsing

;; test/noise-sse-test.el
(require 'ert)
(require 'noise-sse)

(ert-deftest noise-sse--parse-event-basic ()
  "Parse a basic SSE event with data field."
  (let ((event "data: {\"type\":\"message\",\"content\":\"hello\"}\n\n"))
    (let ((result (noise-sse--parse-event event)))
      (should (hash-table-p result))
      (should (equal (gethash "type" result) "message"))
      (should (equal (gethash "content" result) "hello")))))

(ert-deftest noise-sse--parse-event-multiline ()
  "Parse SSE event with multiline data."
  (let ((event "data: {\"type\":\"message\"\ndata: \"content\":\"hello\"}\n\n"))
    (let ((result (noise-sse--parse-event event)))
      (should (hash-table-p result))
      (should (equal (gethash "type" result) "message")))))

(ert-deftest noise-sse--parse-event-empty ()
  "Parse empty SSE event returns nil."
  (let ((event "\n\n"))
    (should (null (noise-sse--parse-event event)))))

Step 2: Run test to verify it fails

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: FAIL with "void-function noise-sse--parse-event"

Step 3: Write minimal SSE parsing implementation

;; noise-sse.el --- SSE client for signal-cli -*- lexical-binding: t; -*-

;;; Commentary:

;; Implements Server-Sent Events client for receiving real-time messages
;; from signal-cli daemon's /api/v1/events endpoint.

;;; Code:

(require 'url)
(require 'json)
(require 'cl-lib)

(defun noise-sse--parse-event (event-string)
  "Parse an SSE EVENT-STRING into a hash table.
Returns nil for empty or invalid events."
  (when (and event-string (not (string-empty-p event-string)))
    (let ((lines (split-string event-string "\n"))
          (data-lines '()))
      (dolist (line lines)
        (when (string-prefix-p "data: " line)
          (push (substring line 6) data-lines)))
      (when data-lines
        (let ((json-str (mapconcat #'identity (nreverse data-lines) "")))
          (condition-case nil
              (json-parse-string json-str)
            (error nil)))))))

(provide 'noise-sse)
;;; noise-sse.el ends here

Step 4: Run test to verify it passes

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: PASS

Step 5: Commit

git add noise-sse.el test/noise-sse-test.el
git commit -m "feat: add SSE event parsing for signal-cli"

Task 2: Add SSE Connection Management

Covers: S1, S2 (connection lifecycle, reconnection)

Files:

Modify: noise-sse.el
Modify: test/noise-sse-test.el
Step 1: Write failing tests for connection state

;; Add to test/noise-sse-test.el

(ert-deftest noise-sse--initial-state ()
  "SSE client starts in disconnected state."
  (let ((noise-sse--process nil)
        (noise-sse--reconnect-timer nil)
        (noise-sse--reconnect-delay 1))
    (should (null noise-sse--process))
    (should (null noise-sse--reconnect-timer))
    (should (= 1 noise-sse--reconnect-delay))))

(ert-deftest noise-sse--reconnect-delay-backoff ()
  "Reconnect delay doubles on each attempt."
  (let ((noise-sse--reconnect-delay 1))
    (noise-sse--increase-reconnect-delay)
    (should (= 2 noise-sse--reconnect-delay))
    (noise-sse--increase-reconnect-delay)
    (should (= 4 noise-sse--reconnect-delay))
    (noise-sse--increase-reconnect-delay)
    (should (= 8 noise-sse--reconnect-delay))))

(ert-deftest noise-sse--reconnect-delay-max ()
  "Reconnect delay caps at max value."
  (let ((noise-sse--reconnect-delay 32)
        (noise-sse--max-reconnect-delay 60))
    (noise-sse--increase-reconnect-delay)
    (should (= 60 noise-sse--reconnect-delay))))

Step 2: Run test to verify it fails

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: FAIL with "void-function noise-sse--increase-reconnect-delay"

Step 3: Implement connection state and reconnection logic

;; Add to noise-sse.el after requires

(defvar noise-sse--process nil
  "Active SSE network process, or nil.")

(defvar noise-sse--reconnect-timer nil
  "Timer for reconnection attempts, or nil.")

(defvar noise-sse--reconnect-delay 1
  "Current delay in seconds before next reconnection attempt.")

(defvar noise-sdefcustom noise-sse-max-reconnect-delay 60
  "Maximum delay in seconds between reconnection attempts."
  :type 'integer
  :group 'noise)

(defun noise-sse--increase-reconnect-delay ()
  "Increase reconnect delay with exponential backoff, capped at max."
  (setq noise-sse--reconnect-delay
        (min (* noise-sse--reconnect-delay 2)
             noise-sse-max-reconnect-delay)))

(defun noise-sse--reset-reconnect-delay ()
  "Reset reconnect delay to initial value after successful connection."
  (setq noise-sse--reconnect-delay 1))

Step 4: Run test to verify it passes

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: PASS

Step 5: Commit

git add noise-sse.el test/noise-sse-test.el
git commit -m "feat: add SSE connection state and reconnection logic"

Task 3: Implement Process Filter for SSE Stream

Covers: S1, S3 (real-time event processing)

Files:

Modify: noise-sse.el
Modify: test/noise-sse-test.el
Step 1: Write failing test for process filter

;; Add to test/noise-sse-test.el

(ert-deftest noise-sse--filter-processes-complete-event ()
  "Process filter parses and calls handler for complete events."
  (let ((processed-events '()))
    (cl-letf (((symbol-function 'noise-sse--handle-event)
               (lambda (event) (push event processed-events))))
      (let ((noise-sse--buffer ""))
        (noise-sse--filter nil "data: {\"type\":\"message\"}\n\n")
        (should (= 1 (length processed-events)))
        (should (hash-table-p (car processed-events)))))))

(ert-deftest noise-sse--filter-buffers-incomplete-event ()
  "Process filter buffers incomplete events until newline."
  (let ((processed-events '()))
    (cl-letf (((symbol-function 'noise-sse--handle-event)
               (lambda (event) (push event processed-events))))
      (let ((noise-sse--buffer ""))
        (noise-sse--filter nil "data: {\"type\":\"message\"")
        (should (= 0 (length processed-events)))
        (noise-sse--filter nil "}\n\n")
        (should (= 1 (length processed-events)))))))

Step 2: Run test to verify it fails

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: FAIL with "void-function noise-sse--filter"

Step 3: Implement process filter

;; Add to noise-sse.el

(defvar noise-sse--buffer ""
  "Buffer for partial SSE event data.")

(defun noise-sse--filter (proc string)
  "Process filter for SSE connection.
Buffers incoming STRING and processes complete events."
  (setq noise-sse--buffer (concat noise-sse--buffer string))
  (while (string-match "\n\n" noise-sse--buffer)
    (let ((event-end (match-beginning 0))
          (event-start 0))
      (let ((event-str (substring noise-sse--buffer event-start event-end)))
        (setq noise-sse--buffer
              (substring noise-sse--buffer (+ event-end 2)))
        (when-let ((event (noise-sse--parse-event event-str)))
          (noise-sse--handle-event event))))))

(defun noise-sse--handle-event (event)
  "Handle a parsed SSE EVENT.
This function will be overridden in noise-receive to process envelopes."
  (message "SSE event received: %s" event))

Step 4: Run test to verify it passes

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: PASS

Step 5: Commit

git add noise-sse.el test/noise-sse-test.el
git commit -m "feat: add SSE process filter for stream parsing"

Task 4: Implement SSE Connection Functions

Covers: S1, S2 (connection establishment, URL construction)

Files:

Modify: noise-sse.el
Modify: test/noise-sse-test.el
Step 1: Write failing test for URL construction

;; Add to test/noise-sse-test.el

(ert-deftest noise-sse--build-url ()
  "Build SSE endpoint URL from base address."
  (let ((noise-signal-cli-address "http://localhost:8080")
        (noise-account "+15551234567"))
    (let ((url (noise-sse--build-url)))
      (should (string-match-p "/api/v1/events" url))
      (should (string-match-p "account=%2B15551234567" url)))))

(ert-deftest noise-sse--build-url-no-account ()
  "Build SSE endpoint URL without account parameter."
  (let ((noise-signal-cli-address "http://localhost:8080")
        (noise-account nil))
    (let ((url (noise-sse--build-url)))
      (should (string-match-p "/api/v1/events" url))
      (should-not (string-match-p "account=" url)))))

Step 2: Run test to verify it fails

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: FAIL with "void-function noise-sse--build-url"

Step 3: Implement URL construction and connection

;; Add to noise-sse.el

(declare-variable noise-signal-cli-address)
(declare-variable noise-account)

(defun noise-sse--build-url ()
  "Build the SSE endpoint URL for signal-cli."
  (let ((base (concat noise-signal-cli-address "/api/v1/events")))
    (if (and (boundp 'noise-account) noise-account)
        (concat base "?account=" (url-hexify-string noise-account))
      base)))

(defun noise-sse--sentinel (proc status)
  "Process sentinel for SSE connection.
Handles connection close and triggers reconnection."
  (when (or (equal status "finished\n")
            (string-match-p "failed" status)
            (string-match-p "deleted" status))
    (message "Noise: SSE connection lost, reconnecting in %ds..."
             noise-sse--reconnect-delay)
    (setq noise-sse--process nil)
    (noise-sse--schedule-reconnect)))

(defun noise-sse--schedule-reconnect ()
  "Schedule a reconnection attempt after the current delay."
  (when noise-sse--reconnect-timer
    (cancel-timer noise-sse--reconnect-timer))
  (setq noise-sse--reconnect-timer
        (run-with-timer noise-sse--reconnect-delay nil
                        #'noise-sse-connect)))

Step 4: Run test to verify it passes

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: PASS

Step 5: Commit

git add noise-sse.el test/noise-sse-test.el
git commit -m "feat: add SSE URL construction and connection management"

Task 5: Implement Connect/Disconnect Functions

Covers: S1, S2 (public API, connection lifecycle)

Files:

Modify: noise-sse.el
Modify: test/noise-sse-test.el
Step 1: Write failing test for connect/disconnect

;; Add to test/noise-sse-test.el

(ert-deftest noise-sse-connect-creates-process ()
  "noise-sse-connect creates a network process."
  (let ((noise-signal-cli-address "http://localhost:8080")
        (noise-account nil)
        (noise-sse--process nil)
        (created-process nil))
    (cl-letf (((symbol-function 'make-network-process)
               (lambda (&rest args) (setq created-process t) (make-identity-process "test"))))
      (noise-sse-connect)
      (should created-process)
      (when noise-sse--process
        (delete-process noise-sse--process)))))

(ert-deftest noise-sse-disconnect-cleans-up ()
  "noise-sse-disconnect cleans up process and timer."
  (let ((noise-sse--process (make-identity-process "test"))
        (noise-sse--reconnect-timer (run-with-timer 1000 nil #'ignore)))
    (noise-sse-disconnect)
    (should (null noise-sse--process))
    (should (null noise-sse--reconnect-timer))))

Step 2: Run test to verify it fails

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: FAIL with "void-function noise-sse-connect"

Step 3: Implement connect/disconnect functions

;; Add to noise-sse.el

;;;###autoload
(defun noise-sse-connect ()
  "Connect to signal-cli SSE endpoint."
  (interactive)
  (noise-sse-disconnect)
  (let* ((url (noise-sse--build-url))
         (parsed (url-generic-parse-url url))
         (host (url-host parsed))
         (port (or (url-port parsed) 80))
         (proc (make-network-process
                :name "noise-sse"
                :host host
                :port port
                :filter #'noise-sse--filter
                :sentinel #'noise-sse--sentinel)))
    (setq noise-sse--process proc)
    (noise-sse--reset-reconnect-delay)
    (process-send-string proc
                         (format "GET %s HTTP/1.1\r\nHost: %s\r\nAccept: text/event-stream\r\n\r\n"
                                 (url-filename parsed)
                                 host))
    (message "Noise: SSE connected to %s" url)))

;;;###autoload
(defun noise-sse-disconnect ()
  "Disconnect from signal-cli SSE endpoint."
  (interactive)
  (when noise-sse--process
    (delete-process noise-sse--process)
    (setq noise-sse--process nil))
  (when noise-sse--reconnect-timer
    (cancel-timer noise-sse--reconnect-timer)
    (setq noise-sse--reconnect-timer nil))
  (setq noise-sse--buffer "")
  (message "Noise: SSE disconnected"))

Step 4: Run test to verify it passes

Run: emacs -batch -l ert -l test/noise-sse-test.el -f ert-run-tests-batch-and-exit Expected: PASS

Step 5: Commit

git add noise-sse.el test/noise-sse-test.el
git commit -m "feat: add SSE connect/disconnect public API"

Task 6: Integrate SSE with Message Reception

Covers: S1, S3, S4 (integration with existing envelope processing)

Files:

Modify: noise-receive.el
Modify: noise-sse.el
Step 1: Add SSE event handler to noise-receive.el

;; Add to noise-receive.el after requires

(require 'noise-sse)

(defun noise-receive--handle-sse-event (event)
  "Handle an SSE EVENT from signal-cli.
Processes the envelope and updates UI."
  (noise-db-init)
  (when-let ((envelope (noise-receive--field event "envelope")))
    (when-let ((conv-id (noise-receive--process-envelope envelope)))
      (noise-conversation-refresh-for conv-id)
      (when-let ((buf (get-buffer "*signal*")))
        (with-current-buffer buf
          (noise-chat-list--render))))))

;; Override the default handler in noise-sse
(setq noise-sse--handle-event #'noise-receive--handle-sse-event)

Step 2: Add SSE start/stop functions

;; Add to noise-receive.el

;;;###autoload
(defun noise-receive-start-sse ()
  "Start receiving messages via SSE."
  (interactive)
  (noise-sse-connect))

;;;###autoload
(defun noise-receive-stop-sse ()
  "Stop receiving messages via SSE."
  (interactive)
  (noise-sse-disconnect))

Step 3: Remove polling functions

;; Remove from noise-receive.el:
;; - noise-receive-interval defcustom
;; - noise-receive--timer defvar
;; - noise-receive--last-poll-error defvar
;; - noise-receive-start-polling function
;; - noise-receive-stop-polling function
;; - noise-receive--poll function

Step 4: Update tests

;; Update test/noise-receive-poll-test.el to test SSE integration
;; or rename to test/noise-receive-test.el

Step 5: Commit

git add noise-receive.el noise-sse.el
git commit -m "feat: integrate SSE with message reception pipeline"

Task 7: Update Noise Mode

Covers: S4 (user-facing mode toggle)

Files:

Modify: noise.el
Step 1: Update noise-mode to use SSE

;; In noise.el, update noise-mode definition:

(define-minor-mode noise-mode
  "Toggle Noise Signal client mode."
  :global t
  :lighter " Noise"
  :group 'noise
  (if noise-mode
      (progn
        (noise-chat-list)
        (noise-receive-start-sse))
    (noise-receive-stop-sse)
    (message "Noise mode disabled")))

Step 2: Add SSE configuration options

;; Add to noise.el defcustom section:

(defcustom noise-sse-max-reconnect-delay 60
  "Maximum delay in seconds between SSE reconnection attempts."
  :type 'integer
  :group 'noise)

Step 3: Commit

git add noise.el
git commit -m "feat: update noise-mode to use SSE instead of polling"

Task 8: Documentation and Cleanup

Covers: S5 (documentation updates)

Files:

Modify: README.md
Modify: AGENTS.md
Step 1: Update README.md

## How it works

Noise uses Server-Sent Events (SSE) to receive messages in real-time from
signal-cli. When a message arrives, it's instantly processed and displayed.

### SSE Connection

- Connects to `signal-cli` at `/api/v1/events`
- Automatic reconnection with exponential backoff on disconnect
- Configurable via `noise-sse-max-reconnect-delay`

### Manual Refresh

You can manually refresh messages with `M-x noise-receive` even while
SSE is running.

Step 2: Update AGENTS.md

## Architecture

- Uses SSE (Server-Sent Events) for real-time message reception
- `noise-sse.el` handles connection and event parsing
- `noise-receive.el` processes envelopes and updates UI
- Automatic reconnection with exponential backoff

Step 3: Commit

git add README.md AGENTS.md
git commit -m "docs: update documentation for SSE migration"

Self-Review Checklist

Spec coverage: All design requirements covered by tasks
Placeholder scan: No TBD/TODO/placeholders in steps
Type consistency: Function names and signatures consistent across tasks
Test coverage: Each component has corresponding tests
Commit frequency: Each task has meaningful commit points

19 KiB Raw Blame History

SSE Migration Implementation Plan

File Structure

Task 1: Create SSE Client Core

Task 2: Add SSE Connection Management

Task 3: Implement Process Filter for SSE Stream

Task 4: Implement SSE Connection Functions

Task 5: Implement Connect/Disconnect Functions

Task 6: Integrate SSE with Message Reception

Task 7: Update Noise Mode

Task 8: Documentation and Cleanup

Self-Review Checklist

19 KiB

Raw Blame History