Engines

What each engine does, what it speaks, and how they work together. The measured comparison with the systems they replace is in parity.md.

	Engine	Speaks	Code
	KeelStore	–	engines/keelstore
	CoreStore	HTTPS/JSON, PostgreSQL protocol	engines/propus-core
	FlashCache	Redis protocol (RESP2/3), HTTPS	engines/propus-cache
	PulseStream	Kafka protocol, HTTPS, WebSocket	engines/propus-stream
	ObjectCore	S3 API, HTTPS	engines/propus-objects
	TrackSense	HTTPS, @@ in SQL	engines/propus-search
	AutoGraph	TypeScript, Python, Java, C#	sdks/*
	Migration Engine	from PostgreSQL, MySQL, SQLite, Redis, Kafka, S3	engines/propus-migrate
	Control Center	browser	console/

#KeelStore

Propus's own storage engine. Copy-on-write B+trees in one file per database, MVCC snapshots (readers never block the writer), checksummed pages (damage is reported, never served), crash-safe commits, and a page cache in memory. A server's data directory holds system.propus (the console account, server admin keys, the list of databases) and databases/<name>.propus.

#CoreStore

SQL. CREATE/ALTER/DROP TABLE, unique and secondary indexes (also on expressions, partial with WHERE, with INCLUDE), INSERT … RETURNING, ON CONFLICT DO NOTHING/DO UPDATE, UPDATE, DELETE, joins, WHERE, GROUP BY, HAVING, aggregates, subqueries, WITH, UNION, CASE, LIKE/ILIKE, IN, BETWEEN, casts and $1 parameters, with three-valued NULL logic. Window functions (row_number, rank, dense_rank, percent_rank, cume_dist, ntile, lag, lead, first/last/nth_value, every aggregate OVER (…) with ROWS/RANGE/GROUPS frames and named windows), aggregates with FILTER, ORDER BY and WITHIN GROUP.

Views. CREATE [OR REPLACE] VIEW, run as the caller so row-level security still applies; dependencies are tracked. CREATE MATERIALIZED VIEW with REFRESH.

Functions and triggers. CREATE FUNCTION and CREATE PROCEDURE in SQL or PL/pgSQL (variables, IF, loops, FOR … IN query, RETURN NEXT/QUERY, RAISE, EXCEPTION blocks, SECURITY DEFINER), set-returning functions, CALL, DO. CREATE TRIGGER BEFORE/AFTER/INSTEAD OF, row and statement level, WHEN, UPDATE OF, TG_* variables; INSTEAD OF makes views writable.

Extensions. vector (pgvector), postgis, plpgsql and uuid-ossp are built in: CREATE EXTENSION and DROP EXTENSION accept them and change nothing, so PostgreSQL scripts run unchanged; any other extension is refused (extension "…" is not available).

Vectors (pgvector). vector(n), the <->, <#> and <=> distances, l2_distance, cosine_distance, inner_product, and USING hnsw (or ivfflat, built as HNSW) indexes for ORDER BY … LIMIT k.

Geometry (PostGIS). geometry and geography with SRIDs, WKT, EWKB and GeoJSON, about 60 ST_ functions (constructors, measurements, relations, ST_DWithin, ST_Buffer, ST_Union, ST_Transform between 4326 and 3857…), && and <->, and USING gist spatial indexes.

Types. Integers, SERIAL, NUMERIC(p,s), REAL/DOUBLE PRECISION, TEXT/VARCHAR/UUID, BOOLEAN, DATE, TIMESTAMP/TIMESTAMPTZ, JSON/JSONB.

Constraints. Primary keys, UNIQUE, NOT NULL, CHECK, and foreign keys with every ON DELETE/UPDATE action.

Transactions. Each request is one ACID transaction; interactive transactions and /transaction batches are all-or-nothing, DDL included, with SAVEPOINT, ROLLBACK TO and RELEASE. EXPLAIN and EXPLAIN ANALYZE (always rolled back) show lookups, scans, joins, the policies that apply and the grants checked.

Row-level security. On for every new table. Non-admin roles see nothing until a policy allows it (CREATE POLICY … USING … WITH CHECK …), and GRANT/REVOKE decide what each role may do. SQL reads the caller with current_setting('propus.tenant'), 'propus.user_id' and 'propus.role'.

Read replicas. A primary streams every commit (and every new or dropped database) to replicas started with --replica-of; a replica starts from a snapshot, serves reads over HTTPS and the Postgres protocol, refuses writes with SQLSTATE 25006, resyncs by itself when it falls behind the primary's log, and becomes a primary with propus promote or, if asked, on its own after the primary has been gone --failover-after seconds. --synchronous-replicas n makes a commit wait until n replicas have it. Caches, streams, objects and migrations are served by the primary only.

Also: encryption at rest, an audit log, verify, verified backups, the change feed, and the PostgreSQL wire protocol (psql, node-postgres and Drizzle are tested).

#FlashCache

Over 230 Redis 7.4 commands: strings, hashes with field expiry, lists, sets, sorted sets, streams with consumer groups, HyperLogLog, bitmaps, geo; MULTI/EXEC/WATCH; blocking commands; pub/sub with patterns; keyspace notifications; Lua scripts and functions; all eight eviction policies. A snapshot plus a journal keeps keys across restarts. Each database has its own keyspace. 866 replies are identical to a real Redis 7.4 in the differential test.

#PulseStream

Topics with partitions and Kafka's key hashing, retention by age and size, compaction, consumer groups with rebalancing, idempotent and transactional producers, every compression codec, dead-letter queues through nack, and 30 Kafka protocol APIs with SASL. A kafkajs session reads the same against PulseStream as against Apache Kafka 3.9. Over HTTPS and WebSocket it also serves the database's change feed.

#ObjectCore

The S3 API at /s3 (SigV4, multipart, ranges, versions), checksums on every object, public buckets served at /o/<db>/<bucket>/<key>, and an event in the database's objects stream for every change.

#TrackSense

POST /search/indexes {"table", "fields", "tenantColumn"} indexes a table; GET /search?q= ranks matches with highlighted snippets; @@ searches from SQL. The index follows the change feed, survives restarts, and shows a tenant only its own rows.

#AutoGraph

The typed layer in every SDK: tables in code, relations, joins, aggregates, sync(), versioned migrations with up and down, introspection and generated types. See sdks.md.

#Migration Engine

propus migrate, or Migrations in the console. It inspects the source and reports what can't carry over, records a change position, copies in resumable batches, follows changes, verifies every row (or key, record, object), fences the source at cutover, and can roll back by writing Propus's later changes back to the source.

From	Into	Follows changes with	Fence at cutover
PostgreSQL 10+	tables	logical decoding	read-only database
MySQL 5.7+, MariaDB	tables	the row binary log	read_only
SQLite	tables	(stop the app)	(stop the app)
Redis, Valkey	the cache	keyspace notifications	CLIENT PAUSE WRITE
Kafka 2.4+	streams, with offsets and consumer groups	reading as a consumer	(stop the producers)
S3 and compatible	the object store, with versions	listing	(stop the app)

Redis clusters are copied slot by slot but not yet tested on a real cluster.

#How the engines work together

Every engine checks identity and tenant itself; none trusts another to have done it. The rules every engine follows are in platform contract.