Parity matrix

What each Propus engine does compared with the system it replaces, measured rather than claimed. A row is "yes" only when a test in this repository exercises it; gaps are listed by name. Updated with every release that changes an engine.

CoreStore vs PostgreSQL 18

Proof: tests/diff/run.sh runs 36,000 random queries and 22 scenarios (views, triggers, functions, windows, savepoints…) on Propus and a real postgres:18 container and compares the results; tests/diff/extensions.sh does the same for vectors against pgvector/pgvector:pg18 and geometry against postgis/postgis:18-3.6. engines/propus-core/tests covers errors, row-level security, recursion limits and index plans; tests/replication/run.sh runs a primary and replicas in separate containers.

AreaPostgreSQL 18CoreStoreNotes
Queriesfullyesjoins, subqueries, WITH (recursive), set operations, CASE, JSON operators
Window functionsyesyesranking, offsets, values, every aggregate over ROWS/RANGE/GROUPS frames, named windows (no frame EXCLUDE)
AggregatesyesyesFILTER, ORDER BY inside, WITHIN GROUP (percentile_cont/disc, mode), string_agg, array_agg, json(b)_agg/object_agg
Viewsyesyesrun as the caller (security_invoker), so row-level security applies; materialized views with REFRESH
IndexesB-tree, expression, partial, INCLUDEyesthe planner uses an expression index when the query repeats the expression, a partial one when the query's WHERE implies its predicate
FunctionsSQL, PL/pgSQL, othersSQL, PL/pgSQLother languages (plpython…) are refused; recursion stops at a stack budget with SQLSTATE 54001
Triggersrow, statement, INSTEAD OF, eventyes, not eventBEFORE/AFTER/INSTEAD OF, WHEN, UPDATE OF, TRUNCATE; event triggers are not provided
Savepointsyesyesalso inside PL/pgSQL EXCEPTION blocks
ExtensionsCREATE EXTENSION, hundreds availablebuilt invector, postgis, plpgsql, uuid-ossp accepted as already there; others (pgcrypto, pg_trgm…) refused with 0A000
pgvectorvector, distances, HNSW, IVFFlatyesHNSW recall ≥ 0.95 tested against exact search; ivfflat is built as HNSW; halfvec and sparsevec are not provided
PostGISgeometry, geography, ~1,000 functions, GiSTthe common ~60ST_Transform knows 4326 and 3857 only; rasters and topology are not provided
Replicationstreaming replicas, synchronous commit, promotionyesprimary and read replicas with a sealed replication secret; failover is a replica's own choice (--failover-after), not a quorum: use one replica with it, or promote by hand
Row-level securitypermissive and restrictive policiespermissiveon by default for every table
Not providedDISTINCT ON, LISTEN/NOTIFY (the change feed does this), restrictive policies, statement_timeout, logical replication slots, foreign data wrappers, table inheritance and partitioning

FlashCache vs Redis 7.4

Proof: tests/cache/run.sh runs the same 866 commands against FlashCache and a real redis:7-alpine (7.4) container and compares every reply (tests/cache/diff.mjs); tests/cache/features.mjs covers what a one-shot comparison cannot (blocking, transactions, pub/sub, notifications, RESP3, clients, monitor, tenants); the runner also checks journal replay after a SIGKILL and snapshot restore. Unit tests cover the data structures (HyperLogLog error bound, geohash against Redis's published values, Lua patterns).

AreaRedis 7.4FlashCacheNotes
ProtocolRESP2, RESP3, inline commands, pipeliningyesHELLO 2/3; RESP3 maps, sets, doubles, booleans, nulls, verbatim, pushes
StringsSET options (NX/XX/GET/EX/PX/EXAT/PXAT/KEEPTTL), GETEX, GETDEL, MSET(NX), INCR family, INCRBYFLOAT, APPEND, SETRANGE/GETRANGE, STRLEN, LCSyes
BitmapsSETBIT, GETBIT, BITCOUNT (BYTE/BIT), BITPOS, BITOP (AND/OR/XOR/NOT), BITFIELD(_RO) with OVERFLOWyesBITOP DIFF/DIFF1/ANDOR/ONE from Redis 8 too
Hashesall commands, HRANDFIELD, HSCAN, field expiry (HEXPIRE family, 7.4)yesHGETDEL/HGETEX/HSETEX from Redis 8 too
Listsall commands, LPOS, LMOVE, LMPOP, blocking BLPOP/BRPOP/BLMOVE/BLMPOP/BRPOPLPUSHyesblocked clients wake on the write, not by polling
Setsall commands, SINTERCARD, SMISMEMBER, SSCANyes
Sorted setsZADD options, ZRANGE (BYSCORE/BYLEX/REV/LIMIT), union/inter/diff (+STORE, WEIGHTS, AGGREGATE), ZINTERCARD, ZRANGESTORE, ZMPOP/BZMPOP, ZRANDMEMBER, ZSCANyessets accepted as inputs to ZUNION and friends, as in Redis
StreamsXADD (ids, *, ms-*, NOMKSTREAM, MAXLEN/MINID), XRANGE/XREVRANGE, XREAD (BLOCK, $, +), consumer groups (XGROUP, XREADGROUP, NOACK, XACK, XPENDING, XCLAIM, XAUTOCLAIM, XINFO, XSETID), XDEL, XTRIMyesentries-read/lag as Redis 7 computes them
HyperLogLogPFADD, PFCOUNT (multi-key), PFMERGE; GET/SET of the HYLL stringyessame hash, registers and estimator as Redis, so counts are identical; GET returns Redis's dense bytes and the PF commands read Redis's dense and sparse bytes, so HyperLogLogs move between Redis and FlashCache with GET and SET (tested both ways at 7 to 40,000 elements)
GeoGEOADD (NX/XX/CH), GEODIST, GEOHASH, GEOPOS, GEORADIUS(BYMEMBER)(_RO), GEOSEARCH, GEOSEARCHSTOREyessearch is a scan over the sorted set; fine for thousands of points, not yet indexed for millions
KeysDEL/UNLINK, EXISTS, TYPE, EXPIRE family with NX/XX/GT/LT, (P)EXPIRETIME, PERSIST, RENAME(NX), COPY, MOVE, DUMP/RESTORE, KEYS, SCAN (MATCH/COUNT/TYPE), RANDOMKEY, TOUCH, OBJECT, SORT/SORT_RO, WAIT, WAITAOFyesDUMP payloads are FlashCache's own (not Redis RDB bytes); COPY … DB and MOVE cross databases
TransactionsMULTI/EXEC/DISCARD, WATCH/UNWATCH, EXECABORT, SELECT inside MULTIyes
ScriptingEVAL/EVALSHA(_RO), SCRIPT LOAD/EXISTS/FLUSH, FUNCTION LOAD/LIST/DELETE/FLUSH/DUMP/RESTORE/STATS, FCALL(_RO), redis.call/pcall/sha1hex/status_reply/error_reply/setresp/log, KEYS/ARGV, cjson, bit, SELECT in scriptsyesLua 5.4 semantics on Piccolo (pure Rust); cmsgpack and struct not provided; SCRIPT KILL/FUNCTION KILL answer NOTBUSY because a script is stopped by the server at lua-time-limit (default 5 s) and its writes until then are kept — Redis instead keeps running and lets other clients kill it only if it has not written
Pub/subPUBLISH, SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE, PUNSUBSCRIBE, PUBSUB CHANNELS/NUMSUB/NUMPAT; shard variantsyesSPUBLISH/SSUBSCRIBE behave as the plain ones (no cluster)
Keyspace notificationsnotify-keyspace-events flags, __keyspace@<db>__/__keyevent@<db>__yesthe channel names the key's own database
Memory boundmaxmemory, all eight maxmemory-policy values, maxmemory-samples, LFU (lfu-log-factor, lfu-decay-time), OOM errors under noevictionyessizes are estimates, as Redis's are
PersistenceRDB snapshots + AOFyes, differentlya KeelStore snapshot every 30 s (and on SAVE/BGSAVE/shutdown) plus a journal with appendfsync always/everysec/no; a restart loses at most the fsync interval; non-deterministic commands (scripts, SPOP, XREADGROUP…) journal their effects, as Redis 7 does
ConfigCONFIG GET/SET/RESETSTAT/REWRITEyessettings persist by themselves (Redis needs CONFIG REWRITE); server-level settings (bind, port, requirepass, dir…) belong to the Propus server
IntrospectionINFO (all sections), DBSIZE, TIME, LASTSAVE, COMMAND (COUNT/LIST/INFO/DOCS/GETKEYS), MEMORY USAGE/STATS/DOCTOR, SLOWLOG, LATENCY, DEBUG OBJECT/SLEEP, ROLE, LOLWUTyesCOMMAND DOCS carries summaries and groups, not the full argument tables; LATENCY histograms come from command statistics
ClientsCLIENT ID/SETNAME/GETNAME/SETINFO/INFO/LIST/KILL (filters)/NO-EVICT/NO-TOUCH/REPLY, MONITORyesCLIENT PAUSE, CLIENT TRACKING (client-side caching) and CLIENT UNBLOCK are not provided
DatabasesSELECT 0–15, MOVE, COPY … DB, SWAPDB, FLUSHDB/FLUSHALL, INFO keyspaceyesevery stored key carries its database, so databases never overlap (KEYS, SCAN, DBSIZE, FLUSHDB see one); pub/sub spans databases, as in Redis
Access controlACL users with command and key rulesdifferentidentities are Propus keys and tokens: a tenant's token is confined to its own key namespace (every key, channel, script and FLUSH), non-admin callers cannot run @dangerous/@admin commands; ACL WHOAMI/CAT/LIST/USERS/GETUSER answer, ACL SETUSER points at the Propus API
Multi-tenancynonemoreone cache per database, tenant namespaces inside it, the same credential as the SQL database
HTTPnonemoreGET/PUT/DELETE /v1/cache/{key}, POST /v1/cache/command
Clusterhash slots, MOVED/ASK, CLUSTER commandsnot yeta standalone's answers (CLUSTER → "cluster support disabled"); horizontal scale is ScaleMesh's job
ReplicationREPLICAOF, PSYNC, WAITnot yetSQL databases have read replicas; a cache lives on the primary only; REPLICAOF/SYNC are refused
ModulesMODULE LOAD, RedisJSON, RediSearch…nothe data types above are built in; JSON documents live in CoreStore (JSONB), search in TrackSense
Client-side cachingCLIENT TRACKING, RESP3 invalidation pushesnoplanned with PulseStream-driven invalidation

Known differences a program could notice: HSCAN/HGETALL return fields in sorted order (Redis: insertion/hash order); error messages from Lua carry no user_script:line: prefix; OBJECT HELP and other HELP texts are shorter; COMMAND lists Redis 8 commands FlashCache also speaks.

PulseStream vs Kafka 3.9

Proof: tests/stream/run.sh runs kafkajs (admin, idempotent and gzip producers, consumer groups shared by two members, transactions with offsets and read_committed, resets, DeleteRecords, group and topic deletion) against PulseStream and against a real apache/kafka:3.9.0 container and compares the two transcripts line by line; confluent-kafka (librdkafka) produces, consumes in a group, commits and reads watermarks; the HTTP suite covers partitions, keys, headers, compaction, retention settings, dead letters, server-assigned groups and WebSocket push. Unit tests cover murmur2 against Kafka's vectors, RecordBatch v2 with CRC-32C and gzip, idempotence, transactions, retention, compaction and the migration of first-release streams.

AreaKafka 3.9PulseStreamNotes
Wire protocolbinary protocol, versioned APIsyesProduce 0–7, Fetch 0–11, ListOffsets 0–5, Metadata 0–8, OffsetCommit 0–7, OffsetFetch 0–5, FindCoordinator 0–2, JoinGroup 0–5, Heartbeat 0–3, LeaveGroup 0–3, SyncGroup 0–3, DescribeGroups 0–4, ListGroups 0–2, SaslHandshake 0–1, ApiVersions 0–3, CreateTopics 0–4, DeleteTopics 0–3, DeleteRecords 0–1, InitProducerId 0–1, AddPartitionsToTxn 0–1, AddOffsetsToTxn 0–1, EndTxn 0–1, TxnOffsetCommit 0–2, DescribeConfigs 0–2, AlterConfigs 0–1, SaslAuthenticate 0–1, CreatePartitions 0–1, DeleteGroups 0–1, IncrementalAlterConfigs 0, OffsetDelete 0; clients negotiate down to these
RecordsRecordBatch v2: keys, values, headers, timestamps, producer id/epoch/sequence, control markersyesCRC-32C checked; batches arrive uncompressed, gzip, snappy (raw or xerial-framed), lz4 or zstd (librdkafka produces each in the tests); fetches return uncompressed batches
Topics and partitionscreate/delete/describe, add partitions, per-topic configsyesup to 1024 partitions; retention.ms, retention.bytes, cleanup.policy, max.message.bytes, delete.retention.ms, message.timestamp.type take effect, other settings are kept and shown; auto-creation on first use (1 partition)
Partitioningmurmur2 of the key, sticky round-robin otherwiseyesthe same hash as Kafka's default partitioner, so keys land where a Kafka client expects
Retentiontime and size based deletionyesenforced every minute and on maintain
Compactionlast record per key, tombstones, delete.retention.msyesa compaction runs when half the records are new or after five minutes; POST /streams/{t}/compact runs it now; DeleteRecords is refused on compacted topics, as Kafka does
Consumer groupsjoin/sync/heartbeat/leave, generations, rebalance and session timeouts, leader-side assignment, static membership ids, DescribeGroups/ListGroups/DeleteGroupsyesthe classic protocol; group.initial.rebalance.delay.ms 500 ms; the server also assigns (range) for HTTP members
OffsetsOffsetCommit/OffsetFetch/OffsetDelete, ListOffsets earliest/latest/by timestamp/max timestampyes
Idempotent producersproducer ids, epochs, sequence checks, duplicate batches answered with their original offsetyes
TransactionsInitProducerId with a transactional id (fencing), AddPartitionsToTxn, AddOffsetsToTxn, TxnOffsetCommit, EndTxn commit/abort markers, last stable offset, aborted-transaction lists, read_committed fetches, transaction timeoutsyes
Fetchlong polling (max_wait_ms, min_bytes), max_bytes budgets, isolation levelsyesfetch sessions are accepted but not kept (every fetch is full)
AuthSASL/PLAIN, SCRAM, OAUTHBEARER, mTLS, ACLsPLAINthe database's connection secret is the password (any username); each database is its own broker-sized world; SCRAM and ACL rules are not provided
Adminconfigs, partitions, records, groupsyesDescribeConfigs for topics and the broker; AlterConfigs and IncrementalAlterConfigs for topics
HTTP and WebSocketnone (REST Proxy is separate)moreJSON produce/read/groups/commit/lag, server-assigned groups, live push
Dead lettersnone (client libraries)morenack counts attempts; at the limit the record goes to <topic>.dlq with the reason and the group moves on
Change feednone (Debezium is separate)moreevery committed CoreStore change is readable and pushed (/changes); ObjectCore events are a keyed objects topic
Encryption at restnonemorethe stream file is sealed with the database's key when the database is encrypted
Replication, multi-broker, KRaft, MirrorMakeryesnot yetstreams live on the primary; SQL read replicas do not carry them
Quotas, throttlingyesno
Kafka Streams, Connect, Schema Registryseparate productsno
Compression on the wiregzip, snappy, lz4, zstdyes, inproducers may use any codec; PulseStream stores records uncompressed and fetches return them so (topic-level compression.type is accepted but not applied)

Known differences a program could notice: fetchTopicOffsets right after createTopics sees the new topic at once (Kafka may take a moment to elect a leader); the group coordinator forgets live membership on restart (committed offsets stay), so consumers rejoin; COMMAND DOCS-style completeness is not a goal here — the protocol table above is what is spoken.

AutoGraph vs Prisma, Drizzle, SQLAlchemy, Entity Framework

Proof: tests/e2e/run.sh runs the same scenario in each SDK (TypeScript, Python, Java, C#) against a live server: tables defined in code, sync and a no-op second sync, inserts with defaults filled in, operator and OR/NOT conditions, enum and column CHECKs refused with SQLSTATE 23514, composite unique refused with 23505, eager loading of one, many and many-through relations (nested in TypeScript and Python), inner and left joins nested under the joined table, group by with count/sum/max/avg, keyset paging, increment, onUpdateNow stamping, row-level security through a scoped client, cascade deletes, versioned migrations generated, applied once and rolled back, (TypeScript, Python) the live schema introspected and generated back as code, nested writes (create, connect, connectOrCreate, set, disconnect, update, delete, upsert on one, many and many-through; rolled back on failure; RLS through as()), middleware (order, changed args and results, short-circuit, soft delete, kept by as()) and renames (detected, refused with drop until decided, data kept, in a migration's up and down; the CLI's --rename / --no-rename in TypeScript).

AreaPrisma / Drizzle / SQLAlchemy / EFAutoGraphNotes
Schema in codeyesyesTS, Python, Java, C#; column types incl. NUMERIC(p,s), VARCHAR(n), UUID, JSONB; primary, unique, not null, defaults, foreign keys with ON DELETE/UPDATE, enums (as CHECK), column and table CHECKs, single and composite indexes and uniques
Row-level security in the schemano (SQL by hand)morepolicies and grants are part of the table definition and sync
Search in the schemanomoresearch: { columns, tenantColumn } keeps a TrackSense index
Push / syncprisma db push, drizzle-kit pushyesadditive by default; { drop: true } also drops columns and tables no longer defined
Versioned migrationsprisma migrate, drizzle-kit generate, Alembic, EF migrationsyesgenerated from the difference, written as SQL files with up and down, applied once each in its own transaction (_propus_migrations), rolled back
Introspection (db pull)yesyesTS and Python return table definitions; the schema keeps each column's declared SQL type
Generated typesyesyesdb.generate() / propus-generate emits TypeScript, Python, Java or C# row types and table definitions; the CLI wraps generate() and is not separately tested
Typed rowsyesTS, C# records via generationTypeScript infers row and insert types from the definitions; Python/Java/C# return dictionaries/maps at runtime, typed through generated code
Queriesfullyeswhere with eq/ne/gt/gte/lt/lte/in/notIn/like/ilike/startsWith/endsWith/contains/between/isNull/matches, AND/OR/NOT; columns, distinct, order, limit/offset, keyset paging, pluck, count, exists
Relationsinclude / with / joinedload / Includeyesone, many, many-through; filtered, ordered, limited and nested eager loading; one query per relation level (no N+1)
Joinsyesyesinner and left, rows nested under the joined table's name or alias
Aggregatesyesyescount (distinct), sum, avg, min, max, group by, having
Writesyesyesinsert, insert many, upsert (ON CONFLICT), update, increment, delete, delete all, RETURNING everywhere
Transactionsyesyesinteractive transactions with the same query API
Raw SQLyesyesquery(sql, params) with $n parameters
Nested writes (create with children)Prismayeson insert create / connect / connectOrCreate for one (parent first, key filled in), many and many-through (join rows); on update also set, disconnect, update, delete, upsert; one transaction as the client's identity, so row-level security applies to every row; any failure rolls back; tested in all four SDKs incl. RLS through as()
Middleware / extensions / soft deletePrisma, EFyesdb.use((params, next) => …) around every operation (writes and the select terminals), onion order, args and results changeable, short-circuit, kept by as() / database() / transactions; soft delete tested in all four SDKs; no $extends-style typed extensions
Lazy loading, change tracking, unit of workSQLAlchemy, EFnoAutoGraph is query-first, like Drizzle
Column renames in migrationsPrisma promptsyesplan().renames lists column (same SQL type) and table (same columns) candidates; decisions emit RENAME COLUMN / RENAME TO (reversed in down), data kept; with drop, an undecided candidate refuses; propus-generate --migration takes --rename / --no-rename (tested) or asks on a terminal (checked by hand)
Other databasesPostgres, MySQL, SQLite…Propus onlythe Postgres wire protocol also lets Prisma, Drizzle and node-postgres talk to Propus directly (tested with Drizzle)

Migration Engine

What moves into Propus, and how. Proof: tests/migrate/run.sh starts throwaway Postgres 18, MySQL 8.4, Redis 7.4, Kafka 3.9 and SeaweedFS (S3) containers and runs engines/propus-migrate/tests (each source end to end, plus a crash in the middle of the copy and a resume) and tests/migrate/api.sh (the same through the server's HTTP API, with a server restart while following).

SourceLands inCopyFollows changesFence at cutoverVerifyRollback
PostgreSQL 10+tableskeyset batches, resumable; types, defaults, NOT NULL, unique, checks, enums, foreign keys, indexes; identity/serial continue after the copied idslogical decoding (test_decoding, a slot recorded before the copy, advanced only after applying)default_transaction_read_only on the database, other sessions endedrow count and SHA-256 per row, merged by keyPropus's changes since the cutover written back in one transaction
MySQL 5.7+ / MariaDBtablesas above; ENUM, JSON, binary as \x hex, BIT, TINYINT(1) as BOOLEANrow binary log (needs binlog_format=ROW, binlog_row_image=FULL)SET GLOBAL read_onlyas aboveas above (REPLACE / DELETE)
SQLitetablesas aboveno (copy with the application stopped)operatoras aboveas above
Redis / Valkey 6+ (standalone; a cluster is read by slot but not yet tested against a real one)the database's FlashCache, db by dbevery type with exact expiries (PEXPIRETIME), hash fields' expiries, streams with groups, consumers and pending entries; HyperLogLogs counted the samekeyspace notifications (turned on for the job, set back after); compare-and-repair after a restart, a dropped connection, and at the cutoverCLIENT PAUSE … WRITEevery key compared on both sideslifts the pause; the source made equal to FlashCache
Kafka 2.4+ (and anything speaking its protocol), plain, TLS, SASL PLAIN or SCRAM-SHA-256/512the database's PulseStreamevery topic (or those named) with its partition count and its own settings; every partition from its earliest offset with offsets, keys, values, headers and timestamps kept; committed data only (aborted transactions stay gaps, as consumers see them); any compression; consumer groups' committed offsetsreading each partition as a consumer does; groups' offsets every 5 s; new topics and partitions picked upnone possible: cutover needs the producers stopped ("fence": false); a fenced cutover is refusedevery record's digest per partition, partition ends, every group's offsetsrecords Propus received after the cutover produced back in order (Kafka gives them the same offsets when nothing else wrote), groups' offsets committed there
S3-compatible (AWS, R2, MinIO, SeaweedFS…)the database's object storeobjects with content type and user metadata; MD5 checked against single-part ETags; multipart ETags kept; in a versioned bucket each object's history, oldest first, as many versions as the store keeps; a bucket readable by anyone (policy status, an AllUsers grant, or a policy letting everyone GET) becomes publiclisting again every 30 snone possible: cutover needs the application stopped ("fence": false); a fenced cutover is refusedETag per object, SHA-256 read again for multipart objects, each versioned object's historythe source buckets made equal to Propus (current objects)

Every job is saved after each batch under <data>/migrations, with its source URL sealed by the server's key; a server restart resumes it. Start one with propus migrate --from <source URL> --to <database URL> or POST /v1/db/<db>/migrations.

Connections to sources use TLS as each one's own clients do: Postgres takes libpq's sslmode (prefer by default; verify-ca/verify-full check the certificate, both also checking the host name) and sslrootcert; MySQL takes ssl-mode (PREFERRED by default, VERIFY_CA, VERIFY_IDENTITY) and ssl-ca; Redis rediss://; S3 HTTPS unless secure=false. The test servers run with a throwaway CA, and MySQL refuses unencrypted connections there, so every MySQL path (copy, binary log, rollback) is shown to run over TLS. Gaps, by name: Postgres views, triggers, functions and row-level security policies are reported, not copied; MySQL triggers and procedures likewise; a Redis FLUSHDB while following sends no per-key notifications and is caught by the cutover's repair, not at once; module types (RedisJSON…) are listed and skipped; S3 lifecycle rules, tags, per-object ACLs and the details of bucket policies are not copied, and an object whose latest version is a delete marker is not copied (Propus keeps versions only under a live object); whether a bucket is public is tested against AWS's documented responses only, because the test server (SeaweedFS) keeps no ACLs or policies; Redis rollback replaces the source's content rather than replaying a change list (writes that reach the source after the cutover are overwritten).