Saas-voc-server

SaaS VOC Server

Independent backend template for the domestic ecommerce VOC product. The first case workspace is Demashi on JD. This repository does not connect to the cross-border production database and does not contain any Fmode, Parse, PostgreSQL, or supplier credential.

Current state

Completed through 2026-07-29:

• Independent Git repository and Node.js 22 / TypeScript service baseline.
• Independent PostgreSQL schema with workspace, source, product, metric, relation, review, import, and sync-job tables.
• Independent Parse Server application configuration on the same dedicated database.
• Strict startup validation for every required credential.
• GET /health database readiness check.
• POST /api/domestic-voc/sync idempotent manual-sync queue entry.
• GET /api/domestic-voc/jobs/:id sync-job status.
• GET /api/domestic-voc/snapshot response compatible with the frontend DomesticDataset interface.
• Server-side Fmode gateway client with timeout, retry, and credential redaction behavior.
• Deterministic identities for products, relations, reviews, and daily sync requests.
• Bounded batch importer for the normalized Demashi dataset.
• Database-free local demo runtime for frontend integration before infrastructure is provisioned.
• Workspace membership and owner / admin / analyst / viewer authorization.
• Parse session authentication for production and a fixed local identity for database-free development.
• Cursor-paginated product, relation, review, import, sync-job, analysis, action, alert, and audit APIs.
• Pending analysis requests, action workflows, alerts, and auditable write operations without fabricated AI output.
• Configurable local workspace identity instead of a Demashi-only runtime branch.
• Manual sync retry/cancel operations plus automatic recovery for stale processing jobs.
• Owner-only owner changes, last-owner protection, and workspace-member action assignment validation.
• Parse REST storage adapter for environments where only a Parse Server URL and application credentials are available.
• Master-key-only Voc* class schemas, resumable bounded imports, direct count verification, and a Parse-backed sync worker.
• Direct streaming import from the source Demashi workbook without a frontend-generated JSON intermediate.
• Idempotent competitor detail backfill through the company ecommerce relay, with mapped placeholders preserved when a request fails.
• Server-side Fmode AI gateway with bounded request validation, JSON/SSE passthrough, timeout handling, and no browser-visible token.
• AI status and live-test endpoints plus a Parse REST-backed DomesticAiPromptConfig store for the reused prompt-management UI.
• Brand/category competitor discovery through the company ecommerce relay, with strict brand relevance filtering and local enriched snapshot output.
• Product knowledge APIs for own products, focus products, competitor material, editable metadata, and audit history.

The current local demo case resolves to 2,817 operating products, 96 unique competitor products, 9,717 daily metrics, 138 relations, and 206 competitor review samples. Of the competitor catalog, 70 products have current market snapshots collected from 27 brand/category queries and the remaining products preserve workbook mapping placeholders. The PostgreSQL importer may create internal relation_stub rows to preserve foreign keys; the Parse REST model stores denormalized relation identities and therefore does not create those extra catalog products.

The worker calls JD product-detail and review paths only through the company /api/voc-e-commerce gateway. It never sends a browser request to a supplier endpoint and never returns raw gateway errors or credentials.

Architecture

Saas-voc frontend
  -> saas-voc-server
      -> shared API and authorization contract
          -> local in-memory repository (development)
          -> dedicated PostgreSQL repository (production, voc schema)
          -> external Parse REST repository (shared development or managed Parse)
      -> dedicated Parse application (production authentication)
      -> sync queue
          -> existing /api/voc-e-commerce gateway
      -> AI gateway
          -> Fmode AI chat completions
          -> DomesticAiPromptConfig via Parse REST

The browser only calls this service. FMODE_API_KEY and FMODE_AI_TOKEN are used only in server-side authorization headers and are never returned to the browser or included in a request URL.

Registered JD gateway contracts:

GET /api/voc-e-commerce/jd/get-item-detail/v1?itemId={productId}
GET /api/voc-e-commerce/jd/get-item-comments/v1?itemId={productId}&page={page}
GET /api/voc-e-commerce/jd/search-item-list/v1?keyword={brand+category}

The first worker run defaults to one review page. JD_REVIEW_MAX_PAGES can raise the bounded limit after quota and response validation.

Runtime

Use Node.js 22.13 or newer within the Node 22 release line. The repository intentionally pins Node 22 because Parse Server publishes explicit supported runtime ranges.

Required configuration is listed in .env.example. Empty secrets are rejected before the HTTP server starts. STORAGE_DRIVER=parse_rest requires the external Parse URL, application id, and master key but no direct database connection. STORAGE_DRIVER=postgres additionally requires DATABASE_URL, a maintenance key, and optionally a separate MIGRATION_DATABASE_URL. Never commit live credentials.

AI analysis is configured independently with:

FMODE_AI_BASE_URL=https://api.fmode.cn
FMODE_AI_TOKEN=<process-only-token>
FMODE_AI_MODEL=deepseek-v4-pro
FMODE_AI_TIMEOUT_MS=120000

When the token is absent, the server still starts and /api/ai/status reports configured: false; completion and test calls return a controlled 503 response.

Parse REST runtime

Use this mode when PostgreSQL is not exposed and the project must connect through an existing Parse Server REST API:

$env:STORAGE_DRIVER = 'parse_rest'
$env:PARSE_SERVER_URL = 'https://parse.example.com/parse'
$env:PARSE_APP_ID = '<application-id>'
$env:PARSE_MASTER_KEY = '<process-only-master-key>'
$env:API_AUTH_MODE = 'disabled' # isolated local development only
$env:FMODE_API_KEY = '<process-only-company-gateway-key>'
npm run bootstrap:parse-rest -- "E:\workspace\Saas-voc\src\assets\data\demashi-summary.json" demashi
npm run import:workbook:parse-rest -- "E:\path\to\德玛仕产品及竟品收集0721.xlsx" demashi
npm run sync:competitors:parse-rest -- demashi --concurrency=3 --reviews --review-pages=1
npm run verify:parse-rest -- demashi jd
npm run dev

The bootstrap command creates or reconciles 16 isolated Voc* classes, seeds the workspace/member/source records, imports the normalized case in request-size-bounded batches, and is idempotent for the same source hash and verified counts. The workbook command performs the same bounded import directly from Excel using a streaming reader. Competitor sync creates all mapped competitor records first, then skips already completed details on later runs. npm run seed:knowledge:parse-rest -- demashi 8 initializes missing top-selling product knowledge without overwriting manual metadata. The verification command checks the schema set, class-level permissions, workspace/import readiness, exact own/competitor/detail counts, product knowledge, and denial of app-id-only reads. See docs/parse-rest-schema.md for the class contract.

The REST client first writes the full schema contract. For managed Parse deployments that reject the required field attribute, it retries the schema write without that attribute while retaining application validation and master-only class permissions. Master-authenticated class and batch requests retry only an exact transient 403 unauthorized response, with a strict attempt bound; ordinary permission failures and browser requests are never retried.

Local frontend integration without a database

The local demo runtime has no PostgreSQL, Parse, or gateway credential dependency. From this repository run:

npm ci
npm run start:local

It loads ../../Saas-voc/src/assets/data/demashi-summary.json by default and listens on http://127.0.0.1:4400. Override the source only when the repositories are stored elsewhere:

$env:LOCAL_DATASET_PATH = 'E:\path\to\demashi-summary.json'
npm run start:local

Local demo sync requests create queryable in-memory completion records that validate whether requested products exist in the packaged dataset. They do not collect external data or persist anything. Production collection and persistence remain in npm run dev.

To regenerate the enriched local snapshot from workbook brand/category relations, configure the company gateway key in the backend process and run:

npm run enrich:competitors:local

The command writes logs/local-enriched-dataset.json. Set LOCAL_DATASET_PATH=logs/local-enriched-dataset.json before starting local mode. Search results must pass the brand/category relevance filter; review failures leave an explicit partial enrichment status without discarding successfully collected product snapshots.

Local mode injects the fixed local-admin owner. It is intentionally database-free and must not be internet-facing. Set LOCAL_WORKSPACE_ID when using a packaged dataset under a workspace other than demashi; omitted API workspace ids then resolve to that configured default.

Database-backed runtime

Local database:

$env:POSTGRES_PASSWORD = '<local-only-password>'
docker compose up -d postgres

The compose database listens only on 127.0.0.1:54329. A local DATABASE_URL therefore uses database saas_voc on port 54329.

Install, migrate, and run:

npm ci
npm run migrate
npm run dev

Import the current normalized case after migration:

npm run import:dataset -- "E:\workspace\Saas-voc\src\assets\data\demashi-summary.json"

001_initial_domain.sql creates the domestic VOC catalog and sync tables. 002_saas_platform.sql adds members, analysis runs, actions, alerts, and audit history. The migration runner records each file in voc.schema_migration and executes each unapplied migration in its own transaction.

Authentication and first administrator

Use API_AUTH_MODE=parse in production. API clients send either X-Parse-Session-Token or Authorization: Bearer <session-token>. The backend validates the session through PARSE_SERVER_URL/users/me and never sends a Parse master or maintenance key to the browser.

For the first production start, set SAAS_BOOTSTRAP_ADMIN_USER_ID to the Parse user object id and optionally set its email and display name. The service upserts that user as owner of SAAS_DEFAULT_WORKSPACE_ID. Clear the bootstrap variables after successful verification so a later restart cannot silently restore that account's owner role.

API_AUTH_MODE=disabled is available for isolated database-backed development only. It uses LOCAL_AUTH_USER_ID, LOCAL_AUTH_USER_EMAIL, and LOCAL_AUTH_USER_NAME, and must not be enabled in a public environment.

Default local API address:

http://127.0.0.1:4400

API

Queue a JD product and review sync:

POST /api/domestic-voc/sync
Idempotency-Key: optional-client-key
Content-Type: application/json

{
  "workspaceId": "demashi",
  "platform": "jd",
  "productIds": ["11266507445"],
  "scopes": ["product", "reviews"]
}

When Idempotency-Key is absent, the service derives a stable key from workspace, platform, products, scopes, and UTC date. The same daily request returns the same job.

Read status and frontend data:

GET /api/domestic-voc/jobs/{jobId}
POST /api/domestic-voc/jobs/{jobId}/retry
POST /api/domestic-voc/jobs/{jobId}/cancel
GET /api/domestic-voc/snapshot?workspaceId=demashi&platform=jd

Catalog and operational APIs:

GET /api/domestic-voc/products
GET /api/domestic-voc/products/{productId}
GET /api/domestic-voc/products/{productId}/reviews
GET /api/domestic-voc/relations
GET /api/domestic-voc/jobs
GET /api/domestic-voc/jobs/{jobId}/events

GET /api/saas/context
GET /api/saas/workspaces
GET|PUT /api/saas/workspaces/{workspaceId}/members[/userId]
GET /api/saas/workspaces/{workspaceId}/data-sources
GET /api/saas/workspaces/{workspaceId}/imports
GET|POST /api/saas/workspaces/{workspaceId}/analyses
GET|POST|PATCH /api/saas/workspaces/{workspaceId}/actions[/actionId]
GET|POST|PATCH /api/saas/workspaces/{workspaceId}/alerts[/alertId]
GET /api/saas/workspaces/{workspaceId}/audit

GET /api/ai/status
POST /api/ai/test
POST /api/ai/chat/completions
GET /api/ai/prompts
PUT /api/ai/prompts/{promptKey}

GET /api/knowledge/products
PUT /api/knowledge/products
DELETE /api/knowledge/products/{productKey}

POST /api/ai/chat/completions accepts a bounded OpenAI-compatible message payload and supports both JSON and SSE responses. Unknown fields are rejected, so a browser cannot supply a token or turn the endpoint into a generic proxy. Prompt configuration is exposed through the dedicated AI routes; the browser never connects to Parse with a master key.

With Parse REST storage, startup reconciles 16 managed Voc* classes and idempotently seeds the 13 domestic AI prompt/model configurations for the default workspace. Existing prompt customizations and product knowledge metadata are preserved.

List endpoints use opaque cursor pagination (limit plus optional cursor). Analysis creation returns a truthful pending record; an analysis worker is not included yet.

An empty database returns a valid empty dataset. It does not invent reviews, ratings, sentiment, pain points, or AI output.

Database rules

• Runtime identities are enforced by composite unique constraints, not by check-then-insert logic.
• Import and sync writers must use INSERT ... ON CONFLICT in batches.
• Foreign-key columns and common workspace/date filters are indexed.
• Workers claim jobs with FOR UPDATE SKIP LOCKED.
• Worker startup recovers processing jobs older than SYNC_JOB_STALE_AFTER_MS; exhausted jobs become failed and eligible jobs return to the queue.
• External HTTP requests must run outside database transactions.
• Workspace membership is checked on every domestic and SaaS business route.
• Disabled workspaces cannot pass authorization even when a membership row remains active.
• Owner and admin can manage ordinary members and read audit history; only owners can change owner membership, and the final active owner is protected. Analyst can run sync/analysis and manage actions/alerts; viewer is read-only.
• Action assignees must be active members of the same workspace.
• Member, sync, analysis, action, and alert writes append an audit entry.
• Production deployments must use a pooled, least-privilege runtime role. Migration credentials should be separate from runtime credentials.
• PostgreSQL DDL is always an explicit migration step. Parse REST mode reconciles only the isolated Voc* class schemas at startup through the schema REST API.

Infrastructure handoff

When the dedicated database and server are available, no route or frontend contract rewrite is required:

1. Set DATABASE_URL to the least-privilege runtime role and MIGRATION_DATABASE_URL to the schema owner.
2. Set independent Parse keys, PARSE_SERVER_URL, and API_AUTH_MODE=parse.
3. Set the company relay base URL/key, worker limits, and exact frontend CORS_ORIGINS.
4. Run npm ci, npm run migrate, then npm run import:dataset -- <dataset-path>.
5. Bootstrap the first owner, start the service, verify /health, /api/saas/context, snapshot totals, and one bounded sync job.
6. Clear the bootstrap administrator variables and place TLS/same-origin proxying in front of /parse and /api.

Security status

parse-server@9.10.0 replaces the initially evaluated v8 line, removing all high and critical audit findings. A ws@8.21.0 override is retained because Parse currently pins an older vulnerable patch. The remaining 14 audit findings are moderate transitive dependencies in Parse push/Firebase and redirect support plus ExcelJS's nested UUID package; push and LiveQuery are not configured in this template. Re-run npm audit --omit=dev before each deployment and do not use npm audit fix --force, which currently proposes breaking Parse/ExcelJS downgrades.

Verification

npm run build
npm test
npm run verify:parse-rest -- demashi jd
npm audit --omit=dev

Current result:

• TypeScript build: passed.
• Unit, AI gateway, REST client, product knowledge, adapter, worker recovery, local-demo, authentication, RBAC, cursor, workflow, audit, and HTTP contract tests: 51 passed.
• Production dependency audit: 0 critical, 0 high, 14 moderate.
• Company gateway health: HTTP 200 on 2026-07-23.
• Credentialed live JD product-detail request: passed through the company gateway for product 11266507445; the adapter extracted the product id, title, and brand from the live double-data envelope.
• Credentialed live JD review requests: all 37 mapped competitors returned usable first-page evidence; 355 sanitized reviews were persisted without nickname, avatar, GUID, or other reviewer identity fields.
• Live credentials are absent from the repository, fixtures, logs, and Git history. No supplier endpoint is contacted directly.
• Dedicated Parse REST development verification (2026-07-27): 16/16 Voc* schemas present, master-key-only access confirmed, 2,817 own products, 37 competitor products with 37 details ready, 9,717 metrics, 40 relations, 355 competitor reviews, and 8 initialized product knowledge records.
• End-to-end local API mode: /health, SaaS context, full snapshot, frontend proxy, desktop navigation, and 390x844 responsive rendering passed against the Parse REST store.
• End-to-end AI mode: status, live test, visual report, follow-up report, prompt loading, desktop rendering, and 390x844 responsive rendering passed through the local gateway with no browser console errors or warnings.
• PostgreSQL integration: pending a newly provisioned database. The local Docker CLI is installed but its engine was unavailable on 2026-07-23; no existing database was contacted.

See TASKS.md for the implementation sequence and acceptance boundary.