Просмотр исходного кода

feat: add database-free local integration mode

gangvy 1 месяц назад
Родитель
Сommit
35b7a12da7

+ 22 - 1
README.md

@@ -17,6 +17,7 @@ Completed on 2026-07-23:
 - Server-side Fmode gateway client with timeout, retry, and credential redaction behavior.
 - Server-side Fmode gateway client with timeout, retry, and credential redaction behavior.
 - Deterministic identities for products, relations, reviews, and daily sync requests.
 - Deterministic identities for products, relations, reviews, and daily sync requests.
 - Bounded batch importer for the normalized Demashi dataset.
 - Bounded batch importer for the normalized Demashi dataset.
+- Database-free local demo runtime for frontend integration before infrastructure is provisioned.
 
 
 The current case import resolves to 2,817 operating products, 9,717 daily metrics, 40 relations, and 38 internal `relation_stub` records required to preserve foreign keys. Stub records are excluded from the frontend product list and are never presented as collected product facts.
 The current case import resolves to 2,817 operating products, 9,717 daily metrics, 40 relations, and 38 internal `relation_stub` records required to preserve foreign keys. Stub records are excluded from the frontend product list and are never presented as collected product facts.
 
 
@@ -50,6 +51,26 @@ Use Node.js 22.13 or newer within the Node 22 release line. The repository inten
 
 
 Required configuration is listed in `.env.example`. Empty secrets are rejected before the HTTP server starts. Generate independent random values for `PARSE_MASTER_KEY` and `PARSE_MAINTENANCE_KEY`; do not reuse any value from `moshengqi-server` or `future-server`. `DATABASE_URL` is the least-privilege runtime connection. `MIGRATION_DATABASE_URL` is the schema-owner connection used only by `npm run migrate`; it falls back to `DATABASE_URL` for local development.
 Required configuration is listed in `.env.example`. Empty secrets are rejected before the HTTP server starts. Generate independent random values for `PARSE_MASTER_KEY` and `PARSE_MAINTENANCE_KEY`; do not reuse any value from `moshengqi-server` or `future-server`. `DATABASE_URL` is the least-privilege runtime connection. `MIGRATION_DATABASE_URL` is the schema-owner connection used only by `npm run migrate`; it falls back to `DATABASE_URL` for local development.
 
 
+### Local frontend integration without a database
+
+The local demo runtime has no PostgreSQL, Parse, or gateway credential dependency. From this repository run:
+
+```powershell
+npm ci
+npm run start:local
+```
+
+It loads `../../Saas-voc/src/assets/data/demashi-summary.json` by default and listens on `http://127.0.0.1:4400`. Override the source only when the repositories are stored elsewhere:
+
+```powershell
+$env:LOCAL_DATASET_PATH = 'E:\path\to\demashi-summary.json'
+npm run start:local
+```
+
+Local demo `sync` requests create queryable in-memory completion records that validate whether requested products exist in the packaged dataset. They do not collect external data or persist anything. Production collection and persistence remain in `npm run dev`.
+
+### Database-backed runtime
+
 Local database:
 Local database:
 
 
 ```powershell
 ```powershell
@@ -132,7 +153,7 @@ npm audit --omit=dev
 Current result:
 Current result:
 
 
 - TypeScript build: passed.
 - TypeScript build: passed.
-- Unit, adapter, worker, and HTTP contract tests: 19 passed.
+- Unit, adapter, worker, local-demo, and HTTP contract tests: 20 passed.
 - Production dependency audit: 0 critical, 0 high, 13 moderate.
 - Production dependency audit: 0 critical, 0 high, 13 moderate.
 - Company gateway health: HTTP 200 on 2026-07-23.
 - Company gateway health: HTTP 200 on 2026-07-23.
 - Credentialed live JD product-detail request: passed through the company gateway for product `11266507445`; the adapter extracted the product id, title, and brand from the live double-`data` envelope.
 - Credentialed live JD product-detail request: passed through the company gateway for product `11266507445`; the adapter extracted the product id, title, and brand from the live double-`data` envelope.

+ 1 - 0
TASKS.md

@@ -12,6 +12,7 @@ Status date: 2026-07-23
 - [x] Add the initial domestic VOC schema, constraints, foreign keys, and indexes.
 - [x] Add the initial domestic VOC schema, constraints, foreign keys, and indexes.
 - [x] Seed the `demashi` workspace and JD source metadata.
 - [x] Seed the `demashi` workspace and JD source metadata.
 - [x] Add health, sync, job-status, and snapshot endpoints.
 - [x] Add health, sync, job-status, and snapshot endpoints.
+- [x] Add a database-free local-demo server for frontend integration using the normalized Demashi snapshot.
 - [x] Add Fmode timeout/retry/authentication behavior behind a server-only client.
 - [x] Add Fmode timeout/retry/authentication behavior behind a server-only client.
 - [x] Add unit tests and compile under strict TypeScript settings.
 - [x] Add unit tests and compile under strict TypeScript settings.
 - [ ] Run migrations against a newly provisioned PostgreSQL database.
 - [ ] Run migrations against a newly provisioned PostgreSQL database.

+ 2 - 0
package.json

@@ -10,7 +10,9 @@
   "scripts": {
   "scripts": {
     "build": "tsc -p tsconfig.json",
     "build": "tsc -p tsconfig.json",
     "dev": "tsx watch src/server.ts",
     "dev": "tsx watch src/server.ts",
+    "dev:local": "tsx watch src/local-server.ts",
     "start": "node dist/src/server.js",
     "start": "node dist/src/server.js",
+    "start:local": "tsx src/local-server.ts",
     "migrate": "tsx scripts/migrate.ts",
     "migrate": "tsx scripts/migrate.ts",
     "import:dataset": "tsx scripts/import-dataset.ts",
     "import:dataset": "tsx scripts/import-dataset.ts",
     "test": "tsx --test test/**/*.test.ts",
     "test": "tsx --test test/**/*.test.ts",

+ 64 - 0
src/local-app.ts

@@ -0,0 +1,64 @@
+import cors from 'cors';
+import express, { type ErrorRequestHandler } from 'express';
+import { ZodError } from 'zod';
+import type { DomesticDataset } from './types/domestic-dataset.js';
+import { LocalSnapshotService } from './modules/domestic-voc/local/local-snapshot.service.js';
+import { LocalSyncJobStore } from './modules/domestic-voc/local/local-sync-job.store.js';
+import { createDomesticVocRouter } from './modules/domestic-voc/routes.js';
+import { SyncService } from './modules/domestic-voc/services/sync.service.js';
+
+export function createLocalDemoApp(input: {
+  dataset: DomesticDataset;
+  corsOrigins: string[];
+}) {
+  const app = express();
+  app.disable('x-powered-by');
+  app.use(cors({
+    origin(origin, callback) {
+      if (!origin || input.corsOrigins.includes(origin)) return callback(null, true);
+      return callback(new Error('Origin is not allowed'));
+    },
+  }));
+  app.use(express.json({ limit: '1mb' }));
+
+  const jobs = new LocalSyncJobStore(input.dataset);
+  const sync = new SyncService(jobs);
+  const snapshot = new LocalSnapshotService(input.dataset);
+
+  app.get('/health', (_request, response) => {
+    response.json({
+      service: 'saas-voc-server',
+      status: 'ok',
+      mode: 'local-demo',
+      database: 'deferred',
+      dataset: {
+        caseName: input.dataset.caseName,
+        platform: input.dataset.platform,
+        products: input.dataset.products.length,
+        reviews: input.dataset.reviews.length,
+      },
+      timestamp: new Date().toISOString(),
+    });
+  });
+
+  app.use('/api/domestic-voc', createDomesticVocRouter({ jobs, sync, snapshot }));
+
+  app.use((_request, response) => {
+    response.status(404).json({ error: 'not_found' });
+  });
+
+  const errorHandler: ErrorRequestHandler = (error, _request, response, _next) => {
+    if (error instanceof ZodError) {
+      response.status(400).json({
+        error: 'invalid_request',
+        issues: error.issues.map((issue) => ({ path: issue.path.join('.'), message: issue.message })),
+      });
+      return;
+    }
+    console.error('[local-http] unhandled request error', error instanceof Error ? error.message : error);
+    response.status(500).json({ error: 'internal_error' });
+  };
+  app.use(errorHandler);
+
+  return app;
+}

+ 64 - 0
src/local-server.ts

@@ -0,0 +1,64 @@
+import { readFile } from 'node:fs/promises';
+import { createServer } from 'node:http';
+import { resolve } from 'node:path';
+import { z } from 'zod';
+import { createLocalDemoApp } from './local-app.js';
+import type { DomesticDataset } from './types/domestic-dataset.js';
+
+const localEnvironmentSchema = z.object({
+  LOCAL_HOST: z.string().min(1).default('127.0.0.1'),
+  LOCAL_PORT: z.coerce.number().int().min(1).max(65535).default(4400),
+  LOCAL_DATASET_PATH: z.string().min(1).optional(),
+  LOCAL_CORS_ORIGINS: z.string().min(1).default('http://localhost:4200,http://127.0.0.1:4200'),
+});
+
+function defaultDatasetPath(): string {
+  return resolve(process.cwd(), '..', '..', 'Saas-voc', 'src', 'assets', 'data', 'demashi-summary.json');
+}
+
+async function loadDataset(path: string): Promise<DomesticDataset> {
+  const parsed = JSON.parse(await readFile(path, 'utf8')) as Partial<DomesticDataset>;
+  if (
+    parsed.schemaVersion !== 1
+    || parsed.platform !== 'jd'
+    || !parsed.caseName
+    || !Array.isArray(parsed.products)
+    || !Array.isArray(parsed.dailyTotals)
+    || !Array.isArray(parsed.mappingGroups)
+    || !Array.isArray(parsed.relations)
+  ) {
+    throw new Error('Local dataset does not match the domestic VOC snapshot contract');
+  }
+  return {
+    ...parsed,
+    reviews: Array.isArray(parsed.reviews) ? parsed.reviews : [],
+  } as DomesticDataset;
+}
+
+async function main(): Promise<void> {
+  const config = localEnvironmentSchema.parse(process.env);
+  const datasetPath = resolve(config.LOCAL_DATASET_PATH || defaultDatasetPath());
+  const dataset = await loadDataset(datasetPath);
+  const app = createLocalDemoApp({
+    dataset,
+    corsOrigins: config.LOCAL_CORS_ORIGINS.split(',').map((origin) => origin.trim()).filter(Boolean),
+  });
+  const server = createServer(app);
+
+  server.listen(config.LOCAL_PORT, config.LOCAL_HOST, () => {
+    console.log(`[local-server] listening on http://${config.LOCAL_HOST}:${config.LOCAL_PORT}`);
+    console.log(`[local-server] loaded ${dataset.products.length} products for ${dataset.caseName}`);
+  });
+
+  const shutdown = (signal: string) => {
+    console.log(`[local-server] received ${signal}; shutting down`);
+    server.close(() => process.exit(0));
+  };
+  process.once('SIGINT', () => shutdown('SIGINT'));
+  process.once('SIGTERM', () => shutdown('SIGTERM'));
+}
+
+main().catch((error) => {
+  console.error('[local-server] startup failed', error instanceof Error ? error.message : error);
+  process.exitCode = 1;
+});

+ 11 - 0
src/modules/domestic-voc/local/local-snapshot.service.ts

@@ -0,0 +1,11 @@
+import type { DomesticDataset } from '../../../types/domestic-dataset.js';
+import type { DomesticSnapshotProvider } from '../routes.js';
+
+export class LocalSnapshotService implements DomesticSnapshotProvider {
+  constructor(private readonly dataset: DomesticDataset) {}
+
+  async getSnapshot(workspaceId: string, platform: string): Promise<DomesticDataset | null> {
+    if (workspaceId !== 'demashi' || platform !== this.dataset.platform) return null;
+    return this.dataset;
+  }
+}

+ 53 - 0
src/modules/domestic-voc/local/local-sync-job.store.ts

@@ -0,0 +1,53 @@
+import type {
+  EnqueueSyncJobInput,
+  SyncJobRecord,
+  SyncJobStore,
+} from '../repositories/sync-job.repository.js';
+import type { DomesticDataset } from '../../../types/domestic-dataset.js';
+
+export class LocalSyncJobStore implements SyncJobStore {
+  private readonly jobs = new Map<string, SyncJobRecord>();
+  private readonly idempotency = new Map<string, string>();
+  private readonly productIds: Set<string>;
+
+  constructor(
+    private readonly dataset: DomesticDataset,
+    private readonly now: () => Date = () => new Date(),
+  ) {
+    this.productIds = new Set(dataset.products.map((product) => product.productId));
+  }
+
+  async enqueue(input: EnqueueSyncJobInput): Promise<SyncJobRecord | null> {
+    if (input.workspaceId !== 'demashi' || input.platform !== this.dataset.platform) return null;
+
+    const existingId = this.idempotency.get(input.idempotencyKey);
+    if (existingId) return this.jobs.get(existingId) ?? null;
+
+    const timestamp = this.now().toISOString();
+    const missingCount = input.productIds.filter((productId) => !this.productIds.has(productId)).length;
+    const job: SyncJobRecord = {
+      id: input.publicId,
+      workspaceId: input.workspaceId,
+      platform: input.platform,
+      status: missingCount ? 'partial' : 'completed',
+      scopes: [...input.scopes],
+      productIds: [...input.productIds],
+      progress: 100,
+      attempts: 0,
+      maxAttempts: 1,
+      errorSummary: missingCount
+        ? `Local demo dataset does not contain ${missingCount} requested product(s)`
+        : null,
+      requestedAt: timestamp,
+      startedAt: timestamp,
+      completedAt: timestamp,
+    };
+    this.jobs.set(job.id, job);
+    this.idempotency.set(input.idempotencyKey, job.id);
+    return job;
+  }
+
+  async findByPublicId(publicId: string): Promise<SyncJobRecord | null> {
+    return this.jobs.get(publicId) ?? null;
+  }
+}

+ 16 - 10
src/modules/domestic-voc/repositories/sync-job.repository.ts

@@ -16,6 +16,20 @@ export interface SyncJobRecord {
   completedAt: string | null;
   completedAt: string | null;
 }
 }
 
 
+export interface EnqueueSyncJobInput {
+  publicId: string;
+  workspaceId: string;
+  platform: string;
+  idempotencyKey: string;
+  scopes: string[];
+  productIds: string[];
+}
+
+export interface SyncJobStore {
+  enqueue(input: EnqueueSyncJobInput): Promise<SyncJobRecord | null>;
+  findByPublicId(publicId: string): Promise<SyncJobRecord | null>;
+}
+
 interface SyncJobRow {
 interface SyncJobRow {
   public_id: string;
   public_id: string;
   workspace_public_id: string;
   workspace_public_id: string;
@@ -32,17 +46,10 @@ interface SyncJobRow {
   completed_at: Date | string | null;
   completed_at: Date | string | null;
 }
 }
 
 
-export class SyncJobRepository {
+export class SyncJobRepository implements SyncJobStore {
   constructor(private readonly database: Queryable) {}
   constructor(private readonly database: Queryable) {}
 
 
-  async enqueue(input: {
-    publicId: string;
-    workspaceId: string;
-    platform: string;
-    idempotencyKey: string;
-    scopes: string[];
-    productIds: string[];
-  }): Promise<SyncJobRecord | null> {
+  async enqueue(input: EnqueueSyncJobInput): Promise<SyncJobRecord | null> {
     const result = await this.database.query<SyncJobRow>(`
     const result = await this.database.query<SyncJobRow>(`
       WITH target_workspace AS (
       WITH target_workspace AS (
         SELECT id, public_id FROM voc.workspace WHERE public_id = $1 AND status = 'active'
         SELECT id, public_id FROM voc.workspace WHERE public_id = $1 AND status = 'active'
@@ -133,4 +140,3 @@ export class SyncJobRepository {
     };
     };
   }
   }
 }
 }
-

+ 8 - 4
src/modules/domestic-voc/routes.ts

@@ -1,9 +1,13 @@
 import { Router } from 'express';
 import { Router } from 'express';
 import { z } from 'zod';
 import { z } from 'zod';
-import type { SyncJobRepository } from './repositories/sync-job.repository.js';
-import type { SnapshotService } from './services/snapshot.service.js';
+import type { SyncJobStore } from './repositories/sync-job.repository.js';
+import type { DomesticDataset } from '../../types/domestic-dataset.js';
 import type { SyncService } from './services/sync.service.js';
 import type { SyncService } from './services/sync.service.js';
 
 
+export interface DomesticSnapshotProvider {
+  getSnapshot(workspaceId: string, platform: string): Promise<DomesticDataset | null>;
+}
+
 const syncRequestSchema = z.object({
 const syncRequestSchema = z.object({
   workspaceId: z.string().min(1).default('demashi'),
   workspaceId: z.string().min(1).default('demashi'),
   platform: z.literal('jd').default('jd'),
   platform: z.literal('jd').default('jd'),
@@ -18,9 +22,9 @@ const snapshotQuerySchema = z.object({
 });
 });
 
 
 export function createDomesticVocRouter(dependencies: {
 export function createDomesticVocRouter(dependencies: {
-  jobs: SyncJobRepository;
+  jobs: SyncJobStore;
   sync: SyncService;
   sync: SyncService;
-  snapshot: SnapshotService;
+  snapshot: DomesticSnapshotProvider;
 }): Router {
 }): Router {
   const router = Router();
   const router = Router();
 
 

+ 2 - 3
src/modules/domestic-voc/services/sync.service.ts

@@ -1,10 +1,10 @@
 import { randomUUID } from 'node:crypto';
 import { randomUUID } from 'node:crypto';
 import { makeSyncIdempotencyKey } from '../domain/identity.js';
 import { makeSyncIdempotencyKey } from '../domain/identity.js';
-import type { SyncJobRecord, SyncJobRepository } from '../repositories/sync-job.repository.js';
+import type { SyncJobRecord, SyncJobStore } from '../repositories/sync-job.repository.js';
 
 
 export class SyncService {
 export class SyncService {
   constructor(
   constructor(
-    private readonly jobs: SyncJobRepository,
+    private readonly jobs: SyncJobStore,
     private readonly now: () => Date = () => new Date(),
     private readonly now: () => Date = () => new Date(),
   ) {}
   ) {}
 
 
@@ -33,4 +33,3 @@ export class SyncService {
     });
     });
   }
   }
 }
 }
-

+ 119 - 0
test/local-app.test.ts

@@ -0,0 +1,119 @@
+import assert from 'node:assert/strict';
+import type { AddressInfo } from 'node:net';
+import test from 'node:test';
+import { createLocalDemoApp } from '../src/local-app.js';
+import type { DomesticDataset, DomesticMetricSummary } from '../src/types/domestic-dataset.js';
+
+const emptyMetrics: DomesticMetricSummary = {
+  gmv: 0,
+  soldUnits: 0,
+  transactionOrders: 0,
+  transactionCustomers: 0,
+  impressions: 0,
+  clicks: 0,
+  views: 0,
+  visitors: 0,
+  cartUnits: 0,
+  orderAmount: 0,
+  orderUnits: 0,
+  orderCount: 0,
+  refundAmount: 0,
+  refundUnits: 0,
+  refundOrders: 0,
+  conversionRate: 0,
+  clickThroughRate: 0,
+  averageUnitPrice: 0,
+  refundToGmvRate: 0,
+};
+
+const dataset: DomesticDataset = {
+  schemaVersion: 1,
+  generatedAt: '2026-07-23T00:00:00.000Z',
+  caseName: 'Demashi',
+  platform: 'jd',
+  source: {
+    sourceFile: 'demashi-summary.json',
+    sourceHash: 'test',
+    sheets: [],
+    dateRange: { start: '2026-07-01', end: '2026-07-23' },
+  },
+  summary: {
+    metricRows: 0,
+    metricProducts: 1,
+    mappingRows: 0,
+    relations: 0,
+    uniqueCompetitorProducts: 0,
+    category2Count: 0,
+    category3Count: 0,
+    reviewCount: 0,
+  },
+  dailyTotals: [],
+  products: [{
+    platform: 'jd',
+    productId: '11266507445',
+    productKey: 'jd:11266507445',
+    asin: '11266507445',
+    role: 'own',
+    brand: 'Demashi',
+    title: 'Local demo product',
+    model: '',
+    category1: '',
+    category2: '',
+    category3: '',
+    source: 'excel',
+    relationCount: 0,
+    summary: emptyMetrics,
+    trend: [],
+  }],
+  mappingGroups: [],
+  relations: [],
+  reviews: [],
+  quality: {
+    orphanMappings: [],
+    mappingsWithoutCompetitor: [],
+    brandWithoutProductId: [],
+  },
+};
+
+test('local demo serves a snapshot and queryable completed sync jobs without a database', async () => {
+  const app = createLocalDemoApp({ dataset, corsOrigins: ['http://localhost:4200'] });
+  const server = await new Promise<ReturnType<typeof app.listen>>((resolve) => {
+    const listening = app.listen(0, '127.0.0.1', () => resolve(listening));
+  });
+
+  try {
+    const address = server.address() as AddressInfo;
+    const baseUrl = `http://127.0.0.1:${address.port}`;
+
+    const health = await fetch(`${baseUrl}/health`);
+    assert.equal(health.status, 200);
+    const healthBody = await health.json() as {
+      mode: string;
+      database: string;
+      dataset: { products: number; reviews: number };
+    };
+    assert.equal(healthBody.mode, 'local-demo');
+    assert.equal(healthBody.database, 'deferred');
+    assert.deepEqual(healthBody.dataset, { caseName: 'Demashi', platform: 'jd', products: 1, reviews: 0 });
+
+    const snapshot = await fetch(`${baseUrl}/api/domestic-voc/snapshot?workspaceId=demashi&platform=jd`);
+    assert.equal(snapshot.status, 200);
+    assert.equal((await snapshot.json() as DomesticDataset).products[0]?.productId, '11266507445');
+
+    const syncResponse = await fetch(`${baseUrl}/api/domestic-voc/sync`, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json', 'Idempotency-Key': 'local-demo-sync-11266507445' },
+      body: JSON.stringify({ productIds: ['11266507445'] }),
+    });
+    assert.equal(syncResponse.status, 202);
+    const syncBody = await syncResponse.json() as { job: { id: string; status: string; progress: number } };
+    assert.equal(syncBody.job.status, 'completed');
+    assert.equal(syncBody.job.progress, 100);
+
+    const jobResponse = await fetch(`${baseUrl}/api/domestic-voc/jobs/${syncBody.job.id}`);
+    assert.equal(jobResponse.status, 200);
+    assert.equal((await jobResponse.json() as { job: { status: string } }).job.status, 'completed');
+  } finally {
+    await new Promise<void>((resolve, reject) => server.close((error) => error ? reject(error) : resolve()));
+  }
+});