| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301 |
- #!/usr/bin/env node
- /**
- * 单一容器多人并发压测(逐级加压找拐点)
- *
- * 用法:
- * node scripts/loadtest-ljtest.mjs --levels 20,30,40,50 --timeout 900
- * node scripts/loadtest-ljtest.mjs --levels 20 --mode text # 只跑文本轮
- * node scripts/loadtest-ljtest.mjs --levels 20 --mode file # 只跑文件轮
- *
- * ## 与 loadtest-agent.mjs 的区别
- *
- * 那个只回答「N 条里几条回了、有没有串台」。本次要回答的是
- * **「多人同时用会不会卡、单一容器上限在哪」**,所以补了:
- * · 优先级:**延迟分位**(p50/p90/p95/max)—— 「卡」的可操作定义
- * · **排队延迟** vs **处理延迟** 拆开(首条 process 事件 - 发出 = 排队)
- * · **两种负载**:纯文本(测调度)/ 文件(测生成+上云+投递)
- * · **逐级加压**,每档独立、冷却后进下一档,满足判停条件即停止
- * · 每档落盘 JSON,便于事后逐会话取证
- *
- * ## 判停条件(满足任一 → 当前档即拐点,不再加压)
- *
- * ① 成功率 < 90%
- * ② p95 端到端耗时 > 首档基线的 3 倍
- * ③ 出现串台(回复里出现别的会话 token)
- * ④ 文件轮:请求了文件但没收到 m.file
- *
- * ## 为什么要等「静默」而不是等 ✅
- *
- * 实测该容器**经常不打 ✅**(2026-11-17 四轮实测 0/4)。
- * 若只等 ✅,每档都要耗到超时。所以:
- * 收到本会话的实质正文 且 该会话 45 秒无新事件 → 视为完成
- * ✅ 到了就立刻算完成(更快)
- */
- import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
- import { homedir } from 'node:os';
- import { join } from 'node:path';
- const FUNCTIONS = 'https://server.fmode.cn/api/functions';
- const FN_PROVISION = 'RP5Ht55NeX';
- const AGENT = 'agent-node-lj-test';
- const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
- function arg(name, fallback) {
- const i = process.argv.indexOf(`--${name}`);
- return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
- }
- const LEVELS = arg('levels', '20,30,40,50').split(',').map(Number);
- const TIMEOUT_S = Number(arg('timeout', '900'));
- const MODE = arg('mode', 'both'); // both | text | file
- const POLL_MS = Number(arg('poll', '3000'));
- const QUIET_MS = Number(arg('quiet', '45000'));
- const COOLDOWN_S = Number(arg('cooldown', '60'));
- const RUN = `lt${Date.now().toString().slice(-6)}`;
- const OUT_DIR = 'docs/03-interface';
- const TOKEN = JSON.parse(readFileSync(join(homedir(), '.fmode/config/user.json'), 'utf8'))?.sessionToken;
- /** 带重试的 fetch(压测期间高并发请求会被限流/瞬时失败,不能一次失败就整轮崩掉) */
- async function fetchRetry(url, opts = {}, tries = 4) {
- let lastErr;
- for (let k = 0; k < tries; k++) {
- try {
- const r = await fetch(url, opts);
- if (r.status < 500) return r;
- lastErr = new Error(`HTTP ${r.status}`);
- } catch (e) {
- lastErr = e;
- }
- await sleep(1000 * (k + 1));
- }
- throw lastErr;
- }
- async function provision() {
- const r = await fetchRetry(FUNCTIONS, {
- method: 'POST',
- headers: { 'Content-Type': 'application/json', 'X-Parse-Application-Id': 'ncloudmaster' },
- body: JSON.stringify({ id: FN_PROVISION, token: TOKEN, params: { agentId: AGENT } }),
- });
- const j = await r.json();
- const d = j.result ?? j;
- if (!d?.accessToken) throw new Error('provision 失败 ' + JSON.stringify(j).slice(0, 200));
- return d;
- }
- const pct = (arr, p) => {
- if (!arr.length) return 0;
- const s = [...arr].sort((a, b) => a - b);
- return s[Math.min(s.length - 1, Math.floor((p / 100) * s.length))];
- };
- const r1 = (n) => Math.round(n * 10) / 10;
- /** 一轮(一档 + 一种负载)的完整执行与统计 */
- async function runWave(level, mode, ctx) {
- const { mx, client, hs, auth, agentMxId, userMxId } = ctx;
- const tag = `L${level}-${mode}`;
- console.log(`\n${'='.repeat(60)}\n档位 ${tag}:${level} 个并发会话(${mode === 'text' ? '纯文本' : '生成文件'})\n${'='.repeat(60)}`);
- // 1) 建 N 个独立房间(串行 + 重试:并发建房会被限流)
- const peers = [];
- for (let i = 1; i <= level; i++) {
- let room;
- for (let k = 0; k < 4 && !room; k++) {
- try {
- room = (
- await client.createRoom({
- invite: [agentMxId],
- name: `${RUN}-${tag}-${i}`,
- preset: mx.Preset.TrustedPrivateChat,
- })
- ).room_id;
- } catch (e) {
- console.log(` 建房 ${i} 第 ${k + 1} 次失败:${String(e.message ?? e).slice(0, 80)}`);
- await sleep(1500 * (k + 1));
- }
- }
- if (!room) {
- console.log(` ⚠️ 建房 ${i} 彻底失败,跳过`);
- continue;
- }
- peers.push({ i, room, token: `T${i}${Math.random().toString(36).slice(2, 8).toUpperCase()}`, sentAt: 0, firstProcessAt: 0, firstBodyAt: 0, ackedAt: 0, lastEventAt: 0, files: [], bodies: [], seen: new Set(), joined: false });
- }
- console.log(` 建房 ${peers.length}/${level}`);
- // 2) 等容器进房(并发)
- const joinDeadline = Date.now() + 180_000;
- while (Date.now() < joinDeadline && peers.some((p) => !p.joined)) {
- await Promise.all(
- peers.filter((p) => !p.joined).map(async (p) => {
- const j = await (await fetchRetry(`${hs}/_matrix/client/v3/rooms/${encodeURIComponent(p.room)}/joined_members`, { headers: auth }).catch(() => null))?.json().catch(() => ({})) ?? {};
- if ((j.joined ?? {})[agentMxId]) p.joined = true;
- }),
- );
- if (peers.some((p) => !p.joined)) await sleep(2000);
- }
- const bound = peers.filter((p) => p.joined);
- console.log(` 容器进房 ${bound.length}/${level}`);
- if (!bound.length) return { level, mode, error: '容器未进任何房间', peers: [] };
- // 3) 同时发送
- const t0 = Date.now();
- const promptFor = (p) =>
- mode === 'text'
- ? `只回复一行:${p.token} 已收到`
- : `在 /opt/data/${RUN}/${tag}/${p.i}/ 下生成 1 个文件 r.txt(三行文字,内容含 ${p.token}),并作为附件发给我。完成后一句话总结。`;
- await Promise.all(
- bound.map(async (p) => {
- const sent = await client.sendEvent(p.room, 'm.room.message', {
- msgtype: 'm.text',
- body: promptFor(p),
- 'm.mentions': { user_ids: [agentMxId] },
- });
- p.sentId = sent.event_id;
- p.sentAt = Date.now();
- }),
- );
- console.log(` ${bound.length} 条已同时发出,开始观测…`);
- // 4) 轮询收集直到全部完成或超时
- const deadline = t0 + TIMEOUT_S * 1000;
- let lastPrint = 0;
- const isProgress = (b) => /^[*·•\s]*(🐍|💻|🔎|📖|✍️|🧠|📝|⚙|🔧|Writing|Reading|Running|terminal|write_file|execute_code)/i.test(b.trim());
- while (Date.now() < deadline) {
- await sleep(POLL_MS);
- await Promise.all(
- bound.map(async (p) => {
- if (p.ackedAt) return;
- const j = await (await fetchRetry(`${hs}/_matrix/client/v3/rooms/${encodeURIComponent(p.room)}/messages?dir=b&limit=80`, { headers: auth }).catch(() => null))?.json().catch(() => ({})) ?? {};
- for (const ev of (j.chunk ?? []).reverse()) {
- if (ev.event_id === p.sentId) continue;
- if (ev.origin_server_ts < p.sentAt) continue;
- if (ev.type === 'm.reaction') {
- const rel = ev.content?.['m.relates_to'];
- if (rel?.event_id === p.sentId && String(rel.key ?? '').includes('✅')) p.ackedAt = ev.origin_server_ts;
- continue;
- }
- if (ev.type !== 'm.room.message' || ev.sender === userMxId) continue;
- /* 🔴 按 event_id 去重:轮询会反复读到同一事件(附件曾被数成 13 个) */
- if (p.seen.has(ev.event_id)) continue;
- p.seen.add(ev.event_id);
- const c = ev.content ?? {};
- const body = String(c.body ?? '');
- p.lastEventAt = ev.origin_server_ts;
- if (c['fmode.kind'] === 'process' || isProgress(body)) {
- if (!p.firstProcessAt) p.firstProcessAt = ev.origin_server_ts;
- continue;
- }
- if (!p.firstBodyAt && body.trim()) p.firstBodyAt = ev.origin_server_ts;
- if (body.trim()) p.bodies.push(body);
- if (c.url && (c.msgtype === 'm.file' || c.msgtype === 'm.image')) p.files.push({ name: body, url: c.url, size: c.info?.size });
- }
- }),
- );
- const now = Date.now();
- const doneCount = bound.filter((p) => p.ackedAt || (p.firstBodyAt && now - p.lastEventAt > QUIET_MS)).length;
- if (now - lastPrint > 20000) {
- lastPrint = now;
- console.log(` [${Math.round((now - t0) / 1000)}s] 完成 ${doneCount}/${bound.length}`);
- }
- if (doneCount === bound.length) break;
- }
- // 5) 统计
- const rows = bound.map((p) => {
- const all = p.bodies.join('\n');
- const leaked = bound.filter((q) => q !== p && all.includes(q.token)).map((q) => q.token);
- const own = all.includes(p.token);
- const finishedAt = p.ackedAt || (p.firstBodyAt ? Math.max(p.firstBodyAt, p.lastEventAt) : 0);
- return {
- i: p.i,
- room: p.room,
- ok: !!p.firstBodyAt && own,
- ownToken: own,
- leaked,
- e2e: finishedAt && p.sentAt ? (finishedAt - p.sentAt) / 1000 : null,
- queue: p.firstProcessAt && p.sentAt ? (p.firstProcessAt - p.sentAt) / 1000 : null,
- files: p.files.length,
- acked: !!p.ackedAt,
- };
- });
- const okRows = rows.filter((r) => r.ok);
- const e2e = okRows.map((r) => r.e2e).filter((v) => v != null);
- const queue = okRows.map((r) => r.queue).filter((v) => v != null);
- const fileOk = mode === 'file' ? rows.filter((r) => r.files > 0).length : null;
- const stats = {
- level,
- mode,
- run: RUN,
- total: bound.length,
- success: okRows.length,
- successRate: r1((okRows.length / bound.length) * 100),
- leakedSessions: rows.filter((r) => r.leaked.length).length,
- p50: r1(pct(e2e, 50)),
- p90: r1(pct(e2e, 90)),
- p95: r1(pct(e2e, 95)),
- max: r1(Math.max(0, ...e2e)),
- queueP50: r1(pct(queue, 50)),
- queueP95: r1(pct(queue, 95)),
- elapsedS: Math.round((Date.now() - t0) / 1000),
- throughputPerMin: r1((okRows.length / Math.max(1, (Date.now() - t0) / 60000))),
- fileOk,
- rows,
- };
- console.log(` 结果:成功 ${stats.success}/${stats.total} (${stats.successRate}%) | p50 ${stats.p50}s p90 ${stats.p90}s p95 ${stats.p95}s max ${stats.max}s | 排队 p50 ${stats.queueP50}s | 串台 ${stats.leakedSessions}${mode === 'file' ? ` | 带文件 ${stats.fileOk}` : ''}`);
- return stats;
- }
- async function main() {
- console.log(`\n=== lj-test 单容器并发压测 ===\n档位: ${LEVELS.join(' → ')}\n模式: ${MODE}\n单档超时: ${TIMEOUT_S}s\n运行号: ${RUN}\n`);
- const prov = await provision();
- const mx = await import('matrix-js-sdk');
- const client = mx.createClient({ baseUrl: prov.homeserver, accessToken: prov.accessToken, userId: prov.userMxId });
- const hs = prov.homeserver.replace(/\/$/, '');
- const ctx = { mx, client, hs, auth: { Authorization: `Bearer ${prov.accessToken}` }, agentMxId: prov.agentMxId, userMxId: prov.userMxId };
- console.log(`用户 ${prov.userMxId} | 容器 ${prov.agentMxId}`);
- const modes = MODE === 'both' ? ['text', 'file'] : [MODE];
- const all = [];
- let baselineP95 = null;
- let stop = false;
- for (const level of LEVELS) {
- for (const mode of modes) {
- if (stop) break;
- const s = await runWave(level, mode, ctx);
- all.push(s);
- if (s.error) { stop = true; break; }
- if (mode === 'text' && baselineP95 == null) baselineP95 = s.p95;
- const reasons = [];
- if (s.successRate < 90) reasons.push(`成功率 ${s.successRate}% < 90%`);
- if (baselineP95 && mode === 'text' && s.p95 > baselineP95 * 3) reasons.push(`p95 ${s.p95}s > 基线3倍(${r1(baselineP95 * 3)}s)`);
- if (s.leakedSessions > 0) reasons.push(`串台 ${s.leakedSessions} 个会话`);
- if (mode === 'file' && s.fileOk < s.total * 0.9) reasons.push(`文件缺失 ${s.total - s.fileOk} 个`);
- if (reasons.length) {
- console.log(`\n⛔ 拐点:${tag(level, mode)} —— ${reasons.join(';')}`);
- s.breakReason = reasons.join(';');
- stop = true;
- }
- }
- if (stop) break;
- if (level !== LEVELS[LEVELS.length - 1]) {
- console.log(`\n冷却 ${COOLDOWN_S}s 后进入下一档…`);
- await sleep(COOLDOWN_S * 1000);
- }
- }
- mkdirSync(OUT_DIR, { recursive: true });
- const file = join(OUT_DIR, `loadtest-ljtest-${RUN}.json`);
- writeFileSync(file, JSON.stringify({ run: RUN, agent: AGENT, levels: LEVELS, mode: MODE, waves: all }, null, 2));
- console.log(`\n===== 汇总 =====`);
- for (const w of all) {
- if (w.error) console.log(`${tag(w.level, w.mode)}: 失败 ${w.error}`);
- else console.log(`${tag(w.level, w.mode)}: 成功 ${w.successRate}% | p50 ${w.p50}s p95 ${w.p95}s max ${w.max}s | 吞吐 ${w.throughputPerMin}/分${w.breakReason ? ` | ⛔ ${w.breakReason}` : ''}`);
- }
- console.log(`\n落盘: ${file}`);
- client.stopClient();
- }
- const tag = (l, m) => `L${l}-${m}`;
- main().catch((e) => { console.error('失败:', e.message ?? e); process.exit(1); });
|