Context and production outcome
The Policy Evidence Knowledge Graph Platform is a collaboration between the NUS School of Computing and NUS Business School. Research requirements were developed under supervisor guidance for a recurring evidence-organisation problem in climate-policy research. The project was completed between January and June 2026.
The source literature creates several linked challenges:
- one policy instrument can appear under several terms;
- outcomes, methods, contextual limits, and decisive estimates are distributed across prose, tables, and appendices;
- compound questions need intervention, outcome, geography, method, source passage, and explanation of disagreement; and
- keyword search can locate papers but cannot preserve those relationships by itself.
The project established a traceable path through that material:
- papers become structured findings with source anchors;
- a knowledge graph connects policy instruments, outcomes, methods, regions, and papers;
- an evidence-constrained service builds concept explanations and cross-paper syntheses from those records; and
- citations, evidence cards, source passages, table context, and graph relationships retain the same stable identities.
The platform launched for internal access, and its highest recorded reach approached 100 visitors. Production use brought long-running ingestion, section streaming, conversation persistence, feedback capture, administrative permissions, and failure recovery into one operating loop. Public entry, ordinary research queries, and administrative mutation use separate access boundaries for the internal research environment.
Project responsibilities and technical delivery
Chunxiao Ren served as the principal implementer, translating the supervisor-guided requirements into the technical implementation and core system development across:
- literature data engineering;
- knowledge-graph construction and governance;
- constrained question-answering services;
- the React research product; and
- the production environment.
Research rules expressed as system rules
- Conclusions: require source evidence and stable citation identities.
- Cross-paper comparisons: retain method, context, effect direction, and source coverage.
- Taxonomy decisions: pass through reviewable states and support undo.
- Answer scope: distinguishes sufficient support, partial support, and unsupported requests.
- Release quality: uses data contracts, stage gates, retrieval rules, section states, governance workflows, and production checks.
Boundary conditions handled explicitly
- Concept-name drift can fragment retrieval, so the platform uses canonical identities.
- Missing paper metadata can weaken citation readability, so metadata remains part of the evidence contract.
- Interrupted ingestion can leave a batch incomplete, so jobs use durable state and terminal reconciliation.
- Proxy buffering can delay streamed sections, so stream behaviour is verified through the production path.
- Browser state can diverge from server-side conversations, so the server remains the source of truth and the browser cache stays bounded.
Research scope and domain judgements remain part of the two-school collaboration and supervisor-guided process.
End-to-end architecture
The platform has three application components with separate responsibilities: one React web application and two FastAPI services.
1. React web application
- Provides bilingual Wiki navigation, paper and evidence detail, graph exploration, T1 to T4 answers, conversation history, ingestion progress, and governance screens.
- Selects a fixed page skeleton from the answer type and renders typed section payloads.
- Receives section content, evidence cards, citation identities, and explicit status values through the service contract.
- Keeps completed sibling sections readable when one section fails.
2. Constrained Agent service
- Classifies questions and produces answer type, subquestions, geography, direction, and table requirements.
- Coordinates retrieval and assembles T1 to T4 response types.
- Streams typed section updates and calculates programmatic confidence for evidence-bearing T1 and T2 answers.
- Uses the graph and data service as a read-only evidence dependency.
- Places unsupported or malformed output into a failed, skipped, or refusal state at the section boundary.
3. Knowledge-graph and data service
- Owns ingestion, review queues, taxonomy, deduplication, semantic retrieval, deterministic graph queries, and evidence records.
- Maintains retrieval vectors, canonical concepts, and graph semantics in one service.
- Controls Neo4j access and the workflows that change corpus data or configuration.
- Separates ordinary research reads from protected governance writes.
Storage responsibilities
| Store | Responsibility | Recovery role |
|---|---|---|
| Neo4j | Papers, findings, evidence, concepts, outcomes, methods, and traversable research relationships | Rebuilt or reconciled from reviewed master artefacts |
| SQLite-backed stores | Pipeline jobs, conversations, feedback, and other ordered operational records | Preserves task and interaction histories independently of graph rebuilds |
| Structured artefacts | Extraction output, table references, validation results, and reviewed evidence master data | Provides an auditable source before and after graph materialisation |
Explicit payload contracts
A finding returned to the Agent carries:
- stable paper and evidence identity;
- policy instrument, outcome, method, and context;
- effect information and evidence references; and
- retrieval relevance.
The Agent response contract carries:
- answer type and expected section list;
- section identifiers, payloads, and status values;
- evidence cards, citation identities, and related papers; and
- confidence metadata when the answer type supports it.
Requests and responses validate language, query scope, stable identities, required fields, enum values, and nested objects. Missing optional research fields remain explicit null values. A section failure retains its identity and reason, allowing the browser to distinguish missing data, generation failure, and network failure.
Two independent data flows
Question answering
- The browser sends question text and interface language.
- The Agent selects an answer strategy and extracts retrieval parameters.
- The data service returns semantic results or deterministic graph-query results.
- The Agent validates evidence and streams typed sections.
- The browser renders the fixed answer structure and source interactions.
- A terminal message reconciles delivered sections, evidence cards, citations, related papers, and confidence.
The generation layer cannot modify corpus data, and the display layer cannot create a citation missing from the payload.
Ingestion and governance
- The administrative interface performs upload pre-checks and creates a durable job.
- Stage artefacts pass schema validation, source-evidence gates, human review, and count reconciliation.
- Trusted records are materialised in Neo4j.
- Operational state and audit history remain in their dedicated stores.
The Agent has no role in this mutation path, so a research question cannot trigger extraction or graph mutation.
Fifteen-stage data and knowledge-graph pipeline
The ingestion workflow gives every stage an observable state and durable output:
- parse each PDF into structured document output;
- flatten the parsed result while retaining page markers;
- run two-stage finding extraction, first discovering candidates and then normalising them to the schema;
- pass extracted findings through an independent critic;
- canonicalise driver names against the existing corpus;
- build citable evidence snippets;
- repair missing taxonomy fields;
- import papers, findings, evidence, drivers, outcomes, and methods into Neo4j;
- repair suspicious bibliographic titles from authoritative metadata;
- classify drivers into the live taxonomy;
- assign paper-level instrument, domain, objective, outcome, actor, and trigger labels;
- generate cross-paper community summaries after classification;
- find and resolve duplicate driver concepts with structural safeguards;
- fill missing Chinese display names; and
- audit those Chinese names for mistranslation, awkwardness, or untranslated text.
Four stage groups
| Group | Stages | Output and gate |
|---|---|---|
| Document and evidence | 1 to 7 | Parsed pages, normalised findings, critic results, canonical names, anchored evidence, and base taxonomy fields |
| Graph and bibliography | 8 to 9 | Reconciled Neo4j records and repaired bibliographic metadata |
| Semantic governance | 10 to 13 | Classified concepts, paper labels, community summaries, and guarded duplicate resolution |
| Bilingual display | 14 to 15 | Completed and audited Chinese display names |
This grouping keeps failure scope legible. A display-name issue changes localisation metadata. A failed evidence gate blocks the affected record from the trusted graph.
Stage invariants and status
- Flattened text retains stable page markers.
- Candidate quotations resolve to parsed source text.
- Findings satisfy the field schema and evidence gates.
- Graph materialisation reconciles the expected paper identities against the records written.
- Cross-paper summaries depend on completed classification labels.
- Concept merges preserve relationships, taxonomy placement, and an undo snapshot.
- Chinese-name stages change display fields only.
Stage status records waiting, running, completed, failed, interrupted, or skipped work. Each status includes timing, input summary, and result summary. The parent run derives its terminal state from all fifteen child stages, and the administrative interface reads the persisted status after refresh or reconnection.
Extraction granularity and table evidence
- Discovery pass: scans full papers for paragraphs and tables that may express policy effects.
- Normalisation pass: creates a strict record centred on one policy instrument, one outcome, applicable context, and research method.
- Effect fields: retain direction, magnitude, statistical detail, and mechanism when supplied by the source.
- Multiple anchors: allow one finding to reference main-text explanation and appendix qualifications.
- Table evidence: retains table identity, row and column context, page location, and a lightweight citation.
- Prose evidence: retains a verifiable source span.
This granularity supports cross-paper comparison while preserving evidence context and keeping graph size maintainable.
Evidence gates and review queue
| Control | Checks | Failure handling |
|---|---|---|
| Deterministic source matching | Verifies that evidence text resolves to the parsed source | Restricted fuzzy re-anchoring handles equation markup, line-break hyphenation, and layout drift |
| Numeric and context checks | Verifies values, page context, and source identity | Unresolved candidates enter review and stay outside the trusted corpus |
| Independent critic | Checks direction, values, method descriptions, qualifications, and source sufficiency | Flags retain reasons and suggested correction fields |
| Human review | Supports accept, correct, and reject transitions | Original extraction output remains preserved for provenance |
Later graph stages read only the trusted set. Pending records remain available for quality analysis, and each correction retains its originating stage and configuration version.
Incremental work and cost control
- Parsing, extraction, evidence generation, and bibliography repair operate on the current paper batch.
- Cross-paper summaries, duplicate discovery, and selected classification rebuilds depend on corpus-wide state.
- Artefact versions, graph predicates, and affected-record sets narrow corpus-wide recomputation.
- Stage-specific configuration allows full-paper discovery, structured normalisation, critic review, short classification, translation, and candidate adjudication to use different cost profiles.
- Low-confidence outputs move to human review.
- Run logs preserve the chosen stage configuration and result summary.
Persistent state, recovery, and reconciliation
- Run batches, stage tasks, logs, and artefact locations use durable state.
- Service restart converts abandoned running work into interrupted work.
- Completed idempotent artefacts can be reused against the original batch identity.
- Targeted retry starts from a selected stage and paper set after validating upstream artefact versions.
- Version mismatch schedules required dependencies again.
- External graph effects use idempotent identities and reconciliation queries to avoid duplicate nodes.
Every critical transformation records expected, processed, and skipped counts plus the exceptional identity set. A successful process exit cannot hide an omitted paper. Reconciliation cross-checks the source batch, structured artefacts, and Neo4j identities before graph materialisation or corpus-wide governance continues.
Data mutations execute serially, while read-only question answering continues independently. Upload pre-checks, append-only logs, checkpoints, terminal reconciliation, restart recovery, and targeted retries form one recoverable ingestion loop.
Evidence-constrained AI question answering
The Agent selects one answer strategy before generation begins:
| Strategy | Request type | Output contract |
|---|---|---|
| T1, concept explanation | Definitions, categories, and related concepts | Structured C0 to C7 concept page |
| T2, cross-paper synthesis | Impact, comparison, consensus, disagreement, and heterogeneity | Structured L0 to L4 synthesis |
| T3, corpus metadata | Counts, lists, distributions, and chart data | Deterministic payload from approved graph-query templates |
| T4, boundary refusal | Unsupported prediction, counterfactual, casual, probing, external, or out-of-corpus requests | Scoped refusal and an evidence-searchable reformulation when available |
Routing parameters and guardrails
The routing payload can include:
- intent and answer type;
- policy instrument and outcome;
- subquestions and effect direction;
- geography and time range; and
- table requirements.
These values become retrieval filters and answer-contract inputs. Guardrails move corpus counts and journal distributions to T3. A recognisable climate-policy object with an impact relation can return from an overly conservative boundary decision to evidence retrieval. Unknown metadata templates fall back to a safe general statistic, while invalid parameter ranges are narrowed or returned with an explicit scope note.
Bilingual semantic retrieval and graph fallback
- T1 and T2 use semantic retrieval as the primary recall path.
- A Chinese question receives a controlled English equivalent, and both forms participate in retrieval.
- Policy instruments, effect directions, regions, and table needs become metadata filters.
- Keyword matches contribute a soft ranking signal.
- Empty or narrow semantic recall activates deterministic graph fallback through canonical concepts and permitted neighbouring relationships.
Each result remains a structured finding with paper metadata and evidence identity. The retrieval contract supports paper-level deduplication, citation validation, graph expansion, and deterministic coverage counts.
For complex questions:
- subqueries and community-summary retrieval can run in parallel;
- results merge in original subquestion order;
- stable identities and overlapping evidence ranges remove duplicates;
- per-paper caps are applied after the merged result set is formed; and
- graph neighbourhoods expand results when seed findings exist.
Records remain separate when they express different outcomes or contexts. Metadata filters preserve inclusion and exclusion reasons, graph fallback records its traversal path, and bounded parallel work ends with deterministic reconciliation.
Structured section contracts and confidence scope
Every T1 and T2 section has a known identifier, status, and payload shape. Renderable fields include definitions, citations, timeline entries, concept edges, corpus metrics, instrument rows, comparison cells, subgroup results, evidence cards, and follow-up questions.
The contract applies these controls:
- deterministic sections use counts or normalised records;
- model-assisted sections receive retrieved evidence and return the expected schema;
- malformed output enters an error state at its section boundary;
- citation markers resolve to evidence cards included in the same response;
- evidence cards link to valid paper metadata; and
- terminal reconciliation removes duplicate unreferenced cards and marks citation gaps.
Programmatic confidence covers T1 and T2 only and uses the sections delivered in the final response.
- T1 inputs: definition consistency, citation support, corpus coverage, and section completeness.
- T2 inputs: conclusion convergence, evidence volume, method quality, citation coverage, and completeness.
- Scope: current corpus and current response.
- Excluded modes: T3 deterministic data and T4 corpus boundaries receive no composite confidence score.
T1: C0 to C7 concept page
| Sections | Purpose | Availability |
|---|---|---|
| C0 and C1 | Definition and category boundaries | Core definition requires adequate citation support |
| C2 and C3 | Implementation context and timeline | Optional when support is sparse |
| C4 | Typed concept relationships | Optional when the graph neighbourhood is weak |
| C5 | Corpus coverage from database records | Core |
| C6 | Representative papers and evidence cards | Core |
| C7 | Follow-up questions grounded in retrieved findings | Optional |
- The overview presents answer type, definition, and category cards.
- Citations sit next to supported claims and open the associated evidence.
- Persistent conversation history preserves the surrounding research context.
- Implementation context and dated events use separate payload shapes.
- A missing timeline can become skipped while the definition remains visible.
- A slower context section can complete independently.
- Typed relationships connect instruments, mechanisms, risks, and outcomes.
- Coverage reports the papers, regions, and methods present in the corpus.
- Evidence gaps remain visible beside the domain model.
- Representative papers bring the concept answer back to individual sources.
- Human-readable author, year, and journal labels retain stable evidence identity.
- Finding and source-snippet views keep the generated explanation auditable.
T1 degrades section by section. Optional context, timeline, graph, or follow-up sections can be skipped. Core definition, coverage, and evidence remain visible when supported. A failed follow-up section leaves completed evidence cards intact, and citation-coverage checks evaluate the substantive content that reached the user.
T2: L0 to L4 cross-paper synthesis
| Layer | Purpose | Construction |
|---|---|---|
| L0 | Evidence base and matched-paper count | Deterministic structured records |
| L1 | Policy-instrument inventory | Deterministic structured records |
| L2 | Consensus and disagreement per instrument | Evidence-constrained synthesis |
| L3 | Comparable fields across instruments | Deterministic alignment plus cited explanation |
| L4 | Variation by region, sector, method, or implementation design | Evidence-constrained synthesis |
- The dashboard shows matched papers, instrument coverage, study-design composition, and the first instrument analysis.
- Source breadth and method mix appear before consensus text.
- Deterministic dashboard sections can reach ready status while synthesis sections continue.
- Comparison rows align effect direction, available magnitude, supporting papers, and limits.
- Evidence-gap cards identify regions, outcomes, or instruments that the current corpus cannot compare.
- Mixed directions and sparse coverage remain explicit.
Citation-coverage checks compare substantive synthesis against the independent sources represented in the retrieved set, with repeated records from the same paper suppressed. Direction differences stay visible in L2, comparable fields align in L3, and contextual variation remains separate in L4.
T2 also degrades by section:
- the evidence base and instrument inventory can render if consensus text times out;
- a sparse instrument remains visible as limited or unsupported;
- mixed directions remain disagreement;
- a failed comparison can return paper-level evidence and a narrower synthesis; and
- a gap statement records the condition that the corpus could not satisfy.
T3 deterministic answers and T4 boundaries
T3
- Uses an allowlist of predefined graph-query templates.
- Extracts approved parameters before querying.
- Returns statistics, lists, or chart payloads from the data service.
- Applies language-specific wording after values are known.
- Keeps paper counts, journal distributions, and concept coverage numerically aligned across Chinese and English.
T4
- Covers future prediction, unsupported counterfactuals, transient external facts, absent local-policy knowledge, casual conversation, and system probing.
- Explains the supported corpus scope.
- Offers an evidence-searchable reformulation when possible.
- Does not add unrelated papers to fill an unsupported answer.
- Receives no programmatic confidence score.
Section-level SSE status
- The first event identifies the answer type and expected section list.
- The browser builds a fixed skeleton.
- Each section reaches ready, error, or skipped status with its payload or reason.
- Independent retrieval and generation work can finish in parallel.
- The terminal message reconciles delivered sections, citations, evidence cards, related papers, and confidence.
- The browser persists the conversation after terminal reconciliation.
Event arrival order does not change section positions, citation identities, or the final evidence collection. If the network fails first, the current message records a connection error and preserves completed sections.
React web product
The React product connects several research entry points to one evidence model.
Research surfaces
- Wiki hierarchy: root themes, driver types, and normalised concept detail pages.
- Concept detail: overview, mechanism, geography, timeline, taxonomy tags, and relevant papers.
- Paper and finding views: metadata, summaries, evidence text, page context, methods, outcomes, and graph links.
- Evidence library: paper and finding modes, search, structured filters, and pagination.
- Knowledge graph: search, node-type filters, node-centred expansion, selected-node detail, and navigation to related papers or findings.
Connected evidence navigation
- Citation cards and graph nodes present the same stable records.
- An in-text citation opens source span, method, and effect fields.
- A graph action centres the related paper or evidence node and requests the needed neighbourhood.
- Returning to the answer preserves the conversation and completed sections.
- Wiki concepts, paper details, and evidence-library records share identity rules across surfaces.
Conversation and feedback state
- Conversations persist on the server with a limited browser cache.
- A one-time migration supports earlier local history.
- Conversation summaries and full messages load separately.
- Viewing history and editing a conversation remain distinct actions.
- Messages persist after the streamed answer reaches a reconciled terminal state.
- Feedback retains the question, visible sections, citations, and confidence snapshot from that moment.
Bilingual interaction
- Chinese and English cover navigation, cards, questions, answers, refusals, metadata templates, and taxonomy display names.
- Language switching preserves stable record identity.
- Chinese concept names pass through completion and audit stages.
- Original source text, translated display names, and canonical English concepts remain distinct.
- Bilingual semantic expressions broaden retrieval while preserving source language.
Caching and lazy loading
- The server is the source of truth; browser caching supports migration and local resilience.
- Stable-read cache keys include language, query parameters, and data version.
- Governance changes update the relevant version so stale concept and taxonomy reads expire.
- Conversation writes use the server response as confirmation.
- The force-graph engine, ingestion screens, and administration screens load on demand.
- Search input is debounced, graph expansion fetches selected neighbourhoods, and paper detail loads separately from result lists.
- Each lazy module has its own error boundary.
Interface states and accessibility
| Concern | Implemented behaviour |
|---|---|
| Empty results | Evidence lists report that no records matched |
| Partial failure | Paper and graph errors preserve surrounding content and existing canvas state |
| Ingestion state | Pre-check, upload, interrupted, reconciliation mismatch, and completed states remain distinct |
| Retry clarity | Copy and actions identify retryable work and data that remains safe |
| Navigation | Current-page state is exposed to assistive technology |
| Controls | Icon-only actions have readable labels and expandable menus expose open state |
| Keyboard and focus | Submission and administrative input support keyboard use, visible focus, disabled states, and busy states |
Administration and data governance
The administration workspace combines ingestion state, evidence quality, and semantic governance.
Ingestion operations
- Upload pre-checks validate file type, batch structure, and duplicate papers before expensive work begins.
- The run view displays batch identity, stage status, append-only logs, expected counts, actual counts, and exceptional sets.
- Interrupted work retains its last completed stage and targeted recovery actions.
- Completed processes still pass terminal reconciliation before the batch becomes complete.
Evidence and taxonomy review
- Low-confidence records and critic flags enter a queue grouped by paper.
- Review combines the structured claim, neighbouring findings, source span, page context, and flag reason.
- Corrected acceptance returns a record to the trusted path for later graph materialisation.
- Rejection removes a record from the trusted candidate set while preserving review state and reason.
- Taxonomy review starts with concept name and description, then adds representative findings when confidence remains low.
- Paper-level labels use the governed vocabulary after concept classification stabilises.
- Chinese-name review compares canonical English names, proposed translations, and representative evidence.
Deduplication, undo, and derived data
- Candidate pairs and connected clusters show lexical similarity, scientific-text representation, taxonomy placement, representative findings, and the proposed canonical name.
- Structural guards block merges across incompatible root categories or semantic groups.
- Applied merges save relationship and property snapshots.
- Governance history records candidate evidence, the decision, graph changes, and undo status.
- Undo performs a structural check and pauses for human confirmation when later governance has changed the same concept.
- Successful restoration rebuilds dependent paper counts, concept neighbourhoods, taxonomy placement, and cross-paper summaries.
Configuration, access control, and feedback
- Stage-specific configuration supports different quality and cost profiles for extraction, critic review, classification, and translation.
- Configuration versions attach to run records for audit.
- Administrative writes require a protected authorization context.
- Ordinary evidence reads remain separate from governance mutation.
- Public browser resources contain no administrative authorization material.
- Answer-level feedback stores the question, verdict, visible sections, citations, and score snapshot.
- Page-level feedback records qualitative product issues.
- Protected filtering, inspection, and export support later quality review and calibration.
- Feedback does not rewrite existing evidence or immediately train the live answer model.
Quality control and anti-hallucination design
| Layer | Control | Visible or operational result |
|---|---|---|
| Source anchor | Exact matching plus restricted fuzzy re-anchoring | Unresolved evidence remains outside the trusted corpus |
| Semantic review | Independent critic checks direction, numbers, method descriptions, qualifications, and sufficiency | Flags retain reason and enter human review |
| Canonical identity | Lexical, scientific-text, taxonomy, model-assisted, and human signals govern concepts | Equivalent concepts can merge without hiding semantic conflicts |
| Answer evidence | T1 and T2 core sections accept retrieved records only | Citations and evidence cards validate against each other |
| Citation coverage | Substantive claims pass a coverage check | Weak support becomes a skipped section, gap, narrowed scope, or T4 refusal |
| Deterministic data | Counts and normalised records remain separate payloads | Generated text cannot rewrite statistics |
| Numeric normalisation | Source value, unit, comparable form, and direction remain linked | Failed normalisation enters review |
| Confidence scope | T1 and T2 use current corpus and delivered sections | T3 and T4 receive no composite score |
| Governance recovery | Merge snapshots and derived-data rebuilds follow identity changes | Incorrect canonicalisation can be reviewed and undone |
Taxonomy and deduplication are answer-quality controls because retrieval depends on concept identity. Fragmented equivalents understate coverage and community summaries. Incorrect merges create false consensus. No single similarity value authorizes a graph mutation.
Engineering and testing
Contract and execution boundaries
- One repository preserves independent dependency boundaries for the web application, Agent service, and graph and data service.
- Pydantic models, JSON Schema, typed section payloads, and taxonomy versions define service contracts.
- Pipeline mutations use durable task state and a serial execution guard.
- Read-heavy retrieval and independent answer sections use bounded parallelism followed by terminal reconciliation.
Test matrix
| Test level | Coverage |
|---|---|
| Pure transformations | Source spans, fuzzy re-anchoring, statistical normalisation, concept canonicalisation, taxonomy mapping, result merging, and section payloads |
| Services | Task transitions, restart recovery, stage reconciliation, permission boundaries, retrieval fallback, feedback persistence, and deterministic metadata templates |
| Integration | Router and retrieval wiring, pipeline execution, graph mutation and undo, SSE event order, terminal conversation persistence, and bilingual answers |
| Frontend | T1 and T2 skeletons, loading and partial-error states, citation navigation, production builds, and internationalisation audit |
| Contracts | Shared field and state interpretation across the data service, Agent, and browser |
| Release | Real entry point, stream flushing, management protection, feedback submission, and static-asset freshness |
Historical failure cases retained as regressions
- Structurally valid data can reference the wrong source passage.
- A graph merge can succeed mechanically while damaging concept meaning.
- A process can exit after omitting a paper.
- An event stream can close with a section still loading.
- A Chinese interface can expose an untranslated label.
Controlled batches exercise completion, partial failure, interrupted recovery, and repeated execution. Graph tests compare relationships before and after merges, taxonomy compatibility, dependent-data rebuilds, and undo history. Question-answering tests fix retrieval input and inspect routing parameters, per-paper caps, schemas, citation coverage, confidence scope, and refusal boundaries. SSE tests cover out-of-order arrival, one-section failure, skipped sections, terminal reconciliation, and partial output after network interruption.
Production builds check module resolution, asset splitting, and page generation. Wording and bilingual tests protect refusal scope, numerical consistency, and research tone. Internationalisation scans cover user-visible text.
Alibaba Cloud deployment and reliability
Production topology
- The production stack runs on Alibaba Cloud ECS in Singapore.
- The React application is published as static assets.
- Two FastAPI services run as independent processes behind a Caddy reverse proxy.
- Public traffic enters through protected ingress.
- Application services bind to the loopback interface.
- The proxy presents one browser origin and routes application traffic internally.
- Agent stream buffering is disabled so section events flush immediately.
Proxy and cache behaviour
- Public builds contain no internal service locations.
- Content-hashed bundles receive long cache lifetimes.
- Entry documents revalidate.
- Large graph and taxonomy responses use compression.
- Event streams bypass buffering.
- Full-path stream checks verify that intermediary behaviour preserves section progress.
Operational safeguards
- health checks;
- process supervision and independent service restart;
- boot recovery;
- separate service logs;
- protected administrative reads and writes;
- static-asset freshness checks; and
- release verification for the frontend, service schemas, streaming, and feedback submission.
Failure diagnosis and recovery
| Boundary | Diagnostic responsibility | Recovery behaviour |
|---|---|---|
| Reverse proxy | Confirms whether a request reached the entry point | Keeps static, graph, and stream traffic behind one origin |
| Agent service | Records classification and streaming state | Restores read-only dependencies after process restart |
| Data service and Neo4j | Records evidence retrieval, graph access, and ingestion state | Reconciles stale tasks before new governance work |
| Process supervision | Detects unavailable application processes | Restarts one failed service without changing graph data |
| Ingestion recovery | Marks abandoned work as interrupted | Waits for log and count review before retrying high-cost stages |
Request correlation, stage identity, and error categories separate browser-connection failures from downstream evidence-query failures. Minimal health checks distinguish unavailable processes without triggering expensive graph or model work.
Layered access control
- Protected ingress guards the public entry.
- Application services remain bound to loopback.
- Administrative mutation requires a second authorization boundary.
- The Agent keeps read-only access to research source data.
- Authorization material stays out of public resources and operational logs.
Static delivery, two application processes, same-origin proxying, health checks, caching, compression, stream flushing, supervision, and boot recovery preserve the service boundaries used during development.
Outcome
The deployed platform connects one complete research workflow:
- papers pass through the fifteen-stage pipeline into reviewable evidence records;
- canonical concepts and taxonomy relationships are materialised in the knowledge graph;
- T1 to T4 routes organise answers under retrieval and section-contract constraints;
- the React product connects Wiki concepts, papers, findings, graph exploration, conversations, ingestion, and governance; and
- researchers can follow citations, evidence cards, and graph relationships back to the source paper.
The main engineering tradeoffs remain visible:
- structured sections add schema and renderer work while supporting partial completion, deterministic cards, and stable evidence links;
- fifteen observable stages add operational state while enabling targeted retry, recovery, and count reconciliation;
- conservative routing narrows visible coverage while keeping corpus boundaries explicit; and
- human review adds governance work that protects concept identity, taxonomy relationships, and cross-paper summaries.
The production result is a traceable research system. Concept definitions connect to measured corpus coverage. Cross-paper synthesis exposes independent sources, method context, and disagreement. Evidence cards return to precise source locations, and concept governance retains decision and undo histories. The full path runs from supervisor-guided research requirements through document processing, evidence modelling, constrained question answering, and a live internal production environment.