01 - Select
Start with the context.
The prototype identifies anchor-like evidence within a long input and produces a reduced context for the underlying language model.
Experimental long-context inference research
Reduce long-context overhead. Measure quality honestly.
SpiralCoreAttention is an experimental context-reduction layer for open-weight LLM inference. It selects a smaller relevant subset of a long input before inference, then measures the latency, memory and quality trade-off against the full-context baseline.
~24%
lower p95 latency
Internal long-context test
~27%
lower peak VRAM
Internal long-context test
17K
input tokens
Synthetic evaluation prompts
10
test cases
Manual output review required
Current evidence
An internal signal, not a production promise.
The current prototype was evaluated on a single RTX 5090 using Qwen2.5-1.5B-Instruct and a synthetic long-context set. Results are workload-specific and require independent validation before any production decision.
MetricBaselineSpiral
p95 latency1677.67 ms1269.83 ms
Mean peak VRAM8.836 GB6.466 GB
10 internal cases, approximately 17K input tokens. Outputs require side-by-side manual review. 70B, multi-GPU and vLLM validation are not yet complete.
02 - Compare
Keep the model unchanged.
Baseline and Spiral runs use the same model and inference settings. The comparison isolates the context-reduction trade-off.
03 - Review
Treat quality as a first-class metric.
Latency and VRAM are only useful if the output remains acceptable. Every evaluation includes baseline-versus-Spiral output review.
Technical validation pilot
A controlled 7-day benchmark inside your environment.
A pilot is designed for teams with document-heavy or long-context open-weight LLM workloads. It is an evaluation, not a production deployment or a guaranteed-performance claim.
Fixed pilot fee: $500
01
Choose the workload
Select one long-context endpoint and provide 10-30 anonymised prompts or a synthetic equivalent.
02
Run a controlled comparison
Use the same model, decoding settings, token limits and GPU environment for baseline and Spiral.
03
Review the evidence
Receive latency, VRAM, token, fallback and side-by-side output comparison results.
No production access required
No raw customer-data export required
No production integration required
Project status
Early-stage, transparent about the boundaries.
SpiralCoreAttention is a research prototype. The next validation target is larger open-weight models and controlled vLLM or multi-GPU environments.
Validated internally
- Single-GPU inference on RTX 5090
- Qwen2.5-1.5B-Instruct
- Synthetic long-context workload comparison
- Latency, peak VRAM and fallback reporting
Still to validate
- 70B models and multi-GPU serving
- vLLM integration
- Customer-specific quality criteria
- Production traffic and deployment