Game Review NLP
SoloSep 2025 – Dec 2025
- 60.7%
- top-1 across 1,232 titles, against a ~0.08% random baseline
- 94.9%
- sentiment accuracy, 0.949 F1, on the natural distribution
- 67 MB
- per model after int8 quantisation, down from 268 MB
- ~120 ms
- typical inference in the browser
The problem
Two questions over the same 4.2 million Steam reviews. The easy one: is this review positive or negative. The hard one: which of 1,232 games is it about, from the text alone, with no metadata.
Approach
- Fine-tuned DistilBERT separately for each task, keeping the 1,232-way classifier to games with at least 501 reviews so every class had enough signal to learn from.
- Exported both to ONNX and quantised the weights to 8-bit integers, cutting each model from about 268 MB to 67 MB, small enough to send to a browser.
- Measured the cost of quantisation instead of assuming it was free. Game identification fell 1.4 points, from 61.4% to 60.0%. Sentiment lost nothing.
- Serve them client-side through ONNX Runtime in a Web Worker, so the demo has no server to sleep, no cold start and no running cost.
A headline accuracy that needed its denominator
60.7% reads as a weak number until you know it is a 1,232-way classification where chance is roughly 0.08%, which makes it about 760 times better than guessing. The errors are also the interesting part: the model's near-misses are usually sibling titles in the same series, which is what you would expect from a model reading vocabulary and tone rather than memorising names.
Stack
- PyTorch
- HuggingFace
- DistilBERT