GreenLlama
Compressed, FPGA-accelerated LLM with a fairness gate
A compressed, FPGA-accelerated LLM that pairs aggressive efficiency with a responsible-AI gate: 8× smaller, 43× less power, and 56% less measured bias — because a configuration that saved power but degraded fairness never shipped.
8× compression
43× power reduction
56% bias reduction
Stack
PyTorch · Quantization · Verilog · FPGA toolchain · BOLD bias eval
the problem
Shrinking an LLM to run on edge hardware saves power — but compression quietly amplifies the model's biases, and almost nobody measures it.
how it works
Start with a large language model, then quantize and compress it ~8× smaller. Inference runs on an FPGA (implemented in Verilog) for a 43× drop in power versus the baseline. Crucially, every compression configuration is scored on bias — using the BOLD benchmark — right alongside latency and power, so fairness is a metric you optimize, not something you discover in production.
bias is a first-class metric
Compression doesn't degrade a model evenly — it amplifies existing biases in ways that don't show up in accuracy numbers. We measured bias at every configuration and blocked any that improved power or size but regressed fairness past a threshold. The result: 56% less measured bias than the uncompressed baseline while still hitting the efficiency targets. Treating fairness as a gate — not a report filed afterward — is the whole point.
my role
I worked on quantization — finding compression configurations that held up under the efficiency targets — and built the bias-evaluation pipeline that scored each one on the BOLD benchmark, so fairness could sit next to power and latency as a first-class objective.
key decisions
Fairness as a gate, not a report
Any configuration that saved power but regressed bias past a threshold was blocked. Bias sat next to latency and power as a hard constraint — not a footnote discovered after the fact.
Efficiency proven on hardware
Inference in Verilog on an FPGA — not a GPU with the power dialed down — so the 43× power number reflects the edge hardware these models actually have to run on.
outcomes
- 8× model compression
- 43× lower inference power on FPGA vs. baseline
- 56% reduction in measured bias (BOLD) vs. the uncompressed baseline
- Bias evaluated as a hard constraint at every compression configuration
up next
Smart Sensing Vest
Wearable gesture-recognition vest