Harness-led Fine-tuning
Where I landed when thinking about how to start fine-tuning: evaluate the harness first, then tune the model around its shortcomings.
This isn’t a guide, but where I landed when thinking about how to start fine-tuning.
The usual reasons to fine-tune are to:
- Reduce costs.
- Improve performance.
- Keep control of your data (data sovereignty).
But fine-tuning requires some work before you start. And no, I’m not saying that you need to label data and whatnot. You need to see where the system surrounding the model falls short, then tune the model around it. You may as well call this harness-led fine-tuning.
So here’s what I’m doing to fine-tune a small model to beat a frontier model at date extraction.
First, I started by naively sending some text to LFM 1.7 and telling it what I expected in return, aka structured output. The model did well in many cases where the text wasn’t cluttered and messy. But it failed to pick the correct values otherwise, especially with numbers and dates.
I noticed right then that this approach wasn’t going to work. I needed a proper way to test the models, so I searched the internet and found the Structured Output Benchmark (SOB). It’s a great way to test the harness against different kinds of data. So I put my data away for a bit and attempted to build a small harness and evaluate it against the benchmark. It scored okay, but it was still way behind what I wanted.
Being a TypeScript addict, I ported the benchmark to TypeScript using promptfoo as the runner, so I only rewrote the assertions, or “metrics,” if you will.