Loading…
20
Shoutout to the Devs Who Explained Why My Model Kept Outputting the Same Response
I spent last Tuesday debugging a small text model that repeated "I am here to help" after every input, no matter what I typed. I assumed it was a data problem, so I kept feeding it new examples for 6 hours. A dev on the project finally asked if I had set the temperature to zero, and yeah, I had, back when I was testing something else 2 weeks ago. I thought temperature only affected creativity, but it also controls whether the model can pick anything except the single most likely next word. Has anyone else wrecked a build with one leftover setting they forgot to change back?
1 comments
Log in to join the discussion
Log In1 Comment
ray_carr19d ago
Reminds me of when I spent a whole weekend tweaking prompts on a chatbot that kept giving me super short, boring answers. Turned out I had max tokens set to like 20 from some quick test days before and never changed it back. I felt pretty dumb when I finally noticed it buried in the config file. Those leftover settings are sneaky like that.
6