- Category
- Research & Data
Find the Best Temperature for a Local Model
Evaluate a local model’s use of web search to find a Deno documentation command, then tune its temperature and system prompt.
By Threads — Published on — Modified on
Prompt template
0 copies
- Format
- Text
- Inputs
- Local_model (text) · Required Web_search_and_fetch_tools (text) · Required
- Outputs
- Evaluation_response (text)
Evaluate a local model to find a suitable temperature for answering a question using web search:
1. Set up the model with no system prompt.
2. Give it access to web search and fetch tools.
3. Ask: “How do I initialize a Svelte project with Deno?”
4. If alignment seems difficult, try a higher level of reasoning.
5. Evaluate the results. If necessary, add a minimal system prompt with instructions for web search and tool use.
6. Fine-tune the temperature until the model gives the command from the tutorial in Deno’s web documentation. Lower temperatures can make it adhere more closely to web results, but temperatures that are too low may cause it to get stuck in a loop. Find the sweet spot.
7. Adjust the system prompt to align the response’s shape and format, while accounting for the format the model was trained to use.
Comments — 0