The author just made an update that makes the generation much better, even with the 7B model:

https://twitter.com/ggerganov/status/1634310199170179075

I tried it out myself (git pull && make) and the difference in results are day and night! It's amazing to play with, although you should prompt it differently than ChatGPT (more like the GPT-3 API).

parameter tuning is pretty necessary, according to anecdotes. People on twitter have got good results by changing the default parameters.

For 13b and 30b, it really needs high temperature to produce good outputs.