7 июл. 2024 г. · Is my understanding of temperature, top_p, max_tokens, frequency_penalty, presence_penalty, and stop, good? Would you recommend any other ...

30 апр. 2023 г. · In my experience it's better than top-p for natural/creative output. --top_k 0 --top_p 1.0 --tfs 0.95 --temp 0.7 were good for me. They also ...

23 июл. 2023 г. · Top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p ...

12 нояб. 2023 г. · [Asking for help]Is there a tutorial on how to adjust the parameters of llama.cpp? ... top_p of 0.95 means only "blue" and "the limit" will ...

18 июл. 2023 г. · Yep, typical sampling, not top_p! Glad to know that the people who know what they're doing are seeing the same results. Thanks!

29 июл. 2024 г. · When I look at the available options when serving a model via llama.cpp, I'm amazed. What options, possibly non-obvious options, do you like to use?

30 мая 2023 г. · I find this behavior surprising and enlightening. Having top_p small enough keeps it mostly coherent while making it have a really difficult time speaking ...

2 дня назад · Speculative decoding just landed in llama.cpp's server with 25% to 60% speed improvements. News.

15 нояб. 2023 г. · I've found that a bad preset can make a model significantly worse or golden depending on the settings.

14 дек. 2023 г. · If you're using llama.cpp, try setting min_p to 0.0 (instead of the default 0.05). In my experience there is something not working well with ...