Customise, run and save LLMs utilizing OLLAMA and the Modelfile

On this article, I’ll present you easy methods to use the Modelfile in Ollama to vary how an current LLM (Llama2) behaves when interacting with it. I’ll additionally present you easy methods to save your newly personalized mannequin to your private namespace on the Ollama server.

I do know it might get a bit complicated with all of the completely different ”llamas” flying round. Simply keep in mind, Ollama is the corporate that lets you obtain and regionally run many various LLMs. Whereas, Llama2 is a specific LLM created by Meta the proprietor of Fb. Aside from this relationship, they don’t seem to be linked in every other manner.

When you’ve by no means heard of Ollama earlier than I like to recommend that you simply take a look at my article beneath the place I am going into depth on what Ollama is and easy methods to set up it in your system.

What’s a modelfile?

In Ollama, a modelfile refers to a configuration file that defines the blueprint to create and share fashions with Ollama.

The modelfile incorporates info comparable to,

1. Base Mannequin Reference. All modefiles will need to have a mannequin that they use as the idea for any new mannequin
2. Parameters. These specify issues such because the temperature, top_k and top_p that needs to be utilized to the brand new mannequin. We’ll discuss extra about these in a while.
3. Template. This specifies what the ultimate immediate shall be that’s handed to the LLM.
4. System. We are able to use this command to find out how the system behaves total.

There are different properties the modelfile could make use of however we’ll solely be utilizing those above. There’s a hyperlink to the Ollama documentation on the finish of the article if you wish to discover out extra about this.

The bottom mannequin

The very first thing we have to do is establish an current mannequin so we will look at its properties and make the adjustments we need to it. For that, I’m going…