Quantize with llama.cpp

./llama-quantize model-f16.gguf model-q4_k_m.gguf q4_k_m