model quality

2 items

RESEARCH↑ trendingReddit r/LocalLLaMA·5/6/2026

Quality comparison between Qwen 3.6 27B quantizations (BF16, Q8_0, Q6_K, Q5_K_XL, Q4_K_XL, IQ4_XS, IQ3_XXS,...)

This content compares the quality of various Qwen 3.6 27B model quantizations using a custom chess game test to find the optimal one for 16 GB VRAM setups. It evaluates the models' ability to track board states and generate accurate SVG images of the chessboard.

VRAM Benchmarking quantization model quality

Quality comparison between Qwen 3.6 27B quantizations (BF16, Q8_0, Q6_K, Q5_K_XL, Q4_K_XL, IQ4_XS, IQ3_XXS,...)

RESEARCHarXiv CS.LG·22d ago

Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels

This study investigates the impact of post-training quantization on Large Language Models (LLMs) quality, revealing that compression can lead to bias emergence. 3-bit quantization caused 6-21% of previously unbiased items to develop new stereotypical behaviors in models like Qwen2.5-7B, Mistral-7B, and Phi-3.5-mini. This follows a clear dose-response pattern across various precision levels.

Model Compression LLMs quantization model quality