When you fine-tune it, do you train just the head/last few layers or do you also...

		titaniczero on May 3, 2023 \| parent \| context \| favorite \| on: Replit's new Code LLM: Open Source, 77% smaller th... When you fine-tune it, do you train just the head/last few layers or do you also unfreeze the model afterwards and retrain the whole model with a very small LR for a few epochs?