I am doing the same thing as you. I am currently playing with gpt2 since it’s extremely small. Then when I am comfortable I plan to play with gptj or other ~7b models. Then finally I kinda want to try something with a 20b model as a final big project maybe since I saw you can fine tune it on 4090.