Hex editing portraits

I was wondering if anyone by chance had figured out how to switch your portrait via hex editing, I managed to sort of do it by editing the .gsi file, all of the information in there pertains to the preview screen before you load the save file, which is nice, I was able to swap between some human portraits, however I have had no luck finding where your actual portrait ingame portrait is stored in the .tfaf file, I have tried to make completely identical characters except for their portrait, and then compare them against eachother in a hex editor (This is how I discovered the number to change in the .gsi file) but the save files have a whole ton of differences that make it impossible to tell, my experience with hex editing is minimal so I have no way to narrow it down, and Cheat Engine could only achieve a temporary change that doesn't save (I text searched for the portrait image name and changed it that way)

Why don't I just use a custom portrait?
I don't really want my save to rely on custom content like that, mainly I'm trying to do this so I can have a Half Elf with the human sprite (Lore wise it does make sense that they use elven sprites but they can look pretty much completely human too) so I can make my character feel more unique without affecting the rest of the game.

Any help would be very much appreciated.