Aye Judy, i just read your posts, what a challenge huh - now listen gf, your are NOT a bad mother/grandmother, a LOT of other girls would give the world to have a mom like you.
I totally understand you, do yourself a favor, and follow your heart! make absoluty sure that this is what you want, be fair to yourself!