I recently told someone I felt guilty spending so much time knitting and crocheting, and she challenged me to think about this in terms of purpose.
What I realized is that most of the things I make are donated, so someone somewhere is (hopefully) keeping a bit warmer and more comfortable because of a blanket I made.
If I make something that I keep, such as a pillow cover, and I enjoy looking at it, and feel proud, then that was well worth my time and effort.
If I make a gift, that is something that makes the recipient happy, so definitely a good feeling all around..
Sure, I could be doing housework or plenty of other things, but knitting or crocheting for a purpose is nothing to feel guilty about.