HTML and XHTML will give you the building blocks of your website. Learning them isn’t hard. On the contrary, it can be fun to discover what new things you can do with your website as you move along.
But why learn this when you can create websites easily by clicking your cursor around in a wysiwyg program, like frontpage? The most important reason is control. That’s right: wysiwyg editors don’t offer complete control of your website. They will do for most basic needs. But you’re bound to encounter a design problem that just can’t be done with it. Then you’ll be glad you know some HTML so you can look at how your website is really made, and edit it the way you need.
Also, if you plan on making your website more dynamic, as you can do with php for example, you’ll need at least some basic knowledge of HTML. Let’s say you found some nice script that creates a counter on your page showing how many visitors you had. Unless you know where to put it, it won’t be made for you.
But as I said, it’s easy and fun. At least it was for me. ;-)