All the phyla are given this category because they comprise animals with a unique 'Bauplan' or body plan, so you're correct.
Of course, the degree of "uniqueness" necessary to recognize a phylum depends ultimately on the decision of the biologists, so you may find that the numbers are somewhat different.
According to the Animal Diversity Web, there are 32 phyla:
http://animaldiversity.ummz.umich.edu/site/accounts/classification/Animalia.html#Animalia
whereas Wikipedia lists 36 phyla: http://en.wikipedia.org/wiki/Animal
To see their characteristics, which are too many to list here, just follow the links. I recommend the Animal Diversity Web site. By the way, even though they're quite different in their organization, most of these phyla comprise what you and I would generally call "worms".
These are some of the most unusual:
Phylum Orthonectida (orthonectids) and Phylum Rhombozoa (rhombozoans): these are animals with very few cells and peculiar (sometimes parasitic) life cycles.
Different types of weird "worms"
Phylum Gnathostomulida (gnathostomulids)
Phylum Chaetognatha (arrow worms)
Phylum Gastrotricha (gastrotrichs)
Phylum Phoronida (phoronids)
Phylum Nemertea (proboscis worms)
Phylum Sipuncula (sipunculan worms)
Phylum Onychophora (velvet worms)
Phylum Kinorhyncha (kinorhynchs)
Phylum Priapula (priapulans)
and some that don't look like worms:
Phylum Loricifera (loriciferans)
Phylum Rotifera (rotifers)
Phylum Tardigrada (tardigrades) - these look like microscopic bears, quite cute!