Well if you really want to get into it, lets start at the top. I've highlighted the tree to corallimorphs.
Phylum Cnidaria
Within Cnidaria you have 3 basic groups
_________________________________
Corals - Class Anthozoa
Hydroids - Class Hydrozoa
Jellyfish - Scyphozoa, Cubozoa, and Staurozoa
Class Anthozoa then splits into two:
___________________________________
SubClass Alcyonaria which has 8 way symmetry (soft corals)
SubClass Zoantharia which has 6 way symmetry (hard corals, anemones, and various "polyps")
Zoantharia is split up into many orders a few noteworthy ones are:
___________________________________________
Scleractinia (Stony Corals / Hard Corals)
Actiniaria (Anemones)
Corallimorpharia (what we are talking about)
So you see, Corallimorphs are no more related to anemones than they are to hard corals. They look very similar to anemones to real people, but to scientists there is no mistaking them.
As far was what the scientists are looking at that makes them so different. I don't know exactly, but I'd bet that even if I did know the exact details, it would not really help you distinguish them visually, ( their insides work a bit differently ) and would require lots of big words that no one other than the scientists understand anyway.
From a practical standpoint: Corallimorphs tend to be smaller and have short tentacles.