Better in what way? Madang has very pretty coral reefs and walls, and the pelagic life at Planet Rock can be prolific when dived at the right time. However many times it's not safe to dive due to strong currents.
Kavieng has more WWII plane wrecks and certain sites in current are better for pelagics like silver tip sharks since they do feeding here. I'm sorry I can't tell you which is cheaper to dive since my prices were quite a while ago.
I like Madang for the colors and variety of fish, but Kavieng has the channels (thus a few more sites with thick fish life)and plane wrecks. Nusa Resort, Malagan Hotel and Lissenung are the 3 places I'm familiar with in Kavieng. All of them can reach the marquee sites. Dorian of Scubaventures (Nusa) knows more of the plane wrecks since my buddy and I went there for that.
Madang, I only dived on a liveaboard and at the Jas Aben. I did stay at the Madang resort but did not dive with the ops there.
If you really have too much time, do the trek from Goroka to Madang and then dive there. Just hire a guide because the security can be a bit iffy. Good luck and safe travels!