Long distance trains in New Zealand are old, slow and antiquated, but often travel through lovely country. They are not generally competitively priced and passenger lines are limited to:
Auckland to Wellington
Picton to Christchurch
Christchurch to Greymouth
If you do the Rail Trail in Otago, there is a train from Dunedin Station to Middlemarch.
To get across Cook Strait from Wellington (North Island) to Picton (South Island):
Use the Interisland ferry or Bluebridge ferry. The Bluebridge is marginally cheaper.
You have to pay for your bike separately. Book on line a day or so earlier and you can generally get a cheaper deal than just rolling up on the day.
The entrance through the fjord into Picton from Wellington is particularly beautiful and a great introduction to the South Island.
In the North Island you can use Intercity Buses. They are not always bike friendly and the company will not book your bike unless you check it with the driver first. If he is grumpy, tell him politely that you will contact the company and he will be a good deal more courteous.
In the South Island you have (along with a few local competitively priced companies) Atomic Shuttles. These are almost always bike friendly and have racks for bikes. They travel all over the south which makes them invaluable if you need to get out of a remote spot or it's raining or you need to get somewhere in a hurry.
New Zealand at present has several airlines and some good deals for local air travel.
Try Air New Zealand, Virgin, or Quantas.
This is often the cheapest way to travel between main centres in New Zealand. Bikes are free (at present) providing they are wrapped or boxed and minus pedals with handlebars turned. I always carry a groundsheet with brass eyelets to tie around my bike frame with string. It looks ugly but gets me on the plane (or bus).
Travel Well!