I'm self employed and will keep some customers here in NZ. Who do I pay tax to? What about charging GST?

You will need to keep you business identity in NZ. You pay tax in whichever country the business is created. If you invoice your NZ customers and they pay freight, then pay their invoice into your australian bank account (beware - international transfer fees!) then you pay the Australian govt tax etc - but if you keep the accounts/transactions in nz then you will need to continue NZ Tax and GST payments. Another point is Taxes vary - best site is ATO.gov.au

All Nz sourced income must be lodged in a non-resident NZ tax return. The same income must be recorded in an Oz tax return as Overseas income but you get a tax credit for NZ tax paid so you are not double-taxed.