9600 Baud rate should be fine, unless you have a very long serial line (and then I mean 500 to 1000 meters long).

You can change the speed using

smitty chtty

select the correct tty device, and set the new speed. I would suggest 4800. If the problem continues, lower by half again, etc..

However, more probably there is a problem with your cable, or another setting.