The boat arrival surprised authorities last week when it reached mainland Australia instead of the usual destination of Christmas Island.
Now, 38 members of the group, who spent an estimated 44 days at sea, are being flown home to Sri Lanka after earlier being transferred to Christmas Island.
They are heading to the Sri Lankan capital Colombo on an Australian Air Force plane.(ABC)
Refugee groups claim the Sri Lankans have been denied access to lawyers.