Sayyidyna Yusuf (Alayhi salaam) returned the barter item as a hint to his father that what he treasured will also be returned back to him... only until after the father saw this - did he allow Yusuf’s brother to go with his brothers...