Can I withdraw funds directly from my USD wallet?

No, your USD wallet is just for receiving crypto. You need to sell your crypto into Naira to withdraw your funds in Naira.