If you're mounting it at bootup via fstab then it's probably owned by root with a restrictive set of rights. You should just need to change either the owner of the mount or the permissions, or both if ...
You should probably edit your fstab file. I believe it is located in your /etc directory. You should have no problems with the fat32 partitions as almost all kernels can see them easy but some have ...